Frequently Asked Questions
The Centers for Medicare & Medicaid Services (CMS) sponsors the eCQI Resource Center, which provides a centralized resource for stakeholders engaged in electronic quality improvement. The eCQI Resource Center draws from existing tools and resources whenever possible to minimize duplication of efforts across programs and websites. The site contains information put forth by federal health agencies such as CMS, the Office of the National Coordinator for Health Information Technology, the National Library of Medicine, and the Agency for Healthcare Research and Quality.
You can find resources for use at various stages of the electronic clinical quality measure (eCQM) lifecycle, digital quality measures, and information about standards and tools to support eCQI, including clinical decision support. The eCQI Resource Center also supports users looking for links to external resources related to eCQMs and data reporting, like ONC Project Tracking System (Jira), the Measure Authoring Tool, Bonnie QDM, Bonnie FHIR, the Measure Authoring Development Integrated Environment, and the Value Set Authority Center.
The eCQI Resource Center includes:
- eCQM specifications and supporting materials
- eCQM Data Element Repository
- Event listings and news related to eCQMs and eCQI
- CMS Quality Reporting Document Architecture (QRDA) Category I and III Implementation Guides, Schematrons, and Sample Files
- Digital Quality Measures
- Measure Collaboration Workspace
- Standards and data models used in conjunction with eCQMs and eCQI such as Clinical Quality Language, Fast Healthcare Interoperability Resources®, Health Quality Measure Format, Quality Data Model (QDM), and QRDA
- Tools and links to resources used in conjunction with eCQMs and eCQI
There is information for implementers located throughout the eCQI Resource Center. The Implementation tab of the Get Started with eCQMs page provides a list of key resources.
Users may compare individual eCQM versions to identify changes between available versions. Navigate to the eCQM you would like to compare by using the "Find an eCQM" search on the home page or by selecting a reporting/performance year and an eCQM title from the Eligible Hospital or Eligible Clinician eCQM tables. From the individual eCQM page, "Select eCQM Years to Compare" and the orange "Compare" button.
To set up an account on the eCQI Resource Center, select “Manage Your Account” under the "Sign in" menu item on the upper right side of any page on the website. This opens the Log in page. Select the tab that reads “Create new account” and set up an account by providing your email address and desired username. You will receive a “Welcome” email with a temporary password. We recommend you go to “Reset your password” and enter the user name or email you used to set up the account and change your password. If you do not get an email, please email the eCQI Resource Center.
Users with active eCQI Resource Center accounts can receive updates on a particular topic of interest by subscribing to a page. Subscribing to a page provides the user with email notifications with the addition of new content or a change on that page. To subscribe to a page, log into the website, navigate to the webpage of interest (e.g., eCQI Tools & Key Resources, QDM, QRDA), and select "Receive updates on this topic" on top left part of the page. When you successfully subscribe to a page, you will notice that “Receive updates on this topic” has been replaced by “Stop receiving updates on this topic.” Select this link if you wish to stop receiving notifications.
You can manage your subscriptions by logging into the eCQI Resource Center and selecting “My account” at the upper right side of the page. Manage your subscriptions on the site by navigating to the “Subscriptions” tab. The “Edit” tab allows you check a box to opt-out of news and events emails. Select “Save” when updates are complete.
Note that not all pages allow subscriptions. If you have any questions on this feature, email the eCQI Resource Center team.
Users with active accounts can update their personal time zone. You can manage your eCQI Resource Center account by logging into the website and selecting “My account” at the upper right side of the page. The “Edit” tab allows you to select your time zone. Select “Save” when updates are complete.
When adding an event using the Outlook.com or Office 365 button you may receive the error
404. The event contains errors.
The event you have tried to add to your calendar contains invalid event information.
The error returned to AddEvent:
REST API is not yet supported for this mailbox.
This is an error caused by the security settings set for your mailbox and not caused by the event from the eCQI Resource Center. If issues persist, please contact your information technology department or local administrator.
See an online article for more information.
The CMS CBE endorses quality measures through a transparent, consensus-based process to foster health care quality improvement. CMS uses CBE-endorsed measures in some of its quality reporting and value-based purchasing programs. Note: not all CBE-endorsed measures are used in CMS programs and not all CMS measures are CBE-endorsed. The new CMS CBE has posted its inventory, the Submission Tool and Repository (STAR), with CBE numbers replacing the NQF number. The numbers did not change. CMS has started making the nomenclature change by using CBE numbers in a proposed rule, in the CMS Measures Inventory Tool (CMIT), and plans to update in the Qualified Clinical Data Registries (QCDR). The Electronic Clinical Quality Improvement (eCQI) Resource Center will be making the change from NQF number to CBE number throughout the site and within key resources in 2023 and finalizing all changes by 2024.
The MC Workspace is a collaborative portal that supports quality and information technology staff at measured entity organizations to help implement and use electronic clinical quality measures (eCQMs). The MC Workspace brings together a set of interconnected resources, tools, and processes to promote transparency and better interaction across stakeholder communities that develop, implement, and report eCQMs. The MC Workspace has three modules to support eCQM collaboration:
- eCQM Concepts module allows users to search for eCQM concepts suggested by others, comment on those suggested eCQM concepts, and suggest new eCQM concepts.
- eCQM Testing Opportunities module allows eCQM developers to announce eCQM testing opportunities and provides stakeholders with available opportunities to participate in eCQM testing.
- eCQM Data Element Repository (DERep) provides data definitions to aid in eCQM implementation and data mapping.
The MC Workspace User Guide provides detailed information on using all of the modules.
Users do not need an account to view information contained within the MC Workspace. You need an eCQI Resource Center account if you would like to subscribe to notifications, add a suggested eCQM concept, or add a comment on an existing suggested eCQM concept.
Please email us with any questions or feedback.
You may request the eCQI Resource Center submit your suggested eCQM concepts to CMS for consideration. These suggested eCQM concepts may help provide ideas for measurement not previously considered by CMS. The submission of suggested eCQM concepts is outside the pre-rulemaking process.
The eCQM DERep is an online searchable repository that provides all the data elements and definitions associated with eCQMs used in CMS quality reporting programs by eligible hospitals, critical access hospitals, and clinicians. The CMS DEL is the centralized resource for CMS assessment instrument data elements (e.g., questions and responses) and their associated health information technology standards, including the Minimum Data Set, Inpatient Rehabilitation Facility Patient Assessment Instrument, Functional Assessment Standardized Items, Outcome and Assessment Information Set, Long-Term Care Hospital Continuity Assessment Record and Evaluation Data Set, and the Hospice Item Set.
CQL is a Health Level Seven International® (HL7®) authoring language standard intended to be human readable. CQL provides the ability to express logic that is human readable yet structured enough for processing a query electronically. eCQM developers with access to the Measure Authoring Tool (MAT) and the Measure Authoring Development Integrated Environment (MADiE) can use the tool to author eCQMs using CQL.
CMS first implemented CQL beginning with the calendar year 2019 performance/reporting period eCQMs. CQL replaces the logic expressions previously defined in the Quality Data Model (QDM). Beginning with QDM v5.3, QDM includes only the conceptual model for defining the data elements (the data model).
CQL allows for a more modular, flexible, and robust expression of the logic. It allows logic to be shared between eCQMs and clinical decision support (CDS).
Explore these resources to get an overview of CQL and learn how to implement CQL.
- Learn more about the fundamentals of CQL and why it is important on the CQL Educational Resources page. Also find past presentations and webinars for the various audiences who might use CQL.
- Attend an upcoming Cooking with CQL, QDM, and FHIR® webinar. The sessions feature open discussion with subject matter experts on the MAT, show how to express eCQMs using CQL, and review questions. Visit the eCQI Events page to find registration information.
- Visit the CQL Formatting and Usage Wiki for introductions, tutorials, and walkthroughs.
- Visit the CQL Formatting and Usage Wiki Cooking with CQL Examples to review common questions and answers on CQL.
- Visit the Connect page to find out how to connect to various CQL resources.
A CQL library is a text document (.cql, .xml, and .json) that contains CQL expressions, definitions, functions, and other declarations that can be used across eCQMs. Each eCQM contains a primary library that defines the criteria used by the populations of the eCQM. The Health Quality Measure Format document references this library and defines the populations by identifying which expressions in the CQL library define each population (e.g., Inpatient Encounter, terminologies). eCQMs may include references to a shared library, which can be either referenced throughout a single eCQM or across different eCQMs (and even Clinical Decision Support rules) to share definitions and functions like "Hospitalization." Library sharing minimizes efforts with eCQM development and implementation.
For more information on libraries, refer to the Using Libraries to Shared Logic section of the CQL Implementation Guide.
You can use CQL with any data model. The examples in the Cooking with CQL, QDM, and FHIR® sessions are focused on using the Quality Data Model (QDM), which has a serialization as Quality Reporting Document Architecture (QRDA). QRDA is similar to the consolidated clinical document architecture (C-CDA) standard. QRDA and C-CDA are HL7 V3 standards.
Yes, you can use return, it is an arbitrary expression. So, whatever you want to return from the “from,” you can.
For example
define "Encounter with Assessment Result":
from
"Valid Encounter" Encounter,
["Assessment, Performed": "Specified Assessment"] Assessment
where Assessment.result in "Specified Assessment Result Valueset"
and Assessment.relevantDatetime during Encounter.relevantPeriod
return Encounter
There is tooling to support authoring, parsing, and validation of CQL. There are open source tools available to evaluate the Expression Logical Model (ELM) for both the JavaScript and Java platforms. There is also a .NET toolkit for ELM that can support translation and evaluation.
Developing the measures in Java, C#, or any other commercially supported languages would not achieve the goal of sharing the eCQM logic in a platform independent way, nor would it serve as a vehicle for communicating the intent of the eCQMs in the way that CQL/ELM does. Traditional programming languages do not support the required set of operations for expressing eCQM logic, and traditional query languages, though closer to the mark, still do not support important aspects like terminology and interval-based timing. Refer to the GitHub tooling repository for additional guidance.
We recommend using Visual Studio (VS) Code. VS Code has a CQL extension. We previously recommended Atom, a desktop general-purpose editor. However, GitHub archived Atom and no longer supports as of December 15, 2022.
ELM is a machine-readable representation of an eCQM's logic and provides the information needed to automatically retrieve data from an electronic health record. The ELM file can be in XML (.xml) or JavaScript Object Notation (JSON) file (.json).
There are a lot of things that CQL supports. Higher-level constructs, such as timing phrases, are translated into a representation in ELM focused on implementation so you do not have to worry about those pieces within ELM. There is nothing that says you could not do all that yourself. But for implementation, using ELM takes those pieces off an implementer’s plate. Individually each are not terribly difficult, but when you put them all together and add them up, and the fact that we maintain ELM as a part of this whole infrastructure, it is a significant advantage to be able to use the ELM directly rather than have to start from a parser. Refer to the CQL Educational Resources page for additional guidance.
The QDM is a conceptual data model that helps eCQM developers and implementers understand the general concepts needed to compute an eCQM. However, QDM only defines the concepts, such as a diagnosis, a laboratory test (with a result), or a physical examination finding (such as blood pressure readings). A method to relate each concept to other concepts requires an expression language. As an example, the QDM data element is similar to a noun in expressing grammar (e.g., a laboratory test) and related adjectives (the time the lab performed the laboratory test). But to create a sentence, one must have verbs (e.g., starts) and adverbs (after the beginning of). The expression language provides the verbs and adverbs. Prior versions of QDM (v.5.3 and earlier) included the expression language and the conceptual data model. The expression language portion of QDM was difficult to understand and more challenging to compute. CQL allows eCQM developers to better express the eCQM calculation. CQL can express that some specific activity happened during an inpatient encounter and that it happened before another activity. However, CQL needs a data model to indicate what is related to what.
- CQL defines the expression – i.e., the relationship between item A and item B
- QDM defines the items
- CQL can work with other data models not just QDM. However, in the current CMS eCQM development activities, QDM is the chosen data model to work with CQL. Other projects could consider other data models.
In the Health Level Seven International® (HL7®) community, the Clinical Quality Improvement and Clinical Decision Support (CDS) Work Groups harmonized efforts to express eCQM (QDM) and CDS artifacts (virtual medical record). The result is an HL7 Fast Healthcare Interoperability Resources® (FHIR®) Implementation Guide, QI-Core, which includes a detailed mapping of QDM concepts. QI-Core-related tooling allows eCQM developers to use it directly to author eCQMs in FHIR. QI-Core specifically builds directly on each FHIR version to assure consistency of eCQM expressions and the evolving method for data interchange and interoperability. The eCQI Resource Center includes further information about FHIR and transition efforts.
The QDM specifications provide tables for each datatype available for use. For more information, visit the About tab of the QDM page to retrieve the correct version of the specification. The eCQM Data Element Repository (DERep) provides the pertinent attributes for datatypes and the eCQMs that use them.
Electronic clinical quality measure (eCQM) developers use the QDM along with CQL to develop eCQMs in the Measure Authoring Tool (MAT) and test the eCQM logic using the Bonnie tool. The QDM documentation provides guidance for how to use the information model to express eCQM content. Please visit the eCQI Resource Center QDM page Connect tab to learn about the Quality Data Implementation (QDI) User Group.
Please look for events on the eCQI Resource Center Upcoming Events. Please also visit the eCQI Resource Center QDM page Connect tab to learn about QDM and to register for the QDI User Group distribution list. The QDI User Group addresses questions about how to use various information models (e.g., QDM, QI-Core) to express information required for authoring, understanding, and implementing eCQMs. Submit all requests for QDI User Group discussions to the ONC Project Tracking System (Jira) QDM project.
You can create an account on the ONC Project Tracking System (Jira). Once logged in, select “Projects” from the top menu bar and chose "View All Projects." Select “QDM Issue Tracker (QDM)” from the list. Select “Create” and enter any necessary details of your question or issue.
Find the current version of the QDM on the QDM About tab. Previous versions of the QDM are available on the QDM page Previous Versions tab.
CMS updates the QDM as needed, but no more frequently than annually, based on the QDI User Group’s recommendations evaluating stakeholder requests. QDI User Group Meeting Notes provide information about future changes to QDM. CMS is using QDM version 5.6 for the 2023 and 2024 reporting/performance periods. MAT version 6.10 and Bonnie version 5.1.2 are for use with QDM 5.6. The current, published QDM version is QDM v5.6 - for use for the 2023 and 2024 reporting/performance periods.
QDM versions prior to v.5.3 included both the conceptual data model (the categories and their context of use) and the expression logic to state eCQM clauses. QDM versions since v.5.3 include only the conceptual data model. This change occurred with eCQMs designed for the 2019 reporting/performance period when eCQM developers started using CQL to express logic. Download current versions of the QDM from the eCQI Resource Center QDM About tab.
Negation has several aspects: (a) assurance a certain condition does not exist (e.g., no known allergies); (b) lack of evidence a condition exists (e.g., no information is available in the electronic health record about allergies); and (c) a measured entity intentionally did not perform an action, with or without a reason (e.g., the clinician did not prescribe the medication due to interaction with other medications a patient is taking). The QDM negation rationale attribute addresses only the third definition, sometimes called action negation. The QDM includes this concept as negation rationale as an attribute of the QDM datatypes. Its inclusion allows an eCQM developer to express an eCQM criterion or exception if the clinician expressed a reason for not performing the action expected. The QDM documentation on the eCQI Resource Center describes how an eCQM developer can use negation rationale. Negation rationale is part of QDM version 5.6 for the 2023 and 2024 reporting/performance periods.
This information is in the CQL-based Health Quality Measure Format (HQMF) Implementation Guide (IG). This document describes constraints on the HQMF Normative edition Release 1 (HQMF R1) header and body elements for the CQL-based HQMF. Volume 3 of the IG references QDM. CMS is using QDM v.5.6 for the 2023 and 2024 reporting/performance periods and CQL-based HQMF STU 4.1 addresses QDM v5.6.
Updates to the QDM occur no more than annually and there are currently several versions of QDM available, although each for a different CMS reporting/performance period. See the eCQM Standards and Tools Versions table on the eCQI Resource Center from the eCQI Tools & Key Resources page for the version by reporting year. Each eCQM documentation includes the QDM version number in the CQL file and the HTML human-readable file.
No, “Medication, not Administered” is the method CQL uses to express the QDM datatype “Medication, Administered” when using negation rationale to indicate a reason the action of administering the medication did not occur. Administered medications do not necessarily originate with orders, for example, for medications with standing orders, or orders initiated outside of the clinical software. In the QDM, negation indicates an action did not occur for a reason. It indicates the clinician recorded they did not administer a medication. In the first implementation of eCQMs, measure concepts allowing negation used the same medication value set to represent the type of medications not administered.
Nine QDM datatypes include the action “Recommended” which represents the clinician's proposal to the patient about a specific treatment or action. The meaning is implicit. Electronic health records vary with respect to how they might represent recommendations. However, Health Level Seven International® (HL7®) Fast Healthcare Interoperability Resources® (FHIR®) provides a method to indicate intent in data exchange. An intent = plan represents a recommendation and an intent = order represents an order. Thus, QDM attributes are consistent with evolving HL7 standards for data exchange. “Recommended” does not address a suggested action based on clinical decision support (CDS), represented in HL7 FHIR as intent = proposal. The QDM action “Performed” indicates completion of an action. The comparable concept in HL7 FHIR includes the resource (e.g., Procedure) plus a status = completed.
"Medication, Discharge" defines medications intended for use when the patient leaves the hospital. As defined in the QDM, "Medication, Order" does not differentiate between an order for medications for administration immediately prior to discharge from those written for the patient to take after departure from the hospital. Further, although there maybe orders for some discharge medications (or prescribed), others may already be present in the home and still others may be over-the-counter substances for which orders are unnecessary. Hence, "Medication, Discharge" refers to reconciled medications listed on the patient’s Discharge Medication List. “Medication, Discharge” events should start and end within the time duration of the episode of care, even though the patient may not start the medication until after the episode ends. The expectation is that, at the time of discharge, discharge medications will be the same as the subsequent home medication lists for those medications that are germane to the eCQM, but you cannot assume this at the time of admission as changes may have occurred since the prior episode of care. United States Core (US Core) representation of HL7 FHIR defines a medication list as one compiled from orders (MedicationRequest) with a status = active and intent = order (for physician orders) or intent = plan (for patient reporting or recommended medications). Thus, US Core represents active medications using order and, further, allows indication of the setting in which the patient should take the medication as category = community. Therefore, using US Core for data interchange, the order (MedicationRequest) can represent all discharge medications by using the category = community. QDM v.5.5 includes a setting attribute that maps directly to the US Core category for "Medication, Discharge" to assist with the transition to FHIR. Quality Improvement Core (QI-Core) includes mapping from all QDM datatypes and attributes to FHIR.
The QDM v.5.4 and subsequent versions added a prescriber attribute to “Medication, Order,” “Medication, Dispensed,” and “Medication, Discharge,” and a dispenser attribute to “Medication, Dispensed.” Therefore, from the 2019 reporting/performance period forward, QDM has enabled expressions indicating a medication’s prescriber and dispenser. QDM v.5.5 and forward further added entities, concepts for use to specify details about an actor (or performer) of any QDM datatype. Entities include Patient, Care Partner, Practitioner, and Organization. V 5.6 added the entity Location. Each entity has specific attributes such that a “Medication, Order” prescriber can reference a Practitioner with a role = physician and a specialty = internist (as an example). eCQM expressions can indicate the “Medication, Order” prescriber is the same individual as the “Encounter, Performed” participant, or add further details indicating the individual’s role or specialty. eCQM developers need to test such expressions in real-world settings to determine the feasibility of retrieving such information.
Yes, the MAT is a web-based tool, which allows eCQM developers to author eCQMs using the QDM to represent clinical concepts and CQL to represent the logic. The tool provides the capability to export eCQMs in several formats supporting both human- and machine-reading. Refer to the MAT User Guide for more information.
There are no formatting conventions specific to the QDM, eCQM developers should review the QDM for specific guidance on using the datatypes and attributes in the conceptual model. However, there is information on conventions for formatting statements and expressions in CQL, which encourage consistency, readability, maintainability, and reusability of the resulting CQL. Find these conventions on the CQL Formatting and Usage Wiki. Additionally, CMS annually publishes a CQL Style Guide, based on the Formatting and Usage Wiki contents. The CQL Style Guide focuses on a set of common best practices implemented across CQL-based eCQMs in the CMS reporting programs and also promotes the use of consistent language within the framework of CQL, including libraries, aliases, definitions, functions, and conventions. eCQM stewards or eCQM developers who are developing or specifying eCQMs for future inclusion in CMS programs should align with these best practices.