Getting Started with eCQMs
Electronic clinical quality measures (eCQMs) use data electronically extracted from electronic health records (EHRs) and/or health information technology systems to measure the quality of healthcare provided. The Centers for Medicare & Medicaid Services (CMS) use eCQMs in a variety of quality reporting and value-based purchasing programs.
There are several benefits of using eCQMs:
- eCQMs use clinical data to assess the outcomes of treatment by measured entities.
- eCQMs reduce the burden of manual abstraction and reporting for measured entities.
- eCQMs foster the goal of access to real-time data for point of care quality improvement and clinical decision support.
Measured entities use eCQMs to provide feedback on their care systems and to help them identify opportunities for clinical quality improvement. eCQMs are reported to CMS, The Joint Commission, federal health agencies, and commercial insurance payers in programs that track and/or reimburse measured entities based on quality reporting.
Review eCQM Basics, eCQM 101 - Getting Started with eCQMs for Quality Reporting Programs, and visit the Educational Resources tab to learn more about eCQMs.
- For hospital reporting guidance, visit QualityNet and the Quality Reporting Center for specific program reporting education.
- For clinician reporting guidance, visit the Quality Payment Program website and the Quality Payment Program Resource Library.
- For information about the Medicare Promoting Interoperability Program, refer to the CMS Promoting Interoperability Program webpage.
For federal health agency programs using eCQMs, refer to specific program requirements.
The Meaningful Measures Initiative
CMS’s comprehensive Meaningful Measures Initiative identifies high priority areas for quality measurement and improvement. Its purpose is to improve outcomes for patients, their families and providers while also reducing burden on measured entities. The Meaningful Measure Areas serve as the connectors between CMS strategic goals and individual measures/initiatives.
The eCQM Strategy Project
The eCQM Strategy Project supported the CMS Patients Over Paperwork initiative to evaluate and streamline regulations with a goal to reduce unnecessary burden, to increase efficiencies, and to improve the beneficiary experience. The project provided CMS with an understanding of eCQM implementation and reporting burden and made recommendations for improvement in the use of eCQMs in CMS quality reporting programs. Read the report here.
Many different organizations may participate in the development and maintenance of electronic clinical quality measures (eCQMs). The CMS Measures Management System is a standardized system and provides details on the process for developing and maintaining the quality measures used in CMS's various quality reporting and incentive programs. Information on the clinical quality measure and eCQM lifecycle is available in the Blueprint for the CMS Measures Management System (Blueprint) and on the eCQM Lifecycle webpage. The Clinical Quality Language (CQL) Style Guide is for use by measure developers. It provides examples to standardize expression of measure concepts across eCQMs and define a uniform look and feel to eCQM logic using CQL. The Guide provides a set of common best practices for implementation across CQL-based eCQMs used in CMS quality reporting and incentive programs.
The eCQM Lifecycle is a series of stages eCQMs progress through development (conceptualization, specification, and testing), implementation, and maintenance. CMS and measure developers provide several opportunities for stakeholders to engage throughout the Measure Lifecycle.
Updates to eCQMs
CMS requires eCQMs approved for CMS programs to be updated annually to reflect changes in evidence-based medicine, code sets, and measure logic. CMS works with measure stewards, measure developers, and eCQMs users to ensure necessary changes are reflected in the Annual Update. Additionally, CMS may provide addenda to the eCQM Annual Update periodically within the year to ensure updated code systems are reflected in the eCQM specifications and supporting materials. Use the Change Review Process (CRP) Guide to participate in the eCQM updates.
Structure of eCQMs
To facilitate reporting from an electronic health record (EHR) and/or other health information technology, eCQMs are made of electronic specifications. Health Quality Measure Format (HQMF), CQL, Expression Logical Model (ELM), and other Health Level Seven International® (HL7) standards, are used for electronically documenting eCQM content.
Coding of information for eCQMs consists of
- Computable representations of the eCQM, which contain important details about the measure, the definition of the data elements, and the underlying logic of the measure calculation. The files include
- HQMF XML syntax (.xml). The HQMF includes a header and a body. The header identifies and classifies the document and provides important metadata about the measure. The MAT User Guide, Chapter 6: Measure Details discusses the metadata, which populates the header. The HQMF body contains eCQM sections (e.g., definitions, population criteria, supplemental data elements).
- Shared CQL libraries (.cql, .xml, and .json). The shared libraries are the basic units of sharing CQL. They consist of a foundation of CQL statements used within a measure. Every measure has at least one main CQL library referenced from HQMF.
- CQL file (.cql). The CQL file provides the expression logic for data criteria, population criteria, and supplemental data elements. It provides a formal description of the computable content in the measure and organized into libraries for reusing or sharing between measures and other artifacts.
- Expression Logical Model (ELM) XML document (.xml). ELM provides a machine-readable representation of the measure’s logic in XML. The intent of the ELM file is for machine processing and provides the information needed to retrieve data from an EHR automatically.
- Human-readable representation of the eCQM displays the eCQM content in a human-readable format directly in a web browser, Hypertext Markup Language (HTML) file (.html). This file does not include the underlying HQMF syntax, but the narrative content at the top of the HTML is an extraction from the HQMF header.
- Value sets and direct reference codes (DRCs) convey specific coded value(s) allowed for the data elements within the eCQM. Identification of value sets is via an object identifier (OID) and each value set includes several metadata elements that describe the inclusion and exclusion criteria for the codes in the set. The value set includes a list of codes (i.e., the value set expansion code set) acceptable or valid for a specific data element in the measure, descriptors of those codes, the code system from which the codes are derived, and the version of that code system. DRCs are specific codes referenced directly in the eCQM logic to describe a data element or one of its attributes. Find value sets and DRCs in the VSAC.
CMS created a unique “CMS eCQM Identifier” to name eCQM files. The naming convention adds the eCQM identifier assigned to the eCQM in the Measure Authoring Tool (MAT) with the “eCQM Version Number”. “CMS” is in front of the number. The eCQM Version Number indicates the published version of the eCQM. For example, based on this naming convention, eligible professional measure (NQF0056-Diabetes: Foot Exam) would be CMS123v1 for the first version of the measure. Use the eCQM Naming and Terminology Guide for guidance regarding common words, phrases, and technical terms used in conjunction with eCQMs.
|eCQM Identifier (MAT)||123|
|eCQM Version number||1|
|CMS eCQM identifier||CMS123v1|
Health IT Certification for eCQM Reporting
The ONC Health Information Technology (IT) Certification Program is a voluntary certification program established by the Office of the National Coordinator for Health Information Technology (ONC) to provide for the certification systems to health IT standards, implementation specifications, and certification criteria adopted by the Health and Human Services (HHS) Secretary (Secretary). The ONC Health IT Certification Program supports the availability of certified health IT for its encouraged and required use under federal, state, and private programs.
The ONC develops the functional and conformance testing requirements for the testing and certification of Health IT Modules to the certification criteria adopted by the Secretary, including certification criteria related to electronic clinical quality measures (eCQMs). Only ONC-approved test methods can be used to test products intended for certification in the ONC Health IT Certification Program (please note that any individual or organization may submit test methods to ONC for approval). Currently, ONC has approved the Cypress open source testing tool and NCQA’s alternative test method. Test methods (test procedures, test data, and test tools) are used by ONC-Accredited Testing Laboratories (ATLs) and ONC-Authorized Certification Bodies (ONC-ACBs) to evaluate the conformance and functionality of Health IT Modules. The 2015 Edition test procedures and certification companion guide aid stakeholders in the development and testing of health IT modules.
Regulation Text Citation
eCQMs – record and export
eCQMs – import and calculate
eCQMs – report
eCQMs – filter
Note. The Certification Criterion uses ‘CQMs’ within the 2015 Edition when referencing eCQMs.
Read the Standards and Certification Regulations.
Learn more regarding the 2015 Edition Final Rule. The Health IT Certification Criteria (2015 Edition) builds on past rulemakings to facilitate greater interoperability for several clinical health information purposes and enables health information exchange through new and enhanced certification criteria, standards, and implementation specifications. Taking into account public comments received on the 2015 Proposed Rule, the final rule continues to focus on the establishment of an interoperable nationwide health information infrastructure. The CQM certification criteria § 170.315(c) (1) – (c) (4) were established in the 2015 Edition Final Rule.
Learn more regarding the ONC Cures Act Final Rule and ONC Interim Final Rule. These rulemakings support the seamless and secure access, exchange, and use of electronic health information. The CQM certification criteria at § 170.315(c)(1), (c)(2) and (c)(4) had minor security modifications in the ONC Cures Act Final Rule, and the criterion at § 170.315(c)(3) was modified by the ONC Interim Final Rule.
Filter Tools & Resources
Atom Text Editor
Atom is an open source text editor. The ‘Language-CQL’ plug-in has functionality for Clinical Quality Language (CQL) syntax highlighting and Fast Healthcare Interoperability Resources® (FHIR) model validation. This free text editor is also capable of performing measure evaluation and testing for eCQMs using the open source CQL Evaluation Engine. Additional information is contained in the Atom User Instructions.
Bonnie is a software tool that allows eCQM developers to test and verify the behavior of their eCQM logic. The Bonnie QDM application allows measure developers to load Quality Data Model (QDM) based measures they have constructed using the Measure Authoring Tool (MAT), helps measure developers execute the measure logic against the constructed patient test deck, and evaluates whether the logic aligns with the intent of the measure.
The value set Binding Parameter Specification (BPS) is generated by the National Library of Medicine (NLM) and is a record of the value set metadata information that defines the code lists specified by current CMS eCQMs. Measure implementers and vendors use the BPS to track versions and other parameters that define the code lists for each eCQM release. Users can access the BPS metadata file for any released set of CMS eCQM value set files through the NLM's Value Set Authority Center Downloadable Resources.
Clinical Decision Support (CDS) Connect is an Agency for Healthcare Research and Quality (AHRQ)-sponsored online platform for sharing interoperable CDS resources. Resources include standards-based CDS logic, implementation guides, pilot demonstration reports, and open-source software tools for CDS developers.
Clinical Decision Support (CDS) Hooks is a Health Level Seven International® (HL7®) specification managed by the HL7 Clinical Decision Support (CDS) Workgroup that provides a way to embed additional, near real-time functionality within a clinician’s workflow of an EHR. A specified event(s) within the clinician’s workflow triggers the CDS service which gathers the appropriate specified data elements through secure Fast Healthcare Interoperability Resources® (FHIR®) services. By utilizing FHIR services, CDS Hooks provides interoperability between multiple stakeholders operating on different platforms.
The FHIR service app can return appropriate guidance such as proposed alternative medication or formulary information. CDS Service vendors can test CDS Service functionality using the CDS Hooks Sandbox. More advanced functionality is under development, such as the multi-stakeholder Da Vinci Project and provides use cases for alerts/notifications, prior authorization, and document exchange. The Agency for Healthcare Research and Quality's CDS Connect Project provides a web-based platform to assist with development of CDS artifacts and CDS Hooks.
Certified Health IT Product List (CHPL) is a comprehensive and authoritative listing of all Health IT modules certified by the ONC Health IT Certification Program. All products listed are tested by an ONC-Authorized Testing Lab (ONC-ATL) and certified by an ONC-Authorized Certification Body (ONC-ACB) to meet criteria adopted by the Secretary of the Department of Health and Human Services (HHS). For additional information on how to navigate the CHPL, please refer to the CHPL Public User Guide.
CMS Measures Inventory Tool (CMIT) is a repository for information about CMS measures. CMS and its partners use the inventory to inform stakeholders, to manage its measure portfolio, and to guide measure development. The functions allow users to find measures quickly, to compile and refine sets of related measures, to identify measures across the continuum of care, and will help coordinate measurement efforts across all conditions, settings, and populations.
CMS EHR Certification Number
A CMS EHR Certification ID is a number generated by the Certified Health Information Technology (IT) Product List (CHPL) and used for reporting to CMS. It represents a single product or combination of products in the CHPL. The CMS EHR Certification ID is different from the CHPL product number. In the CHPL, this is the number generated when the hospital/clinician selects EHR Certification ID for a suite of products that make up the hospital's/clinician's EHR solution. If a product changes, then a different CMS EHR Certification ID will be generated. The CMS EHR Certification ID is only unique to the product suite. If two different hospitals/clinicians happen to use the same products, then they will both have the same CMS EHR Certification ID.
CMS requires the CMS EHR Certification ID in several of its quality reporting programs. Check with individual quality reporting programs for requirements.
CMS Help Desks
The CMS Measures Under Consideration Entry and/ Review Information Tool (CMS MERIT) is the web-based data collection portal that allows a measure developer to submit candidate measures to CMS for consideration. The Tool walks the submitter through the process of providing information. Each submitter has a dashboard that allows them to track the progress of their submitted measure and to communicate with CMS through the comment function. The Tool also provides a view of all measures submitted for the current year. Another purpose of CMS MERIT is to give federal officials, the consensus-based Measure Applications Partnership Workgroups, and other stakeholders more information as they review measures.
CMS QRDA IGs
The CMS Quality Reporting Document Architecture (QRDA) Implementation Guides (IGs) are based on the HL7 QRDA IGs. CMS's QRDA Category I IG is used for Hospital Inpatient Quality Reporting (IQR) and Medicare and Medicaid Promoting Interoperability Program for Eligible Hospitals and Critical Access Hospitals (CAHs). CMS's QRDA Category III IG is used for the Eligible Clinicians and Eligible Professionals Programs that address aggregate reporting requirements for Comprehensive Primary Care Plus (CPC+), Merit-Based Incentive Payment System (MIPS) programs, and Medicaid Promoting Interoperability (PI).
CMS QRDA Pre-Submission Validation Tools
The CMS Quality Reporting Document Architecture (QRDA) Pre-Submission Validation Tools are resources which offers QRDA validation tools available to submitters. The intent is to give users a single point of reference for these tools and assist them in selecting the most appropriate tool to meet their individual needs. The tools cover:
The Clinical Quality Framework (CQF) is a collaborative community of participants from the public and private sectors focused on providing tools, services, and guidance to facilitate the harmonization of standards for clinical decision support (CDS) and electronic clinical quality measures (eCQM) use in clinical quality improvement.
Clinical Quality Language (CQL) is an HL7 standard for trial use (STU) and it is part of the effort to harmonize standards between electronic clinical quality measures (eCQMs) and clinical decision support (CDS). CQL provides the ability to express logic that is human readable yet structured enough for processing a query electronically.
CQL Evaluation Engine
The Clinical Quality Language (CQL) Evaluation Engine is an open source Java-based evaluation engine capable of evaluating the result of any CQL expression.
CQL Execution Framework
CQL Formatting and Usage Wiki
The Clinical Quality Language (CQL) Formatting and Usage Wiki is a collaborative workspace for the development of CQL formatting conventions and usage patterns for the representation of logic within quality measures. All users have edit rights to be able to submit edits, add comments and concerns. Items on the Wiki are a work in progress and subject to change.
Clinical Quality Language (CQL) Runner is an online platform for ad hoc testing of CQL.
CQL Style Guide
The CQL Style Guide provides standardized expressions of measure concepts across eCQMs and promotes a uniform “look and feel” to eCQM logic using CQL. The guide focuses on a set of common best practices that have been implemented across CQL-based eCQMs in CMS quality reporting programs. The style guide also promotes the use of consistent language within the framework of CQL, including libraries, aliases, definitions, and functions, as well as guidance on other conventions, such as operator precedence. Measure stewards or developers who are developing or specifying eCQMs for future inclusion in CMS programs should align with these best practices.
CQL-to-ELM Translator Reference Implementation
The CQL-to-ELM Translator is an open-source reference implementation that translates the high-level CQL syntax into the Expression Logical Model (ELM) representation. The reference implementation is used in support of CQF implementations as a tool to enable CQL output to be uniformly and automatically translated into ELM XML or JSON documents for sharing and distribution to support implementation, integration, translation, and execution of CQL-based artifacts.
For more information or to get involved, visit the Clinical Quality Framework initiative page on HL7.
Cypress with the fully integrated Cypress Validation Utility + Calculation Check (CVU+) is an open source testing tool used by vendors to certify their EHRs and health information technology (IT) modules for calculating eCQMs. Cypress is an official testing tool for the ONC EHR Certification Program. Testing involves Cypress generating synthetic patient records for the subset of published eCQMs selected for certification and testing the ability of the EHR systems and health IT modules to accurately record, import, calculate, filter, and report eCQMs.
Cypress Tech Talks
The Cypress Tech Talks are an open forum for discussing technical implementation and tool issues with the Cypress support team.
De Novo Measure Scan
The De Novo Measure Scan (DNMS) is a new, advanced feature of the Environmental Scanning Support Tool (ESST) on the controlled access CMIT site. The DNMS helps measure developers conduct early and frequent environmental scans while developing new measures. The DNMS uses a Clinical Quality Measure (CQM) ontology to describe a measure and represent quality measure concepts. Components of the ontology include target population, health status or utilization, change concept, expected output of the change concept, and care setting. The DNMS uses structured search terms to represent the measure concepts to build a new measure and uses artificial intelligence to identify and prioritize the most relevant PubMed, PubMed Central, and Cumulative Index of Nursing and Allied Health Literature (CINAHL) related literature.
The Data Exchange for Quality Measures Implementation Guide (IG), or DEQM, provides a framework that defines conformance profiles and guidance to enable the exchange of quality information and quality measure reporting (e.g., for transferring quality information from a health care provider to a payer). The DEQM expects to use quality measures specified in accordance with the Quality Measure IG and Quality Improvement Core Framework (QI-Core).
The DEQM IG supports the data exchange scenario and reporting scenarios including individual reporting, summary reporting, and gaps in care reporting. Individual reporting is supported by the DEQM Individual MeasureReport profile and summary reporting is supported by the DEQM Summary MeasureReport profile, which are similar to the QRDA Category I and QRDA Category III in the QDM-CQL-QRDA paradigm.
eCQM Calculations Libraries and Technical Services
The eCQM Calculations Libraries and Technical Services are open source libraries which provide support for loading and calculating eCQMs, generating Quality Data Model (QDM) types, and other libraries related to Clinical Quality Language (CQL) and Fast Healthcare Interoperability Resources (FHIR). Many of the libraries listed below are used within the Bonnie and Cypress tools and are actively supported.
CQM Models: A software library containing auto generated models based off of QDM.
CQM Execution: NPM module for calculating eCQMS written in CQL, using the CQM-Models library
CQM Reports: A library for import and export of reports for use with eCQMs.
CQM Parsers : A library for parsing CQL-based HQMF documents and parsing Measure Authoring Tool (MAT) eCQM packages into models based off of the CQM-Models library.
CQM Validators: A library for QRDA validation. It contains Schematron validators, and validators for QRDA-specific features (measure IDs, performance rates, result extraction, etc).
CQM Execution Service: A Dockerized Node-based service for calculating eCQMs which uses the CQM Execution module.
CQL to ELM Translation Service: A microservice wrapper for the CQL to ELM conversion library.
CQL Exec FHIR: A FHIR-based data source module for use with the CQL Execution Engine.
eCQM Clinical Workflows
The eCQM Clinical Workflow module allows users to review measure flow and clinical context for CMS eCQMs under development. This allows stakeholders interested in an eCQM to provide comments, clinical workflow concerns, and guidance early in the measure lifecycle. Lessons learned from previous eCQM development efforts help address implementation-specific issues.
eCQM Concepts Module
The Measure Collaboration Workspace Electronic Clinical Quality Measures (eCQM) Concepts Module gives users the ability to comment on eCQM concepts under development, submit new measure concepts, align new measures with Meaningful Measures criteria, and identify whether similar measures exist. Feedback provided through the eCQM Concepts Module can help guide the measure developer to refine the concept and purpose behind a new measure to better suit the needs of all the quality measurement stakeholders.
The eCQM Data Element Repository (DERep) provides clarification for all the data elements associated with published and tested eCQMs used in CMS quality reporting programs as well as the definitions and clinical focus for each data element. An end user can filter information by data elements, eligible hospital/critical access hospital or eligible clinician/professional eCQMs, Quality Data Model (QDM) attributes, QDM datatypes, QDM categories, and QDM entities.
The eCQM Flows are flowcharts designed to assist in interpretation of the eCQM logic and calculation methodology for reporting rates. These flows provide an overview of each of the population criteria components and associated data elements that lead to the inclusion or exclusions into the eCQM’s quality action (numerator).
eCQM Implementation Checklist
The eCQM Implementation Checklist assumes that a health care practice/organization has determined which measures to report on. It provides the necessary technical steps health information technology (IT) developers, implementers, and health care organizations must take to update their systems and processes with the eCQM Annual Update for the upcoming reporting and performance periods. The most recent eCQM Annual Update should be applied to your system for use in electronic quality reporting.
eCQM Known Issues
The eCQM Known Issues dashboard provides implementation information for eCQMs with known technical issues for which a solution is under development but not yet available in a published eCQM specification. This includes discrepancies between eCQM narrative and logic, value sets, and/or technical, standard, or logic-related issues.
eCQM Logic and Implementation Guidance
The eCQM Logic and Implementation Guidance is a resource document which provides guidance for understanding, using, and/or implementing eCQMs.
eCQM Test Results Module
The eCQM Test Results Module of the Measure Collaboration Workspace allows users to participate in eCQM testing by providing eCQM specifications for testing, submitting test results using a template, and viewing test results. Measure developers use testing to assess eCQM feasibility, reliability, validity, and usability. Feasibility testing helps determine and provides transparency into the extent to which the required data elements are available and retrievable in the electronic health record/health information technology systems and whether they can be implemented without undue burden to all stakeholders.
The Environmental Scanning Support Tool (ESST) is a tool within the CMS Measures Inventory Tool (CMIT) environment. It makes it easier to complete the environmental scans required in the information gathering process to develop and maintain quality measures. The ESST reduces the time needed to scan literature from months to hours, saving immense amounts of resources. Conventional literature reviews use keyword searches to find the relevant research that has been published on a topic, a collection that may contain millions of documents. The ESST uses an automated natural language processing (NLP) approach that rapidly scans literature in PubMed and PubMed Central and the Cumulative Index of Nursing and Allied Health Literature (CINAHL) to:
Identify relevant documents (abstracts and full-text articles)
Identify and extract the specific knowledge within each relevant document that applies to the measure’s potential opportunity for improvement.
The Federal Register is the official daily publication for rules, proposed rules, and notices of federal agencies and organizations, as well as executive orders and other presidential documents. The Federal Register is published by the Office of the Federal Register, National Archives and Records Administration (NARA).
Guide for Reading eCQMs
Health IT Certification for eCQM Reporting
HL7 CQL-HQMF IG
The HL7 Implementation Guide (IG) defines an approach to using the Clinical Quality Language (CQL) with the Quality Data Model (QDM) and Health Quality Measures Format (HQMF) to define electronic Clinical Quality Measures (eCQMs). The IG describes the use of CQL to express the logic and the US Domain conceptual data model, QDM, to express the individual data types and their attributes for an eCQM detailed using the HQMF standard.
HL7 QRDA I IG
HL7 QRDA III IG
Hospital Quality Reporting
The Hospital Quality Reporting Portal is the only CMS-approved website for secure communications and healthcare quality data exchange between quality improvement organizations (QIOs), hospitals, physician offices, nursing homes, end stage renal disease (ESRD) networks and facilities, and data vendors. It provides secure access for data exchange such as the CMS Data Element Library, End Stage Renal Disease Quality Reporting System, Quality Improvement Organizations, QIES Business Intelligence Center, and Hospital Quality Reporting.
Interoperability Standards Advisory (ISA) process represents the model by which the Office of the National Coordinator for Health Information Technology (ONC) coordinates interoperability standards and implementation specifications for the industry use to fulfill specific clinical health information technology interoperability needs.
The Measure Authoring Tool (MAT) is a web-based tool that allows measure developers to author eCQMs using either the Clinical Quality Language (CQL) and the Quality Data Model (QDM) or Fast Healthcare Interoperability Resources® (FHIR®). The MAT provides the capability to express complex measure logic and export measures in several formats.
MAT and Bonnie User Group
The MAT and Bonnie User Group is a community of users that benefit from shared experiences and questions. It allows the MAT and Bonnie development team to easily reach out to the community for new releases, propose features, and focus group opportunities. To join, please navigate to the Measure Authoring Tool (MAT) website and go to the Training and Resources tab for information on how to register for the User Group.
MC Workspace User Guide
The MC Workspace User Guide provides detailed instructions for the use of each module in the MC Workspace.
The Measures Management System (MMS) is a standardized system for developing and maintaining the quality measures used in CMS’s various quality initiatives and programs. The primary goal of the MMS is to provide guidance to measure developers to help them produce high-caliber healthcare quality measures.
The Measures Management System (MMS) Blueprint is a guidance document which provides a standardized system for developing and maintaining the quality measures used in various quality initiatives and programs. The Blueprint includes information on measure development concepts, Measure Lifecycle, as well as tools, forms, and templates. Information in the Blueprint is applicable to all measure developers, regardless of level of experience or whether the intent of the measure(s) they develop is for use in a CMS program or some other purpose.
NCQA eCQM Certification Program
The National Committee for Quality Assurance (NCQA) eCQM Certification Program uses an alternative ONC-approved methodology for testing Health IT's ability to accurately calculate eCQMs in order to produce Quality Reporting Document Architecture (QRDA) reports. The NCQA eCQM Certification Program is for use in the ONC HIT Certification Program and is approved as an Authorized Test Lab (ATL).
The National Quality Forum (NQF) Quality Positioning System (QPS) is a web-based inventory tool developed by the NQF to help people find and select NQF-endorsed measures. Users can search QPS for NQF-endorsed measures by NQF measure number, type of measure, title, keyword, or measure steward and view measure-related information. With QPS, a user can find NQF-endorsed measures on particular topics or NQF measure selection attributes, identify measures that are no longer endorsed, track measures that are important to them, see which measures others are using, and provide measure use feedback to NQF.
ONC Project Tracking System (Jira)
The ONC Project Tracking System (Jira) is a collaboration platform hosted by the HHS's Office of National Coordinator for Health Information Technology (ONC) which supports the implementation of health information technology. It provides internal and external users with a common place to transparently log, track, and discuss issues with appropriate subject matter experts on many topics. CMS uses Jira during most phases of the eCQM measure lifecycle, including development, implementation, and maintenance.
Pioneers in Quality™ is a collaborative Joint Commission program to improve health care quality and patient safety through education (e.g., webinars for CEUs), innovation, and recognition of pioneering practices for Joint Commission stakeholders. Started in 2016 to assist hospitals on their eCQM journey to improve eCQM data use for quality improvement, the program has expanded over time to include additional key components related to eCQM and chart-abstracted data.
Public comments ensure that measures are developed and maintained using a transparent process with balanced input from relevant stakeholders. Public comment periods provide an opportunity for the widest array of interested parties to provide input on the measures, both under development and in use, and can provide critical suggestions not previously considered by the measure developer or its Technical Expert Panel (TEP). eCQM public comments can be made in response to a new eCQM's call for comments, eCQMs in the pre-rulemaking and rulemaking process, eCQMs in testing, eCQMs undergoing updates, and updates to standards and tools used with eCQMs.
The Quality Data Model (QDM) is a conceptual information model that defines principal concepts (classes) of information, their related attributes and their relationships. The model provides a standardized way to express what needs to be retrieved from electronic records to enable quality performance measurement. The QDM is the current structure for electronically representing quality measure concepts for stakeholders involved in electronic quality measurement development and reporting.
QDM Known Issues
The Health Level Seven International® (HL7) Quality Improvement Core (QI-Core) Implementation Guide (IG) defines a set of Fast Healthcare Interoperability Resources® (FHIR) profiles with extensions and bindings needed to create interoperable, quality-focused applications. The profiles in this IG derive from and extend the US Core profiles to provide a common foundation for building, sharing, and evaluating knowledge artifacts across quality improvement efforts in the US Realm.
As an HL7® FHIR® IG, changes to this specification are managed by the sponsoring workgroup, Clinical Quality Information and incorporated as part of the standard balloting process. The current roadmap follows closely behind the base FHIR roadmap, and the US Core IG.
Quality Data Model (QDM) to QI-Core Mapping: This version of QI-Core updates mappings from QI-Core to the QDM and is based on US Core version 3.1.1 (based on FHIR R4), FHIR version R4.0.1, and QDM version 5.6.
The Quality Measure Implementation Guide (IG), or QM IG, is intended to provide guidance on the representation of electronic clinical quality measures (eCQMs) using Fast Healthcare Interoperability Resources® (FHIR®) Clinical Reasoning Module, Quality Improvement (QI)-Core IG, and Clinical Quality Language (CQL). It provides requirements for measure structure, metadata, definitions, and logic to be used when authoring and evaluating eCQMs.
QPP Developer Tools
The Quality Payment Program (QPP) Developer Tools allow industry developers, who build software for participating clinicians, to integrate their software directly with CMS applications and data. Available tools include Submissions Application Programming Interface (API), Submissions Developer Preview environment, Measures, Activities, Benchmarks Repository, and QRDA III Conversion Tool Open Source Package. These include documentation for reporting eCQMs. Additionally, there is information on submitting data through the CMS Web Interface and developer documentation on the Eligibility API.
QPP Resource Library
The Quality Payment Program (QPP) Resource Library is the one-stop shop for all QPP resources. It provides educational materials that support clinicians’ participation in QPP.
Key 2021 resources include:
- MIPS Eligibility and Participant Quick Start Guide
- Quality Payment Program Access User Guide
- MIPS Measure Specifications (multiple links), MIPS Quality Benchmarks, MIPS Data Validation Criteria, and Technical User Guides (multiple links)
- QPP Final Rule Resources
All QPP webinar resources, including slides, transcripts and recordings from recent presentations, can be found on the QPP Webinar Library.
The Quality Reporting Document Architecture (QRDA) is a standard document format for the exchange of electronic clinical quality measure (eCQM) data.
QRDA Category I Conformance Statement Resource
The Quality Reporting Document Architecture (QRDA) Category I Conformance Statement Resource assists hospital quality measure data submitters in resolving the most recurrent file upload errors generated by the CMS data receiving system, via the Quality-Net Secure Portal, during the specified CMS reporting period. This resource provides additional explanations and reference materials beyond definitions found in the eCQM Data Receiving Systems Edits documents to resolve common errors and successfully submit quality measure data.
QRDA Category I files are submitted by hospitals to fulfill a portion of the aligned mandatory electronic clinical quality measure (eCQM) reporting requirements for the CMS Hospital Inpatient Quality Reporting (IQR) and Promoting Interoperability (PI) Programs. Hospitals reporting on eCQMs are encouraged to use this resource to support successful reporting.
QRDA III Converter
QRDA Known Issues
QRDA Known Issues provide implementation information for QRDA Implementation Guides (IGs) or supporting documents with known technical issues for which a solution is under development but may not be published. This includes issues such as: discrepancies between conformance assertions and message text, non-critical conformance assertion errors and typographical errors.
QualityNet is a website which provides healthcare quality improvement news, resources, data reporting tools, and applications used by healthcare providers and others. It provides quality improvement information associated with the inpatient (IQR) and outpatient (OQR) hospital quality programs, Ambulatory Surgery Center Quality Reporting (ASCQR), PPS-Exempt Cancer Hospital Quality Reporting (PCHQR), Inpatient Psychiatric Facility Quality Reporting (IPFQR), and End Stage Renal Disease Quality Incentive Program (ESRD QIP).
Synthea™ is an open-source, synthetic patient generator that models the medical history of synthetic patients. It provides realistic, but not real, patient data and associated health records covering every aspect of healthcare. The resulting data is free from cost, privacy, and security restrictions, enabling research with health information technology data that is otherwise legally or practically unavailable.
- Browse and download pre-generated synthetic records
Technical Release Notes (TRN) - eCQM Eligible Hospital/Critical Access Hospital
The Technical Release Notes (TRNs) are resource files which provide an overview of technical changes, e.g., logic and terminology, to each eCQM in the most recent eCQM Annual Update for CMS reporting/performance periods.
Technical Release Notes (TRN) - eCQM Eligible Professional/Clinician
The Technical Release Notes (TRNs) are resource files which provide an overview of technical changes, e.g., logic and terminology, to each eCQM in the most recent eCQM Annual Update for CMS reporting/performance periods.
The Unified Medical Language System® (UMLS®) is a set of files and software that bring together health and biomedical vocabularies and standards to enable interoperability between computer systems. UMLS® is used to enhance or develop applications, such as electronic health records, classification tools, dictionaries and language translators. Use of UMLS® allows end users to view codes used in value sets such as CPT and ICD-10-CM. A free UMLS license is required.
The Value Set Authority Center (VSAC) is a value set authoring and central repository for the official versions of value sets that support the electronic clinical quality measures. The National Library of Medicine maintains the VSAC and provides downloadable access to the value sets and the Binding Parameter Specification. The VSAC provides measure developers with tools to search existing value sets, create new value sets, and maintain value set content consistent with current versions of the terminologies they use. A free UMLS license is required to access the value set details.
VSAC Collaboration Tool, and VSAC API Resources
The Value Set Authority Center (VSAC) Collaboration Tool, and VSAC API Resources provides a central site where value set authors can post value sets for collaborative discussion. Teams can share threaded discussions about the value sets, view recent value set expansions posted by site members, organize their value sets by usage and by team’s workflow needs, collaborate with other measure developers to harmonize value sets, and receive activity and change notifications from VSAC. A free UMLS license is required.
VSAC Support Center
The VSAC Support Center is the Help section of the VSAC website. It provides information on how to use the VSAC, specific information for VSAC authors and stewards (e.g., how to create value sets), and additional information such as release notes and how to use the VSAC Collaboration Tool, and VSAC API Resources.
Introduction to eCQMs
- Pioneers in Quality Video Short - eCQM Basics Series 1 - January 7, 2021
- eCQM Basics - Updated March 2021
- eCQM Communications Resources - Updated March 2021
- eCQM Strategy - eCQI Resource Center - MC Workspace Overview - Updated February 28, 2020
- eCQM 101 - Getting Started with Electronic Clinical Quality Measures for Quality Reporting- Updated March 20, 2019
eCQM Annual Update
- Pioneers in Quality Expert to Expert Webinar eCQM Annual Updates for the 2021 Reporting Period: ED-2 Median Admit Decision Time to ED Departure Time for Admitted Patients - January 2021
- Pioneers in Quality Expert to Expert Webinar eCQM Annual Updates for the 2021 Reporting Period: Stroke Measures & PC-05 Exclusive Breast Milk Feeding - January 2021
- Pioneers in Quality Expert to Expert Webinar eCQM Annual Updates for the 2021 Reporting Period: VTE Measures - January 2021
- Change Review Process (CRP) User Guide - September 2020
- How to Implement eCQM Annual Updates - Updated March 20, 2019
- Measure Collaboration Workspace Series – Session 1 eCQI Resource Center, Hybrid Measure, Measure Collaboration Workspace, and the CMS Measures Inventory Tool - July 15, 2020
- Pioneers in Quality™ Proven Practices: Establish your eCQM A-team - September 5, 2017
- Measure Collaboration Workspace Series –Session 3 - eCQM Data Element Repository (DERep) including health system's experience with using it to implement eCQMs - August 12, 2020
- Measure Collaboration Workspace Series –Session 2 Measure Collaboration Workspace and eCQM Flows and Clinical Flow Prototype – July 29, 2020
- Use of Clinical Dashboards to Drive Performance Improvement for eCQMs - July 31, 2019
- eCQM Implementation Checklist - April 17, 2017
- 36 short educational videos (YouTube) assist providers in implementing CMS Comprehensive Primary Care (CPC), several of which are specifically relevant to electronic clinical quality improvement:
Implementing eCQMs Eligible Hospitals and Critical Access Hospitals
- Pioneers in Quality Expert to Expert Webinar eCQM Annual Updates for the 2021 Reporting Period: Safe Use of Opioids—Concurrent Prescribing (CMS506) - January 2021
- Pioneers in Quality Expert to Expert Series: PC-01 & PC-05 - March 26, 2019
- Pioneers in Quality Expert to Expert Series: CAC-3 & EHDI-1a - March 5, 2019
- Pioneers in Quality Expert to Expert Series: VTE-1 & VTE-2 - February 26, 2019
- Pioneers in Quality Expert to Expert Series: ED-1 & ED-2 - February 12, 2019
- Pioneers in Quality Expert to Expert Series: STK-5 & AMI-8a - January 29, 2019
- Pioneers in Quality Expert to Expert Series: STK-2, -3, & -6 - December 11, 2018
Implementing eCQMs Eligible Professionals and Eligible Clinicians
- Eligible Clinician New eCQMs Finalized for 2019 - March 11, 2019
- Three Eligible Clinician eCQMs with Substantive Changes for the 2019 Performance Year - CMS69, CMS159, and CMS160 - February 27, 2019
- Eligible Clinician eCQM Diabetes Measures - January 30, 2019
- Eligible Clinician eCQM Preventive Care and Screening Measures - January 15, 2019
- ONC's Innovations in the Use of Electronic Health Data for eMeasurement and Quality Improvement webinar YouTube - May 4, 2017
- Health Information Technology (IT) Curriculum Resources for Educators videos were funded by The Office of the National Coordinator for Health Information Technology (ONC) as part of the Workforce Development Programs and provide instructional materials covering areas relevant to improved care delivery. NOTE: Many of these components are over 1 GB in size. Downloading may take up to 10 minutes, depending on your connection. A number of the curriculum components cover electronic clinical quality improvement.
- eCQI Toolkit - March 2017 and eCQI Resources from Mountain-Pacific Quality Health Quality Improvement Organization assist organizations with leveraging health information technology (HIT) and the Plan Do Study Act (PDSA) process improvement methodology to support and advance their health quality improvement initiatives.
eCQI Resource Center
- eCQI Resource Center Updates: CMS Quality Bi-Monthly Forum - May 2021
- eCQM Standards Cycle and eCQI Resource Center Updates: CMS Quality Bi-Monthly Forum - March 2021
- eCQI Resource Center Demonstration: The eCQM Data Element Repository (DERep) breadcrumbs and measure detail page - CMS Quality Bi-Monthly Forum - January 2021
- eCQI Resource Center Overview - July 15, 2020
- Includes tools and resources related to the eCQM annual update for the 2021 reporting and performance period and telehealth eCQM reporting implementation and reporting guidance.
- eCQI RC: eCQM Pages - November 13, 2019
- eCQI RC: Key Tools and Resources - November 13, 2019
CMS provides several opportunities to provide input on eCQM development and updates, public calls for measures, comments on measures under development, and participation on technical expert panels. Visit the Measure Collaboration Workspace, the CMS Measures Management System Get Involved webpage, and the Engage in electronic clinical quality improvement (eCQI) page for a list of current opportunities.
CMS, through the ONC Project Tracking System (Jira), allows stakeholders to submit issues with eCQM implementation and development and receive feedback. Submit eCQM technical questions to the eCQM Issue Tracker. For known issues about eCQMs, see the eCQM Known Issues Jira project.
Find information on various helpdesks and tools for eCQM help is found on the eCQI Resource Center Contact Us page.