eCQI Tools & Key Resources

The eCQM tools and resources referenced in this section are openly available and are provided for stakeholder use. They provide a foundation for the development, testing, certification, publication, implementation, reporting, and continuous evaluation of quality measures and their improvement. Tools and resources are listed in alphabetical order and may also be searched by category. To search by category, select the categories of interest from the drop down menu and select Apply.

Category: Implementation

The Binding Parameter Specification is a metadata file that contains no terminology codes. This file provides a record of the value set metadata information that defines the value set code lists specified by published CMS eCQMs. Measure implementers and vendors can use the BPS to track versions and other parameters that define the value set code lists for each eCQM release. The BPS contains...

Category: Development, Testing

Bonnie is a software tool that allows eCQM developers to test and verify the behavior of their eCQM logic. The main goal of the Bonnie application is to reduce the number of defects in eCQMs by providing a robust and automated testing framework. The Bonnie application allows measure developers to independently load measures that they have constructed using the MAT. Developers can then...

Category: Development, Testing

This user group is a community of Bonnie users that benefit from one another’s experience and questions, and allows the development team to easily reach out to the community regarding new releases, proposed features, and focus group opportunities.  Follow these instructions to join:

Category: Continuous Evaluation/Feedback, Development, Publication

The CDS Connect Project demonstrates a web-based repository service to enable the Clinical Decision Support (CDS) community to identify evidence-based standards of care, translate and codify information into an interoperable standard, and leverage tooling to...

Category: Development, Testing

Cedar is a free, open source tool for testing the strength of electronic Clinical Quality Measure (eCQM) collection systems that receive Quality Reporting Document Architecture (QRDA) files.

Category: Certification

The Certified Health IT Product List (CHPL) is a comprehensive and authoritative listing of all certified Health Information Technology which has been successfully tested and certified by the ONC Health IT Certification program. All products listed on the CHPL have been tested by an ONC-Authorized Testing Labs (ONC-ATLs) and certified by an ONC-Authorized Certification Body (ONC-ACB) to meet...

Category: Continuous Evaluation/Feedback

CQF Framework is a collaborative community of participants from the public and private sectors focused on providing the tools, services and guidance to facilitate the functional exchange of health information. CQF Framework focuses on identifying, defining, and harmonizing electronic standards that promote integration between CDS and eCQM.

Category: Publication

The CMS Measure Inventory Tool (CMIT) is the repository of record for information about the measures which CMS uses to promote healthcare quality and quality improvement. CMS and its partners use this measure inventory to inform stakeholders, to manage its measure portfolio, and to guide measure development. The inventory contains information describing each of the measures including title,...

Category: Reporting

This tool details the different aspects of multiple QRDA validation tools that are available to submitters. It is intended to give users a single point of reference for these tools and assist them in selecting the most appropriate tool to meet their individual needs. The tool covers:

  • Cypress QRDA Validation Utility
  • Pre-Submission Validation Application (PSVA)
  • ...
Category: Reporting

The eCQM 2017 CMS Receiving System Edits Document, XLS-330 KB (11/21/17) is a resource which provides greater insight into the...

Category: Development, Implementation

CQL Online is a tool which provides a "sandbox" where electronic Clinical Quality Measure (eCQMs) developers and implementers can "play" with the Clinical Quality Language (CQL). The tools allows users to get a feel for what CQL looks like and how to write proper expressions in this new and powerful language.

More specifically, CQL Online is a syntax directed editor for writing...

Category: Implementation

The HL7 CQL specification defines both a high-level, author friendly syntax for expressing clinical knowledge, as well as a machine friendly, syntax-independent canonical representation of clinical knowledge called Expression Logical Model (ELM). The high level syntax is designed to support measure and...

Category: Certification, Testing

Cypress is an open source testing tool used by vendors to certify their EHRs and health information technology (IT) modules for calculating eCQMs. Cypress is an official testing tool for the ONC EHR Certification Program. Testing involves Cypress generating synthetic patient records for the subset of published eCQMs selected for certification and testing the ability of the EHR systems...

Category: Continuous Evaluation/Feedback

Cypress Tech Talks are an open forum for discussing technical implementation and tool issues with the Cypress team.

Category: Certification, Testing

The Cypress QRDA Validator provides implementers with the ability to validate the conformance of QRDA Category I and Category III documents to the 2016, 2017, and 2018 CMS implementation guides:

  • 2016 (EP/EH)
  • 2017 (EP/EH)
  • 2018 (EH Only)

The Cypress QRDA Validator provides implementers with the...

Category: Implementation

This webpage is designed to support the eCQI implementation activities of the entire care team.  The page provides links to resources and past presentations on implementing eCQMs.

Category: Publication

This website provides eCQI resources and connections with the community of professionals who are dedicated to electronic clinical quality improvement for better health. It provides eCQM electronic specifications along with tool and resource links. It serves as “the one-stop shop for the most current resources to support electronic clinical quality improvement.”

Category: Continuous Evaluation/Feedback, Testing

These open source libraries provide a proof-of-concept implementation of eCQMs against patient information represented in Fast Healthcare Interoperability Resources (FHIR). These libraries reuse the calculation engine developed within Bonnie and Cypress and adapt it to a FHIR data model. They can be used as a...

Category: Publication

Published by the Office of the Federal Register, National Archives and Records Administration (NARA), the Federal Register is the official daily publication for rules, proposed rules, and notices of Federal agencies and organizations, as well as executive orders and other presidential documents.

Category: Implementation, Reporting

Implementation Guides developed by CMS to support Quality Reporting Document Architecture (QRDA) implementation for Eligible Professionals, Eligible Clinicians, Eligible Hospitals and Critical Access Hospitals for eCQM reporting.

Category: Implementation

Implemenation Guides developed and published by HL7 to provide a standard structure to construct a quality measure including metadata, data criteria, population criteria, etc. based on the QDM.

Category: Continuous Evaluation/Feedback

Kaizen, or continuous improvement, is a long-term approach to work that systematically seeks to achieve incremental changes in order to improve efficiency and quality. It focuses on removing process waste and maximizing value to the customer (in this case the patient). CMS and ONC have focused on eCQM development and implementation.

Category: Development

The MAT is a web-based tool that allows measure developers to author eCQMs using the Quality Data Model (QDM). The tool provides the capability to express measure logic and export measures. The MAT provides the capability to express complex measure logic and export measures in several formats, including a human-readable document which can be viewed in...

Category: Implementation

This document provides guidance for those interested in understanding, using, and/or implementing eCQMs.

Category: Continuous Evaluation/Feedback, Development

The MMS is a standardized system for developing and maintaining the quality measures used in CMS’s various quality initiatives and programs. The primary goal of the MMS is to provide guidance to measure developers to help them...

Category: Continuous Evaluation/Feedback, Development, Implementation, Testing

The Blueprint is a guidance document which provides a standardized system for developing and maintaining the quality measures used in various quality initiatives and programs. Includes information on Measure Development Concepts, Measure Lifecycle, eCQM Lifecycle, as well as Tools, Appendices, and Forms.

Category: Development, Implementation, Publication

The CMS Measures Inventory is a compilation of measures used by CMS in various quality, reporting, and payment programs. The Inventory lists each measure by program and contains descriptive information including, but not limited...

Category: Certification, Testing

The NCQA-provided crosswalk identifies eCQMs that are tested under the NCQA eMeasure (eCQM) Certification program, accepted in the NCQA PCMH Recognition Program, as well as those selected as core America’s Health Insurance Plans (AHIP)/CMS eCQMs and used within NCQA’s Healthcare Effectiveness Data and Information Set (HEDIS®) reporting. 

The same eCQMs that are used for CMS Quality...

Category: Certification, Testing

The National Committee for Quality Assurance (NCQA) eCQM Certifcaiton program which uses ONC-approved methodology for testing Health IT's ability to create accurate eCQMs.  The NCQA eCQM Certification program is for use in the ONC HIT Certification Program and is approved as an Authorized Test Lab (ATL).

Category: Implementation

QPS is a web-based tool developed by the National Quality Forum (NQF) to help people select and use NQF-endorsed measures. It allows a user to search for NQF-endorsed measures in a number of ways: export search results; and view types of measures. Using QPS, a user can find NQF-endorsed measures on particular topics, track and get reminders about measures that are important...

Category: Certification, Continuous Evaluation/Feedback, Implementation, Testing

The ONC Project Tracking System is a JIRA (Atlassian, Inc) based collaboration platform hosted by the HHS’s Office of National Coordinator for Health Information Technology (ONC) that supports the implementation of health information technology. It provides internal and external users with a common place to...

Category: Implementation

Pioneers in Quality (PIQ) is a Joint Commission program to assist hospitals on their journey towards electronic clinical quality measure adoption that includes educational programs (e.g., webinars for CEUs), a resource portal, recognition categories, an advisory council, a modified annual report, speaker’s bureau outreach, a peer–to–peer solution exchange, as well as having a strong focus on...

Category: Reporting

popHealth is an open source software tool that can automate the reporting of summary quality measures and streamline the exchange of summary quality data. popHealth can be used by developers to better understand quality measurement and reporting. The tool has also proven valuable for its individual parts: standardized data importer/exporter; user interface; and most importantly its quality...

Category: Reporting, Testing

PSVA is an Application Program Interface (API) which provides vendors, hospitals and providers with a method for validating eCQM files within their own system/environment prior to submission and for securely transporting valid files to Centers for Medicare & Medicaid Services (CMS). The PSVA tool assists in identifying and correcting Quality Reporting Document Architecture (QRDA)...

Category: Reporting, Testing

As of Calendar Year 2017, Quality Reporting Document Architecture (QRDA) Category I files are submitted by Eligible Hospitals (EHs) to fulfill a portion of the mandatory electronic Clinical Quality Measure (eCQM) reporting requirements for the Centers for Medicare & Medicaid Services (CMS) Hospital Inpatient Quality Reporting (IQR) and Medicare Electronic Health Record (EHR) Incentive...

Category: Reporting, Testing

Designed for IT developers and available in the CMS...

Category: Development

QDM is an information model that defines relationships between patients and clinical concepts in a standardized format to enable electronic quality performance measurement. The model is the current structure for electronically representing quality measure concepts for stakeholders involved in electronic quality measurement development and reporting. The QDM provides the language that...

Category: Reporting

QualityNet provides healthcare quality improvement news, resources and data reporting tools and applications. QualityNet is the only CMS-approved site for secure communications and healthcare quality data exchange between healthcare organizations and facilities.

Category: Reporting

Developer tools for the Quality Payment Program allow industry developers who build software for participating clinicians to integrate their software directly with CMS applications and data. 

The following tools are available:

  • Submissions Application...
Category: Implementation

The Quality Reporting Standard Sandbox is a Standards Implementation & Testing Environment which contains resources and tools for the QRDA Category I and Category III standards. The SITE QRDA Validator provides implementers with the ability to validate the conformance of HL7 QRDA Category I, R2 and QRDA Category III, R3 standards. Note that CMS programs may have additional program specific...

Category: Implementation

Release notes are files which provide an overview of technical changes in the eCQM Annual Update.

Category: Implementation

Release notes are files which provide an overview of technical changes in the eCQM Annual Update.

Category: Development, Implementation, Publication

The United States Health Information Knowledgebase (USHIK) is a registry and repository of health care-related metadata, specifications, and standards. The CMS Quality Programs portal provides the ability to view, download, and compare eCQM specifications, data elements, value sets...

Category: Development

UMLS® is a set of files and software that brings together many health and biomedical vocabularies and standards to enable interoperability between computer systems. UMLS® is used to enhance or develop applications, such as electronic health records, classification tools, dictionaries and language translators.

Category: Development, Implementation, Publication, Testing

The Value Set Authority Center (VSAC) is a central repository for the official versions of value sets that support the electronic Clinical Quality Measures. The National Library of Medicine (NLM) maintains the NLM Value Set Authority Center and provides downloadable access to the value sets and the Binding Parameter Specification. The VSAC provides measure developers with tools to search...

Category: Development

The VSAC Collaboration Tool provides a central site where value set authors can post value sets for collaborative discussion. In that site, teams can share threaded discussions about the value sets, view recent value set expansions posted by site members, organize their value sets by usage and by team’s workflow needs, and receive activity and change notifications from VSAC. VSAC authors and...