download chevron-down chevron-right chevron-with-circle-down chevron-with-circle-up check circle-with-minus circle-with-plus export help-with-circle inbox link mail old-phone star-outlined star user v-card close close close Back to top
Top
Top of Content

eCQI Tools & Key Resources

Tools & Resources

The eCQM Informational Tools, Resources and Collaboration (InfoTRAC) referenced in this section are openly available and are provided for stakeholder use. They provide a foundation for the development, implementation, reporting, help, and feedback of quality measures and their improvement.

Diagram of the eCQM lifecycle showing the progression through the stages of development, implementation, reporting, and help & feedback then returning to development for maintenance.
Find the Tools and Resources You Need
You can browse and search all of the tools and resources by visiting the eCQI Tools and Resources Library.
The standards and tools versions listed for each reporting/performance period are the versions used to create and/or support the implementation of the specific reporting/performance period specifications. Newer versions of the standards and tools may be available, but were not used in the development of the reporting/performance period listed.

The eCQMinfo-icon tools and resources provide a foundation for the development, testing, certification, publication, implementation, reporting, and continuous evaluation of quality measuresinfo-icon and their improvement. You can refine the tool list by selecting a category of interest and/or a role that best describes your needs, or you can also click a specific tool from the list below to view additional details.

Bonnie

Category: Development, Testing

Bonnieinfo-icon is a software tool that allows eCQMinfo-icon developers to test and verify the behavior of their eCQM logic. The Bonnie application allows measure developersinfo-icon to independently load measures that they have constructed using the MATinfo-icon and helps measure developers execute the measure logic against the constructed patient test deck and evaluate whether the logic aligns with the intent of the measure.

Bonnie User Group

Category: Development, Testing

The Bonnie User Groupinfo-icon is a community of users that benefit from shared experiences and questions. It allows the development team to easily reach out to the community for new releases, proposed features, and focus group opportunities. To join:

  • Navigate to https://groups.google.com/forum/#!forum/bonnie
  • Sign in with a google account or create an account tied to an existing email address: under “Choose your username” click “I prefer to use my current email address.”
  • Click “Join group”

Binding Parameter Specification (BPS)

Category: Implementation

The value setinfo-icon Binding Parameter Specificationinfo-icon (BPSinfo-icon) is generated by the National Library of Medicine (NLM) and is a record of the value set metadata information that defines the code lists specified by current CMS eCQMsinfo-icon. Measure implementers and vendors use the BPS to track versions and other parameters that define the code lists for each eCQMinfo-icon release. Users can access the BPS metadata file for any released set of CMS eCQM value set files through the NLM's Value Set Authority Center Downloadable Resources.

CDS Connect

Category: Continuous Evaluation/Feedback, Development, Implementation

Clinical Decision Support (CDSinfo-icon) Connect is a tool that is a web-based repository service which enables the CDS community to identify evidence-based standards of care, translate and codify information into an interoperable standard, and leverage tooling to promote a collaborative model of CDS development.

Certified Health IT Product List (CHPL)

Category: Certification, Implementation

Certified Health ITinfo-icon Product List (CHPLinfo-icon) is a comprehensive and authoritative listing of all Health Information Technologyinfo-icon certified by the ONC Health IT Certification program. All products listed are tested by an ONC-Authorized Testing Lab (ONC-ATL) and certified by an ONC-Authorized Certification Body (ONC-ACB) to meet criteria adopted by the Secretary of the Department of Health and Human Services (HHS). For additional information on how to navigate the CHPL, please refer to the CHPL Public User Guide.

Clinical Quality Framework

Category: Continuous Evaluation/Feedback

The Clinical Quality Frameworkinfo-icon is a collaborative community of participants from the public and private sectors focused on providing tools, services and guidance to facilitate the harmonizationinfo-icon of standards for clinical decision support (CDSinfo-icon) and electronic clinical quality measuresinfo-icon (eCQMinfo-icon) use in clinical quality improvement.

Clinical Quality Language (CQL)

Category: Development

Clinical Quality Language (CQL) is an HL7info-icon standard for trial use (STUinfo-icon) and it is part of the effort to harmonize standards between electronic clinical quality measuresinfo-icon (eCQMsinfo-icon) and clinical decision support (CDSinfo-icon). CQL provides the ability to express logic that is human readable yet structured enough for processing a query electronically.

CMS Measures Inventory Tool (CMIT)

Category: Development, Implementation

CMS Measures Inventory Tool (CMITinfo-icon) is a repository for information about CMS measures. CMS and its partners use the inventory to inform stakeholders, to manage its measure portfolio, and to guide measure development. The functions allow users to find measures quickly, to compile and refine sets of related measures, to identify measures across the continuum of care, and will help coordinate measurement efforts across all conditions, settings, and populations.

CMS Quality Reporting Document Architecture (QRDA) Implementation Guides (IGs)

Category: Reporting

The CMS Quality Reporting Document Architecture (QRDA) Implementation Guides (IGs) are based on the HL7info-icon QRDA IGs. CMS's QRDA Category I IG is used for Hospital Quality Reporting (HQR), and CMS's QRDA Category III IG is used for the Eligible Cliniciansinfo-icon and Eligible Professionalsinfo-icon Programs that address aggregate reporting requirements for Comprehensive Primary Care Plus (CPC+) and Merit-Based Incentive Payment System (MIPS) programs.

CMS QRDA Pre-Submission Validation Tools Guide

Category: Reporting

The CMS Quality Reporting Document Architecture (QRDA) Pre-Submission Validation Tools Guide is a resource which offers multiple QRDA validation tools that are available to submitters. It is intended to give users a single point of reference for these tools and assist them in selecting the most appropriate tool to meet their individual needs. The tool covers:

  • Cypressinfo-icon QRDA Validation Utility
  • Pre-Submission Validation Application (PSVAinfo-icon)
  • CMS Data Receiving System, within the QualityNetinfo-icon Secure Portal

CMS QualityNet Data Receiving Systems Edits Document

Category: Reporting

Each year, the electronic clinical quality measure (eCQM) Data Receiving System Edits Document is developed to provide greater insight into the error messages generated by the CMS data receiving system when test and production Quality Reporting Document Architecture (QRDA) Category I files are submitted to the CMS receiving system within the QualityNet Secure Portal.

CQL Formatting and Usage Wiki

Category: Development, Implementation

The Clinical Quality Language (CQL) Formatting and Usage Wiki is a collaborative workspace for the development of CQL formatting conventions and usage patterns for the representation of logic within quality measures. All users have edit rights to be able to submit edits, add comments and concerns. Items on the Wiki are a work in progress and subject to change. 

CQL Formatting and Usage Wiki - eCQM Known Issues

Category: Development, Implementation, Reporting

The eCQMinfo-icon Known Issues is a section in the Clinical Quality Language (CQL) Formatting and Usage Wiki that provides a listing of known issues relative to CQL-Based eCQM support.

CQL Style Guide

Category: Development

The Clinical Quality Language (CQL) Style Guide provides examples to standardize expression of measure concepts across eCQMs and define a uniform “look and feel” to eCQM logic using CQL. The guide focuses on a set of common best practices implemented across CQL-based eCQMsinfo-icon in CMS reporting programs and also promotes the use of consistent language within the framework of CQL, including libraries, aliases, definitions, functions and conventions.

CQL-to-ELM Translator Reference Implementation

Category: Development, Implementation

The CQL-to- ELM Translator is a specificationinfo-icon that describes a formal mechanism for translating the high-level CQL syntax into the canonical called Expression Logical Model (ELM) representation. The reference implementation is intended to be used in support of CQF implementations as a tool to enable CQL output to be uniformly and automatically translated into ELM XML or JSON documents for sharing and distribution to support implementation, integration, translation, and execution of CQL-based artifacts.

Cypress

Category: Certification, Implementation, Testing

Cypressinfo-icon is an open source testing tool used by vendors to certify their EHRs and health information technology (IT) modules for calculating eCQMs. Cypress is an official testing tool for the ONC EHR Certification Program. Testing involves Cypress generating synthetic patient records for the subset of published eCQMsinfo-icon selected for certification and testing the ability of the EHRinfo-icon systems and health IT modules to accurately record, import, calculate, filter, and report eCQMs.

Cypress Tech Talks

Category: Continuous Evaluation/Feedback, Implementation

The Cypress Tech Talksinfo-icon are an open forum for discussing technical implementation and tool issues with the Cypress support team.

Cypress Validation Utility (CVU)

Category: Certification, Reporting, Testing

The Cypress Validation Utility (CVU) conformance tool provides implementers with the ability to validate the conformance of QRDA Category I and Category III documents to the 2018 and 2019 CMS implementation guides.

eCQI Resource Center

Category: Implementation

The eCQI Resource Centerinfo-icon is a website which provides eCQI resources and connections with the community of professionals who are dedicated to electronic clinical quality improvement for better health. It provides eCQM electronic specifications along with tool and resource links. It serves as “the one-stop shop for the most current resources to support electronic clinical quality improvement.”

eCQM Calculations Libraries and Technical Services

Category: Continuous Evaluation/Feedback, Testing

The eCQM Calculations Libraries and Technical Servicesinfo-icon are open source libraries which provide a proof-of-concept implementation of eCQMsinfo-icon against patient information represented in Fast Healthcare Interoperability Resources (FHIR). These libraries reuse the calculation engine developed within Bonnieinfo-icon and Cypressinfo-icon and adapt it to a FHIR data model. They can be used as a starting point for an implementation of eCQMs from FHIR-based data.

fhir-patient-api maps the current Quality Data Model (QDM) based eCQMs to FHIR resources.

node-qme a “patient source”, which is responsible for using the fhir-patient-api to retrieve patients from a database and send them to eCQMinfo-icon code for calculation.

node-cqm-engine monitors requests for eCQM calculations and executes them. It also aggregates the results from individual patient measure results into a result for the entire patient population.

ecqm server a server application that provides RESTful interfaces for eCQM calculation.

ecqm-frontend a software package that provides a web-based interface for users to create and view the results of eCQM calculations.

eCQM Flows

Category: Implementation

The eCQM Flowsinfo-icon are flowcharts designed to assist in interpretation of the eCQMinfo-icon logic and calculation methodology for reporting rates. These flows provide an overview of each of the population criteria components and associated data elements that lead to the inclusion or exclusions into the eCQM’s quality action (numeratorinfo-icon).

eCQM Measure Logic Guidance Document

Category: Implementation

The eCQMinfo-icon Measure Logic Guidance Document is a resource document which provides guidance for understanding, using, and/or implementing eCQMsinfo-icon.

Federal Register

Category: Continuous Evaluation/Feedback

The Federal Registerinfo-icon is the official daily publication for rules, proposed rules, and notices of federal agencies and organizations, as well as executive orders and other presidential documents. The Federal Register is published by the Office of the Federal Register, National Archives and Records Administration (NARA). 

Category: Reporting

The Centers for Medicare & Medicaid Services (CMS) and The Joint Commission (TJC) released guidance for reporting diagnostic test data in the Diagnostic Study Performed (V4) template of Quality Reporting Document Architecture (QRDA) Category I files. This affects the electronic clinical quality measureinfo-icon (eCQMinfo-icon) CMS31v6 - Hearing Screening Prior To Hospital Discharge only. The 2018 CMS QRDA Category I Implementation Guide (IG) for Hospital Quality Reporting allows for two ways of reporting the result of a diagnostic test in the Diagnostic Study Performed (V4) template:

  1. Using the element “value” inherited from its parent template Procedure Activity Observation (V2), or
  2. Using the contained template Result Observation (V3).

For 2018 reporting, CMS has aligned with TJC to only use the first option. Result data for the Diagnostic Study Performed template must be in the value element in order to be processed by CMS.

Result data provided in the template Result Observation (V3) will not be used by the QualityNetinfo-icon data receiving system during measure calculation of CMS31v6. The QRDA file will not be rejected, nor will a warning be returned, if the Result Observation (V3) template is used, and that the measure outcome of CMS31v6 from CMS may not match what is expected.

Guide to Reading eCQMs

Category: Implementation

The Guide to Reading eCQMsinfo-icon is a resource to assist stakeholders in interpreting and understanding eCQMsinfo-icon. The guide provides information on eCQMs such as file naming conventions, understanding an eCQMinfo-icon human-readable rendition, value setsinfo-icon, QDM data criteria, and more.

Health Quality Measure Format (HQMF)

Category: Development

Health Quality Measureinfo-icon Format (HQMF) is an HL7info-icon standards-based representation of a quality measure as an electronic document. A quality measure expressed in this way is also referred to as an electronic clinical quality measure (eCQMinfo-icon).

HL7 Clinical Quality Language (CQL)-based Health Quality Measure Format (HQMF) Implementation Guide (IG)

Category: Development

The CQL-based HQMF IG constrains the HL7info-icon HQMF IG by utilizing both a domain analysis model called the Quality Data Model (QDM) and an expression logic called CQL. QDM allows quality measureinfo-icon developers to describe clearly and unambiguously the data required to calculate quality measuresinfo-icon. QDM data types are mapped to the HQMF templates. CQL allows the sharing of unambiguous logic among eCQMsinfo-icon and can be used with multiple data models.

HL7 Quality Data Model (QDM)-based Health Quality Measure Format (HQMF) Implementation Guide (IG)

Category: Implementation

The QDM-based HQMF IG constrains the HL7info-icon HQMF IG by utilizing a domain analysis model called the QDM. This allows quality measureinfo-icon developers to describe clearly and unambiguously the data required to calculate quality measuresinfo-icon. QDM data types are mapped to the HQMF templates. No stand-alone QDM-based HQMF IG has been created since October 2016 and its functionality has been superseded by the CQL-based HQMF IG.

HL7 Quality Reporting Document Architecture (QRDA) Category I Implementation Guide (IG)

Category: Reporting

The HL7info-icon Quality Reporting Document Architecture Category I Implementation Guide (HL7 QRDA I IG) describes constraints on HL7's Clinical Document Architecture Release 2 (CDA R2) to create a QRDA I document. QRDA Category I is for reporting quality data at the individual patient level.

HL7 Quality Reporting Document Architecture (QRDA) Category III Implementation Guide (IG)

Category: Reporting

The HL7info-icon Quality Reporting Document Architecture Category III Implementation Guide (HL7 QRDA III IG) describes constraints on HL7's Clinical Document Architecture Release 2 (CDA R2) to create a QRDA III document. QRDA Category III is for reporting aggregated quality measureinfo-icon data.

Measure Authoring Tool (MAT)

Category: Development

The Measure Authoring Tool (MATinfo-icon) is a web-based tool that allows measure developersinfo-icon to author eCQMsinfo-icon using the Quality Data Model (QDM). The MAT provides the capability to express complex measure logic and export measures in several formats.

Measures Management System (MMS)

Category: Development

The Measures Management System (MMS) is a standardized system for developing and maintaining the quality measuresinfo-icon used in CMS’s various quality initiatives and programs. The primary goal of the MMS is to provide guidance to measure developersinfo-icon to help them produce high-caliber healthcare quality measures.

Measures Management System (MMS) Blueprint

Category: Continuous Evaluation/Feedback, Development, Implementation, Testing

The Measures Management System (MMS) Blueprintinfo-icon is a guidance document which provides a standardized system for developing and maintaining the quality measuresinfo-icon used in various quality initiatives and programs. The Blueprint includes information on Measure Development Concepts, Measure Lifecycle, eCQMinfo-icon Lifecycle, as well as Tools, Appendices, and Forms.

NCQA eCQM Certification Program

Category: Certification, Implementation, Testing

The National Committee for Quality Assurance (NCQA) eCQMinfo-icon Certification Program uses ONC-approved methodology for testing Health ITinfo-icon's ability to create accurate eCQMsinfo-icon. The NCQA eCQM Certification Programinfo-icon is for use in the ONC HIT Certification Program and is approved as an Authorized Test Lab (ATL).

NCQA eCQM Crosswalk

Category: Certification, Implementation, Testing

The National Committee for Quality Assurance (NCQA) eCQMinfo-icon crosswalk identifies eCQMsinfo-icon that are tested under the NCQA eMeasure (eCQM) Certification program, accepted in the NCQA Patient Centered Medical Home Recognition Program, as well as those selected as core America’s Health Insurance Plans (AHIP)/CMS eCQMs and used within NCQA’s Healthcare Effectiveness Data and Information Set (HEDIS®) reporting. 

 

ONC Project Tracking System (JIRA)

Category: Certification, Continuous Evaluation/Feedback, Development, Implementation, Testing

JIRA is a collaboration platform hosted by the HHS’s Office of National Coordinator for Health Information Technologyinfo-icon (ONC) which supports the implementation of health information technology. It provides internal and external users with a common place to transparently log, prioritize, and discuss issues with appropriate subject matter experts on many topics. CMS uses JIRA during most phases of the eCQMinfo-icon measure lifecycle, including development, implementation, and maintenance.

Pioneers in Quality

Category: Implementation

Pioneers in Qualityinfo-icon is a Joint Commission program to assist hospitals on their journey towards electronic clinical quality measureinfo-icon adoption including educational programs (e.g., webinars for CEUs), a resource portal, recognition categories, an advisory council, a modified annual report, speaker’s bureau outreach, a peer–to–peer solution exchange, as well as having a strong focus on partnering with hospitals to provide the highest level of quality care for patients and their families.

popHealth

Category: Reporting

popHealthinfo-icon is an open source software tool that can automate the reporting of summary quality measuresinfo-icon and streamline the exchange of summary quality data. popHealth can be used by developers to better understand quality measurement and reporting. The tool has also proven valuable for its individual parts: standardized data importer/exporter, user interface, and most importantly its quality measureinfo-icon engine, which is technology shared with Cypressinfo-icon.

Pre-Submission Validation Application (PSVA)

Category: Reporting, Testing

The Pre-Submission Validation Application (PSVAinfo-icon) is an application program interface (API) which provides vendors, hospitals, and providers with a method for validating eCQMinfo-icon files within their own system/environment prior to submission and for securely transporting valid files to the Centers for Medicare & Medicaid Services (CMS). The PSVA tool has three interfaces as part of the download package and a user is able to select the option that best meets their needs.

Quality Payment Program (QPP) Developer Tools

Category: Reporting

The Quality Payment Program (QPP) Developer Tools allow industry developers, who build software for participating clinicians, to integrate their software directly with CMS applications and data. Available tools include Submissions Application Programming Interface (API), Measures, Activities, and Benchmarks Repository, Submissions Schemas, and QRDA III Conversion Tool Open Source Package. These include documentation for reporting eCQMsinfo-icon used in the QPP program through QRDA III, QPP JSON, or QPP XML formats.

Quality Payment Program (QPP) Resource Library

Category: Reporting

The Quality Payment Program (QPP) Resource Library is the one-stop shop for all QPP resources. It provides participating eligible cliniciansinfo-icon and implementers with general and regulatory QPP information. 
Key resources include
• MIPS Participation & Overview Fact Sheet and Scoringinfo-icon Guide
• Enterprise Identity Data Management (EIDM) User Guide
• MIPS Measures, Benchmarks, Validation Criteria, and Technical User Guides 
• QPP Final Rule, Fact Sheet, and Executive Summary

NQF Quality Positioning System (QPS)

Category: Implementation

The National Quality Forum (NQFinfo-icon) Quality Positioning System (QPSinfo-icon) is a web-based inventory tool developed by the NQF to help people select and use NQF-endorsed measures. It allows a user to search for NQF-endorsed measures in a number of ways: export search results and view types of measures. Using QPS, a user can find NQF-endorsed measures on particular topics, track and get reminders about measures that are important to them, provide feedback on measures, and see which measures others are using.

QRDA Category I Conformance Statement Resource

Category: Reporting

The Quality Reporting Document Architecture (QRDA) Category I Conformance Statement Resource assists data submitters in resolving file upload errors by providing additional explanations and reference material to support successful submission. Since Calendar Year 2017, QRDA Category I files are submitted by Eligible Hospitalsinfo-icon (EHs) to fulfill a portion of the mandatory eCQMinfo-icon reporting requirements for the CMS Hospital Inpatient Quality Reporting (IQR) and Promoting Interoperability Programs.

QRDA III Converter

Category: Reporting, Testing

The Quality Reporting Document Architecture (QRDA) III Converter tool converts QRDA III files to the QPP JSON (JavaScript Object Notation) format for Quality Payment Program (QPP) data submission. The tool can also be used to test whether QRDA III files can be consumed by QPP and allows you to see whether it complies with the most current QRDA III Implementation Guide and can be successfully submitted to QPP.

Quality Data Model (QDM)

Category: Development

The Quality Data Model (QDM) is an information model that defines relationships between patients and clinical concepts in a standardized format to enable electronic quality performance measurement. The model is the current structure for electronically representing quality measureinfo-icon concepts for stakeholders involved in electronic quality measurement development and reporting.

Quality Reporting Document Architecture (QRDA)

Category: Reporting

The Quality Reporting Document Architecture (QRDA) is a standard document format for the exchange of electronic clinical quality measure (eCQM) data.

QualityNet

Category: Implementation, Reporting

QualityNetinfo-icon is a website which provides healthcare quality improvement news, resources and data reporting tools, and applications. QualityNet is the only CMS-approved site for secure communications and healthcare quality data exchange between healthcare organizations and facilities.

Technical Release Notes (TRN) - eCQM Eligible Hospital/Critical Access Hospital

Category: Implementation

The Technical Release Notes (TRN) for Eligible Hospitalinfo-icon and Critical Access Hospitalinfo-icon eCQMsinfo-icon are resource files which provide an overview of technical changes in the eCQM Annual Update and addenda.

Technical Release Notes (TRN) - eCQM Eligible Professional/Clinician

Category: Implementation

The Technical Release Notes (TRN) for Eligible Professionalinfo-icon/Clinician eCQMsinfo-icon are resource files which provide an overview of technical changes in the eCQM Annual Update and addenda.

Unified Medical Language System® (UMLS®)

Category: Development

The Unified Medical Language System® (UMLS®) is a set of files and software that bring together health and biomedical vocabularies and standards to enable interoperability between computer systems. UMLS® is used to enhance or develop applications, such as electronic health records, classification tools, dictionaries and language translators. Use of UMLS® allows end users to view codes used in value setsinfo-icon such as CPT and ICD-10.

The United States Health Information Knowledgebase (USHIK)

Category: Implementation

The United States Health Information Knowledgebase (USHIK) is a registry and repository of health care-related metadata, specificationsinfo-icon, and standards. The CMS Quality Programs portal provides the ability to view, download, and compare eCQMinfo-icon specifications, data elements, value setsinfo-icon, and codes released by CMS and National Library of Medicine. Each eCQM release package is available in a single file, or can be downloaded as eligible hospitalinfo-icon, eligible professionalinfo-icon, and individual measure packages.

Value Set Authority Center (VSAC)

Category: Development, Implementation

The Value Setinfo-icon Authority Center (VSACinfo-icon) is a central repository for the official versions of value setsinfo-icon that support the electronic clinical quality measuresinfo-icon. The National Library of Medicine maintains the VSAC and provides downloadable access to the value sets and the Binding Parameter Specificationinfo-icon. The VSAC provides measure developersinfo-icon with tools to search existing value sets, create new value sets, and maintain value set content consistent with current versions of the terminologies they use.

VSAC Collaboration Tool

Category: Development

The Value Setinfo-icon Authority Center (VSACinfo-icon) Collaboration Tool provides a central site where value set authors can post value setsinfo-icon for collaborative discussion. Teams can share threaded discussions about the value sets, view recent value set expansions posted by site members, organize their value sets by usage and by team’s workflow needs, collaborate with other measure developersinfo-icon to harmonize value sets, and receive activity and change notifications from VSAC.

Last Updated: February 15, 2019