Glossary of eCQI Terms

Alignment
Measure alignment includes using the same quality measures and value sets across settings and within multiple programs when possible. Alignment is achieved when a set of measures works well across settings or programs to produce meaningful information without creating extra work for those responsible for the measurement.
Bonnie
Bonnie is a software tool that allows eCQM developers to test and verify the behavior of their eCQM logic. The main goal of the Bonnie application is to reduce the number of defects in eCQMs by providing a robust and automated testing framework. The Bonnie application allows measure developers to independently load measures that they have constructed using the MAT. Developers can then use the measure metadata to build a synthetic patient test deck for each measure from the clinical elements defined during the measure development process. By using measure metadata as a basis for building synthetic patients, developers can quickly and efficiently create a test deck for a measure. The Bonnie application helps measure developers execute the measure logic against the constructed patient test deck and evaluate whether the logic aligns with the intent of the measure.
Clinical Decision Support (CDS)
Health information technology functionality that builds upon the foundation of an electronic health record (EHR) to provide persons involved in care processes with general and person-specific information, intelligently filtered and organized, at appropriate times, to enhance health and health care.
Clinical Quality Language (CQL)
Clinical Quality Language (CQL) is an HL7 standard for trial use (STU) and it is part of the effort to harmonize standards between electronic clinical quality measures (eCQMs) and clinical decision support (CDS). CQL provides the ability to express logic that is human readable yet structured enough for processing a query electronically.
Clinical Quality Measure (CQM)
A mechanism used for assessing the degree to which a provider competently and safely delivers clinical services that are appropriate for the patient in an optimal time frame. CQMs are a subset of the broader category of performance measures.
Code system
A code system is a managed collection of concepts with each concept represented by at least one internally unique code and a human readable description, e.g., SNOMED CT.

Comment Period

A comment period is the period of time the public has to respond to a request for public comment - whether it is a proposed rule, Request for Information, a new measure posted on the Measure Management System website, or some other document. The Administrative Procedure Act requires that federal agencies give the public an opportunity to participate in rulemaking.  Executive Orders 12866 and 13563 provide that a comment period generally should be no less than 60 days, but the length of the comment period varies if not part of the rulemaking process.

Composite performance measure

A combination of two or more component measures, each of which individually reflects quality of care, into a single performance measure, also called composite measure, with a single score.
Continuous variable
A measure score in which each individual value for the measure can fall anywhere along a continuous scale, and can be aggregated using a variety of methods such as the calculation of a mean or median (for example, mean number of minutes between presentation of chest pain to the time of administration of thrombolytics).
Cypress
Cypress is an open source testing tool used by vendors to certify their EHRs and health information technology (IT) modules for calculating eCQMs. Cypress is an official testing tool for the ONC EHR Certification Program. Testing involves Cypress generating synthetic patient records for the subset of published eCQMs selected for certification and testing the ability of the EHR systems and health IT modules to accurately record, import, calculate, filter, and report eCQMs. The Cypress test data are available in QRDA Category I for import into an EHR system. Cypress tests an EHR system’s ability to generate accurate QRDA Category I and Category III documents for reporting to CMS.
Denominator
The lower part of a fraction used to calculate a rate, proportion, or ratio. It can be the same as the initial population or a subset of the initial population to further constrain the population for the purpose of the measure. Continuous variable measures do not have a denominator, but instead define a measure population.

Direct Referenced Code

A specific code that is referenced directly in the eCQM logic to describe a data element or one of its attributes. Direct referenced code metadata include the description of the code, the code system from which the code is derived, and the version of that code system.
Efficiency measure
A measure that evaluates the resource use (or cost) associated with a specific level of performance with respect to the aims of quality. For example, a provider in the healthcare system would be efficient if it was able to maximize output for a given set of inputs or to minimize inputs used to produce a given output.
Electronic Clinical Quality Measure (eCQM)
An eCQM is a clinical quality measure that is expressed and formatted to use data from electronic health records (EHR) and/or health information technology systems to measure healthcare quality, specifically data captured in structured form during the process of patient care. So they can be reported from an EHR, the Health Quality Measure Format (HQMF) is used to format the eCQM content using the Quality Data Model (QDM) to define the data elements and Clinical Quality Language (CQL) to express the logic needed to evaluate a provider or organization’s performance.
Electronic Health Record (EHR)
This is also known as the electronic patient record, electronic medical record, or computerized patient record. As defined by Healthcare Information Management and Systems Society, “the electronic health record (EHR) is a longitudinal electronic record of patient health information generated by one or more encounters in any care delivery setting. Included in this information are patient demographics, progress notes, problems, medications, vital signs, past medical history, immunizations, laboratory data, and imaging reports.”

Eligible Clinicians

Refers to clinicians who are eligible to participate in the Quality Payment Program through the Merit-based Incentive Payment System (MIPS) and similar participants of other CMS programs using eCQM’s for quality reporting such as Alternative Payment Model (APM) participants.

Eligible Hospitals

Medical facilities that report eCQMs and Improving Interoperability Program data through the CMS QualityNet Secure Portal. They include; Medicaid-eligible Hospital, Medicare-eligible Hospital, Dual-Eligible Hospital (both Medicare and Medicaid) and Critical Access hospitals (CAHs).

Eligible Professional

Health care professionals that report eCQMs and Medicaid Promoting Interoperability Program data through their state-based systems.

Feasibility Criteria
Extent to which the specifications, including measure logic, require data that are readily available or that could be captured without undue burden and can be implemented for performance measurement.
Harmonization
The standardization of specifications for related measures with the same measure focus (for example, influenza immunization of patients in hospitals or nursing homes); related measures for the same target population (for example, eye exam and HbA1c for patients with diabetes); or definitions applicable to many measures (for example, age designation for children) so that they are uniform or compatible, unless differences are justified (in other words, dictated by the evidence). The dimensions of harmonization can include numerator, denominator, exclusions, calculation, and data source and collection instructions. The extent of harmonization depends on the relationship of the measures, the evidence for the specific measure focus, and differences in data sources. Value sets used in measures (especially eCQMs) should be harmonized when the intended meaning is the same. Harmonization of logic in eCQMs is beneficial when the data source in the EHR is the same.
Health Information Technology (health IT)
Per the Health Information Technology for Economic and Clinical Health Act, the term health information technology means hardware, software, integrated technologies or related licenses, intellectual property, upgrades, or packaged solutions provided as services that are designed for or support the use by health care entities or patients for the electronic creation, maintenance, access, or exchange of health information.
Health Level 7 (HL7)
A standards-developing organization that provides a framework and standards for the exchange, integration, sharing, and retrieval of electronic health information that supports clinical practice and the management, delivery, and evaluation of health services.
Health Quality Measure Format (HQMF)
Health Quality Measure Format (HQMF) is an HL7 standards-based representation of a quality measure as an electronic document. A quality measure expressed in this way is also referred to as an electronic clinical quality measure (eCQM).
Impact of a measure (Importance sub-criterion)
Now called High Priority by the National Quality Forum (NQF). The measure topic addresses a specific national health goal or priority; affects large numbers of patients; is a leading cause of morbidity/mortality; high resource use and severity of patient/societal consequences of poor quality. For patient-reported outcomes, there is evidence that the target population values the patient reported outcome (PRO) and finds it meaningful.
Initial population
Refers to all events to be evaluated by a specific performance electronic clinical quality measure (eCQM) involving patients who share a common set of specified characteristics within a specific measurement set to which a given measure belongs. All patients counted (for example, as numerator, as denominator) are drawn from the initial population.
Intermediate outcome
An intermediate outcome is a (measured) change in physiologic state that leads to a longer-term health outcome. There should be a body of evidence that the measured intermediate clinical outcome leads to a desired health outcome.
Inverse measures
Inverse measures are measures where a lower performance rate is better. Therefore, a zero performance rate for these measures is a good score. For example, The National Healthcare Safety Network calculates most Healthcare-Associated Infections (HAI) as a standardized infection ratio (SIR). The SIR compares the actual number of HAIs (the numerator) with the predicted number based on the baseline U.S. experience (e.g., standard population), adjusting for several risk factors that have been found to be most associated with differences in infection rates. The goal is to have the numerator equal to or very close to zero thereby having a SIR equal to or very close to zero.
ONC Issue Tracking System
The ONC Issue Tracking System is a JIRA (Atlassian, Inc) based collaboration platform hosted by the HHS’s Office of National Coordinator for Health Information Technology (ONC) that supports the implementation of health information technology. It provides internal and external users with a common place to transparently log, prioritize, and discuss issues with appropriate subject matter experts on many topics. CMS uses JIRA during most phases of the eCQM measure lifecycle, including development, implementation, and maintenance. JIRA supports projects such as CMS’s annual Measures under Consideration process and ONC’s health IT certification program.
Kaizen
A long-term approach to work that systematically seeks to achieve incremental changes in order to improve efficiency and quality. It focuses on removing process waste and maximizing value to the customer (in this case the patient).
Lean Kaizen
A Japanese phrase meaning continuous quality improvement by eliminating waste. The principles were implemented after World War II, influenced by American quality management teachers who visited Japan.
Meaningful Measures
The Meaningful Measures framework is the Centers for Medicare and Medicaid Services’ initiative which identifies the highest priorities for quality measurement and improvement. It involves only assessing core issues that are the most critical to providing high-quality care and improving individual outcomes. The Meaningful Measure Areas serve as the connectors between CMS goals and individual measures/initiatives that demonstrate how high-quality outcomes for our beneficiaries are being achieved.
Measure
A mechanism to assign a quantity to an attribute by comparison to a criterion. A measure may stand alone or belong to a composite, subset, set, and/or collection of measures. A healthcare performance measure is a way to calculate whether and how often the healthcare system does what it should. Measures are based on scientific evidence about processes, outcomes, perceptions, or systems that relate to high-quality care.
Measure Authoring Tool (MAT)
A publicly available, web-based tool for measure developers to create electronic clinical quality measures (eCQMs). The MAT tool enables measure developers to create their eCQMs in HQMF, a structured document format, without extensive knowledge of HQMF standards. Measure developers use the MAT to express measure criteria using QDM and CQL.
Measure score
The numeric result that is computed by applying the measure specifications and scoring algorithm. The computed measure score represents an aggregation of all appropriate patient-level data (for example, proportion of patients who died, average lab value attained) for the entity being measured (hospital, health plan, home health agency, clinician, etc.). The measure specifications designate the entity that is being measured and to whom the measure score applies.
Measure steward
Is an individual or organization that owns a measure and is responsible for maintaining the measure. Measure stewards are often the same as measure developers, but not always. Measure stewards are also the ongoing point of contact for people interested in a given measure.
Measure testing
Empirical analysis to demonstrate the reliability and validity of a measure as specified including analysis of issues that pose threats to the validity of conclusions about quality of care such as exclusions, risk adjustment/stratification for outcome and resource use measures, methods to identify differences in performance, and comparability of data sources/methods.
Measures Management System (MMS)
In response to an ever-increasing demand for quality measures, the Centers for Medicare & Medicaid Services (CMS) developed a standardized system for developing and maintaining the quality measures used in its various accountability initiatives and programs. Known as the Measures Management System (MMS), measure developers (or contractors) should follow this core set of business processes and decision criteria when developing, implementing, and maintaining quality measures.
Measures Under Consideration (MUC)
A “list of quality and efficiency measures DHHS is considering adopting, through the federal rulemaking process, for use in the Medicare program.” Made publicly available by December 1 each year for categories of measures that are described in section 1890(b) (7) (B) (i) (I) of the [Affordable Care] Act.
National Quality Forum (NQF)
Not-for-profit, nonpartisan, membership-based organization that works to catalyze improvements in healthcare.
Null Performance Rate
If the measure is not applicable for all patients within the sample, the performance rate would be 0/0 (null) and would be considered satisfactorily reporting.
Numerator
The upper portion of a fraction used to calculate a rate, proportion, or ratio. Also called the measure focus, it is the target process, condition, event, or outcome. Numerator criteria are the processes or outcomes expected for each patient, procedure, or other unit of measurement defined in the denominator. A numerator statement describes the clinical action that satisfies the conditions of the performance measure.
Numerator exclusions
Defines instances that should not be included in the numerator data. Numerator exclusions are used in ratio and proportion measures.
Outcome Measure
A measure that assesses the results of healthcare that are experienced by patients: clinical events, recovery and health status, experiences in the health system, and efficiency/cost.
Patient-reported outcome (PRO)
Any report of the status of a patient’s health condition, health behavior, or experience with healthcare that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else. This definition reflects the key domains of:
  • Health-related quality of life (including functional status)
  • Symptoms and symptom burden (e.g., pain, fatigue)
  • Experience with care
  • Health behaviors (e.g., smoking, diet, exercise)
Patient-reported outcome measure (PROM)
Instrument, scale, or single-item measure used to assess the patient reported outcome (PRO) concept as perceived by the patient, obtained by directly asking the patient to self-report.
Patient-reported outcome-based performance measure (PRO-PM)
A performance measure that is based on patient-reported outcome measure (PROM) data aggregated for an accountable healthcare entity.
Measure testing
Measure testing (sometimes referred to as pilot testing) is divided into two main types: alpha testing (also called formative testing), and beta testing (also called field testing).

Prevention and Treatment of Leading Causes of Morbidity and Mortality

Promoting the most effective prevention and treatment practices for the leading causes of mortality, starting with cardiovascular disease.

Process measure
A measure that focuses on a sequence of actions or steps that should be followed to provide high quality evidence-based care. There should be a scientific basis for believing that the process, when executed well, will increase the probability of achieving a desired outcome.
Proportion
A score derived by dividing the number of cases that meet a criterion for quality (the numerator) by the number of eligible cases within a given time frame (the denominator) where the numerator cases are a subset of the denominator cases (for example, percentage of eligible women with a mammogram performed in the last year).
Quality Data Model (QDM)
QDM is an information model that defines relationships between patients and clinical concepts in a standardized format to enable electronic quality performance measurement. The model is the current structure for electronically representing quality measure concepts for stakeholders involved in electronic quality measurement development and reporting. The QDM provides the language that defines the criteria for clinical quality measurement. It allows the electronic definition of a clinical concept via its data elements and provides the vocabulary to relate them to each other. By relating attributes between data elements and using filtering functions, the QDM provides a method to construct complex clinical representations for eCQMs.
Quality measure (or performance measure)
Numeric quantification of healthcare quality for a designated accountable healthcare entity, such as hospital, health plan, nursing home, clinician, etc. A healthcare performance measure is a way to calculate whether and how often the healthcare system does what it should. Measures are based on scientific evidence about processes, outcomes, perceptions, or systems that relate to high-quality care.
Quality Reporting Document Architecture (QRDA)
QRDA is a standard document format for the exchange of electronic clinical quality measure (eCQM) data. The QRDA documents:
  • Contain data extracted from electronic health records (EHRs) and other health information technology systems
  • Can be used to exchange eCQM data between systems
  • Are the data submission standards for a variety of quality measurement and reporting initiatives
  • Were adopted by the Office of the National Coordinator for Health Information Technology (ONC) as the standard to support both QRDA Category I (individual patient) and QRDA Category III (provider’s aggregate) data submission
Ratio
A score that is derived by dividing a count of one type of data by a count of another type of data (for example, the number of patients with central lines who develop infection divided by the number of central line days).

Resource use measures

Also called cost and resource use measures. Refers to broadly applicable and comparable measures of health services counts (in terms of units or dollars) applied to a population or event (broadly defined to include diagnoses, procedures, or encounters). A resource use measure counts the frequency of defined health system resources. Some measures may monetize the health service by applying a dollar amount such as allowable charges, paid amounts, or standardized prices to each unit of resource use.
Respecified Measure
An existing measure is changed to fit the current purpose or use. This may mean changing a measure to meet the needs of a different care setting, data source, or population. Or, it may mean changes to the numerator, denominator, or adding specifications to fit the current use.
Scoring
Method(s) applied to data to generate results/score. Most quality measures produce rates. However, other scoring methods include categorical value, continuous variable, count, frequency distribution, non-weighted score/composite/scale, ratio, and weighted score/composite/scales.
Semantic validation
A method of testing the validity of an electronic clinical quality measure (eCQM) whereby the formal criteria in an eCQM are compared to a manual computation of the measure from the same test database.
Sensitivity
As a statistical term, sensitivity refers to the proportion of actual positives that are correctly identified as such (for example, the percentage of people with diabetes who are correctly identified as having diabetes). See Specificity.
Specifications
Measure instructions that address the following: data elements, data sources, point of data collection, timing and frequency of data collection and reporting, specific instruments to be used (if appropriate), and implementation strategies.

Specificity

As a statistical term, specificity refers to the proportion of negatives that are correctly identified (for example, the percentage of healthy people who are correctly identified as not having the condition). Perfect specificity would mean that the measure recognizes all actual negatives (for example, all healthy people will be recognized as healthy). See Sensitivity.
Standard for Trial Use (STU)
STUs are used to "provide timely compliance with regulatory or other governmental mandate and/or timely response to industry or market demand." STU, following a suitable period for evaluation and comment, are incorporated into fully balloted and accredited version of the standard. Formerly called Draft Standard for Trial Use (DSTU).
Stratification
Divides a population or resource services into distinct, independent groups of similar data, enabling analysis of the specific subgroups. This type of adjustment can show where disparities exist or where there is a need to expose differences in results.
Structural measure 
A structural measure is one that assesses features of a healthcare organization or clinician relevant to its capacity to provide healthcare.

Target population

The numerator (cases) and denominator (population sample meeting specified criteria) of the measure.
United States Health Information Knowledgebase (USHIK)
An on-line, publicly accessible registry and repository of healthcare-related metadata, specifications, and standards. USHIK is funded and directed by the Agency for Healthcare Research and Quality (AHRQ) with management support and engagement from numerous public and private partners.
Usability criteria
Extent to which intended audiences (for example, consumers, purchasers, providers, policy makers) can understand a measure’s results and find them useful for quality improvement and decision making. Usability criteria ask if the measure is strong enough to be used for various types of measurement programs, including public reporting, whether it leads to actual improvement for patients, and whether the benefits of the measure outweigh any potential harms.
Validity (scientific acceptability of measure properties subcriterion)
Measure validity: The measure accurately represents the concept being evaluated and achieves the purpose for which it is intended (to measure quality). For example, the measure:
  • Clearly identifies the concept being evaluated (face validity)
  • Includes all necessary data elements, codes, and tables to detect a positive occurrence when one exists (construct validity)
  • Includes all necessary data sources to detect a positive occurrence when one exists (construct validity)
Data element validity: The extent to which the information represented by the data element or code used in the measure reflects the actual concept or event intended. For example:
  • A medication code is used as a proxy for a diagnosis code
  • Data element response categories include all values necessary to provide an accurate response

Value Set

A value set is a subset of concepts drawn from one or more code systems, where the concepts included in the subset share a common scope of use, e.g., Anticoagulant Therapy.
Value Set Authority Center (VSAC)
The VSAC is a central repository for the official versions of value sets that support electronic Clinical Quality Measures (eCQMs). The National Library of Medicine (NLM) maintains the Value Set Authority Center and provides downloadable access to the value sets and the Binding Parameter Specification. The VSAC provides measure developers with tools to search existing value sets, collaborate with other measure developers to harmonize value sets, to create new value sets, and to maintain value set content consistent with current versions of the terminologies they use.
Value Set Authority Center Collaboration Tool
The VSAC Collaboration Tool provides a central site where value set authors can post value sets for collaborative discussion. In that site, teams can share threaded discussions about the value sets, view recent value set expansions posted by site members, organize their value sets by usage and by team’s workflow needs, and receive activity and change notifications from VSAC. VSAC authors and stewards can access the VSAC Collaboration Tool from within VSAC’s Collaboration Management tab as well as through a direct link to the VSAC Collaboration Tool.

Virtual Medical Record (vMR)

Data model for representing the data that are analyzed and/or produced by clinical decision support (CDS) engines.
eXtensible Markup Language (XML) 
A markup language that defines a set of rules for encoding documents in a format which is both human-readable and machine-readable.

____________________________________________________

[1] Department of Health and Human Services, Agency for Healthcare Research and Quality. Health care efficiency measures: identification, categorization, and evaluation. Final report. Rockville, MD: Agency for Healthcare Research and Quality; Apr 2008; Report No. 08-0030. Available at: http://archive.ahrq.gov/research/findings/final-reports/efficiency/ . Accessed on: March 14, 2016.
[2] National Quality Forum . Measurement framework: evaluating efficiency across patient-focused episodes of care. Washington, DC: National Quality Forum; 2009; Report No. ISBN: 978-1-933875-42-2.
[3] Value Set Authority Center (VSAC). Available at: https://ecqi.healthit.gov/ecqm-tools/tool-library/value-set-authority-center-vsac. Accessed on May 24, 2016.
Last Updated: June 4, 2018