Back to top


Agile development

Agile development is a term used to describe iterative software development in order to shorten the software development lifecycle. Agile development teams execute the entire software development lifecycle in smaller increments, usually called sprints, typically 1-4 weeks long. Agile development is often contrasted with traditional or waterfall development, where larger projects are planned up front and executed against that plan. Agile development is an iterative approach to development with regular feedback loops or intervals. These iterations allow a team to be diverted to and productive in another part of a project while a problem or issue is resolved in another part. . Microsoft. (2021, May 12). What is agile?


Measure alignment includes using the same quality measures and value sets across settings and within multiple programs when possible. Alignment is achieved when a set of measures works well across settings or programs to produce meaningful information without creating extra work for those responsible for the measurement.

Analytic model or framework

An analytic model for developing clinical practice guidelines is a visual representation of a causal pathway showing how the proposed key research question(s) and intervention(s) under consideration as reflected in the PICOTS (P-Patient population; I- Intervention; C- Comparator; O- Outcome; T- Timing; S- Setting) framework are linked to their intended outcomes that require confirmation by evidence review to support related recommendations. The linkages in the model represent critical logical premises and presumed relationships among intermediate, surrogate, and ultimate health outcomes related to a specified clinical question. Woolf, S., Schünemann, H. J., Eccles, M. P., Grimshaw, J. M., Shekelle, P. (2012). Developing clinical practice guidelines: types of evidence and outcomes; values and economics, synthesis, grading, and presentation and deriving recommendations. Implementation Science, 7(61).

Application Programming Interface

An Application Programming Interface (API) is a system of tools and resources in an operating system, enabling developers to create software applications. An API is a software intermediary that allows two applications to talk to each other. Each time you use an app like Facebook, send an instant message, or check the weather on your phone, you are using an API. MuleSoft. (n.d.). What is an API? Retrieved June 2, 2021, from

Artifacts or computational artifact

A computational artifact is anything created by a human using a computer. An artifact can be, but not limited to a code, program, image, audio, video, presentation, or web page file. The University of Rhode Island. (n.d.) Computing as a creative activity and computational artifacts. Retrieved September 21, 2021, from

Business Process Modeling Notation

Business Process Modeling Notation (BPMN) is a flow chart method that models the steps of a planned business process from end to end. A key to Business Process Management, it visually depicts a detailed sequence of business activities and information flows needed to complete a process. BPMN was originally developed by the Business Process Management Initiative (BPMI). It is currently maintained by the Object Management Group (OMG).

Case Management Modeling and Notation

Case Management Modeling and Notation (CMMN) “is a graphical notation used for capturing work methods that are based on the handling of cases requiring various activities that may be performed in an unpredictable order in response to evolving situations.” Visual Paradigm. (n.d.). What is case management and modeling notation (CMMN). Retrieved September 21, 2021, from

Case presentation

A case presentation is a formal communication between health care professionals (e.g., physicians, pharmacists, nurses, therapists, nutritionists) that gives a synthesis of a patient's clinical information. Components of a case presentation typically include: identifying information; reason for consultation/admission; chief complaint (in the patient's own words); recent history; active medical problems; medications; allergies; social history; physical examination findings; relevant test results; assessment; and treatment plans.

CDS Alerts

The most commonly implemented clinical decision support (CDS) alerts prompt clinicians about guidance, e.g., drug-allergy, drug-drug, and drug-disease warnings, or provide dosing guidance. Passive CDS includes order sets, patient data reports, and documentation templates while active CDS includes rules and alerts usually delivered through computerized provider order entry (CPOE) or other functions of the electronic health records (EHRs).

CDS artifacts

CDS artifacts are items that represent medical knowledge from various knowledge sources (e.g., clinical guidelines, peer-reviewed articles, local best practices, and clinical quality measures). The artifacts can take many forms, but the goal is to create computable, interoperable translations using Clinical Quality Language (CQL). As of April 2021, there are 63 CDS artifacts found in the Repository. . A variety of organizations, including federal agencies, contribute CDS artifacts to the Repository and they span a number of topics including anatomy, health care, diseases, etc.

CDS developer

A CDS developer is an individual or organization that translates knowledge to a structured and/or executable tool that aids in making evidence-informed decisions about a patient’s healthcare. CDS developers may or may not be the original knowledge authors (e.g., guideline developers, subject matter experts) or the final implementers. They are responsible for ensuring accurate and consistent reflection of the original clinical knowledge in the appropriate standard coding schemes (e.g., Clinical Quality Language, and terminologies such as Current Procedural Terminology and SNOMED CT), accounting appropriately for intellectual property and licensing.

Change Review Process

The Change Review Process (CRP) provides electronic clinical quality measure (eCQM) users the opportunity to review and comment on draft changes to the eCQM specifications and supporting resources under consideration by the measure steward. The CRP is conducted through the ONC Project Tracking System (Jira) website. The goal of the CRP is for eCQM implementers to comment on the potential impact of changes to measures so the Centers for Medicare & Medicaid Services (CMS) and measure stewards can make improvements to meet CMS’s intent of minimizing provider and vendor burden in the collection, capture, calculation, and reporting of eCQMs. To participate in the CRP, users must have a Jira account and log into the eCQM Issue Tracker where specific CRP tickets are posted for public comment and voting. Users can sign up for an account on the login in page.

Clinical decision support

CDS is health information technology functionality that builds upon the foundation of an EHR to provide persons involved in care processes with general and person-specific information, intelligently filtered and organized, at appropriate times, to enhance health and healthcare.

Clinical Decision Support (CDS)

CDS “alert fatigue” occurs when a provider, after receiving too many alerts or reminders, begins to override or ignore further alerts without attending to them, thus potentially decreasing the care improvements expected from the tools.

Clinical Document Architecture

Clinical Document Architecture (CDA) is a popular, flexible markup standard developed by Health Level Seven International® that defines the structure of certain patient medical records, such as discharge summaries and progress notes, as a way to better exchange this information between healthcare providers and patients. Wallask, S. (n.d.). Clinical Document Architecture (CDA). TechTarget: SearchHealthIT. Retrieved September 21, 2021, from

Clinical Information Systems

Clinical information systems "are computer systems that provide immediate access to current patient data regarding clinical notes, medication history, laboratory reports, images, and reports either directly or via data networks. They are parts of a hospital information system, which facilitates direct patient care." Islam, M. M., Poly, T., N., & Li, Y-C. J. (2018). Recent advancement of clinical information systems: Opportunities and challenges. Yearbook of Medical Informatics, 27(1), 83–90.

Clinical practice guideline representations or expressions L1-L4

L1 Narrative: The knowledge is human-readable and in an unstructured format. This knowledge can be used as an input for crafting policies or making decisions. The structure does not allow machines to utilize knowledge.

L2 Semi-structured: This level has some structure, paving the way for clinical domain experts to interpret. HUMAN READABLE

L3 Structured: This adds a machine-interpretable structure to the knowledge representation. Knowledge can be shared across settings and systems. There may be subjectivity in knowledge interpretation. COMPUTER READABLE

L4 Executable: This format enables not only machine-readable knowledge representation but also machine-executable knowledge representation based on the patient’s clinical context with a specific type of CDS tool. CLINICAL DECISION SUPPORT

Clinical practice guidelines

Clinical practice guidelines (CPGs) are systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances. CPGs are statements that include recommendations intended to optimize patient care. They are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options. Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. (2011). Clinical practice guidelines we can trust. Institute of Medicine. 

Clinical quality measure

A clinical quality measure (CQM) is a mechanism used for assessing the degree to which a provider competently and safely delivers clinical services that are appropriate for the patient in an optimal time frame. CQMs are a subset of the broader category of quality measures.

Code repositories

Code repositories are a file archive and web hosting facility that provides secure storage for code and version control. 

Code system

A code system is a managed collection of concepts with each concept represented by at least one internally unique code and a human readable description, e.g., SNOMED CT.

Coding system

A coding system is the symbolic arrangement of data or instructions in a computer program or the set of such instructions.

Comment period

A comment period is the period of time the public has to respond to a request for public comment, such as a proposed rule, Request for Information, a new measure posted on the Measures Management System website, or some other document. The Administrative Procedure Act requires that federal agencies give the public an opportunity to participate in rulemaking. Executive Orders 12866 and 13563 provide that a comment period generally should be no less than 60 days, but the length of the comment period varies if not part of the rulemaking process.

Communication or health communication activities

Communication activities are the study and use of communication strategies to inform and influence individual and community decisions that affect health. Health communications link the fields of communication and health and are increasingly recognized as necessary elements to improve personal and public health. Examples of communication activities include traditional (e.g., print through manuscripts, professional presentations, issue briefs, white papers, television, radio media) and non-traditional (e.g., social marketing techniques and social media platforms) methods of communication to present and disseminate the clinical guidelines and to encourage their use and adherence. For more information see Centers for Disease Control and Prevention's Gateway to health communication.

Composite measure

A composite measure is a measure that contains two or more individual measures, resulting in a single measure with a single score.

Computable care guidelines

Computable care guidelines are the expression of and sharing of health care guidelines in a grammar that is understood by a software application. Integrating the Health Enterprise. (n.d.). Computable care guidelines. Retrieved September 21, 2021 from

Computer code

Computer code is the symbolic arrangement of data or instructions in a computer program.

Concept maps

Concept maps are visual representations of information that can take the form of charts, graphic organizers, tables, flowcharts, Venn Diagrams, timelines, or T-charts. The Learning Center - University of North Carolina at Chapel Hill. (n.d.). Concept maps. Retrieved September 21, 2021, from

Conflict of interest

"A conflict of interest is a set of circumstances that creates a risk that professional judgment or actions regarding a primary interest will be unduly influenced by a secondary interest" (p. 46). For example, a work group member might have pharmaceutical stock for a vaccine that is recommended as part of a clinical guideline. Committee on Conflict of Interest in Medical Research, Education, and Practice. (2009). Conflict of interest in medical research, education, and practice. In (pp 44-61). Institute of Medicine.

Continuous variable

A continuous variable is a measure score in which each individual value for the measure can fall anywhere along a continuous scale and can be aggregated using a variety of methods such as the calculation of a mean or median (for example, mean number of minutes between presentation of chest pain to the time of administration of thrombolytics).

Critical Access Hospital

A Critical Access Hospital (CAH) is a hospital in a federal program established in 1997 as part of the Balanced Budget Act designed to promote rural health planning, network development, and improve access to health services for rural residents. CAHs represent a separate provider type with their own Medicare Conditions of Participation (CoP) as well as a separate payment method. The Code of Federal Regulations lists the CoPs for CAHs at 42 CFR 485 subpart F.

Crowd sourcing

Crowd sourcing is a method to obtain information or input into a particular task or project by enlisting the services of a large number of people, either paid or unpaid, typically via the Internet.

Data capture

Data capture, or electronic data capture, is the process of extracting information from a paper of electronic document and converting it into data readable by a computer. Hyland. (n.d.). What is data capture? Retrieved September 21, 2021, from

Data element validity

Data element validity is the extent to which the information represented by the data element or code used in the measure reflects the actual concept or event intended. For example

  • Use of a medication code as a proxy for a diagnosis code.
  • Data element response categories include all values necessary to provide an accurate response.

See also measure validity.

Data elements

A data element is any unit of data defined for processing, e.g., account number, name, address, and city.

Data model

A data model is an abstract model that organizes elements of data and standardizes how they relate to one another. For instance, a data model that links guideline information with clinical data for the patient. Guru99. (n.d.). Data modelling: Conceptual, logical, physical model types. Retrieved September 21, 2021, from

Decision Model and Notation

Decision Model and Notation (DMN) is a standard published by the Object Management Group. It is a standard approach for describing and modeling repeatable decisions within organizations to ensure that decision models are interchangeable across organizations. HEFLO. (n.d.) What is decision model and notation (DMN)? Retrieved September 21, 2021 from

Decision tree or algorithm

A decision tree is an upside-down tree that makes decisions based on the conditions present in the data. It is a supervised machine learning algorithm where data are continuously divided at each row based on certain rules until the final outcome is generated. Great Learning Team. (2020, February 13). Decision tree algorithm explained with examples.


The denominator is the lower part of a fraction used to calculate a rate, proportion, or ratio. It can be the same as the initial population or a subset of the initial population to further constrain the population for the purpose of the measure. Continuous variable measures do not have a denominator, but instead define a measure population.

Denominator exception

A denominator exception is any condition that should remove a patient, procedure, or unit of measurement from the denominator of the performance rate only if the numerator criteria are not met. A denominator exception allows for adjustment of the calculated score for those providers with higher risk populations. A denominator exception also provides for the exercise of clinical judgment and should be specifically defined where capturing the information in a structured manner fits the clinical workflow. Denominator exception cases are removed from the denominator. However, the number of patients with valid exceptions may still be reported. Allowable reasons fall into three general categories - medical reasons, patient reasons, or system reasons. A denominator exception is used only in proportion measures.

Denominator exclusion

A denominator exclusion is a case that should be removed from the measure population and denominator before determining if numerator criteria are met. Denominator exclusions are used in proportion and ratio measures to help narrow the denominator. For example, a measure developer would list patients with bilateral lower extremity amputations as a denominator exclusion for a measure requiring foot exams. Continuous variable measures may use denominator exclusions but use the term measure population exclusion instead of denominator exclusion.

Derivative products

Derivative products, with respect to clinical practice guidelines, are products with content that is derived from the content of the practice guideline, e.g., CDS, patient/family guides, pocket cards, phone apps for clinicians, continuing education programs. 

Digital platform

A digital platform is an established device or web-based platform for presenting cloud technology and content, such as Facebook, Twitter, Blogs, Websites, and sometimes short message service (SMS). This is in contrast to an analog platform, e.g., billboards, direct mail, telemarketing, events, word-of-mouth, etc.

Digital Quality Measures

CMS draft definition - A digital quality measure (dQM) is a software that processes digital data to produce a measure score or measure scores. Data sources for dQMs may include administrative systems, electronically submitted clinical assessment data, case management systems, EHRs, instruments (for example, medical devices and wearable devices), patient portals or applications (for example, for collection of patient-generated health data), health information exchanges (HIEs) or registries, and other sources.

Direct reference code

A direct reference code (DRC) is a specific code that is referenced directly in the eCQM logic to describe a data element or one of its attributes. Direct reference code metadata include the description of the code, the code system from which the code is derived, and the version of that code system.

eCQI implementer

An electronic clinical quality improvement (eCQI) implementer does many things to prepare processes and systems such as

  • putting measure data components into systems and workflow
  • using measures when conducting healthcare activities
  • providing information from measures to inform quality improvement (e.g., health information technology implementer, quality analyst, quality reporting validator)
Efficiency measure

An efficiency measure is the cost of care (inputs to the health system in the form of expenditures and other resources) associated with a specified level of health outcome.

Electronic Case Reporting

Electronic case reporting (eCR) is the automated generation and electronic submission of reportable diseases and conditions from an EHR to public health agencies. Each state has public health reporting requirements and relies on healthcare providers to report on certain conditions.

Electronic clinical quality improvement

Electronic clinical quality Improvement (eCQI) is the use of health information technology, the functionality, and data in an EHR and/or other health information technology, along with clinical best practices to support, leverage, and advance quality improvement initiatives.

Electronic clinical quality measure

An electronic clinical quality measure (eCQM) is a clinical quality measure expressed and formatted to use data from EHRs and/or health information technology systems to measure healthcare quality, ideally data captured in structured form during the process of patient care. For the measured entity to report an eCQM from an EHR, eCQM developers format the Health Quality Measure Format (HQMF) using the Quality Data Model (QDM) to define the data elements and CQL to express the logic needed to evaluate a provider or organization’s performance.

Electronic health record

Electronic health record (EHR) is also known as the electronic patient record, electronic medical record, or computerized patient record. An EHR is a “longitudinal electronic record of patient health information generated by one or more encounters in any care delivery setting. Included in this information are patient demographics, progress notes, problems, medications, vital signs, past medical history, immunizations, laboratory data, diagnoses and treatment, medications, allergies, immunizations as well as radiology images and laboratory results.” International Social Security Association. (n.d.). Information and communication technology- Guideline 92 electronic health record system. Retrieved September  21, 2021, from

Eligible clinician

An eligible clinician refers to a clinician who is eligible to participate in the Quality Payment Program (QPP) through the Merit-based Incentive Payment System (MIPS) and similar participants of other CMS programs using eCQMs for quality reporting such as Alternative Payment Model (APM) participants.

Eligible hospital

An eligible hospital is an acute care facility, e.g., Subsection (d) hospitals in the 50 states or District of Columbia paid under the Inpatient Prospective Payment System (IPPS) and critical access hospitals, meeting eligibility requirements for EHR incentive payments by adopting, implementing, or updating certified electronic health record technology.

Eligible professional

An eligible professional is a healthcare professional that reports eCQMs and Medicaid Promoting Interoperability Program data through their state-based systems.

End user

An end user (sometimes end-user) is a person who ultimately uses or intends to ultimately use a guideline, measure, or its derivative products.

Environmental Scan

An environmental scan is the process of systematically reviewing and interpreting data to identify issues and opportunities that will influence prioritization of current or future plans.

Evaluation activities

Evaluation activities are a systematic collection of information about a guideline. It involves collecting and analyzing information about a guideline's activities, characteristics, and outcomes to make judgments about the guideline, to improve its effectiveness, and/or inform decisions about future guideline development.

Expression Logical Model

The Expression Logical Model (ELM) is a machine-readable representation of an eCQM's logic and provides the information needed to automatically retrieve data from an EHR. The ELM file can be in XML (.xml) or JavaScript Object Notation (JSON) (.json).

Feasibility criterion

The Feasibility criterion is the extent to which the specifications, including measure logic, require data that are readily available or that could be captured without undue burden and can be implemented for performance measurement. National Quality Forum. (2019). Measure Evaluation Criteria and Guidance for Evaluating Measures for Endorsement.

FHIR profile

A Fast Healthcare Interoperability Resources (FHIR) profile is a set of requirements and constraints that are placed on a resource. It can describe general features that the system supports for that resource or information that is handled or produced according to a specific use case. Often, they include rules about which API features, terminologies, or resource elements are or are not used.

Future state

Future state is a model that integrates the downstream work of informatics, implementation, communication, and evaluation into the guideline or measure development process.


Git is a free and open source distributed version control system for tracking changes in any set of files among programmers cooperating on source code during software development. It is designed to handle everything from small to very large projects with speed and efficiency.


GitHub, Inc. is a subsidiary of Microsoft which provides hosting for software development and version control using Git.

GRADE Evidence to Decision framework

GRADE Evidence to Decision (EtD) framework is a systematic and transparent approach to making well-informed healthcare choices.

Grading of Recommendations Assessment, Development, and Evaluation

Grading of Recommendations Assessment, Development, and Evaluation (GRADE) is a transparent framework for developing and presenting summaries of evidence and provides a systematic approach for making clinical practice recommendations. It is the most widely adopted tool for grading the quality of evidence and for making recommendations.

Guideline Implementation with Decision Support

Guideline Implementation with Decision Support (G.U.I.D.E.S.) is a tool to assist professionals when implementing guidelines with CDS.

Guideline oversight committee

A guideline oversight or steering committee, selected at the beginning of a project, is a senior advisory body that is made up of senior stakeholders or experts that gain leadership support and are accountable for the proposed guideline. This committee provides guidance on issues that emerge in the guideline development process, including the products that emerge from all guideline working groups.

Guideline recommendation

A guideline recommendation tells the intended end-user of a guideline what they can or should do in specific situations to achieve the best health outcomes possible, individually or collectively. It offers a choice among different interventions or measures having an anticipated positive impact on health and implications for the use of resources. 

Guideline working groups

Guideline working groups include experts who conduct the guideline development work. Working groups can focus on different areas of expertise such as conducting the systematic literature review, developing the computable guideline, designing the implementation process, communicating/disseminating the guideline, or evaluating the impact of the guideline.


Harmonization is the standardization of specifications for related measures with the same measure focus (for example, influenza immunization of patients in hospitals or nursing homes); related measures for the same target population (for example, eye exam and HbA1c for patients with diabetes); or definitions applicable to many measures (for example, age designation for children) so that they are uniform or compatible, unless differences are justified (in other words, dictated by the evidence). The dimensions of harmonization can include numerator, denominator, exclusions, calculation, and data source and collection instructions. The extent of harmonization depends on the relationship of the measures, the evidence for the specific measure focus, and differences in data sources. Value sets used in measures should be harmonized when the intended meaning is the same. Harmonization can also mean adoption of the same standard(s) for different purposes such as use of standards for eCQMs and CDS, for example Clinical Quality Language for logic expression in eCQMs and CDS.

Health information technology

Health information technology (HIT) involves the processing, storage, and exchange of health information in an electronic environment. According to the Health Information Technology for Economic and Clinical Health (HITECH) Act, the term health information technology includes hardware, software, integrated technologies or related licenses, intellectual property, upgrades, or packaged solutions provided as services that are designed for or support the use by healthcare entities or patients for the electronic creation, maintenance, access, or exchange of health information.

Health IT developer/vendor

A health IT developer/vendor is an entity that designs, develops, and/or markets health IT software application(s) for use in hospitals, ambulatory care settings, and/or specialty care delivery settings, e.g., EHR developer, EHR vendor, quality reporting software developer, quality reporting software vendor.

Health Level Seven International®

Health Level Seven International® (HL7) is a standards-developing organization that provides a framework and international standards for the exchange, integration, sharing, and retrieval of electronic health information (including clinical and administrative data) that supports clinical practice and the management, delivery, and evaluation of health services. These standards for transfer of data between healthcare software applications focus on the application layer, which is "layer 7" in the Open Systems Interconnection model (OSI model), a conceptual model that characterizes and standardizes the communication functions of a telecommunication or computing system without regard to its underlying internal structure and technology.

Healthcare consumer

An individual who uses the services of a healthcare provider including patients receiving medical care or treatment. IHI Global Services. (n.d). What is health consumer. Retrieved September 21, 2021, from

Healthcare organization

A healthcare organization is a purposefully designed, structured social system developed for the delivery of healthcare services by specialized workforces to defined communities, populations, or markets.

Healthcare payor

A healthcare payor is any payer of healthcare services other than the insured person, e.g., insurance company, Health Maintenance Organization, Preferred Provider Organization, or the federal government.

Healthcare Service Location

Healthcare Service Location (HSLOC) is a classification of locations and settings where healthcare services are provided which can be used to identify locations within a facility, such as the Emergency Department and medical intensive care unit.

Hybrid measure

A hybrid measure is a quality measure that uses more than one source of data for measure calculation. Current hybrid measures use claims data and electronic clinical data from EHRs to calculate measure results.

Hypertext Markup Language

Hypertext Markup Language (HTML) is the standard markup language for creating web pages and web applications.

Implementation activities

Implementation activities refer to configuration, customization, and other steps that are needed for health information technologies to function for a specific organization or group of end-users. For example, with CDS, the local implementation will consider end-user workflows to identify the points at which to present the CDS, determine which data fields in local databases will map to data fields that drive the CDS, and educate end-users about important features of the CDS. Typically, this involves translation of guideline recommendations from L3 to L4 or other local adaptation (localization).

Importance to measure and report criterion

The Importance criterion is the extent to which the specific measure focus is evidence-based and important to making significant gains in healthcare quality where there is variation in or overall less-than-optimal performance. National Quality Forum. (2019). Measure Evaluation Criteria and Guidance for Evaluating Measures for Endorsement.

Industry standard modeling technique

Industry standard modeling techniques use a standard visual approach to facilitate communication related to business models and software design. For example, Unified Modeling Language (UML) uses integrated diagrams to specify and document the structure and dynamic behavior of information system and software artifacts.

Informatics Activities

Informatics activities include tasks related to knowledge engineering and technical development of CPG artifacts (L2 to L4). These may include, but are not limited to: developing use case diagrams, flow diagrams, and decision trees; identifying relevant data elements, FHIR profiles, terminologies, value sets, libraries, rules, and logic representations; writing software code (e.g., for CDS); and developing comprehensive test cases and testing resources.

Integrated process

With an integrated process, steps related to guideline development, informatics activities, implementation, evaluation, and communications are conducted in parallel, insofar as possible, rather than in sequence.

Intermediate outcome measure

An intermediate outcome measure is a measure that assesses the change produced by a healthcare intervention that leads to a long-term outcome.

Inverse measure

An inverse measure is a measure where a lower performance rate is better. Therefore, a zero performance rate for these measures is a good score. For example, the National Healthcare Safety Network calculates most healthcare-associated infections (HAIs) as a standardized infection ratio (SIR). The SIR compares the actual number of HAIs (the numerator) with the predicted number based on the baseline U.S. experience (e.g., standard population), adjusting for several risk factors that have been found to be most associated with differences in infection rates. The goal is to have the numerator equal to or very close to zero thereby having an SIR equal to or very close to zero.

JavaScript Object Notation

JavaScript Object Notation (JSON) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language Standard ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely language independent, but uses conventions that are familiar to programmers of the C-family of languages including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language. (n.d.). Introducing JSON. Retrieved September 21, 2021, from

Knowledge assets

Knowledge assets (e.g., declarative or models) refer to specific knowledge-related items. These can include declarative (factual) assets or procedural assets that describe how facts will be used. Types of knowledge assets can include data, expertise, lessons learned, policies and procedures, and other knowledge-related documents. Freeze, R.D. & Kulkarni, U. (2007), Knowledge management capability: Defining knowledge assets. Journal of Knowledge Management,11(6), pp. 94-109.

Knowledge management platform

A knowledge management platform automatically structures and organizes content to be quickly findable, navigable, and searchable in a variety of places. 

Living guideline

A living guideline uses the results of a living systematic review and determines whether new guidelines recommendations are needed or whether modifications are needed to existing guideline recommendations as new evidence emerges. Akl, E. A., Meerpohl, J. J., Elliott, J., Kahale, L. A., Schünemann, H. J., Agoritsas, T., ... & Pearson, L. (2017). Living systematic reviews: 4. Living guideline recommendations. Journal of Clinical Epidemiology, 91, 47-53.

Living systematic review

A living systematic review uses the same processes as other systematic reviews but is continually updated, based on frequent searches of the literature, with incorporation of relevant new evidence as it becomes available. Cochrane Community (n.d.) Living systematic reviews. Retrieved September  21, 2021 from

Logic model or analytic framework

A logic model is a graphic depiction (road map) that represents the shared relationships among the resources, activities, outputs, outcomes, and impact for the evaluation of a proposed guideline. This visual framework shows the critical logical premises and presumed relationships among intermediate, surrogate, and ultimate health outcomes related to a specified clinical question. Woolf, S., Schünemann, H. J., Eccles, M. P., Grimshaw, J. M., & Shekelle, P. (2012). Developing clinical practice guidelines: types of evidence and outcomes; values and economics, synthesis, grading, and presentation and deriving recommendations. Implementation Science, 7(1), 1-12.

Machine learning

Machine learning is a branch of artificial intelligence focused on building applications that learn from data and improve their accuracy over time without being programmed to do so. IBM Cloud Education (2020, July 15). Machine learning.

Meaningful Measures Framework

The Meaningful Measures Framework is CMS's initiative which identifies the highest priorities for quality measurement and improvement. It involves only assessing core issues that are the most critical to providing high-quality care and improving individual outcomes. The Meaningful Measure Areas serve as the connectors between CMS goals and individual measures/initiatives that demonstrate how high-quality outcomes for CMS beneficiaries are being achieved.

Measure developer

A measure developer is an individual or organization responsible for the development, implementation, and maintenance of a measure. Measure developers create, edit, and submit measures for consideration by CMS to include in programs. CMS encourages measure developers to use the Blueprint as a guide in creating their measures and to collaborate with other measure developers to share best practices/new learnings freely e.g., CMS measure development contractor, hospital system, medical association, or federal health agency.

Measure score

The measure score is the numeric result that is computed by applying the measure specifications and scoring algorithm. The computed measure score represents an aggregation of all appropriate patient-level data (for example, proportion of patients who died, average lab value attained) for the entity being measured (hospital, health plan, home health agency, clinician, etc.). The measure specifications designate the entity that is being measured and to whom the measure score applies.

Measure steward

A measure steward is an individual or organization that owns a measure and is responsible for maintaining the measure. Measure stewards may also be measure developers. Measure stewards are also the ongoing point of contact for people interested in a given measure e.g., medical specialty society or federal health agency.

Measure testing

Measure testing is empirical analysis to demonstrate the reliability and validity of a measure as specified. It includes analysis of issues that pose threats to the validity of conclusions about quality of care such as exclusions, risk adjustment/stratification for outcome and resource use measures, methods to identify differences in performance, and comparability of data sources/methods.

Measure validity

Measure validity is when the measure accurately represents the evaluated concept and achieves the intended purpose (to measure quality). For example, the measure

  • clearly identifies the evaluated concept (face validity)
  • includes all necessary data elements, codes, and tables to detect a positive occurrence when one exists (construct validity)
  • includes all necessary data sources to detect a positive occurrence when one exists (construct validity)
Measured entities

Measured entities are the front-line clinicians and their organizations, including health information technology, collecting quality measurement data. Measured entities are the implementers of quality measures.

Measures Under Consideration

The Measures Under Consideration (MUC) is a list of quality and efficiency measures the Department of Health & Human Services is considering adopting, through the federal rulemaking process, for use in the Medicare program. The MUC list is made publicly available by December first each year for categories of measures that are described in section 1890(b) (7) (B) (i) (I) of the Social Security Act (SSA) as amended by Section 3014 of the Patient Protection and Affordable Care Act.

Narrative guidelines

A text-based representation of the practice guideline. Narrative guidelines typically include guideline recommendations as well as additional explanatory text.

National Quality Forum

The National Quality Forum (NQF) is a not-for-profit, nonpartisan, membership-based organization that works to catalyze improvements in healthcare.

Needs assessment

Needs assessment is aimed at delineating differences between the current state and the ideal state, followed by determining why these gaps exist and identifying solutions to address them.

Null performance rate

The null performance rate is when all of the denominator eligible instances are attributed to all denominator exceptions. Therefore, the performance rate for satisfactory reporting would be 0/0 (null).


The numerator is the upper portion of a fraction used to calculate a rate, proportion, or ratio. Also called the measure focus, it is the target process, condition, event, or outcome. Numerator criteria are the processes or outcomes expected for each patient, procedure, or other unit of measurement defined in the denominator. A numerator statement describes the action that satisfies the conditions of the quality measure.

Numerator exclusion

A numerator exclusion defines an instance that should not be included in the numerator data. Use numerator exclusions only in ratio and proportion measures.


A formal representation of knowledge within a domain that is machine-readable but also understandable to humans; it typically includes unique hierarchically arranged concepts that have specific attributes and are semantically related to other concepts. Dissanayake, P. I., Colicchio, T. K., & Cimino, J. J. (2020). Using clinical reasoning ontologies to make smarter clinical decision support systems: a systematic review and data synthesis. Journal of the American Medical Informatics Association, 27(1), 159-174.

Outcome measure

An outcome measure is a measure that focuses on the health status of a patient (or change in health status) resulting from healthcare - desirable or adverse.

Parameter estimates

A parameter estimate provides a statistical approximation of specific measure that describes a population. Point estimates provide a single value whereas confidence intervals provide a range of values within which the population estimate is most likely to occur.  

Patient-reported outcome (PRO)

A patient-reported outcomes (PRO) is defined as any report of the status of a patient’s health condition, or health behavior that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else. This definition reflects the key domains of

  • health-related quality of life (including functional status)
  • symptoms and symptom burden (e.g., pain, fatigue)
  • health behaviors (e.g., smoking, diet, exercise)

(Adapted from the Food and Drug Administration. (2009). Guidance for industry PRO measures: Use in medical product development to support labeling claims.

Patient-reported outcome measure (PROM)

Patient-reported outcome measures (PROMs) are an “instrument, scale, or single-item measure used to assess the patient-reported outcome (PRO) concept as perceived by the patient, obtained by directly asking the patient to self-report” (p. 27). National Quality Forum. (2013, January). PROs in performance measurement.

Patient-reported outcome-based performance measure

A patient-reported outcome-based performance measure (PRO-PM) is a performance measure that is based on patient-reported outcome measure (PROM) data aggregated for an accountable healthcare entity. The data are collected directly from the patient using the PROM tool, which can be an instrument, scale, or single-item measure.

Population, Intervention, Comparator group, Outcomes, Time interval, Settings (PICOTS)

Population, Intervention, Comparator group, Outcomes, Time interval, Settings (PICOTS) is a framework for delineating clinical questions that facilitates identification of relevant research evidence. For example, among adults with non-cancer related pain (Population) who receive an opioid pain medication (Intervention) versus a non-opioid pain medication or placebo (Comparator) what are the effects on ratings of pain, functioning, and quality of life (Outcome) with 3 months of treatment (Time) in outpatient settings of care (Setting)? McMaster University Health Sciences Library (n.d.). Resources for evidence-based practice: Forming questions. Retrieved September 21, 2021, from

Process measure

A process measure is a measure that focuses on steps that should be followed to provide good care. There should be a scientific basis for believing that the process, when executed well, will increase the probability of achieving a desired outcome.


A proportion is a score derived by dividing the number of cases that meet a criterion for quality (the numerator) by the number of eligible cases within a given time frame (the denominator) where the numerator cases are a subset of the denominator cases (for example, percentage of eligible women with a mammogram performed in the last year).

QDM attribute

A QDM attribute provides specific details about a QDM data element. QDM versions 5.5 and 5.6 include only datatype-specific attributes, i.e., metadata, or information about each QDM datatype that might be used in eCQM expressions to provide necessary details for calculation.

QDM category

A QDM category consists of a single clinical concept identified by a value set. A category is the highest level of definition for a QDM element. The QDM currently contains 22 categories. Some examples of categories are Medication, Procedure, Condition/Diagnosis/Problem, Communication, and Encounter.

QDM data element

A QDM data element encapsulates a certain category with an associated datatype. It is a discrete unit of information used in quality measurement to describe part of the clinical care process, including a clinical entity and its context of use. It can include criteria for any relevant metadata about a clinical or administrative concept relevant to quality measurement. A QDM data element provides an unambiguous definition and enables consistent capture and use of data for quality measurement. It may be defined for any given measure and reused when the same information is required for another measure. Reuse encourages standardization of quality measures and reduces the generation of additional software requirements for every new measure.

QDM datatype

A QDM datatype is the context in which each category is used to describe a part of the clinical care process. Examples of QDM datatypes include "Medication, Active" and "Medication, Administered" as applied to the QDM Medication category.

QDM entities

QDM entities represent concepts that can be used to specify details about the actor (or performer) of any QDM datatype. An eCQM can use the entities to provide further information required for an individual or organization actor to meet the measure's criteria.

Qualitative assessment

Qualitative assessments use approaches such as focus groups, in-depth interviews, detailed behavioral observations, or analysis of written or spoken text, typically to gain an understanding of motivations or opinions.

Quality measure

A quality measure is a standard for measuring the performance and improvement of population health or of health plans, providers of services, and other clinicians in the delivery of health care services (§931). Patient Protection and Affordable Care Act of 2010, Pub. L. No. 111-148, 124 Stat. 119 (2010).

Quantitative assessment

Quantitative assessments use numerically based methods, typically supported by statistical analyses, to show relationships between independent and dependent variables to facilitate description, prediction, and establishing of causal relationships.


A ratio is a score that is derived by dividing a count of one type of data by a count of another type of data. For example, the number of patients with central lines who develop infection divided by the number of central line days. The key to the definition of a ratio is that the numerator is not in the denominator.

Reliability subcriterion (Scientific Acceptability of the measures properties subcriterion)

The Reliability subcriterion tests whether the measure is well defined and precisely specified so that it can be implemented consistently within and across organizations and allow for comparability. National Quality Forum. (2019). Measure Evaluation Criteria and Guidance for Evaluating Measures for Endorsement. NQF: Measure Evaluation Criteria (

Resource use measure

A resource use measure, also called a cost and resource use measure, is a measure of health services counts (in terms of units or dollars) applied to a population or event (including diagnoses, procedures, or encounters). A resource use measure counts the frequency of use of defined health system resources. Some may further apply a dollar amount (e.g., allowable charges, paid amounts, or standardized prices) to each unit of resource use.

Respecified measure

A respecified measure is an existing measure that is changed to fit the current purpose or use. This may mean changing a measure to meet the needs of a different care setting, data source, or population. Or, it may mean changes to the numerator, denominator, or adding specifications to fit the current use.


A sandbox allows development and testing of a software application in an isolated and controlled environment. For testing of applications such as clinical decision support, a sandbox will need to contain a sufficient amount of realistic data to mimic application functioning in the clinical system. 


The scope delineates what is included and excluded in a project. It may define specific products (also known as deliverables) or specific processes that will occur as part of a project. Scope definitions can also include descriptions of assumptions, constraints on the project, and acceptance criteria. 


Scoring is the method(s) applied to data to generate results/score. Most quality measures produce rates. However, other scoring methods include categorical value, count, continuous variable, frequency distribution, non-weighted score/composite/scale, ratio, and weighted score/composite/scales.

Semantic validation

Semantic validation is a method of testing the validity of an eCQM whereby the formal criteria in an eCQM are compared to a manual computation of the measure from the same test database.


Sensitivity, as a statistical term, refers to the proportion of correctly identified actual positives. For example, the percentage of people with diabetes correctly identified as having diabetes. See Specificity.

SMART objectives

SMART objectives are a structured approach to achieving project goals by focusing on objectives that are Specific, Measurable, Achievable, Realistic, and Time-bound (SMART).

SMART-on-FHIR apps

SMART-on-FHIR apps are APIs that use the Substitutable Medical Applications and Reusable Technologies platform in concert with FHIR to provide a standards- based method for authentication, authorization, and retrieval of clinical data as well as interoperable data exchange with EHRs.


A specification is the measure instructions that address: data elements, data sources, point of data collection, timing and frequency of data collection and reporting, specific instruments to be used (if appropriate), and implementation strategies.


Specificity, as a statistical term, refers to the proportion of correctly identified negatives (for example, the percentage of healthy people who are correctly identified as not having the condition). Perfect specificity would mean that the measure recognizes all actual negatives. For example, all healthy people recognized as healthy. See Sensitivity.


A stakeholder is an individual, group or organization that is affected by the outcome of a project and, thus, has an interest in the project's success.

Standard for Trial Use

A Standard for Trial Use (STU) is used to provide timely compliance with regulatory or other governmental mandate and/or timely response to industry or market demand. An STU, following a suitable period for evaluation and comment, is incorporated into fully balloted and accredited version of the standard. Formerly called Draft Standard for Trial Use (DSTU).

Standard operating procedure

A standard operating procedure is a set of fixed step-by-step instructions or steps that are applicable to routine operations or situations with the intent to improve efficiency, uniformity, and quality of operations.


Stratification divides a population or resource services into distinct, independent groups of similar data, enabling analysis of the specific subgroups. This type of adjustment can show where disparities exist or where there is a need to expose differences in results.

Structure measure

A structure measure, also known as a structural measure, is a measure that assesses features of a healthcare organization or clinician relevant to its capacity to provide healthcare.

Subject Matter Expert

A subject matter expert (SME) is an individual with specialized expertise in a specific area or field.

Systematic review

A systematic review is a scientific investigation that focuses on a specific question and uses explicit, pre-specified scientific methods to identify, select, assess, and summarize the evidence from similar but separate studies. It may include a quantitative synthesis (meta-analysis), depending on the available data.


A tag is a keyword or term assigned to a piece of electronic information that serves as metadata and facilitates search processes.

Target/Initial Population

The target/initial population refers to all events for evaluation by a specific performance measure involving patients who share a common set of specified characteristics within a specific measurement set to which a given measure belongs. Draw all patients counted (e.g., as numerator, as denominator) from the target/initial population.

Technical Expert Panel

A Technical Expert Panel (TEP) is a group of experts and other stakeholders who contribute guidance and thoughtful input to the measure developer or other group seeking advice and expert information from representatives from multiple stakeholder groups for the purpose of obtaining balanced input that represents varied perspectives. Measure developers involve TEPs in every stage of the measure development process, from conceptualization through maintenance.

Terminology system

A terminology system is a set of terms representing the system of concepts of a particular field.

Text mining

Text mining is the process of using artificial intelligence technologies to transform unstructured or semi-structured textual data into meaningful patterns and actionable information.

Unified Modeling Language

Unified Modeling Language (UML) is a standardized methodology-independent approach to specifying, visualizing, modeling, and documenting the structure and design requirements of business processes, such as software development.

Usability and Use criterion

The Usability and Use criterion examines the extent to which potential audiences (e.g., consumers, purchasers, providers, policymakers) are using or could use performance results for both accountability and performance improvement to achieve the goal of high-quality, efficient healthcare for individuals or populations. National Quality Forum. (2019). Measure Evaluation Criteria and Guidance for Evaluating Measures for Endorsement. NQF: Measure Evaluation Criteria (

Use case

A use case is a unique instance of sharing a specific type of information regarding patients and their health. Each use case has a specific purpose, type of data exchanged, and rules for interactions between people and systems. Examples of use cases include immunization records sent to the state government for public health reporting, and admission notifications sent to doctors and other members of a care team when one of their patients is admitted to a hospital.

User experience

User experience is a person's perceptions and responses resulting from the use and/or anticipated use of a product, system, or service.

User story

A user story is a succinct, plain-language description of a desired software feature that is written from the viewpoint of the end-user.

Validity subcriterion (Scientific Acceptability of measure properties subcriterion)

The validity subcriterion tests whether measure specifications are consistent with the measure intent and captures the most inclusive target population. There are two main types of validity, measure validity and data element validity. Measure validity is when the measure accurately represents the concept being evaluated and achieves the purpose for which it is intended (to measure quality). Data element validity is the extent to which the information represented by the data element or code.
See measure validity and data element validity.

Value set

A value set is a list of specific values, terms, and their codes, used to describe clinical and administrative concepts in the quality measures. Value sets provide groupings of unique values along with a standard description or definition from one or more standard vocabularies used to describe the same clinical concept, e.g., diabetes, clinical visit, demographics, within quality measures. Examples of such standard vocabularies that are used to support effective, interoperable health information exchange include SNOMED CT, RxNORM, and Logical Observation Identifiers Names and Codes (LOINC).

Virtual Medical Record

The Virtual Medical Record (vMR) is a data model for representing the data that are analyzed and/or produced by clinical decision support (CDS) engines.


A workflow is a repeatable sequence of steps or tasks that are needed to complete a specified process. In a health care delivery setting, workflow refers to a directed series of physical and cognitive activities performed by care delivery team or equipment/computers to assess, change, or maintain the health of a patient as part of the delivery of clinical services. 

Workflow diagram

A workflow diagram is a visual representation of the sequence of steps or tasks that are needed to complete a specified process.


eXtensible Markup Language (XML) is a markup language that defines a set of rules for encoding documents in a format which is both human-readable and machine-readable.