CMS-10294 QECP MEasure Production QA Worksheet

OMB No. 0938-1144
Exp. 04/30/2015

MEASURE PRODUCTION QUALITY ASSURANCE (QA) WORKSHEET
The purpose of this information collection worksheet (starting on page 2 of this document) is to
provide the QECP team with a comprehensive and concise view of the QEs’ measurement
process from data integration to final reporting (Exhibit 1). The intent of the collected
information is to present and describe the QE’s error identification and correction approach in
its measure production process (i.e., QE’s quality assurance process). QEs must use this
worksheet to describe how their organization:
 integrates data into a repository to calculate reported performance measures;
 addresses data completeness, accuracy, and consistency; and
 identifies, corrects, and mitigates errors during the measure calculation and reporting
process once Medicare data are integrated.
The information QEs provide in this worksheet address several (but not all) of the evidence
requirements for Element 6A. Refer to page 12 of the “Revisions to Phase 3 Assessments and
Evidence” document for a complete list of evidence requirements for Element 6A. QEs must
complete and upload this worksheet, along with relevant policy/procedural supporting
documents, to their secure QECP application portal under Element 6A in order to comply with
Phase 3 of the QECP Minimum Requirements Review.
Exhibit 1: Measure Production Quality Assurance
(Standard 6)

(Standard 2)
Medicare
FFS Data

Measure Verification Are measures calculated as
specified?

DATA
REPOSITORY

Other Data
Source 1

Other Data
Source 2

QECP Measure Production QA Worksheet

(Standard 7)

(Standards 4 & 5)
REPORTS
AGGREGATE
MEASURE
CALCULATIONS

Provider A
Provider B
Provider C
Provider D

1

OMB No. 0938-1144
Exp. 04/30/2015

Table 1: Department/Vendor Responsibilities
For each function listed in the table below, provide the name of the department or vendor responsible for completing each task; the
name, credentials, and title of the responsible personnel; and the application or software used to complete each task.
General Measure
Production Functions

Internal Department
Name or Vendor Name1

Internal Staff
(Name, Credentials, and
Title)

Contracted Staff
(Name, Credentials,
and Title)

Application/Software1

Data Integration

Medicare Data Integration
(if different than above)
Data Warehouse
Maintenance
Measure Repository
Maintenance
Measure Calculation

Setting Quality Assurance
(QA) Standards or Policies

1

If applicable, documentation and/or purchase orders of the software and/or systems vendor utilized in the applicant’s QA process must be uploaded as
evidence to Element 6A in the QE’s application portal.
QECP Measure Production QA Worksheet

2

OMB No. 0938-1144
Exp. 04/30/2015

Conducting QA Reviews

Reviewing QA Reports

Other
(Specify:_______________)
Information Request

Response

List measures calculated for the first time (list only
those that will include Medicare data).
List and briefly describe your organization’s QA policy {List the name of each QA policy document, and provide a brief
document(s).
description of its contents.}
[A complete QA policy may be uploaded as evidence for
Element 6A in the QE’s application portal]
What data circumstance (e.g. performance above or
below expected performance - 3rd standard deviation)
triggers additional review and scrutiny for accuracy by
your organization?
How does your organization determine compliance
with measure specifications?

QECP Measure Production QA Worksheet

3

OMB No. 0938-1144
Exp. 04/30/2015

Table 2: Data Files and Data Accuracy
Describe each source of data combined with Medicare data; add rows as necessary (i.e., two sources of enrollment data will require
two rows of “Enrollment or Membership”).

Data Sources

File Description
(e.g., Enrollment data
from Happy Health
Plan)

Size of File
(e.g., # of claims
lines or covered
lives)

Date Loaded
into Data
Repository

Date
Integrated
with Medicare
Data

Proportion of Data File
in Final QE Data File
(% of all claim lines or
covered lives in
integrated data file)

Enrollment or
Membership
Practitioner

Claim or Encounter

Behavioral Healthcare

Pharmacy

Public Registry
(e.g. immunizations)

QECP Measure Production QA Worksheet

4

OMB No. 0938-1144
Exp. 04/30/2015

Other
(Specify:______________)
Information Request

Response

Are standard code sets used (e.g. ICD-9, CPT)? If
not, how is consistency between data elements
ensured?
How is data loading and mapping assessed for
accuracy and completeness?
What methods are used to QA each data source?
Were any data excluded from measure reporting?
Why were data excluded (e.g., incomplete data
from a claims vendor)?
What percentage of members, practitioners, or
claims was affected by these exclusions?
How does your organization assess whether
individual elements are missing, incomplete or
invalid?

QECP Measure Production QA Worksheet

5

OMB No. 0938-1144
Exp. 04/30/2015

Table 3: QA Reports
Information Request
What types of QA reports are produced by
your organization?

Response
{List the file name of each QA sample report, and provide a brief description of its
contents.}

[Sample reports generated by the QA process
must be attached as evidence to Element 6A
in QE’s application portal]
Who receives the QA reports?
Are corrective actions assigned as part of the
QA process?
How does your organization correct errors in
the measurement and reporting processes?

{If response can be found in a QA policy document, list the file name, page number,
and relevant section of the document.}

[A complete QA policy may be uploaded as
evidence for Element 6A in the QE’s
application portal]
If an error is identified during the QA process,
how does your organization update the
reports to the providers and consumers?

{If response can be found in a QA policy document, list the file name, page number,
and relevant section of the document.}

[A complete QA policy may be uploaded as
evidence for Element 6A in the QE’s
application portal]

QECP Measure Production QA Worksheet

6