CMS-10394 QECP Measure Production QA Worksheet

Revised 8/16/17

MEASURE PRODUCTION QUALITY ASSURANCE (QA) WORKSHEET
The purpose of this information collection worksheet (starting on page 2 of this document) is to
provide the QECP team with a comprehensive and concise view of the QEs’ measurement process
from data integration to final reporting (Exhibit 1). The intent of the collected information is to
present and describe the QE’s error identification and correction approach in its measure
production process (i.e., QE’s quality assurance process). QEs must use this worksheet to describe
how their organization:




integrates data into a repository to calculate reported performance measures;
addresses data completeness, accuracy, and consistency; and
identifies, corrects, and mitigates errors during the measure calculation and reporting
process once QE Medicare data are integrated.

The information QEs provide in this worksheet address several (but not all) of the evidence
requirements for Element 6A. Refer to page 88 of the 2016 Operations Manual for a complete
list of evidence requirements for Element 6A. QEs must complete and upload this worksheet,
along with relevant policy/procedural supporting documents, to their secure QECP application
portal under Element 6A in order to comply with Phase 3 of the QECP Minimum Requirements
Review.

QECP Measure Production QA Worksheet

1

PRA Disclosure Statement: According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of
information unless it displays a valid OMB control number. The valid OMB control number for this information collection is CMS#10394 (09381144) Exp. XX/XX/XXXX. The time required to complete this information collection is estimated to average 500 hours per response, including
the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If
you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security
Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

Table 1: Department/Vendor Responsibilities
For each function listed in the table below, provide the name of the department or vendor responsible for completing each task; the
name, credentials, and title of the responsible personnel; and the application or software used to complete each task.

General Measure
Production Functions

Internal Department
Name or Vendor Name1

Internal Staff
(Name, Credentials, and
Title)

Contracted Staff
(Name,
Credentials, and
Title)

Application/Software1

Data Integration

QE Medicare Data
Integration
(if different than above)
Data Warehouse
Maintenance
Measure Repository
Maintenance
Measure Calculation

1

If applicable, documentation and/or purchase orders of the software and/or systems vendor utilized in the entity’s QA process must be uploaded as evidence
to Element 6A in the QE’s application portal.
QECP Measure Production QA Worksheet
CMS#10394 (0938-1144) Exp. XX/XX/XXXX

2

Setting Quality Assurance
(QA) Standards or Policies
Conducting QA Reviews

Reviewing QA Reports

Other
(Specify:_______________)

QECP Measure Production QA Worksheet
CMS#10394 (0938-1144) Exp. XX/XX/XXXX

3

Information Request

Response

List measures calculated for the first time (list only those
that will include QE Medicare data).
List and briefly describe your organization’s QA policy {List the name of each QA policy document, and provide a brief description
document(s).
of its contents.}
[A complete QA policy may be uploaded as evidence for
Element 6A in the QE’s application portal]
What data circumstance (e.g. performance above or
below expected performance - 3rd standard deviation)
triggers additional review and scrutiny for accuracy by
your organization?

How does your organization determine compliance with
measure specifications?

Table 2: Data Files and Data Accuracy
Describe each source of data combined with QE Medicare data; add rows as necessary (i.e., two sources of enrollment data will require
two rows of “Enrollment or Membership”).

QECP Measure Production QA Worksheet
CMS#10394 (0938-1144) Exp. XX/XX/XXXX

4

Data Sources

File Description
(e.g., enrollment data
from Happy Health
Plan)

Size of File
(e.g., # of claims
lines or covered
lives)

Date Loaded
into Data
Repository

Date
Integrated
with Medicare
Data

Proportion of Data File
in Final QE Data File
(% of all claim lines or
covered lives in
integrated data file)

Enrollment or
Membership
Practitioner

Claim or Encounter

Behavioral Healthcare

Pharmacy

Public Registry
(e.g. immunizations)
Other
(Specify:______________)

QECP Measure Production QA Worksheet
CMS#10394 (0938-1144) Exp. XX/XX/XXXX

5

Information Request

Response

Are standard code sets used (e.g. ICD-9, CPT)? If
not, how is consistency between data elements
ensured?
How is data loading and mapping assessed for
accuracy and completeness?
What methods are used to QA each data source?

Were any data excluded from measure reporting?
Why were data excluded (e.g., incomplete data
from a claims vendor)?
What percentage of members, practitioners, or
claims was affected by these exclusions?
How does your organization assess whether
individual elements are missing, incomplete or
invalid?

QECP Measure Production QA Worksheet
CMS#10394 (0938-1144) Exp. XX/XX/XXXX

6

Table 3: QA Reports
Information Request
What types of QA reports are produced by
your organization?

Response
{List the file name of each QA sample report, and provide a brief description of its
contents.}

[Sample reports generated by the QA
process must be attached as evidence to
Element 6A in QE’s application portal]
Who receives the QA reports?
Are corrective actions assigned as part of the
QA process?
How does your organization correct errors in
the measurement and reporting processes?

{If response can be found in a QA policy document, list the file name, page number,
and relevant section of the document.}

[A complete QA policy may be uploaded as
evidence for Element 6A in the QE’s
application portal]
If an error is identified during the QA
process, how does your organization update
the reports to the providers and consumers?

{If response can be found in a QA policy document, list the file name, page number,
and relevant section of the document.}

[A complete QA policy may be uploaded as
evidence for Element 6A in the QE’s
application portal]

QECP Measure Production QA Worksheet
CMS#10394 (0938-1144) Exp. XX/XX/XXXX

7