Form CMS-10305 Medicare Part C and Part D Measure Organizational Assess

APPENDIX 2: Medicare Part C and Part D Measure
Organizational Assessment Instrument

August 26, 2010

DRAFT

Prepared by:

Centers for Medicare & Medicaid Services

Center for Drug and Health Plan Choice

DRAFT – Meets Section 508 Compliance Standards

TABLE OF CONTENTS

1.0 OBJECTIVES ..................................................................................................................1

2.0 INSTRUCTIONS .............................................................................................................1

2.1
2.2

Instructions for Data Validation Contractor................................................................1

Instructions for Organizations ....................................................................................1

3.0 GENERAL QUESTIONS ................................................................................................2

3.1
3.2
3.3

Organization Information ...........................................................................................2

Contact Information ...................................................................................................2

Part C and Part D Measures Undergoing Validation ...................................................3

4.0 UNDERLYING DATA SOURCES AND REPORTING PROCESSES.........................4

4.1
4.2
4.3

Underlying Data Sources ...........................................................................................4

Programming and Software........................................................................................6

Supplemental Questions Regarding Reporting Processes ...........................................6

5.0 DATA VALIDATION DOCUMENTATION REQUEST ..............................................8

5.1
5.2
5.3
5.4

Request for Programming Code and Example Output ................................................8

Request for Data Dictionary .......................................................................................8

Request for Analysis Plan, Reporting Process Flows, and Diagrams ..........................9

Request for Standard Operating Documents: Standard Operating Procedures

(SOPs), Policies and Procedures, or Other Work Instructions .....................................9

6.0 DATA VALIDATION DOCUMENT LOG.....................................................................9

APPENDIX A: DOCUMENT LOG TEMPLATE .................................................................10

APPENDIX B: DATA DICTIONARY EXAMPLE...............................................................11

DRAFT – Meets Section 508 Compliance Standards

i

1.0

OBJECTIVES

CMS is providing this Organizational Assessment Instrument (OAI) as a tool for data validation
contractors (reviewers) to understand organizations’ reporting processes more completely and to
request documentation that will be evaluated during the review process. The information
collected in this OAI will help prepare reviewers and will reduce resources required for the on-site
portion of the review. While not mandatory, it is strongly recommended that organizations
complete the OAI to add efficiencies to the review process. If an organization does not elect to
complete the OAI, the reviewer will use the same tool to collect this information during the on-site
review, extending the length of the review.

2.0

INSTRUCTIONS

2.1 Instructions for Data Validation Contractor
In the early stage of the data validation review process, and prior to the on-site visit, the reviewer
should request that the organization begin completion of the OAI. It is recommended that each
section of the OAI be completed prior to the on-site visit; however, if preferred, the reviewer may
determine an alternative sequence (e.g., the reviewer may request that an organization complete
Sections 3.0 and 4.0 prior to the on-site, but complete Sections 5.0 and 6.0 following the on-site).
The OAI is designed to capture preliminary information about the organization’s processes for
collecting and reporting data per the CMS reporting requirements. It is assumed that the reviewer
will follow-up on any incomplete or ambiguous responses during the on-site portion of the review.
The OAI should be electronically distributed to the organization undergoing a review. Following
the organization’s completion of this document, the reviewer must attach a completed copy of the
OAI in a file of all data validation review work papers that will be shared with the organization.

2.2 Instructions for Organizations
Organizations electing to complete the OAI should complete each section in advance of the onsite review, or according to the set timeline of the reviewer. All documentation and responses to
questions should reflect the organization’s systems and processes that were in place during the
reporting period(s) undergoing the data validation review. For organizations with multiple
contracts, only one OAI should be completed. If the information provided in the OAI varies by
contract, the organization should specify the differences within the OAI to allow the reviewer to
more easily identify differences that may impact measure calculations or reporting for a given
contract.
All responses are to be submitted to the reviewer electronically. The organization is responsible
for ensuring that it has established mutually agreeable methods for sharing proprietary and/or
secure (PHI/PII) information with the reviewer and that the reviewer complies with all HIPAA
privacy and security requirements.
The completed OAI and any additional information provided as a result of this request will be
assessed by the reviewer. If your organization has any questions while completing the OAI,
contact the reviewer. Each stage of the data validation review should entail a collaborative effort
between the organization and reviewer.

DRAFT – Meets Section 508 Compliance Standards

1

3.0

GENERAL QUESTIONS

The information gathered below will provide a better understanding of the scope for the
organization’s data validation review, including which contract(s) will be reviewed and which Part
C and/or Part D measures the organization is reporting for validation.

3.1 Organization Information
Complete the following table, indicating each Medicare contract that your organization held during
the reporting period(s) undergoing the data validation review. For the “Contract Type” field, select
from the following list:
•
•
•
•

•
•
•

CCP
PFFS
MSA
Employer/Union Direct Contract

1876 Cost
Demo
PDP

Also indicate whether the contract includes the Part C and/or Part D benefit and provide the
number of plan benefit packages (PBP) associated with each contract. Finally, indicate if any of
the PBPs associated with the contract are Special Needs Plans or Employer/Union “800 Series”
plans. The organization may add rows to this table as necessary, but should not manipulate the
columns.
Table 1: Organizational Information
Parent Organization Name:

CMS Contract Number
Example: Contract 123
Example: Contract 123
[add rows as required]

Includes
Part D?
(Y/N)

No. of Plan
Benefit
Packages

Includes
SNP PBP(s)?
(Y/.N)

Includes
Employer/Union
“800 Series”
PBP(s)?
(Y/N)

Y

Y

3

N

N

Y

Y

1

Y

N

Contract
Type

Includes
Part C?
(Y/N)

PFFS
CCP

3.2 Contact Information
Complete the following tables, indicating your organization’s primary and secondary points of
contact responsible for the Part C and Part D reporting requirements data validation review for
each contract included in this OAI.
Table 2: Contact Information
Primary Part C Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

Secondary Part C Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

DRAFT – Meets Section 508 Compliance Standards

2

Primary Part D Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

Secondary Part D Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

3.3 Part C and Part D Measures Undergoing Validation
Complete the following table for the contract(s) included in this OAI, indicating which of the 2010
Part C and/or Part D measures your organization has submitted for data validation review, the
applicable contract numbers (Column B), and whether your organization is able to report on all
required data elements per the CMS Part C and Part D Reporting Requirements Technical
Specifications (Column C).
Table 3: Measures Undergoing Validation

A. Measure

B. CMS Contract
Number(s)

C. Are all required data
elements captured by
your data system(s)?
(Yes/No)

Part C1:
Benefit Utilization
Procedure Frequency
Serious Reportable Adverse Events (SRAEs)
Provider Network Adequacy
Grievances
Organization Determinations/Reconsiderations
Employer Group Plan Sponsors
Plan Oversight of Agents
Special Needs Plans (SNPs) Care Management
Part D2:
Retail, Home Infusion, and Long-Term Care Pharmacy Access
Medication Therapy Management Programs
Grievances
Coverage Determinations and Exceptions
Appeals
Long-Term Care (LTC) Utilization
Employer/Union-Sponsored Group Health Plan Sponsors
Plan Oversight of Agents
1 The following Part C measures are required for CMS reporting but are not included in the data validation review: PFFS Plan

Enrollment Verification Calls and PFFS Provider Payment Dispute Resolution Process.

2The following Part D measures are required for CMS reporting but are not included in the data validation review: Enrollment,

Access to Extended Day Supplies at Retail Pharmacies, Prompt Payment, Pharmacy Support of Electronic Prescribing, Pharmacy &

Therapeutics (P&T) Committees/Provision of Part D Functions, Pharmaceutical Manufacturer Rebates, Discounts, and Other Price

Concessions, Licensure and Solvency, and Fraud, Waste, and Abuse Compliance Programs.

DRAFT – Meets Section 508 Compliance Standards

3

4.0

UNDERLYING DATA SOURCES AND REPORTING
PROCESSES

The questions below address the underlying data sources and reporting processes used to
produce the Part C and Part D measures.

4.1 Underlying Data Sources
Complete the following table for the contract(s) included in this OAI, indicating the name of the
data source(s) used to generate each Part C and Part D measure (Column B). If additional rows
are required to list the data sources for a given measure, insert new rows into the table.
Please indicate all underlying data sources involved in the reporting process, beginning with the
originating data systems (e.g., claims adjudication system, enrollment system) and including all
other data sources used for data collection and storage, data processing, analysis, and reporting.
Table 4: Underlying Data Sources
A. Measure

B. Data Source Name
(e.g., Claims, Enrollment, Provider Information)

Part C:
Example Part C Measure

Claims Adjudication System ABC
Enrollment System DEF
Reporting Data Warehouse GHI
Reporting Data Warehouse JKL
Bob’s Individual Desktop Database MNO

Benefit Utilization

Procedure Frequency

Serious Reportable Adverse
Events (SRAEs)

Provider Network Adequacy

Grievances

Organization Determinations /
Reconsiderations

DRAFT – Meets Section 508 Compliance Standards

4

A. Measure

B. Data Source Name
(e.g., Claims, Enrollment, Provider Information)

Employer Group Plan
Sponsors

Plan Oversight of Agents

Special Needs Plans (SNPs)
Care Management

Part D:
Retail, Home Infusion, and
Long-Term Care Pharmacy
Access

Medication Therapy
Management Programs

Grievances

Coverage Determinations and
Exceptions

Appeals

Long-Term Care (LTC)
Utilization

Employer/Union-Sponsored

Group Health Plan Sponsors

DRAFT – Meets Section 508 Compliance Standards

5

B. Data Source Name
(e.g., Claims, Enrollment, Provider Information)

A. Measure
Plan Oversight of Agents

4.2 Programming and Software
Specify the programming languages and software used to generate the measure data for
reporting (e.g., MS Access, SAS, SQL, Crystal Reports, Cognos) for the contract(s) included in
this OAI.
Table 5: Programming Software Specifications
A. Measure
Part C:
Benefit Utilization
Procedure Frequency
Serious Reportable Adverse Events (SRAEs)
Provider Network Adequacy
Grievances
Organization Determinations/Reconsiderations
Employer Group Plan Sponsors
Plan Oversight of Agents
Special Needs Plans (SNPs) Care Management
Part D:
Retail, Home Infusion, and Long-Term Care Pharmacy Access
Medication Therapy Management Programs
Grievances
Coverage Determinations and Exceptions
Appeals
Long-Term Care (LTC) Utilization
Employer/Union-Sponsored Group Health Plan Sponsors
Plan Oversight of Agents

B. Programming Code/Software

4.3 Supplemental Questions Regarding Reporting Processes
The questions below address additional information required to review the processes used to
compile and report the Part C and Part D measures.
4.3.1

How does your organization ensure it meets the reporting requirements deadline for the
contract(s) included in this OAI? Who is responsible for submitting the data into HPMS
(i.e., responsible department, delegated entity or first tier/downstream contractor)?

4.3.2

What is your organization’s process for correcting or revising data results that have been
returned/rejected by CMS for the contract(s) included in this OAI? Who is responsible
(i.e., responsible department, delegated entity, or first tier/downstream contractor)?

4.3.3

Did your organization receive any outlier notifications from CMS in the previous reporting
year for any of the measures that are currently undergoing data validation review (as
DRAFT – Meets Section 508 Compliance Standards

6

identified in Section 3.3) for the contract(s) included in this OAI? If so, please describe
any such notices received for the measure and any corrective actions taken to address
the issue.

4.3.4

For the contract(s) included in this OAI, how does your organization track CMS-issued
changes to the Part C and/or Part D Reporting Requirements Technical Specifications?
Who is responsible (i.e., responsible department, delegated entity or first tier/downstream
contractor)? How are these changes incorporated into your organization’s data collection
and reporting systems?

4.3.5

Describe any process or quality improvement activities your organization has
implemented since the prior reporting year/period that may affect measure results
submitted to CMS (e.g., development of steering committees, identification of
inefficiencies) for the contract(s) included in this OAI.

DRAFT – Meets Section 508 Compliance Standards

7

5.0

DATA VALIDATION DOCUMENTATION REQUEST

The purpose of the documentation request is to obtain documents that will assist the reviewer in
determining that data elements for each measure are accurately identified, calculated, and
documented. This request is applicable to all organizational processes used in creating the final
HPMS submission for the Part C and Part D reporting requirements.
The organization is responsible for ensuring that it has established mutually agreeable methods
for sharing proprietary and/or secure (PHI/PII) information with the reviewer and that the reviewer
complies with all HIPAA privacy and security requirements. Instructions for logging the
information provided by the organization are included in Section 6.0.
Please reference the Reporting Requirements Technical Specifications for the data elements that
will require supporting documentation.

5.1 Request for Programming Code and Example Output
For the contract(s) included in this OAI, organizations should provide programming code/source
code and example output for computer programs used to calculate the data elements collected
for each of the CMS data measures that are currently undergoing data validation review (as
identified in Section 3.3). Such code may include the following:
•
•
•
•

Programming language for extracting data from the source (including any exclusion
criteria)
Joins between multiple data sources (including validation checks)
Data preparation (such as cleansing and missing data)
Manipulation to produce the final reports

The following are examples of the types of documents and files required:
•
•
•

If using SAS, SPSS, or similar software, provide the programming code, the log file that
shows the results of the compiled programming code, and the list file that shows the
output (e.g., tables and listings) generated by the programming code.
If using MS Access, SQL Server, Oracle, or other database systems, provide the code
used to generate the database query, results of the compiled query, and the output
generated by the query (e.g., saved data queries).
If using MS Excel or other spreadsheet programs, provide the Visual Basic code that
produced the spreadsheets (if applicable), and the actual workbooks with all formulas
used to calculate the values contained in each spreadsheet.

Submitted programming code should ideally be neatly structured and documented so that a third
party can easily read it and understand the programming logic. Best practice is to include
comments within the code; however, if not possible, provide documentation (e.g., work
instructions) that enables the reviewer to interpret the programming logic.

5.2 Request for Data Dictionary
Organizations should provide a data dictionary or any such documentation that provides file
layouts, field definitions, explanation of calculations, and other information about the underlying
data that is used in creating the data submission for the Part C and Part D reporting requirements
for the contract(s) included in this OAI. Appendix B includes an example data dictionary which
should at a minimum include the field name, data type, field description, and additional notes
regarding the data field values.

DRAFT – Meets Section 508 Compliance Standards

8

5.3 Request for Analysis Plan, Reporting Process Flows, and Diagrams
Organizations should provide a copy of their analysis plan, reporting process flows, diagrams, and
any other related documents. These documents should include a description or illustration of the
analysis requirements, analysis methods, and processes used for generating all measure-specific
output reports for the Part C and Part D reporting requirements for the contract(s) included in this
OAI.

5.4

Request for Standard Operating Documents: Standard Operating
Procedures (SOPs), Policies and Procedures, or Other Work Instructions

Organizations should provide a copy of the documentation that describes their data and reporting
systems and processes for the contract(s) included in this OAI. Documents of interest include:





6.0

Work instructions, policies and procedures for the compilation, administration, and/or
submission of the Part C and Part D reporting requirements
Information Systems SOPs (e.g., system maintenance, upgrade, validation procedures)
Data Processing SOPs (e.g., data collection and storage process and frequency)
Data Archive/Restoration SOPS (e.g., disaster recovery plans)

DATA VALIDATION DOCUMENT LOG

The Data Validation Document Log is intended to be used as inventory for all documents and files
provided by the organization as per Section 5.0. Organizations should complete the Document
Log (see Document Log Template in Appendix A) in order to facilitate review of documentation
and files associated with the different stages of the reporting process.
Measure: Measure for which document or file has been provided. For example, if submitting
programming code that generates the Procedure Frequency measure, then indicate “Procedure
Frequency” in this column. Otherwise indicate “N/A” (note that IT system SOPs may be N/A).






Document Name: Electronic file name of document.
Document Type: Type of document or file (e.g., work instruction, policy and procedure,
programming code, programming output/report, data dictionary/file layout, reporting process
diagram).
Reporting Stage: Stage in the reporting process to which the document applies. This
usually applies to programming code, data queries, and programming output and reports.
Examples of stages include, but are not limited to: data extract from adjudication system,
data input into internal database, output/report from internal database, data analysis to
summarize data for reporting, or final report for HPMS entry. Otherwise indicate “N/A” (note
that IT SOPs may be N/A).
Document Description: Description of the document. Work instructions, policies, and
procedures are usually self-explanatory. However, for programming code, organizations
should include a description of the input data sources, the applicable stage in the reporting
process, the intended output, and name of the output file. For data dictionaries/file layouts,
indicate the name of the applicable database and source tables containing the data fields.
For screen shots, process flows, and diagrams provide the relevant description of the
indicated charts, diagrams, and process flows.

DRAFT – Meets Section 508 Compliance Standards

9

APPENDIX A: DOCUMENT LOG TEMPLATE
To the extent possible, list the documents in a logically-ordered fashion so the reviewer can identify sets of documents relative to the reporting
process stage for each measure.
Table 6: Document Log
A. Measure

B. Document File Name

C. Document Type

D. Reporting Stage

Example: SRAEs

SRAE_SOP.doc

SOP

N/A

Example: SRAEs

SRAE_Data_Load.SAS

SAS Program Code

Example: SRAEs

SRAE_Summary.SAS

SAS Program Code

Example: SRAEs

SRAE Summary Report.xls

Example Report Output

Load SRAE data into Data
Warehouse
Summarizes data for HPMS
reporting
Summarized data report
example for HPMS entry

E. Document Description
SOP documents the methods used to gather, analyze, and
report SRAE data according to CMS reporting requirements
SAS program extracts data from adjudication system and
loads into internal data warehouse
SAS program cleans and summarizes data for entry into
HPMS
Example of report output from the SRAE_Summary.SAS
program for HPMS entry

DRAFT – Meets Section 508 Compliance Standards

10

APPENDIX B: DATA DICTIONARY EXAMPLE
Please refer to the following as an example of the information required in a Data Dictionary.
Table 7: Data Dictionary Example
A. Field Name

B. Type

C. Description

D. Additional Notes

Example: AppealID

Long Integer

Unique ID for each case

Example: DateCreated

Date

Date the case was created

Example: AppealStatus

Integer

Status of the case

1=Open; 2=Closed

Example: AppealOutcome

Integer

Outcome of the case

1=Overturned; 2=Withdrawn; 3=Dismissed;
4=Upheld

DRAFT – Meets Section 508 Compliance Standards

11