CMS-10305 Organizational Assessment Instrument

Medicare Part C and Part D Reporting Requirements
Data Validation Procedure Manual
Appendix E: Organizational Assessment Instrument

Prepared by:
Centers for Medicare & Medicaid Services
Center for Medicare
Medicare Drug Benefit and C & D Data
Group
Last Updated: March 2022

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of
information unless it displays a valid OMB control number. The valid OMB control number for this
information collection is 0938-1115 (Expires: 04/30/2023). The time required to complete this information
collection is estimated to average 30 hours per response, including the time to review instructions, search
existing data resources, gather the data needed, and complete and review the information collection. If you have
comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write
to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore,
Maryland 21244-1850.
i

Appendix E: Organizational Assessment Instrument

Table of Contents

1.
2.
2.1
2.2
3.
3.1
3.2
3.3
4.
4.1
4.2
4.3
5.
5.1
5.2
5.3
5.4
6.

OBJECTIVES ................................................................................................................................................................... 1
INSTRUCTIONS............................................................................................................................................................... 1
INSTRUCTIONS FOR DATA VALIDATION CONTRACTORS ....................................................................................... 1
INSTRUCTIONS FOR SPONSORING ORGANIZATIONS .............................................................................................. 1
GENERAL QUESTIONS .................................................................................................................................................. 2
ORGANIZATION INFORMATION .................................................................................................................................... 2
CONTACT INFORMATION .............................................................................................................................................. 3
PART C AND PART D REPORTING SECTIONS UNDERGOING VALIDATION ...........................................................4
UNDERLYING DATA SOURCES AND REPORTING PROCESSES .............................................................................. 4
UNDERLYING DATA SOURCES .................................................................................................................................... 4
PROGRAMMING AND SOFTWARE ............................................................................................................................... 6
SUPPLEMENTAL QUESTIONS REGARDING REPORTING PROCESSES ..................................................................7
DATA VALIDATION DOCUMENTATION REQUEST ...................................................................................................... 8
REQUEST FOR PROGRAMMING CODE AND EXAMPLE OUTPUT ............................................................................. 8
REQUEST FOR DATA DICTIONARY .............................................................................................................................. 8
REQUEST FOR ANALYSIS PLAN, REPORTING PROCESS FLOWS, AND DIAGRAMS ............................................9
REQUEST FOR STANDARD OPERATING DOCUMENTS: STANDARD OPERATING PROCEDURES (SOPS),
POLICIES AND PROCEDURES, OR OTHER WORK INSTRUCTIONS ........................................................................ 9
DATA VALIDATION DOCUMENT LOG ......................................................................................................................... 10

List of Tables

Table 1: Timeline of OAI Activities for Performing Data Validation............................................................................................... 2
Table 2: Organization Information ................................................................................................................................................ 3
Table 3: Part C Contact Information ............................................................................................................................................. 3
Table 4: Part D Contact Information ............................................................................................................................................. 3
Table 5: Reporting Sections Undergoing Validation ..................................................................................................................... 4
Table 6: Underlying Data Sources................................................................................................................................................ 5
Table 7: Programming Software Specifications............................................................................................................................ 6

ii

Appendix E: Organizational Assessment Instrument

1. OBJECTIVES
CMS is providing this Organizational Assessment Instrument (OAI) as a tool for data validation
(DV) contractors to understand Part C and Part D sponsoring organizations’ (SOs’) reporting
processes and to request documentation that will be evaluated during the review process. The
information collected in this OAI will help prepare DV contractors and will reduce resources
required for the site (on-site or virtual) visit portion of the review.

2. INSTRUCTIONS
2.1

INSTRUCTIONS FOR DATA VALIDATION CONTRACTORS

Review the OAI prior to the on-site or virtual visit for preliminary information about the SO’s
processes for collecting and reporting data. If necessary, obtain follow-up information during the
on-site or virtual visit. The OAI captures preliminary information about the SO’s processes for
collecting and reporting data per the CMS reporting requirements. The DV contractor must
analyze the OAI prior to the site visit (on-site or virtual) and follow-up on any incomplete or
ambiguous responses during the site visit portion of the review. The OAI must be electronically
distributed to the SO undergoing a review. Following the SO’s completion of this document, the
DV contractor must attach a completed copy of the OAI in a file of all data validation review
work papers that will be shared with the organization.

2.2

INSTRUCTIONS FOR SPONSORING ORGANIZATIONS

Sponsoring Organizations must complete each section of the OAI in advance of the DV review
period, or according to the set timeline of the reviewer. The SO should complete the OAI and
provide documentation to the DV contractor as early as possible at the start of the DV review
period so that data validation can begin on April 1. All documentation and responses to
questions should reflect the SO’s systems and processes that were in place during the reporting
period(s) undergoing the data validation review. SOs with multiple contracts should complete
only one OAI. If the information provided in the OAI varies by contract, the SO should specify
the differences within the OAI to allow the DV contractor to identify differences that may affect
reporting section calculations or reporting for a given contract.
The SO must submit the OAI, documentation, and any additional information to the reviewer
electronically. The organization is responsible for ensuring that it has established mutually
agreeable methods for sharing proprietary and/or secure (PHI/PII) information with the reviewer
and that the reviewer complies with all HIPAA privacy and security requirements.
The completed OAI and any additional information provided because of this request will be
assessed by the reviewer. If an SO has any questions while completing the OAI, it should contact
the DV contractor. Each stage of the data validation review should entail a collaborative effort
between the SO and DV contractor. An overview of the timeline related to OAI activities is
outlined in Table 1.

1

Appendix E: Organizational Assessment Instrument
Table 1: Timeline of OAI Activities for Performing Data Validation
Step

Responsible
Party

Data Validation Activities

Timeline

1

SO

Complete Organizational Assessment Instrument
(OAI) and provide appropriate documentation to
selected reviewer per the OAI’s documentation
request

No earlier than 30 days

2

DVC, SO

Analyze OAI

Allow two weeks after
receipt.

3

DVC, SO

Prepare for on-site or virtual visit (site visit agenda,
resource needs, and logistics)

Early April

4

DVC, SO

Conduct on-site or virtual review (convene entrance Early April (allow for up
conference, conduct interviews with SO staff,
to 1 week)
observe SO’s reporting processes, and obtain
census and/or sample files)

5

DVC

Request additional documents following on-site or
virtual visit (if applicable)

prior to the start of the
DV cycle on April 1.

Mid/Late April

3. GENERAL QUESTIONS
The information gathered below will provide a better understanding of the scope for the SO’s
data validation review, including which contract(s) will be reviewed and which Part C and/or
Part D reporting sections the SO is reporting for validation.

3.1

ORGANIZATION INFORMATION

Complete Table 2, indicating each Medicare contract that the SO held during the reporting
period(s) undergoing the data validation review. Also, indicate whether the contract includes
the Part C and/or Part D benefit and provide the number of plan benefit packages (PBP)
associated with each contract. Indicate if any of the PBPs associated with the contract are
Special Needs Plans or Employer/Union “800 Series” plans. The SO may add rows to this table
as necessary but should not manipulate the columns.
For the “Contract Type” field, select from the following list:
•
•
•
•
•
•
•

2

CCPP
FFS
MSA
1876 Cost
Employer/Union Direct Contract (800 Series)
PDP
Demo

Appendix E: Organizational Assessment Instrument
Table 2: Organization Information
Parent Organization Name:

No data

No data

No data

No data

No data

No data

CMS Contract Number

Contract
Type

Includes
Part C?
(Y/N)

Includes
Part D?
(Y/N)

No. of Plan
Benefit
Packages

Includes SNP
PBP(s)?
(Y/N)

Includes
Employer/Union
“800 Series”
PBP(s)?
(Y/N)

Example: Contract 123

PFFS

Y

Y

3

N

N

Example: Contract 123

CCP

Y

Y

1

Y

N

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

[add rows as required]

3.2

CONTACT INFORMATION

Complete Table 3 and Table 4 indicating the SO’s primary and secondary points of contact
responsible for the Part C and Part D reporting requirements data validation review for each
contract included in this OAI.
Table 3: Part C Contact Information
Primary Part C Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

Secondary Part C Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

Table 4: Part D Contact Information
Primary Part D Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

3

Secondary Part D Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

Appendix E: Organizational Assessment Instrument

3.3

PART C AND PART D REPORTING SECTIONS UNDERGOING
VALIDATION

Complete Table 5 for the contract(s) included in this OAI. Indicate which of the Part C
and/or Part D reporting sections the SO has submitted for data validation review, the
applicable contract numbers (Column B), and whether the SO is able to report on all
required data elements per the CMS Part C and Part D Reporting Requirements Technical
Specifications (Column C).
Table 5: Reporting Sections Undergoing Validation
A. Reporting Section

B. Contract
Number(s)

No data
Part C:
No data
Grievances
Organization Determinations/Reconsiderations No data
Special Needs Plans (SNPs) Care Management No data
No data
Part D:
No data
Medication Therapy Management Programs
No data
Grievances
Coverage Determinations and Redeterminations No data
No data
Improving Drug Utilization Review Controls

C. Are all required D.
data elements
captured by
your internal
data
system(s)?
(Yes/No)

If the answer
to Column C.
is no, please
indicate which
delegated
entities’ data
systems
contain the
data elements

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

4. UNDERLYING DATA SOURCES AND REPORTING
PROCESSES
The questions below address the underlying data sources and reporting processes
used to produce the Part C and Part D reporting sections.

4.1

UNDERLYING DATA SOURCES

Complete Table 6 for the contract(s) included in this OAI, indicating the name of
the data source(s) used to generate each Part C and Part D reporting section
(Column B). If additional rows are required to list the data sources for a given
reporting section, insert new rows into the table.
Please indicate all underlying data sources involved in the reporting process, beginning with the
originating data systems (e.g., claims adjudication system, enrollment system) and including all
other data sources used for data collection and storage, data processing, analysis, and reporting.

4

Appendix E: Organizational Assessment Instrument
Table 6: Underlying Data Sources
A. Reporting Section
Part C:
Example Part C Reporting
Section
No data
No data
No data
No data
Grievances

No data
No data
No data
Organization
Determinations /
Reconsiderations

No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data

Special Needs Plans
(SNPs) Care Management

No data
No data
No data
N No data o data
No data
No data
No data

5

B. Data Source Name
(e.g., Claims, Enrollment, Provider Information)
No data
Claims Adjudication System ABC
Enrollment System DEF
Reporting Data Warehouse GHI
Reporting Data Warehouse JKL
Bob’s Individual Desktop Database MNO
No data
No data
No data
No data
No data

No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data
No data

Appendix E: Organizational Assessment Instrument
A. Reporting Section
Part D:
Medication Therapy Management
Programs

No data
No data
No data

Grievances

No data
No data
No data

Coverage Determinations,
Redeterminations (including At–
Risk Redeterminations under a
Drug Management Program), and
Reopenings

No data
No data
No data
No data
No data
No data
No data
No data
No data
No data

No data
No data
No data

No data
No data
No data
No data

No data
No data
No data

No data
No data
No data

Improving Drug Utilization Review
Controls

4.2

B. Data Source Name
(e.g., Claims, Enrollment, Provider Information)

PROGRAMMING AND SOFTWARE

In Table 7, specify the programming languages and software used to generate the reporting
section data for reporting (e.g., MS Access, SAS, SQL, Crystal Reports, Cognos, SPSS) for
the contract(s) included in this OAI.
Table 7: Programming Software Specifications
A. Reporting Section
No data
Part C:
No data
Grievances
No data
Organization Determinations/Reconsiderations
No data
Special Needs Plans (SNPs) Care Management
No data
Part D:
No data
Medication Therapy Management Programs
No data
Grievances
Coverage Determinations, Redeterminations (including At– No data
Risk Redeterminations under a Drug Management
Program), and Reopenings
No data
Improving Drug Utilization Review Controls

6

B. Programming Code/Software

Appendix E: Organizational Assessment Instrument

4.3

SUPPLEMENTAL QUESTIONS REGARDING
REPORTING PROCESSES

The questions below address additional information required to review the processes used to
compile and report the Part C and Part D reporting sections.

7

4.3.1

How does your organization ensure it meets the reporting requirements deadline for
the contract(s) included in this OAI? Who is responsible for submitting the data into
the HPMS Plan Reporting Module (i.e., responsible department, delegated entity or
first tier/downstream contractor)?

4.3.2

What is your organization’s process for correcting or revising data results that
have been returned/rejected by CMS for the contract(s) included in this OAI? Who
is responsible (i.e., responsible department, delegated entity, or first
tier/downstream contractor)?

4.3.3

If your organization received an outlier/data integrity notification for any of the
reporting sections that are currently undergoing data validation review (as
identified in Table 5 for the contract(s) included in this OAI), were those
notifications used to accurately report the data into HPMS? For the contract(s)
included in this OAI, how does your organization track CMS- issued changes to the
Part C and/or Part D Reporting Requirements Technical Specifications? Who is
responsible (i.e., responsible department, delegated entity or first tier/downstream
contractor)? How are these changes incorporated into your organization’s data
collection and reporting systems?

4.3.4

Describe any process or quality improvement activities your organization has
implemented since the prior reporting year/period that may affect reporting section
results submitted to CMS (e.g., development of steering committees, identification
of inefficiencies) for the contract(s) included in this OAI.

Appendix E: Organizational Assessment Instrument

5. DATA VALIDATION DOCUMENTATION REQUEST
The purpose of the documentation request is to obtain documents that will assist the reviewer in
determining that data elements for each reporting section are accurately identified, calculated, and
documented. This request is applicable to all organizational processes used in creating the final
HPMS submission for the Part C and Part D reporting requirements.
The SO is responsible for ensuring that it has established mutually agreeable methods for sharing
proprietary and/or secure (PHI/PII) information with the reviewer and that the reviewer complies
with all HIPAA privacy and security requirements. Instructions for logging the information
provided by the SO are included in Section 6.

5.1

REQUEST FOR PROGRAMMING CODE AND EXAMPLE OUTPUT

For the contract(s) included in this OAI, SOs should provide programming code/source code and
example output for computer programs used to calculate the data elements collected for each of
the CMS reporting sections that are currently undergoing data validation review (as identified in
Table 5). Such code may include the following:
•
•
•
•

Programming language for extracting data from the source (including any
exclusion criteria)
Joins between multiple data sources (including validation checks)
Data preparation (such as cleansing and missing data)
Manipulation to produce the final reports

The following are examples of the types of documents and files required:
•

•

•

If using SAS, SPSS, or similar software, provide the programming code, the log file that
shows the results of the compiled programming code, and the list file that shows the
output (e.g., tables and listings) generated by the programming code.
If using MS Access, SQL Server, Oracle, or other database systems, provide the
code used to generate the database query, results of the compiled query, and the
output generated by the query (e.g., saved data queries).
If using MS Excel or other spreadsheet programs, provide the Visual Basic code that
produced the spreadsheets (if applicable), and the actual workbooks with all formulas
used to calculate the values contained in each spreadsheet.

Submitted programming code should be neatly structured and documented so that a third party
can easily read it and understand the programming logic. Best practice is to include comments
within the code; however, if not possible, provide documentation (e.g., work instructions) that
enables the reviewer to interpret the programming logic.

5.2

REQUEST FOR DATA DICTIONARY

Organizations should provide a data dictionary or any such documentation that provides file
layouts, field definitions, explanation of calculations, and other information about the underlying
data that are used in creating the data submission for the Part C and Part D reporting
requirements for the contract(s) included in this OAI. Appendix B of the OAI includes an
example data dictionary that at a minimum should include the field name, data type, field
description, and additional notes regarding the data field values.

8

Appendix E: Organizational Assessment Instrument

5.3

REQUEST FOR ANALYSIS PLAN, REPORTING PROCESS
FLOWS, AND DIAGRAMS

Organizations should provide a copy of their analysis plan, reporting process flows, diagrams,
and any other related documents. These documents should include a description or illustration of
the analysis requirements, analysis methods, and processes used for generating all reporting
section output reports for the Part C and Part D reporting requirements for the contract(s)
included in this OAI.

5.4

REQUEST FOR STANDARD OPERATING DOCUMENTS:
STANDARD OPERATING PROCEDURES (SOPS),
POLICIES AND PROCEDURES, OR OTHER WORK
INSTRUCTIONS

Organizations should provide a copy of the documentation that describes their data and reporting
systems and processes for the contract(s) included in this OAI. Documents of interest include:
•
•
•
•

9

Work instructions, policies and procedures for the compilation, administration,
and/or submission of the Part C and Part D reporting requirements
Information Systems SOPs (e.g., system maintenance, upgrade, validation procedures)
Data Processing SOPs (e.g., data collection and storage process and frequency)
Data Archive/Restoration SOPs (e.g., disaster recovery plans)

Appendix E: Organizational Assessment Instrument

6. DATA VALIDATION DOCUMENT LOG
The Data Validation Document Log is used as inventory for all documents and files provided by
the SO as per Section 5. Organizations should complete the Document Log (see Document Log
Template in Appendix A of the OAI) in order to facilitate review of documentation and files
associated with the different stages of the reporting process.
•

•
•

•

•

10

Reporting Section: Provide reporting section for document or file. For example, if
submitting programming code that generates the SNPs reporting section, then indicate
“SNPs” in this column. Otherwise, indicate “N/A” (note that IT system SOPs may be
N/A).
Document Name: Electronic file name of document.
Document Type: Type of document or file (e.g., work instruction, policy and procedure,
programming code, programming output/report, data dictionary/file layout, reporting
process diagram).
Reporting Stage: Stage in the reporting process to which the document applies. This usually
applies to programming code, data queries, and programming output and reports. Examples
of stages include, but are not limited to, data extract from adjudication system, data input
into internal database, output/report from internal database, data analysis to summarize data
for reporting, or final report for HPMS entry. Otherwise, indicate “N/A” (note that IT SOPs
may be N/A).
Document Description: Work instructions, policies, and procedures are usually selfexplanatory. However, for programming code, SOs should include a description of the
input data sources, the applicable stage in the reporting process, the intended output, and
name of the output file. For data dictionaries/file layouts, indicate the name of the
applicable database and source tables containing the data fields. For screen shots, process
flows, and diagrams provide the relevant description of the indicated charts, diagrams, and
process flows.

APPENDIX A: DOCUMENT LOG TEMPLATE
To the extent possible, list the documents in a logically ordered fashion so the reviewer can identify sets of documents relative to the reporting
process stage for each reporting section.
A. Reporting
Section

11

B. Document File Name

C. Document Type

D. Reporting Stage

E. Document Description

Example: Grievances

Grievances_SOP.doc

SOP

N/A

SOP documents the methods used to gather, analyze,
and report Grievances data according to CMS reporting
requirements

Example: Grievances

Grievances_Data_Load.SAS

SAS Program Code

Load grievance data into
Data Warehouse

SAS program extracts data from adjudication
system and loads into internal data warehouse

Example: Grievances

Grievances_Summary.SAS

SAS Program Code

Summarizes data for HPMS
reporting

SAS program cleans and summarizes data for entry
into HPMS

Example: Grievances

Grievances Summary Report.xls Example Report Output

No data

No data

No data

Summarized data report
example for HPMS entry
No data

Example of report output from the Grievances
Summary. SAS program for HPMS entry
No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

APPENDIX B: DATA DICTIONARY EXAMPLE
Please refer to the following as an example of the information required in a Data Dictionary.
A. Field Name

B. Type

C. Description

D. Additional Notes

Example: Redetermination ID

Long Integer

Unique ID for each case

No data

Example: Date Created

Date

Date the case was created

No data

Example: Redetermination Status

Integer

Status of the case

Example: Redetermination Outcome

Integer

Outcome of the case

1=Open; 2=Closed

No data

No data

No data

1=Overturned; 2=Withdrawn; 3=Dismissed;
4=Upheld
No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

No data

12