CMS-10305 Organizational Assessment Instrument

Appendix E: Organizational Assessment Instrument

Medicare Part C and Part D Reporting Requirements Data
Validation Procedure Manual

Appendix E: Organizational Assessment Instrument

Prepared by:
Centers for Medicare & Medicaid Services
Center for Medicare
Medicare Drug Benefit and C&D Data Group

Effective Date: 2024 Data Validation

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it
displays a valid OMB control number. The valid OMB control number for this information collection is 0938-1115 (Expires:
12/31/2025). The time required to complete this information collection is estimated to average 30 hours per response, including the
time to review instructions, search existing data resources, and gather the data needed, and complete and review the information
collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please
write to: CMS, 7500 Security Boulevard, Attn: PRA Reports clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

1

Appendix E: Organizational Assessment Instrument

Table of Contents
1.

Objectives.............................................................................................................................................................. 3

2.

Instructions ............................................................................................................................................................ 3

2.1.

Instructions for DVCs ...................................................................................................................................... 3

2.2.

Instructions for SOs ........................................................................................................................................ 3

TABLE 1: TIMELINE OF OAI ACTIVITIES FOR PERFORMING DATA VALIDATION ............................................................... 4
3.

General Questions ............................................................................................................................................... 4

3.1.

Organization information ................................................................................................................................ 4

TABLE 2: ORGANIZATION INFORMATION ....................................................................................................................... 5
3.2.

Contact information ......................................................................................................................................... 5

TABLE 3: PART C CONTACT INFORMATION ................................................................................................................... 5
TABLE 4: PART D CONTACT INFORMATION................................................................................................................... 5
3.3.

Part C and Part D Reporting Sections Undergoing DV ................................................................................ 6

TABLE 5: REPORTING SECTIONS UNDERGOING VALIDATION .......................................................................................... 6
4.

Underlying Data Sources and Reporting Processes ......................................................................................... 6

4.1.

Underlying Data Sources ................................................................................................................................ 6

TABLE 6: UNDERLYING DATA SOURCES ....................................................................................................................... 7
4.2.

Programming and Software ............................................................................................................................ 8

TABLE 7: PROGRAMMING SOFTWARE SPECIFICATIONS ................................................................................................ 8
4.3.
5.

Supplemental Questions Regarding Reporting Processes .......................................................................... 9
DV Document Request ........................................................................................................................................ 9

5.1.

Request for Programming Code and Example Output ................................................................................. 9

5.2.

Request for Data Dictionary ......................................................................................................................... 10

5.3.

Request for Analysis Plan, Reporting Process Flow, and Diagrams ......................................................... 10

5.4.
Request for Standard Operating Documents: Standard Operating Procedures (SOPs), Policies and
procedures, or Other Work Instructions.................................................................................................................... 10
6.

DV Document Log.............................................................................................................................................. 11

2

Appendix E: Organizational Assessment Instrument

1. OBJECTIVES
CMS is providing this Organizational Assessment Instrument (OAI) as a tool for data validation (DV) contractors
to understand Part C and Part D sponsoring organizations’ (SOs’) reporting processes and to request
documentation that will be evaluated during the review process. The information collected in this OAI will help
prepare DV contractors (DVCs) and will reduce resources required for the site (on-site or virtual) visit portion of
the review.

2. INSTRUCTIONS
2.1.

INSTRUCTIONS FOR DVCS

Review the OAI prior to the on-site or virtual visit for preliminary information about the SO’s processes
for collecting and reporting data. If necessary, obtain follow-up information during the on-site or virtual
visit. The OAI captures preliminary information about the SO’s processes for collecting and reporting
data per the Medicare Part C and Part D Reporting Requirements. The DVC must analyze the OAI prior
to the site visit (on-site or virtual) and follow-up on any incomplete or ambiguous responses during the
site visit portion of the review. The OAI must be electronically distributed to the SO undergoing a review.
Following the SO’s completion of this document, the DVC must attach a completed copy of the OAI in a
file of all DV review work papers that will be shared with the organization.

2.2.

INSTRUCTIONS FOR SOS

SOs must complete each section of the OAI in advance of the DV review period, or according to the set
timeline of the DVC. The SO should complete the OAI and provide documentation to the DVC as early
as possible at the start of the DV review period so that data validation can begin on April 1. All
documentation and responses to questions should reflect the SO’s systems and processes that were in
place during the reporting period(s) undergoing the data validation review. SOs with multiple contracts
should complete only one OAI. If the information provided in the OAI varies by contract, the SO should
specify the differences within the OAI to allow the DVC to identify differences that may affect reporting
section calculations or reporting for a given contract.
The SO must submit the OAI, documentation, and any additional information to the DVC electronically.
The organization is responsible for ensuring that it has established mutually agreeable methods for
sharing proprietary and/or secure (PHI/PII) information with the DVC and that the DVC complies with all
HIPAA privacy and security requirements.
The completed OAI and any additional information provided because of this request will be assessed by
the DVC. If an SO has any questions while completing the OAI, it should contact the DVC. Each stage of
the data validation review should entail a collaborative effort between the SO and DVC. An overview of
the timeline related to OAI activities is outlined in Table 1.

3

Appendix E: Organizational Assessment Instrument
TABLE 1: TIMELINE OF OAI ACTIVITIES FOR PERFORMING DATA VALIDATION
Responsible
Party

Step

Data Validation Activities
Complete Organizational Assessment Instrument
(OAI) and provide appropriate documentation to
the selected DVC per the OAI’s documentation
request
Analyze OAI

Timeline

1

SO

No earlier than 30
days prior to the start
of the DV cycle on
April 1.
Allow two weeks after
receipt.

2

DVC, SO

3

DVC, SO

Prepare for on-site or virtual visit (site visit agenda, Early April
resource needs, and logistics)

4

DVC, SO

5

DVC

Conduct on-site or virtual review (convene
Early April
entrance conference, conduct interviews with SO
staff, observe SO’s reporting processes, and obtain
census and/or sample files)
Request additional documents following on-site or Mid/Late April
virtual visit (if applicable)

3. GENERAL QUESTIONS
The information gathered below will provide a better understanding of the scope for the SO’s data validation
review, including which contract(s) will be reviewed and which Part C and/or Part D reporting sections the SO is
reporting for validation.

3.1.

ORGANIZATION INFORMATION

Complete Table 2 indicating each Medicare contract that the SO held during the reporting period(s)
undergoing the data validation review. Also, indicate whether the contract includes the Part C and/or
Part D benefit and provide the number of plan benefit packages (PBP) associated with each contract.
Indicate if any of the PBPs associated with the contract are Special Needs Plans or Employer/Union
“800 Series” plans. The SO may add rows to this table as necessary but should not manipulate the
columns.
•

For the “Contract Type” field, select from the following list:

•

CCPP

•

FFS

•

MSA

•

1876 Cost

•

Employer/Union Direct Contract (800 Series)

•

PDP

•

Demo

4

Appendix E: Organizational Assessment Instrument
TABLE 2: ORGANIZATION INFORMATION
Parent Organization Name:
Contract
Type

Includes
Part C?
(Y/N)

Includes
Part D?
(Y/N)

No. of
Plan
Benefit
Packages

Includes
SNP
PBP(s)?
(Y/N)

Includes
Employer/Union
“800 Series”
PBP(s)?
(Y/N)

Example: Contract 123

PFFS

Y

Y

3

N

N

Example: Contract 123

CCP

Y

Y

1

Y

N

CMS Contract Number

[add rows as required]

3.2.

CONTACT INFORMATION

Complete Table 3 and Table 4 indicating the SO’s primary and secondary points of contact responsible
for the Part C and Part D Reporting Requirements data validation review for each contract included in
this OAI.
TABLE 3: PART C CONTACT INFORMATION
Primary Part C Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

Secondary Part C Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

TABLE 4: PART D CONTACT INFORMATION
Primary Part D Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

Secondary Part D Point of Contact
Name:
Title:
Company:
Address:
City, State, Zip:
Telephone:
Fax:
Email:

5

Appendix E: Organizational Assessment Instrument

3.3.
PART C AND PART D REPORTING SECTIONS
UNDERGOING DV
Complete Table 5 for the contract(s) included in this OAI. Indicate which of the Part C and/or Part D
reporting sections the SO has submitted for data validation review, the applicable contract numbers
(Column B), and whether the SO is able to report on all required data elements per the CMS Part C and
Part D Reporting Requirements and Technical Specifications (Column C).
TABLE 5: REPORTING SECTIONS UNDERGOING VALIDATION
A. Reporting Section

B. Contract
Number(s)

C. Are all
required data
elements
captured by your
internal data
system(s)?
(Yes/No)

D. If the answer to Column C. is no,
please indicate which delegated
entities’ data systems contain the
data elements

Part C:
Grievances
Organization
Determinations/Reconsiderations
Special Needs Plans (SNPs)
Care Management
Part D:
Medication Therapy Management
Programs
Grievances
Coverage Determinations,
Redeterminations (including AtRisk Redeterminations Under a
Drug Management Program), and
Reopenings
Improving Drug Utilization Review
Controls

4. UNDERLYING DATA SOURCES AND REPORTING PROCESSES
The questions below address the underlying data sources and reporting processes used to produce the Part C
and Part D reporting sections.

4.1.

UNDERLYING DATA SOURCES

Complete Table 6 for the contract(s) included in this OAI, indicating the name of the data source(s) used
to generate each Part C and Part D reporting section (Column B). If additional rows are required to list
the data sources for a given reporting section, insert new rows into the table.
Please indicate all underlying data sources involved in the reporting process, beginning with the
originating data systems (e.g., claims adjudication system, enrollment system) and including all other
data sources used for data collection and storage, data processing, analysis, and reporting.
6

Appendix E: Organizational Assessment Instrument
TABLE 6: UNDERLYING DATA SOURCES
A. Reporting
Section
Part C:
Example Part C
Reporting Section

B. Data Source Name
(e.g., Claims, Enrollment, Provider Information)
Claims Adjudication System ABC
Enrollment System DEF
Reporting Data Warehouse GHI
Reporting Data Warehouse JKL
Bob’s Individual Desktop Database MNO

Grievances

Organization
Determinations/
Reconsiderations

Special Needs Plans
(SNPs) Care
Management

7

Appendix E: Organizational Assessment Instrument
A. Reporting Section

B. Data Source Name
(e.g., Claims, Enrollment, Provider Information)

Part D:

Medication Therapy Management Programs No Data

Grievances

Coverage Determinations,
Redeterminations (including At– Risk
Redeterminations under a Drug
Management Program), and Reopenings

No Data
No Data
No Data
No Data
No Data
No Data
No Data
No Data

No Data
No Data
No Data
Improving Drug Utilization Review Controls No Data
No Data
No Data
No Data

4.2.

PROGRAMMING AND SOFTWARE

In Table 7, specify the programming languages and software used to generate the reporting section data
for reporting (e.g., MS Access, SAS, SQL, Crystal Reports, Cognos, SPSS) for the contract(s) included
in this OAI.
TABLE 7: PROGRAMMING SOFTWARE SPECIFICATIONS
A. Reporting Section

B. Programming Code/Software

Part C:

No Data

Grievances
Organization Determinations/Reconsiderations
Special Needs Plans (SNPs) Care Management
Part D:
Medication Therapy Management Programs
Grievances
Coverage Determinations, Redeterminations (including At–
Risk Redeterminations under a Drug Management Program), and
Reopenings
Improving Drug Utilization Review Controls

No Data
No Data
No Data
No Data
No Data
No Data
No Data
No Data

8

Appendix E: Organizational Assessment Instrument

4.3.
SUPPLEMENTAL QUESTIONS REGARDING REPORTING
PROCESSES
The questions below address additional information required to review the processes used to compile
and report the Part C and Part D reporting sections:
4.3.1.

How does your organization ensure it meets the Reporting Requirements deadline for the
contract(s) included in this OAI? Who is responsible for submitting the data into the HPMS Plan
Reporting Module (i.e., responsible department, delegated entity, or first tier/downstream
contractor)?

4.3.2.

What is your organization’s process for correcting or revising data results that have been
returned/rejected by CMS for the contract(s) included in this OAI? Who is responsible (i.e.,
responsible department, delegated entity, or first tier/downstream contractor)?

4.3.3.

If your organization received an outlier/data integrity notification for any of the reporting sections
that are currently undergoing data validation review (as identified in Table 5 for the contract(s)
included in this OAI), were those notifications used to accurately report the data into HPMS? For
the contract(s) included in this OAI, how does your organization track CMS-issued changes to
the Part C and/or Part D Reporting Requirements and Technical Specifications? Who is
responsible (i.e., responsible department, delegated entity, or first tier/downstream contractor)?
How are these changes incorporated into your organization’s data collection and reporting
systems?

4.3.4.

Describe any process or quality improvement activities your organization has implemented since
the prior reporting year/period that may affect reporting section results submitted to CMS (e.g.,
development of steering committees, identification of inefficiencies) for the contract(s) included in
this OAI.

5. DV DOCUMENT REQUEST
The purpose of the documentation request is to obtain documents that will assist the DVC in determining that
data elements for each reporting section are accurately identified, calculated, and documented. This request is
applicable to all organizational processes used in creating the final HPMS submission for the Part C and Part D
Reporting Requirements.
The SO is responsible for ensuring that it has established mutually agreeable methods for sharing proprietary
and/or secure (PHI/PII) information with the DVC and that the DVC complies with all HIPAA privacy and security
requirements. Instructions for logging the information provided by the SO are included in Section 6.

5.1.
REQUEST FOR PROGRAMMING CODE AND EXAMPLE
OUTPUT
For the contract(s) included in this OAI, SOs should provide programming code/source code and an
example output for computer programs used to calculate the data elements collected for each of the
CMS reporting sections that are currently undergoing data validation review (as identified in Table 5).
Such code may include the following:
•

Programming language for extracting data from the source (including any exclusion criteria)

•

Joins between multiple data sources (including validation checks)
9

Appendix E: Organizational Assessment Instrument

•

Data preparation (such as cleansing and missing data)

•

Manipulation to produce the final reports

The following are examples of the types of documents and files required:
•

If using SAS, SPSS, or similar software, provide the programming code, the log file that shows
the results of the compiled programming code, and the list file that shows the output (e.g.,
tables and listings) generated by the programming code.

•

If using MS Access, SQL Server, Oracle, or other database systems, provide the code used to
generate the database query, the results of the compiled query, and the output generated by
the query (e.g., saved data queries).

•

If using MS Excel or other spreadsheet programs, provide the Visual Basic code that produced
the spreadsheets (if applicable), and the actual workbooks with all formulas used to calculate
the values contained in each spreadsheet.

Submitted programming code should be neatly structured and documented so that a third party can
easily read it and understand the programming logic. The best practice is to include comments within the
code; however, if not possible, provide documentation (e.g., work instructions) that enables the DVC to
interpret the programming logic.

5.2.

REQUEST FOR DATA DICTIONARY

SOs should provide a data dictionary or any such documentation that provides file layouts, field
definitions, explanation of calculations, and other information about the underlying data that are used in
creating the data submission for the Part C and Part D Reporting Requirements for the contract(s)
included in this OAI. Appendix B of the OAI includes an example data dictionary that at a minimum
should include the field name, data type, field description, and additional notes regarding the data field
values.

5.3.
REQUEST FOR ANALYSIS PLAN, REPORTING PROCESS
FLOW, AND DIAGRAMS
SOs should provide a copy of their analysis plan, reporting process flows, diagrams, and any other
related documents. These documents should include a description or illustration of the analysis
requirements, analysis methods, and processes used for generating all reporting section output reports
for the Part C and Part D Reporting Requirements for the contract(s) included in this OAI.

5.4.
REQUEST FOR STANDARD OPERATING DOCUMENTS:
STANDARD OPERATING PROCEDURES (SOPS), POLICIES AND
PROCEDURES, OR OTHER WORK INSTRUCTIONS
SOs should provide a copy of the documentation that describes their data and reporting systems and
processes for the contract(s) included in this OAI. Documents of interest include:
•

Work instructions, policies and procedures for the compilation, administration, and/or
submission of the Part C and Part D Reporting Requirements
10

Appendix E: Organizational Assessment Instrument

•

Information Systems SOPs (e.g., system maintenance, upgrade, validation procedures)

•

Data Processing SOPs (e.g., data collection and storage process and frequency)

•

Data Archive/Restoration SOPs (e.g., disaster recovery plans)

6. DV DOCUMENT LOG
The DV Document Log is used as inventory for all documents and files provided by the SO as per Section 5. SOs
should complete the Document Log (see Document Log Template in Appendix A of this OAI) in order to facilitate
review of documentation and files associated with the different stages of the reporting process.
•

Reporting Section: Provide reporting section for the document or file. For example, if
submitting programming code that generates the SNPs reporting section, then indicate “SNPs”
in this column. Otherwise, indicate “N/A” (note that IT system SOPs may be N/A).

•

Document Name: Electronic file name of the document.

•

Document Type: Type of document or file (e.g., work instruction, policy and procedure,
programming code, programming output/report, data dictionary/file layout, reporting process
diagram).

•

Reporting Stage: Stage in the reporting process to which the document applies. This usually
applies to programming code, data queries, and programming output and reports. Examples of
stages include but are not limited to, data extract from the adjudication system, data input into
the internal database, output/report from the internal database, data analysis to summarize data
for reporting, or final report for HPMS entry. Otherwise, indicate “N/A” (note that IT SOPs may
be N/A).

•

Document Description: Work instructions, policies, and procedures are usually selfexplanatory. However, for programming code, SOs should include a description of the input
data sources, the applicable stage in the reporting process, the intended output, and the name
of the output file. For data dictionaries/file layouts, indicate the name of the applicable database
and source tables containing the data fields. For screenshots, process flows, and diagrams
provide the relevant description of the indicated charts, diagrams, and process flows.

11

Appendix E: Organizational Assessment Instrument

APPENDIX A: DOCUMENT LOG TEMPLATE
To the extent possible, list the documents in a logically ordered fashion so the DVC can identify sets of documents relative to the reporting process stage for
each reporting section.
Reporting Section

Document File Name

Document Type

Reporting Stage

Document Description

Example: Grievances

Grievances_SOP.doc

SOP

N/A

SOP documents the methods used to gather,
analyze, and report Grievances data according
to CMS Reporting Requirements

Example: Grievances

Grievances_Data_Load.SAS

SAS Program Code

Load grievance data into
Data Warehouse

Example: Grievances

Grievances_Summary.SAS

SAS Program Code

Summarizes data for HPMS
reporting

SAS program extracts data from
adjudication system and loads into internal
data warehouse
SAS program cleans and summarizes data
for entry into HPMS

Example: Grievances

Grievances Summary
Report.xls

Example Report Output

Summarized data report
example for HPMS entry

Example of report output from the
Grievances Summary. SAS program for
HPMS entry

12

Appendix E: Organizational Assessment Instrument

APPENDIX B: DATA DICTIONARY EXAMPLE
Please refer to the following as an example of the information required in a Data Dictionary.
A. Field Name

B. Type

C. Description

D. Additional Notes

Example: Redetermination ID

Long Integer

Unique ID for each case

Example: Date Created

Date

Date the case was created

Example: Redetermination Status

Integer

Status of the case

1=Open; 2=Closed

Example: Redetermination
Outcome

Integer

Outcome of the case

1=Overturned; 2=Withdrawn;
3=Dismissed; 4=Upheld

13