Download:
pdf |
pdfMedicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
Medicare Part C and Part D Reporting
Requirements Data Validation Procedure Manual
Prepared by:
Centers for Medicare & Medicaid Services
Center for Medicare
Medicare Drug Benefit and C&D Data Group
Effective Date: 2024 Data Validation
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid
OMB control number. The valid OMB control number for this information collection is 0938-1115 (Expires: 4/30/2025). The time required to
complete this information collection is estimated to average 30 hours per response, including the time to review instructions, search existing
data resources, and gather the data needed, and complete and review the information collection. If you have comments concerning the
accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports
clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.
1
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
Table of Contents
1.
INTRODUCTION ................................................................................................................................................. 6
1.1. Data Validation Requirement ........................................................................................................................... 6
1.2. DV Scope ......................................................................................................................................................... 6
1.3. Types of Organizations Required to Undergo DV ........................................................................................... 7
1.4. Required DV Manual and Tools ....................................................................................................................... 7
1.5. Organization of the Procedure Manual ............................................................................................................ 8
EXHIBIT 1. DATA VALIDATION PROGRAM PHASES ......................................................................................................... 8
EXHIBIT 2. DDV PROGRAM ACTIVITIES ......................................................................................................................... 9
2.
PART C AND PART D REPORTING SECTIONS REQUIRING DV ................................................................. 10
2.1. Parts C and D Reporting Sections Requiring DV ........................................................................................... 10
EXHIBIT 3. PART C AND PART D REPORTING SECTIONS REQUIRING DV ........................................................................ 10
2.2. Reporting Sections Excluded from the DV Requirement ............................................................................... 10
EXHIBIT 4. PART C AND PART D REPORTING SECTIONS EXCLUDED FROM DV ............................................................... 11
3.
PLANNING FOR DV ACTIVITIES ..................................................................................................................... 11
3.1. Select appropriate contractor based on Standards for Selecting a DVC ....................................................... 11
3.1.1.
Standards for Selecting a DVC ......................................................................................................... 11
3.1.2.
Timing of DVC Selection................................................................................................................... 11
3.1.3.
Requesting a Contractor Change Mid-Review ................................................................................. 11
3.2.
Notify CMS of DVC Selection/Request Access to HPMS PRDVM .......................................................... 12
3.2.1.
Documentation of DVC Selection Process ....................................................................................... 12
3.2.2.
Request Access to HPMS PRDVM .................................................................................................. 12
3.2.2.1.
PROCESS FOR OBTAINING HPMS ACCESS FOR SOS AND DVCS .......................................................... 12
3.2.2.2.
PROCESS FOR SOS ............................................................................................................................ 12
3.2.2.3.
PROCESS FOR DVCS .......................................................................................................................... 13
3.2.2.3.1.
HPMS PRDVM Access .......................................................................................................................... 13
3.2.2.3.2.
Instructions for Requesting Plan Access via EFI ......................................................................................... 13
3.2.2.3.3.
Existing CMS Enterprise User Administration (EUA) User IDs and HPMS Access ............................................ 13
3.2.2.3.4.
Submission of Findings to CMS: Annual DV Review .................................................................................... 13
3.3. Complete the Web-based DV Training .......................................................................................................... 13
3.4. Review all DV Documents .............................................................................................................................. 14
3.4.1.
Introduction to the EES ..................................................................................................................... 14
3.4.2.
EES (Appendix B) ............................................................................................................................. 14
3.4.2.1.
DV STANDARDS INSTRUCTIONS ........................................................................................................... 14
3.4.2.2.
DV STANDARDS 1 - 7 .......................................................................................................................... 14
3.4.2.2.1.
Standard 1 ........................................................................................................................................... 14
2
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 5. STANDARD 1: REQUIRED DATA FIELDS ARE ACCURATELY CAPTURED AND PROPERLY DOCUMENTED............ 15
4.
3.4.2.2.2.
Standard 2 ........................................................................................................................................... 15
3.4.2.2.3.
Standard 3 ........................................................................................................................................... 15
3.4.2.2.4.
Standards 4 and 5 ................................................................................................................................. 15
3.4.2.2.5.
Standards 6 and 7 ................................................................................................................................. 15
PERFORMING DV ACTIVITIES ....................................................................................................................... 16
4.1. COMPLETE OAI (APPENDIX E) AND PROVIDE APPROPRIATE DOCUMENTATION TO SELECTED DVC PER THE OAI
DOCUMENTATION REQUEST ..................................................................................................................................... 16
4.2. ANALYZE OAI RESPONSES ............................................................................................................................... 16
4.2.1.
Perform OAI Gap Analysis................................................................................................................ 16
4.2.2.
Review Source Code and Other Documentation.............................................................................. 17
4.2.3.
Prepare Interview Discussion Guide (IDG) (Appendix F) ................................................................. 17
4.3. PREPARE FOR ON-SITE OR VIRTUAL VISIT ......................................................................................................... 17
4.3.1.
Select Dates and Appropriate Location(s) of On-Site or Virtual Visit ............................................... 17
4.3.2.
Develop Agenda for Site Visit ........................................................................................................... 17
4.3.3.
Data Extraction and Sampling .......................................................................................................... 17
4.4. CONDUCT SITE VISIT ........................................................................................................................................ 18
4.4.1.
Conduct Entrance Conference ......................................................................................................... 18
4.4.2.
Conduct Interviews with Organization Staff ...................................................................................... 18
4.4.3.
Observe Reporting Processes .......................................................................................................... 18
4.4.4.
Conduct Exit Conference .................................................................................................................. 19
4.5. REQUEST ADDITIONAL DOCUMENTS (IF REQUIRED)............................................................................................ 19
5.
ANALYZING RESULTS AND SUBMISSION OF FINDINGS ............................................................................ 19
5.1. REPORTING DV RESULTS: HOW TO DETERMINE SCORING .................................................................................. 19
5.2. THE EES ......................................................................................................................................................... 19
5.2.1.
Reporting Findings for Standards Using Binary Scale ..................................................................... 19
EXHIBIT 6. EXAMPLES OF CALCULATIONS TO DETERMINE MINIMUM THRESHOLD OF CORRECT SAMPLE/CENSUS RECORDS
FOR “YES” FINDING .................................................................................................................................................. 20
EXHIBIT 7. EXAMPLE OF HOW TO DETERMINE MINIMUM THRESHOLD OF IMPLEMENTED POLICIES OR PROCEDURES FOR
“YES” FINDING ......................................................................................................................................................... 20
5.2.2.
Reporting Findings for Standards Using Likert Scale ....................................................................... 20
EXHIBIT 8. EXAMPLE OF HOW TO DETERMINE MINIMUM THRESHOLD OF IMPLEMENTED POLICIES OR PROCEDURES FOR
FINDINGS USING BINARY AND LIKERT SCORE............................................................................................................. 21
5.2.3.
Guidance for Interpreting Standards and Making a Findings Determination from Review of the EES
21
5.2.3.1.
STANDARD 1 ....................................................................................................................................... 21
EXHIBIT 9. EXAMPLE ROWS FROM EES (APPENDIX B) AND GUIDANCE FOR STANDARD 1 ............................................... 22
5.2.3.2.
STANDARD 2 ....................................................................................................................................... 22
5.2.3.3.
STANDARD 3 ....................................................................................................................................... 25
3
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 11. EXAMPLE ROWS FROM EES (APPENDIX B) FOR STANDARD 3 ..................................................................... 25
5.2.3.4.
STANDARD 4-7.................................................................................................................................... 26
EXHIBIT 12. EXAMPLE ROWS FROM EES (APPENDIX B) FOR STANDARDS 4 THROUGH 7 ................................................. 27
5.3. SUBMIT DV REVIEW FINDINGS VIA HPMS PRDVM ............................................................................................ 28
6.
5.3.1.
Review Draft Findings with Sponsoring Organization ...................................................................... 28
5.3.2.
Submit DV Review Findings via HPMS PRDVM .............................................................................. 28
Post-DV Activities .............................................................................................................................................. 29
6.1. COMPILE ARCHIVE OF DV WORK PAPERS.......................................................................................................... 29
EXHIBIT 13. MINIMUM DOCUMENTATION REQUIRED FOR DV ARCHIVE .......................................................................... 29
6.2. REVIEW FINAL DV RESULTS.............................................................................................................................. 29
6.2.1.
Pass/Not Pass Determination ........................................................................................................... 29
6.2.2.
Passing DV – Minimum threshold .................................................................................................... 30
6.2.3.
SO Appeal of DV Determination ....................................................................................................... 30
4
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
List of Appendices
•
Appendix A: Standards for Selecting a Data Validation Contractor
•
Appendix B: Examination Engagement Standards
•
Appendix C: Model Language for Letter to Confirm Selection of Data Validation Contractor
•
Appendix D: Example Application for Access to CMS Computer Systems
•
Appendix E: Organizational Assessment Instrument
•
Appendix F: Interview Discussion Guide
•
Appendix G: Example Site Visit Agenda for On-site or Virtual Visits
•
Appendix H: Data Extraction and Sampling Instructions
•
Appendix I: Example Data File Inventory Log
•
Appendix J: Data Validation Pass/Not Pass Determination Methodology
5
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
1. INTRODUCTION
1.1. Data Validation Requirement
The Centers for Medicare & Medicaid Services (CMS) requires that sponsoring organizations (SOs) contracted to
offer Medicare Part C and/or Part D benefits be subject to an independent yearly audit to validate certain data
reported to CMS to determine its reliability, validity, completeness, and comparability in accordance with
specifications developed by CMS. 1
Data validation (DV) allows CMS to use plan-reported data to:
• Respond to inquiries from Congress.
• Conduct oversight and provide information to agencies and the public about an SO’s performance
using indicators such as operations, costs, availability and use of services.
The data allows CMS to:
• Monitor and compare the performance of SOs over time.
• Manage Star Ratings and other performance measures.
SOs can review the DV results to assess their performance and improve internal data, systems, and reporting
processes. The Procedure Manual (Manual) provides SOs and data validation contractors (DVCs) with
information about the Part C and Part D Reporting Requirements (RR) Data Validation program. The Manual
describes:
• Background information.
• An overview of the DV program.
• Scope and timeframe required for DV.
• Tools and processes used for conducting DV.
1.2. DV Scope
CMS requires that SOs subject the plan-reported data to an independent DV once per year. DV will:
• Take place from April 1st – June 15th.
• Incorporate all data submitted to CMS in the Health Plan Management System (HPMS) by the
submission deadline (Exhibit 3) or the March 31st re-submission deadline for the reporting period.
DVCs cannot audit any RR data an SO submits or re-submits in HPMS after March 31st or RR data not
submitted in HPMS. The reviewer must submit findings from the annual DV review to CMS by June 15th.
1
See 42 CFR §422.516(g) and §423.514(j)
6
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
1.3. Types of Organizations Required to Undergo DV
CMS requires Part C and Part D SOs that report Part C and/or Part D data to CMS per the RR, regardless of
enrollment size, to undergo DV. The only SOs exempt from participating in the DV are:
• Program of All-Inclusive Care for the Elderly (PACE) SOs and Part C Health Care Prepayment Plans
(HCPPs).
• Organizations/sponsors that non-renew or terminate a contract during the measurement year. 2
• A contract that terminates before July 1 in the following year after the contract year (CY) reporting
period. 3
Any SO that delegates the data collection, calculation, and/or reporting for any reporting section or data element
must have the DVC include the delegated entity’s data and reporting processes in its DV review for each
applicable contract. For example, CMS requires delegated entities to provide applicable policies, procedures,
and source data to the DVC for validation if they submit data to an SO used for any reporting section.
1.4. Required DV Manual and Tools
DVCs must use this manual and its appendices to conduct DV. The manual includes the following documents:
1. Standards for Selecting a Data Validation Contractor (Appendix A).
2. Examination Engagement Standards (EES) (Appendix B).
3. Model Language for Letter to Confirm Selection of Data Validation Contractor (Appendix C).
4. Example Application for Access to CMS Computer Systems (Appendix D).
5. Organizational Assessment Instrument (OAI) (Appendix E).
6. Interview Discussion Guide (IDG) (Appendix F).
7. Example Site Visit Agenda for On-site or Virtual Visits (Appendix G).
8. Data Extraction and Sampling Instructions (Appendix H).
9. Example Data File Inventory Log (Appendix I).
10. Pass/Not Pass Determination Methodology (Appendix J).
The EES and other documentation associated with the DV program assess an SO’s information systems
capabilities, and overall processes for collecting, storing, compiling, and reporting the data as specified by the
Part C and Part D RR.
The most current versions appear at Medicare Part C and Part D Data Validation and within the HPMS Plan
Reporting Data Validation Module (PRDVM). Before conducting DV, all SOs and DVCs should confirm that
they’re using the most recent DV documentation available on the CMS DV website.
2 If a PBP (Plan) under a contract terminates at any time in the CY reporting period and the contract
remains active through July 1 of the following year, the contract must still report data for all PBPs,
including the terminated PBP. If otherwise applicable, the contract must also undergo DV.
3 If a contract terminates before July 1 in the following year after the contract year (CY) reporting period,
CMS does not require the contract to report any data for the respective two years – the CY reporting
period, and the following year.
7
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
1.5. Organization of the Procedure Manual
Exhibit 1 below illustrates the organization of the Manual. The four phases that comprise the DV process
structure and the document’s content. The graphic shows the phases in the order in which the annual DV cycle is
conducted.
EXHIBIT 1. DATA VALIDATION PROGRAM PHASES
Planning for
DV Activities
Performing DV
Activities
Analyzing
Results and
Submission of
Findings
Completing
Post-DV
Activities
Each phase of the DV review process contains several activities. Exhibit 2 shows the activities in order,
beginning with the selection of an appropriate DVC and ending with the appeal of DV determinations.
During the DV process:
• The SO provides necessary documentation and information to the DVC.
• The DVC must serve as an independent reviewer and may not provide consultative services to the SO.
Reference 3.1.1 for more information.
We describe each of these steps in more detail throughout the Manual.
8
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 2. DDV PROGRAM ACTIVITIES
DV Phase
Planning for DV
Activities
Performing DV
Activities
Analyzing Results
and Submission of
Findings
Completing
Post- Activities
Step
1
Responsible
Party
SO
2
DVC, SO
3
DVC, SO
4
DVC, SO
5
DV Activities
Timeline
Select independent DVC based on Standards for DecemberSelecting a Data Validation Contractor (Appendix March
A).
Notify CMS of DVC Selection/Request Access to January-April
HPMS PRDVM.
Enter/Update the DVC organization in HPMS.
If applicable, enter/update the pre-assessment
consultant in HPMS.
Complete the web-based Data Validation
Training.
Review all DV documents.
February-March
SO
Complete OAI (Appendix E) and provide
appropriate documentation to selected DVC per
the OAI’s documentation request.
March-April
6
DVC, SO
Analyze OAI (Appendix E) Responses.
March or later
7
DVC, SO
Prepare for on-site or virtual visit (site visit
agenda, resource needs, and logistics).
Early April
8
DVC, SO
Conduct on-site or virtual review (convene
Early April
entrance conference, conduct interviews with SO
staff, observe SO’s reporting processes, and
obtain census and/or sample files).
9
DVC
Request additional documents following on-site or Mid/Late April
virtual visit (if applicable).
10
DVC
11
DVC
Determine compliance with Data Validation
April-June 15
Standards by recording findings in the EES
(Appendix B).
Submit EES findings to CMS via HPMS PRDVM No later than
and receive DV scores.
June 15
14
DVC, SO
Compile archive of DV work papers.
No later than
June 15
15
DVC
Provide draft findings to the SO.
No later than
June 15
16
DVC, SO
Review EES findings and make necessary
technical and factual corrections.
No later than
June 15
17
SO
Appeal DV determination(s) (if applicable).
January-March
5 business days
following the DV
deadline
* References to December refer to the calendar year before the DV review; all other
references to months refer to the same calendar year as the DV review.
9
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
2. PART C AND PART D REPORTING SECTIONS REQUIRING
DV
This section provides an overview of the Part C and Part D reporting sections that will undergo DV.
2.1. Parts C and D Reporting Sections Requiring DV
EXHIBIT 3. PART C AND PART D REPORTING SECTIONS REQUIRING DV
Reporting Section
Reporting Period(s)
Data Submission Due
Date(s) to CMS
Part C
Grievances
1/1 -3/31
4/1 -6/30
7/1 -9/30
10/1 -12/31
First Monday of February
Organization Determinations/ Reconsiderations
1/1 -3/31
4/1 -6/30
7/1 -9/30
10/1 -12/31
Last Monday of February
Special Needs Plans (SNPs) Care Management
1/1 -12/31
Last Monday of February
Part D
No data
Medication Therapy Management Programs
1/1 -12/31
Last Monday of February
Grievances
1/1 -3/31
4/1 -6/30
7/1 -9/30
10/1 -12/31
First Monday of February
1/1 –3/31
Coverage Determinations, Redeterminations (including At4/1 -6/30
Risk Redeterminations Under a Drug Management Program), 7/1 -9/30
and Reopenings
10/1 -12/31
Last Monday of February
Improving Drug Utilization Controls
1/1 -3/31
1/1 -6/30
1/1 -9/30
1/1 -12/31
No data
Last Monday of February
2.2. Reporting Sections Excluded from the DV Requirement
Exhibit 4 lists the reporting sections excluded from the DV review at this time.
10
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 4. PART C AND PART D REPORTING SECTIONS EXCLUDED FROM DV
Part C Reporting Sections
•
•
•
•
3.
Enrollment/Disenrollment
Employer Group Plan Sponsors
Rewards and Incentives Programs
Payments to Providers
Part D Reporting Sections
•
•
Enrollment/Disenrollment
Employer/Union-Sponsored Group Health Plan
Sponsors
PLANNING FOR DV ACTIVITIES
3.1. Select appropriate contractor based on Standards for Selecting a DVC
The SO is required to select an independent, external DVC to complete DV and report the findings to CMS.
3.1.1.
Standards for Selecting a DVC
CMS provides a set of Standards for Selecting a DVC (Appendix A) for SOs to use in acquiring a
contractor. The selected DVC must have these standards, which describe the minimum:
• Qualifications,
• Credentials,
• Resources, and
• Conduct guidelines.
SOs must acquire one DVC to conduct the DV and, if necessary, the DVC may subcontract to ensure that
it has the expertise required for each DV area and to meet the minimum standards. SOs may use their
own staff only to assist the DVC in obtaining the information, data, and documents needed to complete the
DV review.
The Standards for Selecting a DVC also contain best practices that CMS expects DVCs to follow during
the review. CMS requires the DVC to:
• Remain objective and independent in all its activities.
• Keep the SOs’ privileged information confidential.
The DVC may not perform mock audits, pre-assessments, and any other types of review before the DV
review period from April-June. However, the DVC can start preparing for the DV review before April 1 so
that DV can start as soon as possible on or after April 1. Examples of preparation activities may include:
• Meeting with the SO to discuss the DV process, resource needs, timeline, etc.
• Providing the SO with a list of documents, data, and materials needed to complete the review.
Any specific questions regarding organizational independence should be directed to
[email protected].
3.1.2.
Timing of DVC Selection
An SO must select a DVC before the beginning of the DV period. SOs should allow enough time for the
DVC to perform all the requirements of the review and submit findings to CMS via the PRDVM in HPMS
during the required timeframe.
3.1.3.
Requesting a Contractor Change Mid-Review
An SO may not change its DVC during the formal review period from April 1 to June 15 unless there are
11
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
conditions that are unrelated to DV findings such as negligence or malfeasance on the part of the DVC. If
an SO needs to change its contractor, CMS requires the new DVC to complete the DV review in its
entirety, starting with the OAI (Appendix E) analysis and through the submission of findings to CMS by
June 15th.
3.2. Notify CMS of DVC Selection/Request Access to HPMS PRDVM
3.2.1.
Documentation of DVC Selection Process
SOs must document their DVC selection process and be able to show, upon CMS’ request, how their
chosen DVC meets the minimum qualifications, credentials, and resources described in the Standards for
Selecting a DVC. This includes maintaining a copy of the documentation that all DVC staff assigned to the
applicable DV review team completed the CMS Data Validation Training program (see Section 3.3). The
SO must retain this documentation for 10 years as required under 42 CFR §§ 422.504(d) and 423.505(d).
DVC staff must complete this training each calendar year.
If an SO chooses the same DVC it used for a previous year’s DV review, it must still document the
selection process as described above.
CMS does not require the SO to submit this information to CMS, but the information should be available if
CMS requests it.
3.2.2.
Request Access to HPMS PRDVM
Once the SO selects a DVC and documents the selection process, the DVC requests staff access to the
PRDVM in HPMS. This module allows users to gain access to pertinent documents, record DV results,
and submit findings to CMS. The credentials CMS assigns to a user will allow that individual to access
only those associated SO(s)/contract(s). The DVC staff will use these credentials to access the
appropriate documents within the PRDVM starting no earlier than April 1 of the calendar year.
3.2.2.1.
PROCESS FOR OBTAINING HPMS ACCESS FOR SOS AND DVCS
CMS requires users who access the HPMS to have a user ID. Send any questions you have
regarding the user ID process to [email protected]. Read more about obtaining
HPMS access at the CMS HPMS website.
3.2.2.2.
PROCESS FOR SOS
To gain access to the PRDVM, the SO sends their DVC an official letter in either hardcopy or an
emailed PDF format attachment with the following information:
•
The SO’s acknowledgment that it has contracted with the selected DVC to
complete the review,
•
The name of each individual who needs access (up to 5 users),
•
The type of functionality that each user requires,
•
Acknowledgement that the users have completed the web-based DV Training,
•
The contract number(s) the DVC will need access to, and
•
The SO’s Chief Executive Officer’s (CEO) signature.
You can find model language for this letter in the Model Language for Letter to Confirm Selection
of a Data Validation Contractor (Appendix C).
If an SO chooses the same DVC it used for a previous year’s DV review, it must still provide the
DVC with this signed letter for the current year’s DV activities.
12
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
3.2.2.3.
PROCESS FOR DVCS
3.2.2.3.1.
HPMS PRDVM Access
DVC staff must get individual access to the HPMS PRDVM. If the designated user(s) from the DVC
does not have active access to HPMS, each user should download the Application for Access to
CMS Computer Systems at: Access to CMS Data & Application and follow the instructions
provided in Example Application for Access to CMS Computer Systems (Appendix D) for
requesting reviewer access to the HPMS PRDVM.
CMS allows up to 5 users from each DVC to have access to this Module on behalf of each SO.
The DVC must create their user ID using EFI and then email the letter from each SO for which they
are under contract to complete the DV review. Since CMS will process the request for a new user
ID first, the user will only get access to the HPMS main home page until we receive a DV letter.
You can send the letters as email attachments to [email protected].
3.2.2.3.2.
Instructions for Requesting Plan Access via EFI
Follow the instructions for getting a user ID by clicking the Instructions for Requesting Plan Access
via EFI link under the Download section of the CMS HPMS website.
When the user gets to the “I am a” question, the user must select Data Validation Consultant from
the drop down. The system will automatically fill out the company name and Plan number. The
HPMS team created an instructional video to walk a user through getting a new CMS user ID.
3.2.2.3.3.
Access
Existing CMS Enterprise User Administration (EUA) User IDs and HPMS
Users who already have active CMS Enterprise User Administration (EUA) User IDs and HPMS
access, do not need a new Application for Access to CMS Computer Systems. Instead, they must
modify their current credentials to allow access to the PRDVM. For this access, users need to
ensure that the letter from each SO linking the DVC to the SO includes the user’s current User ID
and an explanation that the user already has HPMS Access. Users must send this letter to CMS
via email to [email protected]. If a user had PRDVM access as a DVC
previously, they do not need to resend letters on their behalf. CMS only requires letters for
changes to a DVC user’s account.
3.2.2.3.4.
Submission of Findings to CMS: Annual DV Review
DVCs must submit the findings from the annual DV review to CMS by June 15th of each calendar
year. To assure timely access to the HPMS PRDVM to meet this DV timeframe, submit requests
for HPMS PRDVM access by early April. It will take about 4 weeks for the designated individuals to
get the credentials (CMS EUA User IDs and passwords) to access the PRDVM.
3.3. Complete the Web-based DV Training
CMS developed a web-based DV Training for SOs and DVCs to learn more about the DV program and its
specific requirements.
During the DV preparation phase, CMS recommends that all SO staff involved in DV to complete the CMS webbased DV Training to familiarize themselves with the DV process and requirements.
Also, CMS requires all DVC staff to take the CMS web-based DV Training prior to working on the DV project.
They must complete this training each year. Once they complete training, they will receive a generated certificate
of completion. DVC staff should provide training certificates to the SO before starting work on the DV.
13
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
3.4. Review all DV Documents
Refer to Section 1.4 for the documents you need to review before DV. This section will focus specifically on the
DV standards, which the EES (Appendix B) further describes.
3.4.1.
Introduction to the EES
The DV Standards include general and elemental standards that DVC staff must use to determine whether
the data each SO reported to CMS per the Part C and Part D RR and Technical Specifications (TS) are
reliable, valid, complete, and comparable in accordance with the specifications developed by CMS.
The standards assess an SO’s information systems capabilities and its processes for collecting, storing,
compiling, and reporting Part C and/or Part D data. They also assess whether SOs follow the applicable
RR and TS to compile data, consider appropriate data exclusions, and verify calculations, computer code,
and algorithms.
3.4.2.
EES (Appendix B)
The DV Standards include general and elemental standards that DVC staff must use to determine whether
the data each SO reported to CMS per the Part C and Part D RR and TS are reliable, valid, complete, and
comparable in accordance with the specifications developed by CMS.
The standards assess an SO’s information systems capabilities and its processes for collecting, storing,
compiling, and reporting Part C and/or Part D data. They also assess whether SOs follow the applicable
RR and TS to compile data, consider appropriate data exclusions, and verify calculations, computer code,
and algorithms.
3.4.2.1.
DV STANDARDS INSTRUCTIONS
The DV Standards include identical instructions relating to the types of information that DVCs must
review for each reporting section and a set of validation standards based on the applicable RR and
TS.
The DVC uses these standards with the Data Extraction and Sampling Instructions (Appendix H)
and the EES (Appendix B) to evaluate the SO’s processes for producing and reporting the
reporting section’s data. CMS strongly recommends that the DVC and SO staff responsible for RR
and/or DV review the DV Standards documentation before and during the review of each reporting
section to ensure that they thoroughly understand the standards. This will also help to ensure
DVCs extract all applicable data fields for each reporting section.
The top portion of each set of standards details the documents and reports what the DVC uses to
determine compliance with the standards for each specific reporting section. The gray box
underneath the name of the applicable reporting section lists the documents and reports.
This section also contains notes to the DVC regarding a specific reporting section, if applicable,
and any nuances or differences that DVCs may encounter during the review of that reporting
section. The second section of each set of standards is identical for all Part C and Part D reporting
sections.
3.4.2.2.
DV STANDARDS 1 - 7
3.4.2.2.1.
Standard 1
Standard 1 (see Exhibit 5) provides general and specific criteria for DVCs to review the database
management and structure of source documentation used by the SO.
14
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 5. STANDARD 1: REQUIRED DATA FIELDS ARE ACCURATELY CAPTURED AND PROPERLY DOCUMENTED
DV STANDARD 1
1.
Database management and structure: A review of source documents (for example, programming code,
spreadsheet formulas, analysis plans, saved data queries, file layouts, process flows) shows that all
source documents accurately capture required data fields, and the SO has properly documented them.
Criteria for Validating Source Documents:
a.
b.
c.
d.
e.
f.
g.
SOs properly secured source documents and output and have referenced data file locations correctly so
that DVCs can retrieve source documents at any time to validate the information submitted to CMS via
CMS systems.
SOs clearly and adequately documented source documents.
SOs accurately captured required data fields including all data fields for RR in source documents. All
data fields have meaningful, consistent labels (for example, label field for patient ID as Patient ID, rather
than Field1 and maintain the same field name across data sets).
Source documents are error-free (for example, programming code and spreadsheet formulas have no
messages or warnings indicating errors).
SOs appropriately applied version control of source documents.
SOs captured the appropriate date range(s) for the reporting period(s). SO, reports data based on the
required reporting period of1/1-12/31.
SOs assigned data at the applicable level (for example, plan benefit package (PBP) or contract level).
3.4.2.2.2.
Standard 2
Standard 2 instructs the DVC to review data base extraction functions and validate the underlying
source data, preliminary data sets, and interim data sets used by the SO to populate each
reporting section. Standard 2 requires that the population for each reporting section are accurately
identified, processed, and verified (including calculations for the number of members, claims,
grievances, procedures, etc.). The DVC must review that QA checks/thresholds are applied by the
SO to detect outlier or erroneous data.
3.4.2.2.3.
Standard 3
DVCs use Standard 3 to determine whether an SO implements policies and procedures for
submission of their final stage data into HPMS. The DVC validates that each data element for each
reporting section is valid, complete and accurate by reviewing expected counts and that the SO
accurately captures data by applying data integrity/ logical checks.
3.4.2.2.4.
Standards 4 and 5
For Standards 4 and 5, the DVC must verify that the SO implements policies and procedures for
regular data system updates, archiving of source, intermediate, and final stage data, and
restoration. The Standards ensure that SOs keep data systems up to date and that they have
systems in place for accurate and timely data submission or re-submission in the event of data
loss.
3.4.2.2.5.
Standards 6 and 7
DVCs apply Standards 6 and 7 only in certain situations. Standard 6 is applicable if an SO’s data
systems underwent any changes during the reporting period. If so, the DVC must examine and
confirm whether the changes were properly implemented and that they did not adversely impact
the reported data.
15
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
Standard 7 applies if the SO outsourced any of the data collection or validation processes to
another entity. This standard assesses whether the SO has policies and procedures in place to
routinely monitor the delegated entity’s work to ensure quality and timeliness of the data.
The DVC should mark “Not Applicable” in EES (Appendix B) if Standard 6 or 7 is not applicable to
the reporting section or contract under review.
4.
PERFORMING DV ACTIVITIES
4.1. COMPLETE OAI (APPENDIX E) AND PROVIDE APPROPRIATE DOCUMENTATION TO
SELECTED DVC PER THE OAI DOCUMENTATION REQUEST
The OAI (Appendix E) focuses on how the SO collects, stores, and reports data. We recommend SOs complete
the OAI before the DV, as the DV review relies significantly on the information in this tool. CMS estimates that the
OAI should take a minimum of two weeks to complete and submit it to the DVC no later than early April. SOs
may not send their completed OAI or source code, SOPs, etc. to their DVCs before the start of the DV cycle on
April 1.
CMS requires each SO to provide its DVC with the basic information of its Medicare contracts and which Part C
and/or Part D reporting sections each contract submits to CMS. SOs with more than one contract with CMS only
need to complete one version of the OAI that covers all its contracts. If the SO varies the information they provide
in the OAI allows for the flexibility to identify the differences for the DVC in applicable sections.
The SO should reflect its systems and processes in place during the reporting period(s) undergoing the DV
review in all documentation and responses to questions in the OAI. For example, if the data CMS reviews are for
the 2024 reporting period, the SO should include only diagrams of the information systems in place in 2024 or
the programming code they used in 2024 to calculate the reporting sections, despite DV occurring in 2025.
The SO and its DVC must determine mutually agreeable methods for sharing and protecting proprietary data,
such as data requested in the OAI, and protected health information. The Standards for Selecting a Data
Validation Contractor (Appendix A) includes minimum security requirements, and the DVC’s facility, equipment,
and processes must comply with those requirements. CMS requires the SO to ensure that the DVC complies with
all Health Insurance Portability and Accountability Act (HIPAA) privacy and security requirements.
4.2. ANALYZE OAI RESPONSES
CMS recommends DVCs perform a preliminary review of the documentation submitted in the OAI in advance of
each site visit (on-site or virtual) so that DVCs can follow-up with any documentation during the site visit. CMS
requires the documentation submitted by the SO be adequate for an effective review. The amount of detail SOs
provide in the documentation determines the ease of the review process, especially for the review of
programming code/source code.
Additionally, the OAI provides supplemental questions to help the DVC better understand the processes the SO
uses to compile and submit its reporting sections. The SO’s responses to these questions will provide insight as
to who is responsible for the quality control and submission of the data, the processes for incorporating CMS
updates to the RR and/or TS into the SO’s systems, and descriptions of any past issues during the reporting
process.
4.2.1.
Perform OAI Gap Analysis
When the DVC receives the completed OAI, the DVC should review the document for completeness and
accuracy. The DVC should note the sections of the OAI that the SO missed or did not complete, and the
DVC should follow-up with the SO to complete these sections. CMS requires the DVC to determine
whether any identified gaps in the OAI responses require the SO to address them before or during the site
visit portion of the review.
16
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
4.2.2.
Review Source Code and Other Documentation
Data dictionaries and source codes are critical for allowing the DVCs to map ambiguous field names and
internal status codes to meaningful descriptions. Well organized and structured documentation of the
reporting and data extraction processes for the various reporting sections will assist the DVC in gaining a
more thorough understanding of the SO. The DVC should be familiar with data systems and processes
detailed by the SO in the OAI to ensure thorough preparation for the site visit.
4.2.3.
Prepare Interview Discussion Guide (IDG) (Appendix F)
The Interview Discussion Guide (IDG) (Appendix F) is intended to help the discussion between the DVC
and the SO’s report owners and subject matter experts. The IDG is a dynamic tool that has both general
and reporting section questions that can guide an effective discussion about an SO’s underlying data
systems and reporting processes. If the DVC discovers evidence that may indicate errors in the SO’s data
or reporting processes during review of the documentation the SO provided in response to the OAI, the
DVC should change the IDG used for that SO with new questions that may show any vulnerabilities or
opportunities for repeated errors with data collection or reporting. Additionally, the IDG should serve as a
“guide” for the DVC; it is at the DVC’s discretion to include additional questions and/or details in the
document to discuss during on-site or virtual interviews and to ensure they document the additional details
accordingly.
4.3. PREPARE FOR ON-SITE OR VIRTUAL VISIT
4.3.1.
Select Dates and Appropriate Location(s) of On-Site or Virtual Visit
Site visits may be on-site or virtual. The DVC determines the most appropriate location(s) of the site visit
(for example, virtual, SO’s facility, PBM’s facility, other delegated entity’s facility). During the site visit, the
DVC: (1) conducts interviews with SO staff, (2) observes the SO’s reporting processes, and (3) obtains
census and/or sample files to support the validation of Part C and Part D reporting sections. SOs and
DVCs must determine mutually agreeable dates for performing the site visit or holding a virtual site
meeting.
4.3.2.
Develop Agenda for Site Visit
CMS expects the DVC and SO to work together to prepare a site visit agenda. The time required to
complete a site visit may be contingent upon the size of the SO, efficiency of the SO’s operations, level of
reporting automation, and scope of the DV review. CMS requires the SO’s report owner(s) to schedule a
DVC session for each reporting section to allow enough time for the SO to provide an overview of each of
the relevant data systems they used gathering data and producing reports, as well as to complete the data
extraction/sampling process.
4.3.3.
Data Extraction and Sampling
CMS urges the DVC to review the completed OAI and, if necessary, hold conference calls with the SO to
discuss the SO’s processes. Calls with the SO, specifically with each reporting section’s report owner, can
also provide an opportunity for the DVC to review the Data Extraction and Sampling Instructions (Appendix
H) in more detail and for the report owners to seek clarification as needed. These discussions can also
inform the DVC about the SO’s data systems and the sources from which the SO pulls data.
The DVC can use two methods to extract data for each reporting section. The first is to extract the full
census of data for a reporting section, meaning they extract every data record that is relevant to a
reporting section. When possible, DVCs should attempt to extract the full census. The DVC extracting the
census will enable them to determine with the greatest precision whether SOs submitted reporting
sections accurately in HPMS. DVCs can use the second method of extracting a random sample, if the size
or complexity of a database creates an unusual time burden on the DVC or SO, or both.
17
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
DVCs should use their best judgment to decide if extracting a full census is feasible, or if selecting a
random sample will provide the data necessary for the DV review. Refer to Appendix H for further details
regarding these two methods. CMS recommends that the DVC records details about each reporting
section’s data set into a Data File Inventory Log (Appendix I). Appendix I contains an example log that the
DVC can use. It includes details such as the reporting section name, report owner, data file name, type of
data file (for example, source, intermediate, or final stage data file), number of rows or records, and a
description of the file. By completing this log, the DVC will be able to easily reference the data files during
its post-site visit assessment of data.
The SO should write all data files to tab-delimited or comma-delimited text files with variable names in the
first row and transfer these files to the DVC’s secure storage device for each reporting section’s data. The
SO must also provide the DVC a file layout or data dictionary for the data files in either Word documents or
Excel spreadsheets on the same secure storage device. The SO and DVC must ensure that they have
established mutually agreeable methods for sharing protected health information and that the DVC
complies with all HIPAA privacy and security requirements.
4.4. CONDUCT SITE VISIT
4.4.1.
Conduct Entrance Conference
The entrance conference provides an opportunity for the DVC and the SO’s management and individual
report owners to introduce themselves and discuss expectations for the on-site or virtual visit. At the
entrance conference, the DVC should describe the objectives for the review and discuss any
administrative needs of the team. Optionally, the SO may provide a high-level overview of its organization,
focusing on its operations with respect to meeting the Parts C and D RR.
4.4.2.
Conduct Interviews with Organization Staff
The DVC must conduct interviews with the subject matter experts and data owners for each reporting
section. These interviews provide a first-hand opportunity for the DVC to gain a thorough understanding of
each SO’s data collection and reporting processes with meeting the Parts C and D RR. The DVC should
reference the IDG to ensure that they address all key topics during the interviews. In addition, they should
address and identify any questions and follow-up items during the analysis of the OAI responses during
the interviews.
4.4.3.
Observe Reporting Processes
Designated SO staff (such as report owners) must provide demonstrations of the data systems and
reporting processes including data extraction from originating data sources, data analysis, quality
assurance processes, and processes for entering or uploading final data into CMS systems. The following
is a sample list of the parts of the process that they should demonstrate:
• Contact information and location of report owner and data providers,
• Function and location of all data warehouses,
• Types of data used (format, number of tables),
• Links and joins to other areas/departments/data,
• Types of programming used to create the reports,
• Review processes and oversight,
• Timeframes for the process (amount of time it takes to run specific parts of the report),
• Approximations of report volume,
• Updates to the process and system changes, and
• Storage locations (for example, building or server), security, and access constraints.
18
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
The demonstrations provide a clear illustration of the reporting processes, provide the DVC with insight
into the SO’s ability to ensure reliable, valid, complete, and comparable data, and allow an opportunity to
get immediate responses to any questions or concerns about the reported data.
4.4.4.
Conduct Exit Conference
The on-site or virtual visit should conclude with an exit conference, where the DVC should provide the SO
with a summary of next steps and note any follow-up that may need to occur.
4.5. REQUEST ADDITIONAL DOCUMENTS (IF REQUIRED)
CMS recognizes that it may not be possible for DVCs to get all the required data and documentation during the
scheduled on-site or virtual visit and follow-up conversations may be required. The DVC should make every
attempt to gather all required data and documentation during the on-site or virtual visit. If they need to follow-up
after the conclusion of the scheduled on-site or virtual visit due to not enough information, the DVC should have
more conversations with the SO and/or make requests for documentation. DVCs and SOs should understand
that the DV is an iterative process, and SOs should be ready to provide more data and documentation after the
on-site or virtual visit.
5.
ANALYZING RESULTS AND SUBMISSION OF FINDINGS
5.1. REPORTING DV RESULTS: HOW TO DETERMINE SCORING
After the site visit, the DVC assesses the documentation and census/sample data from the SO, as well as the
information they gained during the interviews and demonstrations of the SO’s reporting processes and
information systems. The DVC determines compliance with the DV standards and records findings in the EES.
They must score and record DV of all reporting sections using the EES.
5.2. THE EES
In the scope of review, the DVC completes the EES (Appendix B) to record the findings for each contract and
uploads as a data file into the PRDVM. The EES (Appendix B) allows the DVC to record notes, data sources they
referenced, and findings for the different standards for a given reporting section. The DVC will record any
reporting section-level and data element-level findings for each reporting section. After the DVC has submitted
in HPMS, the SO can view the findings via the “Review Data Validation Findings Report”.
When using the EES (Appendix B), DVCs should only complete areas displayed in white for data sources, review
results, and findings. Areas in gray are not applicable for scoring. In the "Data Sources and Review Results"
column, the DVC should enter the data sources and review results for each standard or sub-standard. Next to
this column, in the "Findings" column, the DVC selects the appropriate choice based on whether the plan met the
requirements for the standard or sub-standard.
5.2.1.
Reporting Findings for Standards Using Binary Scale
For all standards except 1.b, 1.c, 1.d, 2.a, 3.a and 3.b, DVCs report the findings using a binary scale. The
DVC selects "Y" if the requirements for the standard or sub-standard have been completely met. The DVC
must select "N" if any requirement for the standard or sub-standard was not met.
CMS expects that there will be situations when the DVC finds that an SO is not fully compliant with specific
DV standards. CMS has established a threshold where a minimum of 90% of records are accurate (for
example, sample or census records, source documents, policies and procedures, data entry records) to
record a “Yes” finding for any standard. DVCs apply this threshold to standards that require the review of
policies and procedures when possible to quantify the adherence to or implementation of policies and
procedures (see Exhibit 6). Exhibit 7 shows examples of how to calculate this minimum threshold
specifically for Standard 1 e., where the DV involves samples or the complete census of records and/or
data values.
19
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 6. EXAMPLES OF CALCULATIONS TO DETERMINE MINIMUM THRESHOLD OF CORRECT SAMPLE/CENSUS
RECORDS FOR “YES” FINDING
Sample/
Census
Size
Calculation for
Minimum Threshold
Minimum Threshold of Correct Records for “Yes” Finding
150
0.90 x 150 = 135
At least 135 of the records are correct for the reviewer to record “Yes” to
the standard.
205
0.90 x 205 = 184.5
At least 185 of the records are correct for the reviewer to record “Yes” to
the standard (round 184.5 to 185).
EXHIBIT 7. EXAMPLE OF HOW TO DETERMINE MINIMUM THRESHOLD OF IMPLEMENTED POLICIES OR PROCEDURES
FOR “YES” FINDING
Standard
Standard Description
1 e.
Version control of source
documents is
appropriately applied.
5.2.2.
Minimum Threshold for a “Yes” Finding
SO appropriately applies version control of source documents.
Eleven out of the 12 months in the contract year, the SO
implemented the enrollment system update policy as written (11/12 =
91.6%).
Reporting Findings for Standards Using Likert Scale
For standards 1.b, 1.c, 1.d, 2.a, 3.a and 3.b, the scoring uses a five-point Likert-type scale. DVCs review
the percentage of records that meet the standards, and enter a score based on the Likert- type scale. The
scale corresponds to the percentage of errors that are found in plan data as shown below:
1. Contract data has more than 20 percent error in records – Contract receives a score of 1 for
the given standard/sub-standard.
2. Contract data has between 15.1 percent and 20 percent error in records – Contract receives a
score of 2 for the given standard/sub-standard.
3. Contract data has between 10.1 percent and 15 percent error in records – Contract receives a
score of 3 for the given standard/sub-standard.
4. Contract data has between 5.1 percent and 10 percent error in records – Contract receives a
score of 4 for the given standard/sub-standard.
5. Contract data has less than or equal to a 5% error in records – Contract receives a score of 5
for the given standard/sub-standard.
Exhibit 8 shows examples of different scenarios a DVC might face and corresponding scores DVCs assign
to the contract for the standards using either Likert or binary scoring.
20
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 8. EXAMPLE OF HOW TO DETERMINE MINIMUM THRESHOLD OF IMPLEMENTED POLICIES OR PROCEDURES
FOR FINDINGS USING BINARY AND LIKERT SCORE.
Percentage of Errors That the DVC Found in
Plan Data
Standard 4
DV Response (DV
Findings
the DVC Reported on
Column 'F' in
Appendix B)
Yes
1.a, 1.e, 1.f, 1.g, 4, 5, 6, 7
Fewer than 10 percent error in contract data for the
given reporting section/ data element(s)
1.a, 1.e, 1.f, 1.g, 4, 5, 6, 7
More than 10 percent error in contract data for the
given reporting section/ data element(s)
No
6, 7
If any of the listed standards is not applicable
N/A
Percentage of Errors That the DVC Found in
Plan Data
Standard
1.b, 1.c, 1.d, 2.a, 3.a, 3.b
More than 20 percent error in contract data for the
given reporting section/data element(s)
1
1.b, 1.c, 1.d, 2.a, 3.a, 3.b
Between 15.1 percent and 20 percent error in
contract data for the given reporting section/data
element(s)
2
1.b, 1.c, 1.d, 2.a, 3.a, 3.b
Between 10.1 percent and 15 percent error in
contract data for the given reporting section/data
element(s)
3
1.b, 1.c, 1.d, 2.a, 3.a, 3.b
Between 5.1 percent and 10 percent error in
contract data for the given reporting section/data
element(s)
4
1.b, 1.c, 1.d, 2.a, 3.a, 3.b
5 percent error or less in contract data for the given
reporting section/data element(s)
5
5.2.3.
DV Response (DV
Findings
the DVC Reported on
Column 'F' in
Appendix B)
Guidance for Interpreting Standards and Making a Findings
Determination from Review of the EES
To ensure consistency with the review process, CMS lists these descriptions of the data sources and
criteria that DVCs use to determine findings for each of the DV standards/sub-standard.
5.2.3.1.
STANDARD 1
DVCs assess this validation standard at the reporting section level and use it to determine that all
source documents accurately capture required data fields and the SOs properly documented them.
Exhibit 9 describes the guidance for evaluation Standard 1, which illustrates an example of the
EES (Appendix B) and how to evaluate Standard 1. The DVC will assess this standard at the
reporting section level and determine a finding for each of the seven sub-standards contained in
Standard 1.
4
Standard 6 is not scored.
21
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 9. EXAMPLE ROWS FROM EES (APPENDIX B) AND GUIDANCE FOR STANDARD 1
Standard/
SubStandard/Sub-standard Description
standard
ID
1
Guidance
Database management and structure:
•
• Determine if the SO’s source documents (for example,
programming code, spreadsheet formulas, analysis plans,
A review of source documents
saved data queries, file layouts, process flows) accurately
(for example, programming
capture the data fields required for each reporting section
code, spreadsheet formulas,
under review and are documented with the necessary detail
analysis plans, saved data
and information to create data file sets and other outputs.
queries, file layouts, process
•
Ensure
that all source documentation is legible, descriptive,
flows) shows that all source
and
understandable,
including each of the following:
documents accurately capture
required data fields, and the SO • Standard Operating Procedures (SOPs) include detailed
workflows and processes related to managing, producing, and
has properly documented them.
tracking source documents.
.
• Titles and footnotes SOs used in programs and the reported
output is legible and correspond to HPMS reports and tables.
• If used, they properly documented macros.
• SOPs, file-naming conventions, dates of source documents
and output reports reflect application of version control.
• SOs reference data file locations correctly within source code
(such as, these files can be located using the references that
exist within the source code).
• Dated HPMS entries match the source document(s) to create
the data entered in the HPMS.
• Verify that SOs selected data at the proper level (for example,
either the contract or the PBP level).
• Check date ranges, demographic information, and eligibility
information to examine proper data filtering.
• Ensure that the DVC uses the current documentation, and it is
relevant to the time period of the RR.
Please note that DVCs should address Standards 1 and 2 concurrently since the evaluation of
source documents directly impacts the quality of the underlying data and vice versa so that DVCs
accurately identify, process, and calculate elements for each reporting section. For example, the
DVC should ensure that all source documentation (file layouts, data dictionaries, programming
code, work instructions, SOPs, etc.) is available, and it allows for the complete validation of each
reporting section.
5.2.3.2.
STANDARD 2
DVCs assess whether SOs accurately identified, processed, and verified (including calculations for
the number of members, claims, grievances, procedures, etc.) the source document’s data
populated for each reporting section for this validation standard. Each DVC should ensure that it
has staff fluent in the SO’s programming language (i.e., SQL, SAS, or Microsoft VBA). Standard 2
requires the DVC to assess reporting section-level findings for Sub-Standard 2.a. Exhibit 19
illustrates an example of the EES (Appendix B) and guidance for Standard 2, Sub-Standards 2.a.
Since the DVC conducts the DV reviews at the contract level, for the reporting sections that require
reporting at the PBP level, if the DVC finds that the SO incorrectly identified, processed, or
22
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
calculated the data reported for any of the PBPs included under a contract, then the DVC must
assign a 1, 2, or 3 finding in the EES (Appendix B) for the entire contract for the applicable substandard or data element (for Sub-Standard 2.a).
Exhibit 10 provides several examples of how to review source codes and evaluate the integrity of
the data. However, the DVC may use other methods of DV to ensure a comprehensive and
complete review of the source codes and census/sample data. The DVC must clearly document all
errors they found in programming codes, referring to the program examined, the precise location in
the program, the nature of the error, and the impact of the error in the “Data Sources and Review
Results” section of the EES (Appendix B). Likewise, they must clearly document in the applicable
section of the EES (Appendix B) any evidence from the review of census/sample data that leads to
a negative finding.
While DVCs should carefully inspect the source codes to detect most errors and outliers in the reported data, they
should conduct a careful review of the census or sample data gathered from the SO to minimize the chance that
they didn’t detect a programming error. DVCs can check many of the same items as they review the source codes
and analyze the extracted data sets. The DVC must also review that QA checks/thresholds are applied by the SO
to detect outlier or erroneous data.
23
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 10. EXAMPLE OF THE EES (APPENDIX B) AND GUIDANCE FOR STANDARD 2, SUB-STANDARDS 2.A.
Standard/
Substandard
ID
2
Standard/Sub-standard Description
Guidance
Database Extraction Function:
• Review of source data, preliminary data
sets, and interim data sets (for
example, programming code,
spreadsheet formulas, analysis plans,
saved data queries, file layouts,
process flows) indicates the following:
o SOs accurately identify, process,
and verify the population for each
reporting section (including
calculations for the number of
members, claims, grievances,
procedures, etc.).
o They apply QA checks/thresholds
to detect outlier or erroneous data.
Assess the programming code to determine if SOs
extracted the data from the system properly and if the
calculations SOs used in reporting data to CMS are
accurate according to the reporting section applicable
under review.
A thorough review of source codes must examine every line
of code to ensure the following for each reporting section
under review:
• Data are extracted from the appropriate source system:
o Verify that all data sets found in the programming
code can be traced back to the appropriate source
data sets.
• SOs filtered data sets correctly:
o Verify that SOs applied data inclusion and exclusion
criteria according to the reporting section criteria.
o For example, proper inclusion of records would
ensure that source code indicates that only those
records falling within the reporting period date range
are included in the reported data. An example of
correct exclusion would document source code that
indicates beneficiaries are not eligible for a particular
benefit (for example, Medication Therapy
Management Program).
o Individual data sets are joined or merged correctly
(this is especially important when moving data from
source data sets to intermediate data sets):
o Verify that SOs used the correct key data field to
generate the new data set and that they used the
correct type of join (or data merge) to avoid creating
duplicate records or improperly combining records
from various data sets.
• Data set progression is accurate:
o Verify that required data fields in both the source
and final stage files allow for file comparison and
understanding of data production from source
system through the final stage file.
o If full census data is not extracted, verify that the
sample size is sufficient and representative of the
population of interest.
o While the Data Extraction and Sampling Instructions
provide minimum sample sizes, DVCs often will
need larger data sets to check for errors that occur
infrequently. Statisticians should rely on standard
statistical practices when determining the proper
sample sizes that any estimates generated are
statistically significant.
• All preliminary data sets and interim data sets are
accurate:
o Verify that each data set is consistent with the
standard/sub-standard description.
24
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
5.2.3.3.
STANDARD 3
Standard 3 contains two Sub-Standards. Sub-Standard 3.a and 3.b requires the DVC to assess
data element-level findings which examines each data element at their final stage for compliance
with the data elements reported into HPMS by the SO. In addition, DVCs assess Standard 3 at
the data element-level for reporting sections that are manually entered into the HPMS Plan
Reporting Module (PRM) because it confirms that there were no manual data entry errors for
each data element and that the SO accurately captures data by applying data integrity/logical
checks, and for reporting sections reported as file uploads, it confirms that the SO used the
correct file layout. Exhibit 11 shows an example of the EES (Appendix B) and guidance for
evaluating Standard 3.
EXHIBIT 11. EXAMPLE ROWS FROM EES (APPENDIX B) FOR STANDARD 3
Standard/
Substandard
ID
3
Standard/Sub-standard Description
Guidance
Organization implements policies and procedures in Determine who is responsible for entering/uploading
their final stage data sets for submission into HPMS, data into CMS systems for each reporting section
including the following:
under review and if the SO wrote work instructions or
policies and procedures for the entry or submission
• Expected counts - Data elements are valid,
of the Part C and Part D RR.
complete, and accurate according to the source
document that SOs use to upload/enter data into Evaluate Sub-Standard 3.a and 3.b by performing the
following actions:
the HPMS; ranges of data fields are verified; all
calculations (for example, derived data fields) are
• Assess the Submission Activity Report from the
verified; they properly address missing data;
HPMS PRM by performing the following:
reporting output matches corresponding source
o Request a copy of the contract’s Submission
documents (for example, programming code,
Activity Report from the SO: This report
saved queries, analysis plans); they appropriately
displays information about the original
apply version control of reported data elements.
submission and all subsequent resubmissions
• Organization accurately captures data by applying
for a particular contract or contracts. The
data integrity/logical checks; They apply QA
report also displays Reporting Period, Contract
checks/thresholds to detect outlier or erroneous
Number, Plan ID, Submission Version, Due
data.
Date and Date Submitted for each section.
For either of the above scenarios, the DVC must
clearly document the circumstances in the “Data
Sources and Review Results” section of the EES
(Appendix B).
Assess the census/sample data provided by the SO
to determine each of the following for each reporting
section under review:
• SOs selected data records properly:
o Perform frequency calculations to list all
unique occurrences of data fields pertinent to
the calculation of the data element to verify
they contain values within an acceptable range
for the data field.
o Calculating frequency of occurrence for certain
data fields might also alert the DVC to obvious
mistakes in the data extraction.
25
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 11. EXAMPLE ROWS FROM EES (APPENDIX B) FOR STANDARD 3 (CONT.)
Standard/
Substandard
ID
3
Standard/Sub-standard Description
Guidance
• SOs merged or joined individual data sets correctly to
create final stage data sets:
o Sample a few records, when individual data sets
are available (most likely for intermediate data
sets), from the individual data sets to confirm that
they were joined properly.
o Check for duplicate records and determine if
record counts for the component data sets agree
with those found in the merged data set.
• SOs calculated all data elements accurately:
o Recalculate the data fields that the SO used to
calculate the data elements and refer to CMS
guidance documents and TS.
o Calculate sums of the individual records within
each reporting section to ensure that they equal
those reported to CMS.
o Compare the data file created for submission to
CMS with the data entered in the HPMS to
confirm no manual data entry errors. For file
uploads, confirm that the data file adheres to the
record layout specified in HPMS.
For the reporting sections that require reporting at the
PBP level, if the DVC finds that the SO did not
accurately enter and/or upload data reported for any of
the PBPs included under a contract, then the reviewer
must assign a “No” finding in the EES (Appendix B) for
the entire contract for the applicable data element(s) for
Sub-Standard 3.a and 3.b.
If a reporting section requires both a file upload and
data entry, both occur for an SO to meet Sub-Standard
3.a and 3.b.
DVCs to confirm that the data does not have any logical
errors. The DVC must perform data integrity checks to
verify at the data element level. These data integrity
checks include confirming that a data element does not
include outlier records.
5.2.3.4.
STANDARD 4-7
Standards 4 through 7 assess policies and procedures for periodic data system archiving and
updates; an SO will most likely have these policies and procedures in place for an entire reporting
section, as opposed to having them in place for only certain data elements. Exhibit 12 shows
example rows from the EES (Appendix B) and guidance for Standards 4 through 7.
26
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 12. EXAMPLE ROWS FROM EES (APPENDIX B) FOR STANDARDS 4 THROUGH 7
Standard/
SubStandard/Sub-standard Description
Standard
ID
4
All source, intermediate, and final stage data sets
and other outputs relied upon to enter data into
CMS systems are archived.
Guidance
• Determine if the SO has a policy or procedure for
archiving all source, intermediate, and final stage data
sets relied upon to enter data into CMS systems and
confirm that the SO implemented this policy for the
reporting section under review.
5
Organization implements policies and procedures
for the following:
• Periodic data system updates (for example,
changes in enrollment, provider/pharmacy
status, and claims adjustments).
• Restoring data in each data system (for
example, disaster recovery plan).
6
If organization’s data systems underwent any
• Review documentation on data system changes and
changes during the reporting period (for example,
determine if changes to an SO’s data system
because of a merger, acquisition, vendor change or
adversely impacted data reported by conducting the
upgrade):
following activities:
o Determine if there were any changes to data
• Organization provided documentation on the
sources used for data collection and storage, data
data system changes and, upon review,
processing, analysis, and reporting for the
changes were implemented correctly and did not
adversely impact the reported data.
reporting section under review.
• Determine if the SO has policies and procedures in
place for performing periodic updates for each data
system used for the reporting section under review
that ensures reported data are accurate and timely.
• Determine if the SO implements and adheres to the
policies and procedures referenced above (such as,
was any data for the reporting section under review
negatively impacted by a failure to implement or follow
these policies and procedures?).
• Determine if the SO has policies and procedures in
place for archiving and restoring data in each data
system used for the reporting section under review
that ensures timely data submission or re-submission
in the event of data loss.
• Determine if the SO implements and adheres to the
policies and procedures referenced above (such as,
was any data for the reporting section under review
negatively impacted by a failure to implement or follow
these policies and procedures?).
o Determine if data system changes were the root
cause of any outlier notices received from CMS for
the reporting section under review.
o Determine if the SO implemented any process or
quality improvement activities during the reporting
period specifically related to the data system
change for the reporting section under review.
o Determine if any changes to data systems during
the reporting period adversely impacted the
validity of the SO’s data.
27
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
EXHIBIT 12. EXAMPLE ROWS FROM EES (APPENDIX B) FOR STANDARDS 4 THROUGH 7 (CONT.)
Standard/
SubStandard/Sub-standard Description
Standard
ID
7
If data collection and/or reporting for this reporting
section is delegated to another entity:
• Organization regularly routinely monitors the
quality and timeliness of the data collected
and/or reported by the delegated entity or first
tier/downstream contractor.
5.3.
Guidance
• Assess the following if data collection and/or reporting
for a reporting section is delegated to another entity:
o Determine if the SO has policies and procedures
in place for overseeing the delegated entity’s
reporting process/results for the reporting section
under review.
o Determine if the SO implements and adheres to
the policies and procedures referenced above
(such as, was any data for the reporting section
under review negatively impacted by a failure to
implement or follow these policies and
procedures?).
o Plans are not expected to replicate the delegated
entity’s process and recalculate all their numbers
but are expected to have policies and procedures
in place for routine monitoring to ensure quality
and timeliness of the data. CMS expects that
these policies and procedures are implemented as
frequently as needed to verify the delegated
entities’ reporting.
• SOs are responsible for a delegated entities
calculations and numbers and therefore if they’re
incorrect, the responsibility ultimately falls on the SO.
SUBMIT DV REVIEW FINDINGS VIA HPMS PRDVM
5.3.1.
Review Draft Findings with Sponsoring Organization
The SO and DVC should build time into the April-June 15 DV schedule to allow sufficient time for the
review of the DV findings. Any issues identified during the review of the DVC findings must be resolved
prior to the data validation contractor’s June 15 deadline for submitting findings to CMS. To maintain
independence of the DV audit completed by the DVC, the review of draft findings between the SO and the
DVC should be limited to technical clarifications and/or corrections required for factual accuracy of the DV
review. SOs should not provide new data or request changes to the findings that are not evidence-based.
The formal appeals process described in Section 6.2.3. remains an avenue for dispute resolution.
5.3.2.
Submit DV Review Findings via HPMS PRDVM
Following the conclusion of the DV review and the finalization of findings, the DVC must report the findings
by uploading the EES (Appendix B) to CMS via the PRDVM in HPMS by June 15th. The PRDVM Quick
Reference Guide contains instructions for using this module, which is available in the PRDVM. The EES
(Appendix B) includes review results and/or data sources that were reviewed for each standard or
substandard, as well as the Yes, No, or Not Applicable finding, or a 1-5 Likert scale associated with each
standard or substandard.
DVCs should also indicate which extraction method (full census or sample) was used for each standard.
28
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
6.
POST-DV ACTIVITIES
6.1. COMPILE ARCHIVE OF DV WORK PAPERS
The DVC must prepare a complete archive of work papers associated with the annual DV and provide it to the
SO. At a minimum, this archive must contain the documentation described in Exhibit 30. The DVC should also
retain a complete copy of this archive in accordance with its contract with the SO.
When the SO receives the archive from the DVC, the SO must add the documentation of its DVC selection
process to the archive, including how its chosen DVC meets the minimum qualifications, credentials, and
resources listed in Appendix A. Federal regulations require the SO to retain this complete archive for the 10-year
retention period. CMS may ask for copies of Appendix I and work papers associated with the annual DV. CMS
reviews the information to verify compliance with the data validation process pursuant to 42 CFR §423.514(j) and
OMB control number 0938-1115.
EXHIBIT 13. MINIMUM DOCUMENTATION REQUIRED FOR DV ARCHIVE
DV ARCHIVE
•
Documentation of Data Validation
•
Contractor Selection Process
•
Documentation of completion of CMS Data
Validation Training for all staff assigned to the data
validation team
No data
•
Copies of any formal presentations during site
visit
•
Notes on staff interviews and demonstrations
during site visit
•
Census/sample data
•
Completed OAI (Appendix E), including all
documentation provided in response to OAI
•
Additional documentation provided by SO
during/after site visit
•
Final Site Visit Agenda
•
Draft findings in EES (Appendix B)
•
Completed Sign-in Sheets from site visit (if used)
•
•
Final IDG used during site visit
Notes on issues resulting in changes to draft
findings
•
Final EES (Appendix B)
6.2. REVIEW FINAL DV RESULTS
6.2.1.
Pass/Not Pass Determination
For each of the standards, sub-standards, and/or data elements, the DVC records the appropriate
“Yes/No” finding or a score using a 1-5 Likert scale. The EES findings are submitted to CMS via the
PRDVM in HPMS. Then, CMS analyzes the findings submitted by the DVC and applies a scoring
methodology to make Pass/Not Pass determinations.
CMS assigns percentage points for each finding and these can vary depending on the standard, substandard, and/or data element being scored in each section. A “No” or 1, 2 or 3 finding, however, will
always result in a score of zero percentage points. The Data Validation Pass/Not Pass Determination
Methodology (Appendix J) identifies the individual score CMS has assigned to each standard, substandard, and/or data element for all reporting sections. Additionally, after the DVC has submitted findings
in HPMS, the SO can view the findings via the “Review Data Validation Findings Report”.
SOs can also view their final DV results in HPMS. To access this page, from the top menu select
“Monitoring,” then “Plan Reporting Data Validation.” Select the appropriate contract year. Select the
29
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
PRDVM Reports. Select “Score Detail Report.” Select the applicable reporting section. If you cannot see
the Plan Reporting Data Validation module, contact [email protected].
6.2.2.
Passing DV – Minimum threshold
CMS has established 95% as the passing DV threshold for all reporting sections. SOs may view their
individual contract’s validation results in HPMS.
6.2.3.
SO Appeal of DV Determination
The SO may submit an appeal if they disagree with the DVC’s findings. SOs must submit appeals 5
business days following the DV deadline.
An SO has the right to appeal:
•
Reporting section score of less than 95%
•
Non-compliant data validation standards/sub-standards such as, a "No" or a 1, 2, or 3 on
the 5-point Likert scale in the specific data element's data validation
If the SO wishes to appeal, it must submit an appeal 5 business days following the DV deadline to CMS.
Submissions must be sent to CMS via the [email protected] resource
mailbox and must contain all the following information in order to be considered:
•
Email subject line must state: “Data Validation: Appeal”
•
Content of email must include the information below, in list format and in the following
order:
•
▪
Name of SO
▪
CMS contract number(s)
▪
SO’s contact name, title, phone number and email address
▪
Name of DVC organization
For each appeal, list the following information:
▪
Justification for appeal
▪
Include as attachment any documentation supporting the justification for appeal.
The documentation must have been in existence at the time of the DV. For
example, if after the DV, the SO resubmits corrected data, revises a policy and
procedure, or corrects a programming code that caused it to improperly calculate
reported data; the SO cannot submit documentation of these corrections to
appeal.
Once CMS receives the appeal, we will carefully consider the justification and any supporting
documentation. CMS has not established a timeframe for the consideration of SO appeals.
30
Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual
Glossary of Terms
Acronym
Description
CEO
Chief Executive Officer
CMS
Centers for Medicare and Medicaid Services
CY
Contract Year
DV
Data Validation
DVC
Data Validation Contractor
EES
Examination Engagement Standards
EUA
Enterprise User Administration
HIPAA
Health Insurance Portability and Accountability
HPMS
Health Plan Management System
IDG
Manual
Interview Discussion Guide
Procedure Manual
OAI
Organizational Assessment Instrument
PRDVM
Plan Reporting Data Validation Module
PRM
Plan Reporting Module
RR
Reporting Requirements
SO
Sponsoring Organization
TS
Technical Specifications
31
| File Type | application/pdf |
| File Title | Medicare Part C and Part D Reporting Requirements Data Validation Procedure Manual |
| Subject | Data Validation Procedure Manual |
| Author | Centers for Medicare and Medicaid Services |
| File Modified | 2023-07-26 |
| File Created | 2023-07-26 |