REV_6 DRAFT Supporting_Statement_20100826_508

Supporting Statement for Paperwork Reduction Act Submissions:

Medicare Part C and Part D Data Validation (42 C.F.R. §422.516(g) and §423.514(g))

1. Background

Organizations contracted to offer Medicare Part C and Part D benefits are required to report data to the Centers for Medicare & Medicaid Services (CMS) on a variety of measures. In order for the data to be useful for monitoring and performance measurement, the data must be reliable, valid, complete, and comparable among sponsoring organizations. To meet this goal, CMS is developing reporting standards and data validation specifications with respect to the Part C and Part D reporting requirements. These standards and specifications will provide a review process for Medicare Advantage Organizations (MAOs), Cost Plans, and Part D sponsors to use to conduct data validation checks on their reported Part C and Part D data. The data validation will be primarily “retrospective,” first occurring in 2011 for 2010 data. In order to ensure the independence of the data validation, organizations will not use their own staff to conduct the data validation. Instead, MAOs, Cost Plans, and Part D sponsors will be responsible for acquiring external data validation resources. CMS will also provide a set of standards for selecting a data validation organization for MAOs, Cost Plans, and Part D sponsors to use in acquiring a data validation contractor. These standards describe the minimum qualifications, credentials, and resources that the selected data validation contractor must possess.

2. Justification

1. Need and Legal Basis

Under sections 1857(e) and 1860D-12 of the Social Security Act (“the Act”), CMS has the authority to establish information collection requirements with respect to MAOs and Part D sponsors. Under section 1857(e)(1) of the Act, MAOs are required to provide the Secretary with such information as the Secretary may find necessary and appropriate. Section 1857(e)(1) of the Act applies to Prescription Drug Plans (PDPs) as indicated in section 1860D-12. Pursuant to our statutory authority, we codified these information collection requirements in regulation at §422.516(g) and §423.514(g), respectively.

Consistent with our regulatory authority to collect information, we developed specific Part C and Part D reporting requirements to assist in monitoring the Part C and D programs, and to respond to questions from Congress, oversight agencies, and the public. These inquiries cover a variety of topics including costs, availability of services, beneficiary use of available services, patient safety, grievance rates, and other factors pertaining to MAOs and Part D Plans. We began collecting Part D information at the inception of the program. Over time, we have modified the data elements collected as we gained more experience with the program. The current Part D reporting requirements (OMB 0938-0992) may be accessed at http://www.cms.hhs.gov/PrescriptionDrugCovContra/08_RxContracting_ReportingOversight.asp.

We also require routine reporting of specific data elements by MAOs and 1876 Cost Plans. Beginning in January 2009, these organizations were required to report information across 13 measures ranging from benefit utilization to agent training and testing. Similar to the Part D reporting requirements, these measures are designed to enable us to monitor plan performance and to respond to inquiries. The current Part C reporting requirements (OMB 0938-1054) may be accessed at http://www.cms.gov/HealthPlansGenInfo/16_ReportingRequirements.asp#TopOfPage. In 2010, we suspended reporting requirements for two Part C measures and five Part D measures. There are currently eleven Part C measures and sixteen Part D measures.

In order for CMS to effectively use the data provided by MAOs, Cost Plans, and Part D sponsors, the data must be accurate, valid, reliable, and comparable across plans. Because we have received data of questionable validity from some Part D sponsors, we stated in the 2010 Call letter (http://www.cms.hhs.gov/prescriptiondrugcovcontra) that the agency "has received many inquiries from Congress, oversight agencies, and the public about costs, availability of services, beneficiary use of available services, patient safety, grievance rates, and other factors pertaining to MAOs and PDPs. However, to date, we have not been able to address many of these inquiries due to either an absence of data with respect to MAOs or, despite collecting over three years' worth of data, data of questionable validity submitted by Part D sponsors." Accordingly, to meet the goals of data validity, reliability, and comparability, we indicated in the Call Letter that, "to better enable CMS to respond to inquiries and manage our programs, sponsoring organizations should undertake a data validation audit on reported Part C and Part D data effective for 2010." Given the importance of the new Part C and Part D data reporting requirements, we are proposing to require MAOs and Part D sponsors to undertake an independent data validation audit in accordance with CMS specifications on reported Part C and Part D data that would be effective for CY 2011. We believe that only an independent data validation audit conducted by an external entity under contract to the MAO or Part D sponsoring organization would ensure that the results of the audit are in accordance with CMS specifications, that data used to develop plan performance measures are credible to other stakeholders, and that information used to respond to Congressional and public inquiries are reliable.

In the CMS-HHS proposed rule “Medicare Program: Policy and Technical Changes to the Medicare Advantage and the Medicare Prescription Drug Benefit Programs” (CMS-4085-P; RIN 0938-AP77), published in the Federal Register on October 22, 2009, we proposed to amend §422.516(g) and §423.514(g) to state that each Part C and Part D sponsor be subject to an independent yearly audit of Part C and Part D measures (collected pursuant to our reporting requirements) to determine their reliability, validity, completeness, and comparability in accordance with specifications developed by CMS.

In the proposed rule, we referenced our work with a contractor to develop data validation specifications to ensure that the goals of reliability, validity, completeness, and comparability are met at the conclusion of the data validation audit. These specifications focus on how organizations and sponsors compile the reported data, take into account appropriate data exclusions, and verify calculations, source code, and algorithms. In addition, they will be used to inform how the MAOs, Cost Plans, and Part D sponsors collect, store, and report data. These specifications will be utilized by the auditors hired by MAOs, Cost Plans, and Part D sponsors to conduct the data validation audits, the results of which will be provided to us.

The data collection instruments that will support the Part C and Part D data validation are contained in the appendices. Appendix 1 contains the “Findings Data Collection Form for Data Validation Contractors.” The Findings Data Collection Form is a tool for the data validation contractor (reviewer) to record the validation findings for each contract included in the scope of the review.  The form mirrors the content of the Data Validation Standards document, but allows the reviewer to record notes, data sources referenced, and findings for the different standards and criteria specified for a given measure.

Using the Findings Data Collection Form, the reviewer will conduct the review and record measure-level, and in some cases data element-level, findings for each measure’s standards. Once the findings have been finalized, the reviewer will share the findings with the organization and then submit the completed Findings Data Collection Form to CMS, who will evaluate the measure- or data element-level findings for each measure’s standards to derive an overall “Pass” or “Not Pass” determination.

Appendix 2 contains the “Organizational Assessment Instrument” (OAI). The OAI is a tool for the data validation contractor (reviewer) to understand organizations’ reporting processes and to request documentation that will be evaluated during the review process. The information collected in this OAI will help prepare the reviewer and will reduce resources required for the on-site portion of the review. While not mandatory, it is strongly recommended that organizations complete the OAI to add efficiencies to the review process. If an organization does not elect to complete the OAI, the reviewer will use the same tool to collect this information during the on-site review, extending the length of the review.

Appendix 3 contains the “Data Validation Standards.” This document includes general standards and measure-specific criteria that the data validation contractor (reviewer) will use to determine whether the organization’s data reported to CMS per the Part C/Part D Reporting Requirements are accurate, valid, and reliable.  Each measure’s data validation standards include identical instructions relating to the types of information that will be reviewed, a set of validation standards (also identical for each measure), and measure-specific criteria that are based on the applicable Part C/Part D Reporting Requirements Technical Specifications.

The reviewer must use these standards in conjunction with the Data Extraction and Sampling Instructions for Data Validation Contractors and the Findings Data Collection Form for Data Validation Contractors to evaluate the organization’s processes for producing and reporting the measures. It is strongly recommended that the reviewer and report owner/data provider use the Data Validation Standards documentation before and during the review of a measure to ensure that all applicable data fields are extracted for each measure. Upon review of the information and documentation provided by the organization and completion of the review, the reviewer will determine compliance with each of the standards and, using the Findings Data Collection Form, record the appropriate finding to each standard and criteria.

Appendix 4 contains the “Data Extraction and Sampling Instructions for Data Validation Contractors.” The purpose of this document is to provide guidance to the reviewer regarding drawing and evaluating census and/or sample files to support validation of Part C and Part D measures.

2. Information Users

Data collected from MAOs and Part D sponsors are an integral resource for oversight, monitoring, compliance and auditing activities necessary to ensure quality provision of the benefits provided by MA plans and Part D sponsors to enrollees. Data will be validated, analyzed, and utilized for trend reporting by CMS. If outliers or other data anomalies are detected, the CMS division with primary responsibility for overseeing the applicable reporting requirement will work in collaboration with CMS components for follow-up and resolution.

3. Use of Information Technology

Data validation entities contracting with MAOs and Part D sponsors will utilize the Health Plan Management System (HPMS) to submit or enter findings for 100% of the data elements and processes that will be validated to ensure the completeness, reliability, and accuracy of the Part C and Part D reporting requirements. HPMS is the current conduit by which the sponsoring organizations submit data to CMS; for example, application materials, bids, and formularies (if offering Medicare Part D). CMS and its subcontractors, in turn, communicate to MAOs and Part D sponsors regarding this information, including approval and denial notices and other related announcements through HPMS. HPMS, therefore, is already a familiar tool to MAOs and Part D sponsors. Access to HPMS must be granted to each user, and is protected by an individual login and password; electronic signatures are unnecessary.

4. Duplication of Efforts

This collection does not contain duplication of similar information.

5. Small Businesses

There are no small businesses involved.

6. Less Frequent Collection

If the collection is not conducted or is conducted less frequently, the reliability, validity, completeness, and comparability of the Part C and Part D reporting requirements data will not be ensured. CMS could not confidently use the data for public reporting and the value of the data for just monitoring would be questionable.

7. Special Circumstances

As mandated by 42CFR §422.504 (d) and § 423.505(d), MAOs and Part D sponsors must agree to maintain for 10 years books, records, documents, and other evidence of accounting procedures and practices. CMS could potentially require clarification around submitted data, and therefore CMS may need to contact MAOs and Part D sponsors within 60 days of data submission.

8. Federal Register/Outside Consultation

The document will be forwarded to the Office of the DHHS Secretary for submission to OMB.

9. Payments/Gifts to Respondents

There are no payments/gifts to respondents associated with this information collection request.

10. Confidentiality

CMS will adhere to all statutes, regulations, and agency policies. CMS will not be requesting any beneficiary identification information for the Medicare Part C and Part D Data Validation program. Social security numbers will not be collected.

11. Sensitive Questions

CMS will adhere to all statutes, regulations, and agency policies.

12. Burden Estimates (Hours & Wages)

Based on our experience and in consultant with program experts, we developed an estimate of the hourly burden. We used May 2009 wage statistics supplied by the Department of Labor, Bureau of Labor Statistics (BLS) to develop estimates of direct wages, fringe benefits, and overhead costs. Two labor categories, analyst and manger, were used when calculating the cost burden. The estimated mean cost per hour is $62.84 for an analyst and $76.33 for a manager (wages, fringe benefits, and overhead). Travel costs for the data validation contractor, including airfare, lodging, car rental, and meals were estimated to be $4,150 per review.

Figure 1 summarizes the estimated cost burden for the three scenarios where an organization reports data for 1) Part C data only, 2) Part D data only, and 3) both Part C and Part D data. Travel time for data validation contractors has been factored into the estimate burden.

Figure 1. Burden Estimates for Typical Sponsoring Organization

Assumption/Estimate	Part C Only (1 Contract)	Part D Only (1 Contract)	Part C and Part D (1 Contract)
Hourly Wage*: Analyst (a)	$62.84	$62.84	$62.84
Hourly Wage*: Manager (b)	$76.33	$76.33	$76.33
Sponsor Org. Analyst Hours (c)	111.00	100.00	199.00
Sponsor Org. Manager Hours (d)	132.75	129.00	177.75
Total Sponsor Hours (e) (c) + (d)	243.75	229.00	376.75
DV Contractor: Analyst Hours (f)	233.00	208.00	433.00
DV Contractor: Mgr. Hours (g)	84.75	80.00	142.75
Total DV Contractor Hours (h) (f) + (g)	317.75	308.00	575.75
Sponsor Org.: Analyst Cost (i) (a) x (c)	$6,975	$6,284	$12,505
Sponsor Org.: Manager Cost (j) (b) x (d)	$10,133	$9,847	$13,568
Total Sponsor Org. Cost (k) (i) + (j)	$17,109	$16,131	$26,073
DV Contractor: Analyst Cost (l) (a) x (f)	$14,641	$13,070	$27,209
DV Contractor: Mgr. Cost (m) (b) x (g)	$6,469	$6,107	$10,897
DV Contractor: Travel Exp. (n)	$4,150	$4,150	$4,150
Total DV Contractor Cost + Travel Exp. (o) (l) + (m) + (n)	$25,261	$23,327	$42,256
Total Cost Burden per Review (p) (k) + (o)	$42,369	$39,458	$68,329
Additional Hours per Additional Contract (q)	30.04	26.70	56.74
Additional Cost per Additional Contract (r)	$1,929	$1,714	$3,643

*Includes fringe benefits and overhead costs

Estimates for the total cost burden across all Part C and Part D sponsoring organizations relied upon the number of measures validated, the number of sponsoring organizations, and the average number of contracts per organization as shown in Figure 2.

Figure 2. Part C and Part D Contracts per Sponsoring Organization

Sponsoring Organization Type	Number of Measures Validated	Number of Organizations	Number of Contracts	Avg. Number of Contracts per Org.
Part C Only	9	15	18	1.20
Part D Only	8	56	72	1.29
Part C and Part D	17	188	544	2.89

Estimates for the total cost burden across all organizations are shown in Figure 3 and factor in the average number of contracts per sponsoring organization.

Figure 3. Total Cost Burden by Sponsoring Organization Type

Assumption/Estimate	Part C Only	Part D Only	Part C and Part D
Total Number of Organizations (a)	15	56	188
Average Number of Contracts per Org. (b)	1.20	1.29	2.89
Hourly Wage*: Analyst (c)	$62.84	$62.84	$62.84
Hourly Wage*: Manager (d)	$76.33	$76.33	$76.33
Total Hours per Review: Sponsoring Org. Analysts (e)	111.00	100.00	199.00
Total Hours per Review: Sponsoring Org. Managers (f)	132.75	129.00	177.75
Total Hours per Review: DV Contractor Analysts (g)	238.40	214.96	529.39
Total Hours per Review: DV Contractor Managers (h)	85.36	80.78	153.59
Total Cost per Review: Sponsoring Org. (i) (c) x (e) + (d) x (f)	$17,108	$16,131	$26,073
Total Cost per Review: DV Contractor (j) (c) x (g) + (d) x (h)	$21,497	$19,674	$44,990
Travel Expense per Review (k)	$4,150	$4,150	$4,150
Total Cost per Review (l) (i) + (j) + (k)	$42,755	$39,955	$75,213
Total Cost: All Sponsoring Orgs., All Reviews (m) (a) x (i)	$256,621	$903,312	$4,901,690
Total Cost: All DV Contractors + All Travel Exp, All Reviews (n) (a) x (j) + (a) x (k)	$384,699	$1,334,145	$9,238,394
Total Cost: All Reviews (o) (m) + (n)	$641,319	$2,237,457	$14,140,083
Grand Total – All Reviews, All Orgs.	$641,319 + $2,237,457 + $14,140,083 = $17,018,860

Using the information from Figure 2 and Figure 3, the total estimated annual hours associated with the data validation review are 237,127.

Figure 4. Total Hours

Total Hours (Number of Organizations x Estimated Hours)	Part C Only	Part D Only	Part C and Part D	Total
Total Hours Sponsoring Org. Analysts	1,665.00	5,600.00	37,412.00	44,677.00
Total Hours Sponsoring Org. Managers	1,991.25	7,224.00	33,417.00	42,632.25
Total Hours DV Contractor Analysts	3,576.00	12,037.76	99,525.32	115,139.08
Total Hours DV Contractor Managers	1,280.36	4,523.68	28,874.92	34,679.96
Total Hours	8,512.61	29,385.44	199,229.24	237,127.29

13. Capital Costs

There is a one-time capital cost to develop the software that will allow data entry into HPMS. This is estimated at $100,000 or $0.1 million for 2010.

14. Cost to Federal Government

It will cost an estimated $100,000 to develop the module in the Health Plan Management System (HPMS) that will be used to enter the results of the data validation.

15. Changes to Burden

The total hours associated with the April 2010 burden estimate were 231,410, and the total cost was $32,080,682. The assumptions from the April 2010 burden estimate are listed in Figure 5.

Figure 5. April 2010 Burden Estimate Assumptions

Assumption	Procuring Contractor	Conducting Review
Hourly Wage (a)	$43.14	$194.21
Total Hours/Contract (b)	120	206
Total Contracts (c)	710	710
Total Hours (b x c)	85,200	146,210
Total Cost (a x b x c)	$3,675,528	$28,405,154
Total Hours (Procuring and Review)	231,410 (85,200 + 146,210)
Total Cost (Procuring and Review)	$32,080,682 ($3,675,528 + $28,405,154)

The updated total hours are 237,127, and the updated total cost is $17,018,860. This updated burden estimate methodology was changed to allow for a more detailed build-up of hours and cost. Assumptions of the updated methodology are outlined in Section 12. A list of critical assumptions that changed from April 2010 burden estimate is included in Figure 6 below.

Figure 6. Key Assumption Changes, Previous and Current Supporting Statement

Assumption	Previous Estimate	Current Estimate
Hourly Wage*: DV Contractor	$194.21	$62.84 (Analyst) and $76.33 (Manager)
Hourly Wage*: Sponsoring Organization	$43.14	$62.84 (Analyst) and $76.33 (Manager)
Travel Expenses per Review	Not Included	$4,150
Total Number of CMS Contracts (Part C and Part D)	710	634
Total Number of Sponsoring Organizations	Not used, estimate used number of contracts	259

*Includes fringe benefits and overhead costs

16. Publication/Tabulation Dates

This data collection is intended to ensure that MAOs and Part D sponsors are reporting data that are reliable, valid, complete, and comparable among organizations. The data collection should help organizations to improve reporting. CMS will also be tabulating the data to determine which contracts “pass” and which contracts “fail” the data validation. Contracts that “fail” may be required to develop a corrective action plan or a performance improvement plan, or be subject to other enforcement actions.

17. Expiration Date

This collection does not lend itself to displaying an expiration date.

18. Certification Statement

MAOs and Part D sponsors that have terminated their contracts before the start of the data collection year are exempt from this collection process.

Supporting Statement – Part B

Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sam­pling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corre­sponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

All Part C organizations and Part D sponsors are required to undergo the data validation review process. Because 100% of organizations will undergo the data validation process, sampling is not applied for respondent selection.

Within the data validation process, data validation contractors will be encouraged to collect the entire data set (the census) relied on by organizations to meet Part C and D reporting requirements. If collecting the census proves impractical due to an unusual time burden placed on organizations during data extraction, there is a sampling task that each organization will be required to perform in collaboration with the data validation review contractor. Each organization will draw an initial sample, at a minimum, of either 205 or 150 administrative records, depending on the Part C or Part D measure. Sample sizes may be larger and will be determined by the data validation contractor using standard statistical methodologies. All relevant records associated with these samples will then be selected for review (for example, all claims for a random sample of 205 members). In cases where the population is smaller than the required sample size, records for the entire population should be provided for evaluation.

This sampling process is described in detail in the document titled “Data Extraction and Sampling Instructions for Data Validation Contractors.”

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the pur­pose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

For measures requiring sampling, simple random samples are used in the data validation review. The underlying standard is a quantifiable error rate in key fields. This is assumed to have a binomial distribution. The sample sizes are designed to detect error rates of 15% or more, with a one-tailed Type I error rate (α)=.05, except for samples based on eligibility. In those cases, because more confidence is needed, α is set at .025. A standard normal approximation to the binomial distribution is used to establish critical values. A finite population correction (FPC) factor has been included in sample size calculations. The variation formula below is solved for n to obtain sample size.

,

where is the desired precision (5%), N is the number in the population, p is the assumed proportion (.15), Z is the appropriate critical value from the normal curve (either 1.645 or 1.96, depending on α and n is the sample size.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sam­pling, a special justification must be provid­ed for any collection that will not yield 'reliable' data that can be generalized to the uni­verse studied.

Since extraction of the full census or use of the sampling process is required for all organizations, survey-related issues such as non-response bias are not applicable.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separate­ly or in combination with the main collection of information.

Pilot tests of all methodology, including sampling and all supporting documents, have been conducted with one Part C organization and one Part D sponsor.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Terry Lied, Ph.D.

Centers for Medicare & Medicaid Services

Medicare Drug Benefit and Part C and D Data Group

Division of Drug Plan Policy and Quality

410-786-8973

INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW:

This information has not been publicly disclosed and may be privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to receive the information. Unauthorized disclosure may result in prosecution to the full extent of the law.

18