CMS-10406.Supporting Statement - Part B

Supporting Statement – Part B

Collections of Information Employing Statistical Methods

1. Potential Respondent Universe and Sample Selection Method

The sample will be composed of a geographically-stratified random sample of 2,000 home health agencies (HHAs) with one claim drawn from each provider.  The home health agency, referring provider, and beneficiary associated with each claim will be interviewed. Interviews will include questions and collection of documentation aimed at confirming that the beneficiary received the service prescribed, that the service was medically necessary, that the beneficiary was eligible to receive the service and that there is no evidence of intent to defraud Medicare on the part of the HHA and/or the referring provide.

To reduce data collection issues related to beneficiary recall of services rendered, the sampling strategy for the Probable Fraud Measurement Pilot will focus on recently rendered services. In addition, the pilot will be conducted in two waves, with the second wave drawn approximately three months after the first wave. This will minimize the gap between each claim’s submission date and the date of data collection.

The sample for each wave will be drawn in two stages. In the first stage, a home health agency will be randomly selected; in the second stage, one claim will be randomly selected for each agency. The sampling frame for the first stage consists of all home health agencies that submitted claims (defined as paid final claims appearing in Medicare’s Common Working File) during the sampling period (defined as one quarter prior to the sampling date).¹ The sampling frame for the second stage consists of all paid claims submitted by these home health agencies during the sampling period. The universe of potential respondents for the referring provider interview consists of all providers who were reported as the attending provider on claims submitted during the sampling period, while the universe of potential respondents for the beneficiary interview consists of all beneficiaries listed as recipients of the home health services on claims submitted during the sampling period.

The sample will be stratified first by seven zones, and the total sample will be drawn in proportion to the number of providers in the universe for each zone. Each zone will be further subdivided into two sets of areas representing low and high cost areas for information collection, based on information provided by the contractors conducting the interviews. Specifically, a low-cost area is defined as a location that the contractor can reach by car without needing to make an overnight stay, and a high-cost area is defined as a location that would require the contractor to stay overnight if traveling by car. To decrease the cost of collecting data for the pilot, the low-cost areas will be oversampled. Given knowledge of the stratification used to draw the sample and the universe of Medicare home health claims, the sampling frame can be used to produce nationally representative populations of Medicare home health claims, beneficiaries and providers.

Table 1: Geographic Strata

Strata	States
1	AS, CA, GU, HI, MP, and NV
2	AK, AZ, IA, ID, KS, MO, MT, NE, ND, OR, SD, UT, WA and WY
3	IL, IN, KY, MI, MN, OH and WI
4	CO, NM, OK and TX
5	AL, AR, GA, LA, MS, NC, SC, TN, VA and WV
6	CT, DC, DE, MA, MD, ME, NH, NJ, NY, PA, RI and VT
7	FL, PR and VI

Expected response rates for home health agencies and referring providers are close to 100 percent because these entities are required to respond to requests for documentation. Under section 1833(e) of the Social Security Act, the Department of Health and Human Services has the authority to obtain sufficient information to substantiate claims for reimbursement. Additionally, under 42 C.F.R. § 424.516(f)(1), providers who furnish home health services are required to maintain documentation for 7 years from the date of service and to provide access to that documentation upon the request of CMS or a Medicare contractor. Under 42 C.F.R.424.516(f)(2), the same requirements apply to physicians who refer patients for home health services.

Because beneficiary participation is voluntary, response rates in the beneficiary interview are likely to be considerably lower. Previous efforts to collect similar information from beneficiaries by mail or phone, conducted by the Office of Inspector General, have had response rates ranging from 36 percent to 55 percent.² The Home Health Consumer Assessment of Healthcare Providers and Systems Survey, which surveys the home health population regarding their perception of quality of care issues, is expected to have response rates of 39 percent, 46 percent, and 52 percent for mail, telephone, and mixed modes respectively.³ The use of in-person data collection for this pilot may increase response rates, so estimates of response rates using these sources are likely to be conservative.

2. Information Collection and Statistical Procedures

Most interviews will be unannounced and in-person. Trained and experienced interviewers with knowledge of the Medicare program will conduct in-person unannounced interviews of home health agencies and collect documentation from those agencies. Interviews with all referring providers will occur by appointment, but the interviewers will not disclose to providers in advance the purpose of the interview. The pilot will use a mixed mode for referring provider interviews; half of the interviews will be conducted in-person and half by telephone. Beneficiary interviews will be unannounced and in-person. The pilot uses unannounced interviews when feasible to reduce the opportunity for fraudulent providers to alter or fabricate records or to coach beneficiaries in answering questions. As described in Part A of this Supporting Statement, a member of a Review Panel will use information from the interviews in concert with other information to determine whether a sampled claim meets the pilot’s definition of probable fraud.

The estimated rate of payments made based on claims that are probable fraud will be calculated for the population of home health claims by using standard methods to develop weights for each sampled claim. In the non-stratified version of this two-stage design, the sampling procedure described above implies a weight for sample claim i, denoted by ω_i, equal to (1/n)*N_p*N_k,, where n is the number of claims in the sample, N_p is the number of providers in the population, and N_k is the number of claims in the population for provider k (the rendering provider on sample claim i).⁴

To estimate the total amount of payments associated with probable fraud in the population using sample data,⁵ calculate

where n is the number of claims in the sample, Y_i is the payment amount on claim i, p_i is equal to 1 if the claim is classified as probable fraud and 0 otherwise, and ω_i is the weight for claim i. The estimated rate of payments based on probable fraud in the population is then given by

where P is the total amount of payments in the population of claims.

The variance of the estimated rate of payments based on probable fraud is given by

To determine the sample size required for the pilot, the pilot sponsors examined a series of simulations estimating the variance of the rate of probable fraud payments in the sample under a variety of assumptions, including different assumptions regarding the relationship between probable fraud and payment amount. The simulations indicate that with a total sample size of 2,000 claims (i.e., 2,000 home health agencies, 2,000 referring providers, and 2,000 beneficiaries), the width of a 95 percent confidence interval around the probable fraud rate would be two percentage points under the assumptions that 7.5 percent of claims in the population represent probable fraud and that the amount of likely fraudulent payment is equal to the full payment amount on the claim.⁶ CMS has determined that this sample size will provide a sufficiently accurate estimate of probable fraud in home health payments. The sample size was chosen to balance conflicting factors, including the level of effort required to collect information from the three parties for each claim and the need for a representative national sample.

3. Methods to Maximize Response Rates

Because home health agencies and referring providers are required by 42 C.F.R. § 424.516(f)(1)-(2) to provide documentation supporting payment, non-response issues are expected to be negligible for these respondents. Providers who refuse to provide documentation or are unavailable during reasonable business hours in multiple instances may be in violation of Medicare’s conditions of participation; lack of response is therefore crucial information that will be weighed heavily in determining whether a claim represents probable fraud.

The pilot design seeks to maximize beneficiary response rates while maintaining the voluntary nature of beneficiary participation. Interviewers will make two attempts to contact beneficiaries in person and leave contact information to encourage follow-up beneficiary response. At the start of each interview, beneficiaries will be given a letter of authorization that describes the purpose of the pilot, states who is collecting the information, and provides contact information should the beneficiary have questions or require additional information. Interviewer scripts also contain the information supplied in this letter. Interviewers will be provided with and undergo training on responses to frequently asked questions that cover likely beneficiary concerns. Prepared responses include confirming that beneficiary response is voluntary, that benefits will not be affected by responses, and that personal health information will be protected. Interviewers will also undergo training to ensure that they can answer beneficiary questions regarding how the information collected will be used.

To ensure sufficient accuracy and reliability of information collected, interviews will be conducted by contractors or CMS staff that have extensive experience interviewing beneficiaries and providers as well as experience collecting supporting documentation required by Medicare. Contractors and CMS staff will conduct their interviews as part of their normal course of work.

4. Tests of Procedures

CMS tested the data collection procedures and instruments in December 2011 by using the proposed protocols to collect information from nine beneficiaries, nine home health agencies, and nine referring providers. Nine home health agencies were randomly selected from the Los Angeles area for the testing, which was conducted by trained CMS investigators in the Center for Program Integrity (CPI) Los Angeles Field Office.

During the test the investigators successfully interviewed all nine home health agencies and all nine referring providers included in the sample. They successfully interviewed three of the nine beneficiaries in the sample. Investigators reported that interviews with beneficiaries lasted about 15 minutes. Interviews with home health agencies lasted between 1 and 1.5 hours, and interviews with referring providers lasted about 45 minutes. Improvements were made to both the data collection instrument and protocols based on feedback from investigators.

CMS will conduct a pre-test of the full probable fraud pilot methodology for a small sample of 130 claims to ensure that review panel members have adequate information to make consistent probable fraud determinations. These claims will be drawn from a limited number of geographic areas selected to increase the likelihood that the small sample includes substantial numbers of both probable fraud and non-probable fraud claims. Pre-test sample claims will undergo the full data collection process described in the pilot design.

CMS will use these 130 claims to test all components of the probable fraud methodology and make any needed adjustments to ensure the success of the full pilot. First, once data collection is complete, the review panel will enter a training period in which they will use 30 of the claims from the pre-test sample to refine the review instrument. At the conclusion of this training period, each of the 100 remaining cases will undergo independent review by two randomly chosen panel members. After this double-blind review is complete, CMS will measure inter-rater reliability of the two reviewers using Cohen’s Kappa, a common measure of inter-rater reliability in medical and social science research; a Kappa value of 0.60 or above is generally considered an indicator of a good level of agreement. Next, CMS will convene the review panel members for a facilitated session to provide feedback on whether there were any additional information sources that would be helpful to assessing each case. Based on the feedback from the review panel, the inter-rater reliability rating, and other lessons learned during the pre-test, CMS will make any necessary adjustments to the pilot methodology.

5. Individuals Consulted

The sampling design was prepared by Acumen, LLC. The pilot design as a whole, including the sampling design, has been reviewed by the CMS/ASPE Executive Committee overseeing the pilot as well as by a Technical Expert Panel. The members of the Technical Expert Panel include:

• Dr. Laurie Feinberg, Department of Justice

• Dr. Floyd J. Fowler, University of Massachusetts-Boston

• Dr. Steven C. Hill, Agency for Healthcare Research and Quality (AHRQ)

• Dr. Michael Larsen, George Washington University

• Dr. Jennifer Madans, National Center for Health Statistics (NCHS)

• Dr. Adrian Oleck, formerly of AdminaStar

• Dr. Malcolm Sparrow, John F. Kennedy School of Government, Harvard University

Additional information about this information collection may be obtained from Kelly Gent, Deputy Director, Data Analytics and Control Group, Center for Program Integrity, (410) 786-0198, [email protected].

1 Depending on the date that the sample is drawn, the sampling time period may be adjusted to account for seasonality in claims submission.

2 United States Department of Health & Human Services Office of the Inspector General, “Beneficiary Awareness of Medicare Fraud: A Follow-up,” 2001, and “Medicare Payments for Power Wheelchairs,” 2004.

3 United States Department of Health & Human Services Centers for Medicare & Medicaid Services, “Home Health Care CAHPS Survey Protocols and Guidelines Manual,” September 2010. Available at: https://homehealthcahps.org/SurveyandProtocols/SurveyMaterials.aspx#catid1

4 The stratified random sample that will be used for this pilot is the combination of a series of simple random samples, with weights adapted accordingly.

5 Calculating the rate of probable fraud in claims follows an analogous procedure.

6 Confidence intervals produced by the simulations are not symmetric.

4