CMS-10194.Supporting Statement Part B

Supporting Statement Part B

C. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

Rather than employing statistical sampling of SNPs for the mail survey, CMS intends to

conduct a survey of the entire population of plans operating in 2006, the evaluation period. Only about 200 such plans are operating in 2006,2 and they vary widely in the types of beneficiaries served (dual eligibles, institutionalized beneficiaries, and those with chronic illnesses). Plans also vary in other respects, for example, the use of so-called passive enrollment and the specific arrangements made with state Medicaid agencies. In consequence, there is no effective means of stratifying a sample of SNPs that would not require all or nearly all SNPs within those strata to be surveyed. CMS has therefore chosen to survey the entire population of SNPs. We will also not employ any statistical sampling of SNP enrollees for the focus groups. Although our ability to generalize from focus group responses will therefore be limited, the responses can both inform the evaluation and provide valuable input to SNPs. Focus groups will be conducted with beneficiaries from each type of SNP.

1. Potential respondent universe and any sampling or other respondent technique to be used

The universe of SNPs for the mail survey is approximately 200 as of April 2006. This is based on CMS’s final list of approved SNPs in 2006. Contact information for SNPs will come from CMS’s contract and plan contact databases maintained in the Health Plan Management System. We plan to survey all SNPs and so will not sample from this population. For each of the 15 focus groups, we plan to identify 18 beneficiaries from each SNP for which a focus group will be conducted, to achieve a group size of 8 to 10 enrollees. The sampling frame for focus groups will come from will come from lists of enrollees and contact information from SNP plans, which will provide the most up-to-date information on all SNP enrollees. The enrollees chosen for recruitment will not be a representative sample of SNP enrollees; however, they will be chosen based on time enrolled in the plan (at least six months if possible) and on additional criteria within SNP types. For example, for chronic care SNPs that target beneficiaries with several different diagnoses or disabling conditions, we will attempt to include members with each targeted condition.

2. Procedures for the collection of information

CMS wishes to estimate the proportion of SNPs using certain approaches to enrollment (such as the proportion using passive enrollment) to within +/-0.025 with 95% confidence for a proportion near 0.50. For a population of 200, the necessary sample to achieve this precision, employing the finite population correction, is 177. Because this number is so close to the total population size and because there will be some nonresponse in any event, CMS has decided to survey the entire population of 200 SNPs. Therefore, no statistical sampling will be used for the mail survey of plans or for the focus groups. This is a one-time data collection and will not be repeated.

2 There are currently 276 approved SNPs in 2006, but many of these are the same plan offering the same

coverage in distinct counties.

3. Methods to maximize response rates

A number of steps will be taken to increase response rates in the mail survey of SNPs. First, CMS will inform provider groups, such as the National Health Policy Group (SNP Alliance), of the importance of the survey. The cover letter that will accompany survey mailings will be personally addressed, will be written on CMS letterhead, will include contact information and the signature of the CMS Project Officer, and a toll-free number at which to reach the Mathematica Survey Director. The letter will describe the evaluation and the purpose of the mail survey, and will provide instructions and a timeline for responding to the survey. The letter will also indicate that the survey is voluntary and will give the estimated time to complete the survey.

Reminder postcards and follow-up telephone calls by trained interviewers (during which plans can complete the survey) extend our strategy for maximizing response rates. We will send one questionnaire by mail and will place a follow-up call if the plan has not responded in four weeks. The questionnaire is relatively short and has as few open-ended response categories as possible. There are clear instructions on the first page. We considered making the survey available on the web but concluded that the response might be lower for this modality; we believe that a mail survey will be convenient for respondents because they may need to check administrative records as they complete the questionnaire.

To increase participation in the focus groups, the contractor will mail a letter of invitation and a fact sheet about the focus group to each enrollee selected for the focus group. The letter will be printed on CMS letterhead and signed by a CMS official and will outline the purpose of the group, explain how we intend to ensure confidentiality, offer the $50 incentive payment, and alert the enrollees that they will receive a follow-up (recruitment) call. Trained interviewers will place recruitment calls. Recruits will be sent a confirmation letters including the focus group date, time, and address of the location, and will also receive a reminder call the day before their group is scheduled to meet. Based on past experience, approximately 18 beneficiaries will be recruited to achieve the desired group size of 8 to 10 persons.

The response rate for the mail survey will be calculated as the completion rate – the number of SNPs that complete the questionnaire (either by returning the mail questionnaire or by telephone) divided by the total number of SNP that were mailed surveys (all unique SNPs). Because we know the universe of approved, unique SNPs, the denominator of the response rate does not include ineligible plans or plans whose eligibility is unknown.

4. Tests of Procedures or Methods

A total of nine SNPs were selected to pretest the survey instrument. The plans will were

selected to represent a mix that varies in size and population served. The pretest identified some items that were burdensome or difficult to respond to, and these items were removed or revised accordingly. An average response time estimate from the pretests was 45 minutes, which is figured in our response burden estimate in section B.12 above.

5. Individuals involved in design

The following individuals have contributed to the design of the mail survey and focus groups: Dr. Robert Schmitz, a MPR senior fellow and study project director (617-301-8976), Dr. Angela Merrill, a senior researcher at MPR (617-301-8977), Ms. Jennifer Schore, also a senior researcher at MPR (609-275-2380), and Ms. Julita Milliner-Waddell, a survey researcher at MPR, (609) 275-2206.

Mr. James Hawthorne (410-786-6689), Project Officer, Office of

Research, Demonstrations, and Information, is supervising the study for the government.