Supporting Statement B

B.

Collection of Information Employing Statistical Methods

B1. Respondent Universe and Sample
A total of 120,000 disenrollees from July 2010 through March 2011 will be sampled (covering 9
months of disenrollment activity) using a “modified probability proportional to size (PPS)
sampling approach” with a floor on the sample per contract, and larger samples drawn from the
biggest contracts. The advantage of this approach is that it provides an efficient nationally
representative sample of all enrollees, while allowing for adequate sample in smaller included
contracts to identify contracts with outlying scores on relevant measures. This approach will
generate more precision in larger contracts, where so much of total enrollment lies (particularly
for MA-PD plans). We need to have enough sample from these large contracts so as not to hinder
our ability to make accurate national subgroup estimates. In some cases, we anticipate that there
will be no sample in some very small contracts, although we will strive to make inferences for as
many small contracts as possible.
The basic PPS design (used successfully in sampling for Medicare CAHPS) is implemented in
two stages – first selection of included contracts or sponsors, and second selection of individuals
within selected contracts. Let N represent the number of eligible disenrollees in a contract, and n
the sample size that will be drawn (by simple random sampling) within that contract.
Furthermore let N* be a cutoff disenrollee count and n* a minimum sample size. The sampling
proceeds with slightly different rules depending on N:
 Large contracts (by size of disenrollment ) (N > N*):
o Probability of contract selection = 1
o Sample size n = (N/N*) n*.

[enlarged sample proportional to size]

 Medium contracts (n* < N < N*):
o Probability of contract selection = N/N*
o Sample size n = n* [standard target sample size]
 Small contracts ( N < n* ):
o Probability of contract selection = n*/N*
o Sample size n = N [sample all available cases]
Exact sampling figures will be determined upon receipt of data regarding disenrollment patterns.
Sampling weights reflect the probability that each beneficiary is selected for the survey;
nonresponse weights reflect the probability that a sampled beneficiary responds to the survey;
poststratification weights make the respondent sample’s characteristics more similar to the
population. Sampling weights are readily calculated as the ratio of eligible to sampled
beneficiaries in each contract. Simple contract-level poststratification weights will be calculated
as the ratio of eligible to responding beneficiaries in each contract. More complex individuallevel nonresponse or poststratification weights will be developed using logistic regression and

raking/log linear models, respectively. We will develop weights appropriate to national and
subgroup comparisons.
B2. Information Collection Procedures
Because it is important to survey disenrollees relatively soon after their disenrollment
experience, there will be nine sample cohorts reflecting nine months of disenrollees. Each cohort
will have a 12 week data collection period. The primary mode of data collection will be a mail
survey. Beneficiaries will be mailed a survey packet with a cover letter signed by the CMS
privacy officer (see attachment 1 for a copy of the wave 1 cover letter). Four weeks after the
initial survey mailing, beneficiaries will be mailed a follow-up survey packet with a modified
cover letter signed by the CMS privacy officer (see attachment 2 for a copy of the wave 2 letter).
We will use computer assisted interviewing (CATI) as a secondary or non-response mode. The
CATI mode will be implemented approximately eight weeks after the initial survey mailing.
B3. Methods to Maximize Response Rates
We anticipate a response rate of 60 percent, based on prior disenrollment surveys and recent
experience with surveys of Medicare beneficiaries. We will employ multiple mail contacts and
multiple modes (mail and CATI) to minimize non-response.
B4. Tests of Procedures or Methods
No tests of procedures or methods will be undertaken as part of this data collection.
B5. Statistical and Questionnaire Design Consultants
The survey, sampling approach, and data collection procedures were designed by the RAND
Corporation under the leadership of:
Cheryl Damberg, Ph.D.
RAND Corporation
1776 Main Street
PO Box 2138
Santa Monica, CA 90407-2138
310-393-0411
Data will be collected by the survey vendor CSS Research under the direction of:
Jeff Burkeen
CSS Research
1625 K Street NW, 8th Floor
Washington, DC 20006
202-454-3005