Supporting Statement B

Supporting Statement B.doc

Data Collection for Administering the Medicare Health Improvement Survey

OMB: 0938-1014

Document [doc]

Download: doc | pdf

C. Collection of Information Employing Statistical Methods

C.1 Respondent Universe and Response Rates

The target population for the Medicare Health Improvement Survey is Medicare beneficiaries who have high health care costs (i.e., greater than or equal to $5,000) and/or high disease severity measured by the Hierarchical Conditions Categories (HCC) (i.e., greater than or equal to 1.7) who have been assigned to either the intervention or comparison groups for this program. Those beneficiaries who are deceased will not be included in our sample.

Project statisticians will use CMS’ Enrollment Database (EDB) file in order to obtain current addresses as well as other demographic variables that will be used in the analysis. The data elements from these files that will be used for sampling, data collection and/or analysis will include name (given name, middle name and surname), county of residence and zip code, current mailing address, demographic information including birth date, gender, race and ethnicity, health insurance claim number and social security number.

C.2 Procedures for the Collection of Information

Statistical Methodology for Stratification and Sample Selection

Among the six CMO programs, intervention and comparison populations are identified in two distinct ways. CMS analyzed 2004 claims data to identify Medicare beneficiaries eligible for each program based upon costs, disease severity, and diagnostic criteria, unique to each site. For two programs, CMS was able to randomly assign beneficiaries to intervention and control groups. Eligibility for the remaining four programs was related to beneficiary geographic location as well as receipt of services, or loyalty to, the sponsoring organization. Therefore CMS randomly selected a group of intervention beneficiaries for each of these CMOs from an identified population and then selected a group of similar beneficiaries to serve as a matched comparison group. We employed HCC risk score and cost criteria (varies in detail by site) as the basis for matching populations, and comparisons between the two populations across demographic categories and utilization patterns i.e., using BETOS categories applied to claims, were made to insure similarity and avoid biases.

Care Management Organization	Intervention	Comparison
Randomized comparison group
RMS	5,057	2,024
CLM	15,194	6,084
Matched comparison group
Accent	1,600	1623
MGH	2,500	2,500
Montefiore	2,989	1875
Texas Tech	5,063	5106

Statistical Power

The sample size for the beneficiary survey is designed to be large enough to allow us to reliably detect substantively important differences between the CMHCB intervention and comparison groups in each program. To make these sample size assessments, we focus on the prevalence of a particular outcome in the intervention group and its associated comparison group. These percentages can be created by dichotomizing any of the variables in our survey, such as the percentage of respondents who report they have discussed their care plan recently, the percentage who take their medication as prescribed, or the percentage who rate their experience with their health care team as “very good” or “excellent.”

Table C-1 below shows the estimates which account for survey design effects and anticipated response rates at each mailing. The estimates show that we will initially need to send questionnaires to 815 respondents (2,280 in the CMHCB intervention group and 2,610 in the control group) in each program site to detect a 5% difference.

Table C-1.
Sample sizes for each site and a total for all sites

	Per Site			Total
	Intervention	Control	Total	Intervention	Control	Total
Surveys in initial mailing	380	435	815	2,280	2,610	4,890
Response Rate	80%	70%		80%	70%	74.7%
Projected surveys	305	305	608	~1,830	~1,830	~3,650

*For power=.80, alpha = .05, one-sided test

Survey Eligibility

Medicare beneficiaries who live in institutions, such as jails and prisons, will be ineligible to participate in the sample, because Medicare is the secondary payor for health care services used by this population.

Survey Type

The Medicare Health Improvement Survey is a cross-sectional design. As described earlier, the survey process will consist of a pre-notification letter on CMS letterhead sent out prior to the first questionnaire mailing, the first questionnaire mailing, a thank you/reminder postcard, and a second questionnaire mailing. We will conduct telephone follow up of non-respondents, and an overnight package will be sent to those non-respondents for whom we can not find a telephone number.

C.3 Maximizing Response Rates

To maximize the response rate on the Medicare Health Improvement Survey, we plan to use “best mail survey practices” which have been proven in methodological research to improve response rates, as well as our experience fielding of other Medicare surveys. Research on mail survey design suggests that the following factors usually increase response rates⁴:

pre-notification letters,
number of follow-up contacts,
survey sponsorship,
saliency of the survey topics to the target population,
personalization of the correspondence with sample members, and
postage-paid return envelopes.

The most important of these factors is the number of contacts, including the pre-notification and cover letters, thank you/reminder letters and other follow-up mailings that include a replacement questionnaire. Our data collection approach will employ nearly all of these features, creating a multi-pronged, comprehensive strategy that avoids the weaknesses of reliance upon mail or telephone contact alone.

The Medicare Health Improvement Survey is designed as a mail survey with phone follow-up of non-respondents. The mail survey component of each project will consist of mailing a pre-notification letter to all sample members, followed by an initial questionnaire package seven days later. A Thank You/Reminder Letter designed to thank sample members who have already responded, and to remind others who have not to do so, will be sent ten days after the initial questionnaire is mailed. A second questionnaire mailing will be sent to sample members who do not return a completed questionnaire within four weeks after the pre-notification letter is mailed. All cases that do not return a completed questionnaire during mail survey data collection period will be transferred to the telephone survey unit, where interviewers will attempt to contact the sample member and complete the interview by phone.

All of the materials that we design for use in the mail surveys will employ best mail survey practices suggested by Jenkins and Dillman (1997)⁵ and lessons learned from other surveys of Medicare beneficiaries to ensure high rates of cooperation. These include:

that all materials sent to the sample members will be written at no higher than the fifth- or sixth-grade level, if possible, in order to minimize the burden
the correspondence will briefly describe the survey and emphasize its importance to the sample member; each subsequent letter will address these aspects in stronger language than in the previous mailing
the correspondence will also describe the confidential nature of the survey and that participation is voluntary. Breach of confidentiality by project staff might be seen as a risk and therefore as a potential burden
the print style of the letters and questionnaires will be kept simple, with limited mixtures of fonts
we will use questionnaire boxes to help respondents determine where to mark their answers and to navigate through the instrument
the questionnaires will be printed in large typeface (font size of 13 pt) to facilitate participation among sample members with poor vision
the cover of each questionnaire will include an attractive graphic that will appeal to older Americans
all correspondence will include the name and telephone number of a survey contact person that sample members can call toll-free for information about the survey.

Proxy responses will be accepted from spouses or adult children of sampled respondents. Those who are too cognitively or physically impaired to complete the survey, or whose primary language is not English or Spanish (and who have no acceptable proxy relative), will still be considered eligible, although we will remove them from the mail and calling sample if no proxy is available.

We will attempt to complete an interview with all mail survey non-respondents. Following up by telephone with mail survey non-respondents, as well as using best practices for questionnaire design and data collection, are intended to increase the response. Analysis weights will be created to adjust for non-response. These site specific weights will enable design-consistent estimation of population parameters by scaling the disproportionalities between the sample and the underlying population. These weights may be viewed as inflation factors that account for the number of beneficiaries in the target population that a sample member represents. The initial component of an analysis weight is the inverse of the selection probability that is specified by the sample design. Typically, adjustments are then made to the weights to compensate for potential biases attributable to differential response and coverage among sample members based on relevant characteristics of the sample members.

Response Propensity Analysis. Once data collection has been completed, we will conduct an analysis of response rates to determine whether survey participation was influenced by intervention status or other background characteristics. This response propensity analysis is based on a logistic regression model in which the binary outcome is coded 1 if the sampled beneficiary completed the survey and 0 if the beneficiary did not return a survey. The explanatory variables in the model will consist of factors that are available for all beneficiaries from the CMHCB programs and secondary sources, such as demographic characteristics, HCC scores, chronic disease diagnoses, and intervention status. Factors contributing to differential participation will be incorporated in the sample weights for the study.