CAHPS Health Literacy Supporting Statement Part B

SUPPORTING STATEMENT

Part B

CAHPS Pretest of Proposed Health Literacy Questions and Methodology

0935-0124

October 2007

Agency of Healthcare Research and Quality (AHRQ)

B. Statistical Methods 3

1. Potential Respondent Universe and Sample Selection Method 3

2. Information Collection Procedures .3

3. Methods to Maximize Response Rate 3

4. Tests of Procedures 4

5. Statistical Consultation and Independent Review 5

B. STATISTICAL METHODS

1. Potential Respondent Universe and Sample Selection Method

Potential respondent universe will be persons enrolled in one of up to three health plans for at least one year prior to the survey. The sample will be selected at random from the universe of enrollees provided by each field site by a vendor experienced in conducting the CAHPS survey.

The draw will be a sample large enough to yield approximately 352. It is assumed that approximately 704 enrollees will be drawn from all field sites with response rate of 50%. To increase response rate among Spanish speaking residents, a Spanish version of the survey and telephone transcript will be available.

Plans to be selected to meet the following requirements:

• As much geographic distribution as possible (one on the east coast, one on the west coast or in the southern United States).

• Substantial number of enrollees with limited health literacy

• Substantial number of Spanish speaking enrollees

• Substantial number of minority enrollees

2. Information Collection Procedures

We anticipate a mixed mail-telephone mode of data collection which will include these steps:

• Mailing of the questionnaire and cover letter

• Postal card reminder

• A second mailing of the questionnaire to non-respondents.

• Up to 10 telephone calls to every mail non-respondent approximately two weeks after the final mailing.

3. Methods to Maximize Response Rate

Every effort will be made to maximize the response rate, while retaining the voluntary nature of the effort. We will include a letter explaining what the survey is about, who is doing it and why, and providing contact information for questions. The second mailing and telephone follow-up will produce significant increases in response. We will also make every effort to maximize the response rate among Spanish-speaking respondents. We developed a Spanish version of the questionnaire, cover letters, and the reminder card, as well as Spanish version of telephone transcript. The cover letters in English include a note instructing respondents to call a toll free number if they would like to receive a copy of the survey in Spanish. In addition, we will ask participating field sites with information on language preference and/or race/ethnicity of sample members so that we can tailor the mailing of the survey for Spanish-speakers. If the field sites are able to identify sample members as Spanish-speaking, we will mail those respondents a survey in Spanish. If information on language preference is not available but the sites have information at the individual level on race/ethnicity, we will mail those sample members identified as Hispanic or Latino a survey in both English and Spanish. If the field sites don’t have information on either language preference or race/ethnicity, then we will identify all sample members with Spanish surnames and will mail them both an English and a Spanish version of the survey.

Finally, phone follow-up to respondents who don’t complete the survey by mail will be conducted by bilingual interviewers so that we can accommodate those who want to complete the survey by telephone in either English or Spanish.

4. Tests of Procedures

To achieve the purposes of the field test the following analyses will be done:

• Psychometric analysis focusing on the reliability and construct validity of the items included in the analyses. Items will be assessed for their ability to discriminate among clinicians and plans on their CAHPS performance. Items will also be assessed in terms of their associations with existing CAHPS items and domains using correlations and factor analysis. The domain structure of the survey will be assessed.

• Assessment of the equivalence of survey responses completed in one of two modes: mail and telephone. Surveys completed in each of the modes will be compared for the distribution of respondent characteristics and for overall performance of plans on the two modes.

• Evaluation of potential case mix adjustors. Results of respondents categorized by gender, age, education, self-reported health status, and whether someone helped complete the survey. These variables have been shown to be significantly associated with CAHPS reports and ratings in earlier versions. The relationship of CAHPS results to these variables will be reviewed. Also, unadjusted and adjusted results will be compared.

5. Statistical Consultation and Independent Review

Input from statisticians will be obtained to develop, design, conduct, and analyze the information collected from this survey. This statistical expertise will be available from Marc Elliott, PhD of RAND.

5