Supporting Statement Part B -CAHPS Health Plan Health Literacy Project03-14-2011

SUPPORTING STATEMENT

Part B

Health Literacy Item Set Supplemental to CAHPS Health Plan Survey - Pretest of Proposed Questions and Methodology

Revised March 2011

Agency of Healthcare Research and Quality (AHRQ)

B. Statistical Methods 3

1. Potential Respondent Universe and Sample Selection Method 3

2. Information Collection Procedures .3

3. Methods to Maximize Response Rate 3

4. Tests of Procedures 4

5. Statistical Consultation and Independent Review 5

B. STATISTICAL METHODS

Universe and Sample Selection Method

Inclusion criteria: Potential respondent universe will be individuals 18 years or older who have been enrolled in a commercial health plan for 12 months or longer, with no more than one 45-day break in enrollment during the previous 12 months, and who have had at least one primary care visit within the previous 12 months.

Exclusion criteria: Excluded from the study will be those who are under 18 years old, those not currently enrolled in the health plan, individuals with only other types of coverage, (like a dental-only plan) and those who did not have a primary care visit to a health plan doctor in the previous 12 months. In addition, we will exclude subjects who are unable to provide informed consent, who are unable to speak either English or Spanish well enough to complete the survey, and health plan enrollees with a foreign home address (the U.S. territories – Virgin Islands, Puerto Rico, Guam, and Northern Mariana Islands are not considered foreign addresses and therefore, are not excluded).

Testing sites will be selected purposively based on several considerations, including ability to execute the activities necessary to participate in the pilot; size (measured by number of health plan enrollees) and type of health plan. In selecting field test partners, we will also aim for as much geographic distribution as possible, and will aim to include health plans with substantial numbers of enrollees with limited health literacy, and substantial numbers of minority enrollees, including enrollees whose primary language is Spanish.

The draw will be a sample large enough to yield approximately 1000 completes. It is assumed that approximately 2000 patients will be sampled across 3 field sites with response rate of ~50%. To increase the response rate among Spanish speaking respondents, a Spanish version of the survey and telephone transcript will be available.

2. Information Collection Procedures

We anticipate a mixed mail-telephone mode of data collection which will include these steps:

• Mailing an advanced notification letter

• Mailing of the questionnaire and cover letter

• Postcard reminder

• A second mailing of the questionnaire to non-respondents.

• Up to 10 telephone calls to every mail non-respondent approximately two weeks after the final mailing.

3. Methods to Maximize Response Rate

Every effort will be made to maximize the response rate, while retaining the voluntary nature of the effort. We will provide an advanced notice prior to sending the survey. We will include a letter explaining what the survey is about, who is doing it and why, and providing contact information for questions. The second mailing and telephone follow-up are expected to result in significant increases in response. All outgoing envelopes will include an address update request from the United States Postal Service to ensure that we have as many updated addresses as possible for the field test sample.

We will also make every effort to maximize the response rate among Spanish-speaking respondents. We developed a Spanish version of an advance notice, the questionnaire, cover letters, and the reminder card, as well as Spanish version of telephone transcript. The cover letters in English include a note in Spanish instructing respondents to call a toll free number if they would like to receive a copy of the survey in Spanish. In addition, we will ask participating field sites for information on language preference and/or race/ethnicity of sample patients so that we can tailor the mailing of the survey for Spanish-speakers.

If the field sites are able to identify sample members as Spanish-speaking, we will mail those respondents a survey in Spanish. If information on language preference is not available but the sites have information at the individual level on race/ethnicity, we will mail those sample members identified as Hispanic or Latino a survey in both English and Spanish. If the field sites don’t have information on either language preference or race/ethnicity, then we will identify all sample patients with Spanish surnames and will mail them both an English and a Spanish version of the survey.

Finally, phone follow-up to respondents who don’t complete the survey by mail will be conducted by bilingual interviewers so that we can accommodate those who want to complete the survey by telephone in either English or Spanish.

Surveys generally do not yield complete responses from every individual sampled from the population. There are two basic types of survey nonresponse. Unit nonresponse is the failure of a member of the sample to respond to the entire survey. Item nonresponse is the failure of a respondent to answer one or more survey items that he or she is eligible to answer. In this analysis, we will examine and model patterns of both unit and item nonresponse and assess the potential impacts of nonresponse bias. Common set administrative variables (e.g., age, gender, race/ethnicity) will be used to predict unit nonresponse. These variables and others collected on the survey itself will be used as predictors of item nonresponse. We will use case mix adjustment and nonresponse weights to more accurately reflect consumer experiences with health care in the field test hospitals.

We will estimate multivariate logistic regression models to analyze the factors associated with unit nonresponse and item nonresponse. The initial models will include the full set of potential predictor variables. In the subsequent models, we will alter the model to consider possible interactions and to determine the most parsimonious specifications. Inverse probability weights will be generated from the prediction of the final unit nonresponse model. Case-mix models will be parameterized by linear age, race/ethnicity indicators, linear education, and linear self-reported health status.

4. Tests of Procedures

To achieve the purposes of the field test the following analyses will be done:

• We will examine overall participation rates and the proportion of respondents obtained from the mail and the telephone modes of data collection.

• Psychometric analysis focusing on evaluation of item-scale correlations (convergence within scales and discrimination across scales), internal consistency reliability for hypothesized multi-item composites, and correlations of items and composites with the global ratings.

• Case-mix analyses will be conducted to identify those variables that are significantly associated with reports and ratings of care. Potential case mix variables include respondent gender, age, education, self-reported health status, and whether someone helped complete the survey. Unadjusted and adjusted results will be compared. Other analysis will include examination of predictors of unit and item nonresponse as well as characteristics of early versus later respondents.

5. Statistical Consultation and Independent Review

Input from statisticians will be obtained to develop, design, conduct, and analyze the information collected from this survey. This statistical expertise will be available from Marc Elliott, PhD of RAND.

5