Supporting Statement Part B -Health Literacy HCAHPS 11-05-2009

SUPPORTING STATEMENT

Part B

Health Literacy Item Set Supplemental to CAHPS Hospital Survey - Pretest of Proposed Questions and Methodology

0935-0124

October 2009

Agency of Healthcare Research and Quality (AHRQ)

B. Statistical Methods 3

1. Potential Respondent Universe and Sample Selection Method 3

2. Information Collection Procedures .3

3. Methods to Maximize Response Rate 3

4. Tests of Procedures 4

5. Statistical Consultation and Independent Review 5

B. STATISTICAL METHODS

1. Potential Respondent Universe and Sample Selection Method

Potential respondent universe will be persons who had at least one overnight stay at a hospital within the previous five months. Excluded from the study will be those who were under 18 years old at the time of their admission, had a psychiatric diagnosis, were discharged to a hospice facility or died during the hospitalization. Testing sites will be selected purposively based on several considerations, including ability to execute the activities necessary to participate in the pilot; number of beds, number of discharges for medical, surgical, and obstetric patients, average length of stay, location (urban versus rural), profit status, and academic medical center status.

The sample will be drawn from 3 participating hospitals. Two of these hospitals are urban teaching hospitals located on the west coast while the third is a rural non-teaching hospital located in the mid-west. The draw will be a sample large enough to yield approximately 1000 completes. It is assumed that approximately 2000 patients will be sampled from the 3 hospitals with response rate of ~50%. To increase the response rate among Spanish speaking respondents, a Spanish version of the survey and telephone transcript will be available.

2. Information Collection Procedures

We anticipate a mixed mail-telephone mode of data collection which will include these steps:

• Mailing an advanced notification letter

• Mailing of the questionnaire and cover letter

• Postcard reminder

• A second mailing of the questionnaire to non-respondents.

• Up to 10 telephone calls to every mail non-respondent approximately two weeks after the final mailing.

3. Methods to Maximize Response Rate

Every effort will be made to maximize the response rate, while retaining the voluntary nature of the effort. We will provide an advanced notice prior to sending the survey. We will include a letter explaining what the survey is about, who is doing it and why, and providing contact information for questions. The second mailing and telephone follow-up are expected to result in significant increases in response. We will also make every effort to maximize the response rate among Spanish-speaking respondents. We developed a Spanish version of an advance notice, the questionnaire, cover letters, and the reminder card, as well as Spanish version of telephone transcript. The cover letters in English include a note in Spanish instructing respondents to call a toll free number if they would like to receive a copy of the survey in Spanish. In addition, we will ask participating field sites for information on language preference and/or race/ethnicity of sample patients so that we can tailor the mailing of the survey for Spanish-speakers. If the field sites are able to identify sample members as Spanish-speaking, we will mail those respondents a survey in Spanish. If information on language preference is not available but the sites have information at the individual level on race/ethnicity, we will mail those sample members identified as Hispanic or Latino a survey in both English and Spanish. If the field sites don’t have information on either language preference or race/ethnicity, then we will identify all sample patients with Spanish surnames and will mail them both an English and a Spanish version of the survey.

Finally, phone follow-up to respondents who don’t complete the survey by mail will be conducted by bilingual interviewers so that we can accommodate those who want to complete the survey by telephone in either English or Spanish.

Surveys generally do not yield complete responses from every individual sampled from the population. There are two basic types of survey nonresponse. Unit nonresponse is the failure of a member of the sample to respond to the entire survey. Item nonresponse is the failure of a respondent to answer one or more survey items that he or she is eligible to answer. In this analysis, we will examine and model patterns of both unit and item nonresponse and assess the potential impacts of nonresponse bias. Common set administrative variables (e.g., age, gender, race/ethnicity) will be used to predict unit nonresponse. These variables and others collected on the survey itself will be used as predictors of item nonresponse. We will use case mix adjustment and nonresponse weights to more accurately reflect consumer experiences with health care in the field test hospitals.

We will estimate multivariate logistic regression models to analyze the factors associated with unit nonresponse and item nonresponse. The initial models will include the full set of potential predictor variables. In the subsequent models, we will alter the model to consider possible interactions and to determine the most parsimonious specifications. Inverse probability weights will be generated from the prediction of the final unit nonresponse model. Case-mix models will be parameterized by linear age, race/ethnicity indicators, linear education, and linear self-reported health status.

4. Tests of Procedures

To achieve the purposes of the field test the following analyses will be done:

• We will examine overall participation rates and the proportion of respondents obtained from the mail and the telephone modes of data collection.

• Psychometric analysis focusing on evaluation of item-scale correlations (convergence within scales and discrimination across scales), internal consistency reliability for hypothesized multi-item composites, and correlations of items and composites with the global ratings.

• Case-mix analyses will be conducted to identify those variables that are significantly associated with reports and ratings of care. Potential case mix variables include respondent gender, age, education, self-reported health status, and whether someone helped complete the survey. Unadjusted and adjusted results will be compared. Other analysis will include examination of predictors of unit and item nonresponse as well as characteristics of early versus later respondents.

5. Statistical Consultation and Independent Review

Input from statisticians will be obtained to develop, design, conduct, and analyze the information collected from this survey. This statistical expertise will be available from Marc Elliott, PhD of RAND.

5