Supporting Statement Part A -Health Lit Item Set Sup to CAHPS Health Plan Survey - Pretest 9-20-2011

SUPPORTING STATEMENT

Part A

Health Literacy Item Set Supplemental to CAHPS Health Plan Survey - Pretest of Proposed Questions and Methodology

September 20^th, 2011

Agency of Healthcare Research and Quality (AHRQ)

Table of contents

A. Justification 3

1. Circumstances that make the Collection of Information Necessary 3

2. Purpose and use of information 4

3. Use of Improved Information Technology 5

4. Efforts to Identify Duplication 5

5. Involvement of Small Entities 5

6. Consequences if Information Collected Less Frequently 5

7. Special Circumstances 5

8. Consultation outside the Agency 6

9. Payments/Gifts to Respondents 7

10. Assurance of Confidentiality 7

11. Questions of a Sensitive Nature 7

12. Estimates of Annualized Burden Hours and Costs 7

13. Estimates of Annualized Respondent Capital and Maintenance Costs 8

14. Estimates of Annualized Cost to the Government 8

15. Changes in Hour Burden 8

16. Time Schedule, Publication and Analysis Plans 8

17. Exemption for Display of Expiration Date 9

List of Attachments 9

A. Justification

1. Circumstances that Make the Collection of Information Necessary

The Healthcare Research and Quality Act of 1999 (see Attachment A) states that the mission of the Agency for Healthcare Research and Quality (AHRQ) is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:

1. Research that develops and presents scientific evidence regarding all aspects of health care; and

2. The synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and

3. Initiatives to advance private and public efforts to improve health care quality.

Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (a) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (b) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.

The Consumer Assessment of Healthcare Providers and Systems (CAHPS^®) program is a multi-year initiative. AHRQ first launched the program in October 1995 in response to concerns about the lack of good information about the quality of health plans from the enrollees' perspective. Numerous public and private organizations collected information on enrollee and patient satisfaction, but the surveys varied from sponsor to sponsor and often changed from year to year. The CAHPS^® program was designed to:

• Make it possible to compare survey results across sponsors and over time; and

• Generate tools and resources that sponsors can use to produce understandable and usable comparative information for consumers, health providers and for quality improvement purposes.

Over time, the program has expanded beyond its original focus on health plans to address a range of health care services and to meet the various needs of health care consumers, purchasers, health plans, providers, and policymakers. Based on a literature review and an assessment of currently available questionnaires, AHRQ identified the need to develop a health literacy module for the CAHPS® Health Plan Survey. The intent of the health literacy module is to examine health plan enrollees’ perspectives on how well health information is communicated to them by health plans and by healthcare professionals in the health plan setting. The objective of the new module is to provide information to health plans, clinicians, group practices, and other interested parties regarding the quality of health information delivered to patients. The health literacy module will be pre-tested as a supplement to the CAHPS ® Health Plan Survey.

This pre-test has the following goals:

1) Analysis of item wording – Assess candidate wordings for items.

2) Analysis of participation rate – Evaluate the overall response rate and the proportion of that obtained from mail versus telephone modes of data collection.

3) Case mix adjustment analysis – Evaluate variables that need to be considered for case mix adjustment of scores.

4) Psychometric Analysis – Provide information for the revision of the health literacy item set based on the assessment of the reliability and validity.

5) Dissemination of the CAHPS Health Plan Health Literacy supplemental item set.

To achieve the goals of this pre-test the CAHPS Health Plan Health Literacy Survey will be implemented with a sample of persons from the surveys’ target population, consumers of health care services offered by health plans. The questionnaire is included in Attachments B and C (English and Spanish versions) and the Advanced Notification Letter, Cover Letter, Reminder Post Card, Reminder Letter and Telephone Introductory Script (English and Spanish versions) are included in Attachment D.

This study is being conducted by AHRQ through its contractor, the RAND Corporation, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).

2. Purpose and Use of Information

The data from this pre-test will be used to refine the health literacy module questions and will ensure that the future data collection yield high quality data and ensure a minimization of respondent burden, increase agency efficiency, and improve responsiveness to the public. The survey items will be added to currently available CAHPS ® surveys and will enhance the ability of health plans and health professionals working in a health plan primary care setting to assess the quality of their services.

3. Use of Improved Information Technology

The field test will be conducted using a mixed mode of data collection that involves mail with telephone follow-up to non-responders as necessary (using Computer Assisted Telephone Interviewing), as is done for other CAHPS^® surveys and field tests.

4. Efforts to Identify Duplication

Work carried out under this clearance will be designed to reflect specific customer population needs for which the work is being conducted and will not duplicate any other survey/questionnaire design or pretest work being done by AHRQ or other Federal agencies. During the development of these voluntary instruments, groups within and outside of AHRQ were consulted. Plans to conduct surveys will be reviewed prior to implementation, and any potential duplication will be identified in the review and approval process.

5. Involvement of Small Entities

Survey respondents are consumers of health care services offered by health plans.

The survey instruments and procedures for completing the instruments will be designed to minimize burden on all respondents and will not have a significant impact on small businesses or other small entities.

After completion of the field test, AHRQ expects to reduce the burden on potential respondents through revision and reduction of the number of items based upon the psychometric characteristics of the data.

6. Consequences if Information Collected Less Frequently

This is a one-time field test data collection.

7. Special Circumstances

This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.

This field test is designed to assess a draft survey instrument, not to generalize the results to a population. The data will be used only to assess the quality of the items in the instrument. It will not be used to describe or regulate agencies or to set policy.

8. Federal Register Notice and Outside Consultations

8.a. Federal Register Notice

As required by 5 CFR 1320.8(d), the notice was published in the Federal Register on July 7^th, 2011 for 60 days and again on September 15^th, 2011 for 30 days. No comments were received. See Attachment E.

8.b. Outside Consultations

Input into the design and content of the supplemental item set was collected via key informant interviews with health literacy researchers and health plan representatives including Julie Gazmararian, Associate Professor of Epidemiology at Emory University, Ellen Kuntz health plan representative from Highmark, Steven Rush, health plan representative from UnitedHealth Plan, George Isham, health plan representative from Health Partners, and a health plan consumer recruited from an adult literacy center. In addition, we sought input into the design and content of the health literacy supplemental item set via two Stakeholder meetings attended by the following people:

Edward Sekscenski	Centers for Medicare & Medicaid Services
Dr. Steve Rush	United Healthcare
Winston Lawrence	Literacy Assistance Center
Melanie Brennan	Highmark
Eileen Scheye	Aetna
Vanessa Harwood	Aetna
Dr. Christina Cordero	The Joint Commission
Dr. Barry Weiss	University of Arizona College of Medicine
Sarah Hudson-Scholle	National Committee for Quality Assurance
Elizabeth Goldstein	Centers for Medicare & Medicaid Services
Dr. Cynthia Baur	Office of Disease Prevention and Health Promotion, U.S. Department of Health and Human Services
Frank Funderburk	Centers for Medicare & Medicaid Services
Susan Pisano	American Health Insurance Plans
Joyce Dubow	AARP Public Policy Institute
Dennis White	National Business Coalition on Health
Dr. Alexander Li	LA Care
Sheryl Maletic	Highmark
Ellen Kuntz	Highmark
Dr. Dexter Campinha-Bacote	Aetna
Dr. Steven Rush	United Healthcare
Lee Craig	United Healthcare
Ken Jazel	United Healthcare
Jeanine Urdahl	United Healthcare
Jenny Williams	F.E.G.S. Health and Human Service System
Dr. Jim Galliher	American Academy of Family Physicians

9. Payments/Gifts to Respondents

No payment or gift will be given to respondents.

10. Assurance of Confidentiality

Individuals and organizations will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.

11. Questions of a Sensitive Nature

There are no questions of a sensitive nature on this survey.

12. Estimates of Annualized Burden Hours and Costs

Exhibit 1 shows the estimated annualized burden for the respondents' time to participate in this data collection. About 1000 persons will complete the CAHPS Health Plan Survey Health Literacy Module. The estimated response time of 25 minutes is based on the written length of the survey and AHRQ’s experience with previous CAHPS^® surveys of comparable length that were fielded with similar samples. The total burden hours are estimated to be 417 hours.

Exhibit 2 shows the respondents' cost burden associated with their time to participate in this data collection. The total cost burden is estimated to be $8,715.

Exhibit 1. Estimated annualized burden hours

Form Name	Number of Respondents	Number of responses per respondent	Hours per response	Total Burden hours
CAHPS Health Plan Health Literacy Module	1000	1	25/60	417
Total	1000	1	na	417

Exhibit 2. Estimated annualized cost burden

Form Name	Number of Respondents	Total Burden hours	Average Hourly Wage Rate*	Total Cost Burden
CAHPS Health Plan Health Literacy Survey	1000	417	$20.90	$8,715
Total	1000	417	na	$8,715

*Based upon the average wages, “National Compensation Survey: Occupational Wages in the United States, May 2009,” U.S. Department of Labor, Bureau of Labor Statistics.

13. Estimates of Annualized Respondent Capital and Maintenance Costs

Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.

14. Estimates of Annualized Cost to the Government

Exhibit 3 shows the total and annualized cost to conduct this research. The total cost for this project is approximately $299,000. Since the data collection period is less than one year, the total and annualized costs are identical.

Exhibit 3.  Estimated Annualized Cost

Cost Component	Total Cost	Annualized Cost
Review of literature	$20,000	$20,000
Cognitive interviews	$60,000	$60,000
Field test	$90,000	$90,000
Data analyses	$40,000	$40,000
Finalize survey	$39,000	$39,000
AHRQ project management	$50,000	$50,000
Total	$299,000	$299,000

15. Changes in Hour Burden

This is a new collection of information.

16. Time Schedule, Publication and Analysis Plans

There are no plans for publication of the results of this field test. The purposes of this survey effort are to revise the Health Literacy item set, to evaluate participation rates, and to assess case mix adjustment variables. The data will be used internally by the design team in order to achieve these goals.

The forecasted timeline is as follow:

Field testing completed within 120 days from the date of OMB approval

Final analysis report and survey instrument – 60 days from the date of completing field-testing.

Dissemination plan – July 6, 2012

Data Analysis

To achieve the purposes of the field test the following analyses will be done:

• We will examine overall participation rates and the proportion of completed surveys obtained by mail vs. telephone as well as the proportion of surveys completed in English and Spanish overall and by mode of data collection.

• We will examine item missing data and item distribution (including ceiling and floor effects).

• Psychometric analysis focusing on evaluation of item-scale correlations (convergence within scales and discrimination across scales), internal consistency reliability for hypothesized multi-item composites, and correlations of items and composites with the global ratings. Health plan-level reliability will be estimated for items and composites.

• Case-mix analyses will be conducted to identify those variables that are significantly associated with reports and ratings of care. Potential case mix variables include respondent gender, age, education, self-reported health status, and whether someone helped complete the survey. Unadjusted and adjusted results will be compared. Other analysis will include examination of predictors of unit and item non-response as well as characteristics of early versus later respondents.

In addition, patterns of both unit and item non-response will be examined and modeled, and the potential impact of non-response bias assessed. A common set of administrative variables (e.g., age, gender, race/ethnicity) will be used to predict unit non-response. These variables and others collected on the survey itself will be used as predictors of item non-response. Case mix adjustment and non-response weights will be used to more accurately reflect consumer experiences with health plans. Multivariate logistic regression models will be used to analyze the factors associated with unit non-response and item non-response.

17. Exemption for Display of Expiration Date

AHRQ does not seek this exemption.

List of Attachments:

Attachment A: AHRQ's Authorizing Legislation

Attachment B: CAHPS Health Plan Health Literacy Survey-- English

Attachment C: CAHPS Health Plan Health Literacy Survey –Spanish

Attachment D: Advanced Notification Letter, Cover Letter, Reminder Post Card, Reminder Letter and Telephone Introductory Script – English & Spanish

Attachment E: Federal Register Notice

2