Letter to Dr. Schwab

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

September 23, 2013

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the NCHS Questionnaire Design Research Laboratory (QDRL) (OMB No. 0920-0222, exp. 06/30/2015), in collaboration with the Survey Planning and Special Surveys Branch of the Division of Health Interview Statistics, plans to test new questions, terms, and concepts pertaining to the Affordable Care Act (ACA). In August 2013 OMB clearance was received to allow the QDRL to cognitively test a brief set of questions on health insurance marketplaces and basic terminology to inform the development of questions for future federal surveys including the National Health Interview Survey (NHIS) (Information Collection Title: 2013 Comparative Disability Questions and Mode Effect Study). The first round of these interviews has been completed and the analysis of the health insurance questions (alongside the other components of the previously-approved research) is proceeding. Following, and as a complement to that work, we request approval to conduct (1) a wider scale test of ACA-related questions and concepts using the internet as the mode of data collection; and (2) a cognitive interviewing project that will exclusively explore potential ACA-related concepts such as health insurance exchanges and health insurance subsidies and iteratively test ACA-related questions. We propose to start both of these related sub-projects in October 2013.

Proposed Project: 2013 NHIS Health Insurance Web and Cognitive Interview Study

As a principle source of data on the nation’s health, the NHIS routinely collects detailed information about the health insurance coverage of the US population. Detailed questions concern current coverage, recent periods of non-coverage, barriers to obtaining insurance, type and cost of coverage, household members covered, and adequacy of coverage. Estimates of the uninsured population are published quarterly to track progress toward national objectives. These estimates are critical to accurately monitor the impact of ACA on health insurance coverage.

With Health Insurance Marketplaces (HIM) opening in October 2013 and coverage effective in January 2014, how health care coverage is obtained and paid for will rapidly change as will the number of uninsured persons. There also will be dramatic changes in the way health care coverage is understood by the public. In order to keep the NHIS questionnaire synchronized with the systemic changes and comprehensible to respondents, NCHS wishes to develop and test new questions and insurance-related concepts. 

Sub-Project 1: NHIS Web Study

Two sources will be used to identify adult respondents aged 18 years and older for this experiment. Former participants of the National Health Care Interview Survey (NHCIS, OMB number 0920-0214, exp. 08/31/2014), a follow-back survey of persons who participated in the 2012 NHIS, agreed to be re-contacted for future web-based studies. The second group will be obtained via a commercially available survey research panel. By expanding the reach of this experimentation beyond the physical location of the QDRL, a larger number of respondents can be accessed quickly. Much is known about the characteristics of the NHCIS respondents from their previous survey participation. Likewise, the commercial service provides demographic information key to understanding the results of the experiment. Approximately 1,500 responses will be sought, 500 from the NHCIS sample and 1,000 from the commercial source (500 in October/November and 500 in December).

Respondents will be issued an invitation to participate in testing questions via email with one email reminder sent a week after the first contact. Because these online interviews will be self-administered, a statement has been added to the informed consent language to inform respondents that the security of information transmitted through the internet cannot be guaranteed (see Attachment 1a). The questionnaire will be administered using Survey Monkey software to both the NHCIS and commercial panels. No remuneration or incentive will be offered. This testing mechanism helps address the need for efficient and quick turnaround testing to evaluate critical survey questions and emergent topics.

The survey will take about 10 minutes to complete. This test will seek to understand what people know about commonly used terms and provisions of the law.

The questions (see Attachment 1a) will explore recognition of vocabulary used to describe the new health insurance programs (e.g., marketplace, exchange, subsidy, premium, tax credits, and Medicaid expansion) and experience with obtaining exchange information. In addition to understanding the new programs and where to get information, the public will need to complete detailed forms on-line, by phone, or with federally funded assistance. The questions expand on those currently being tested and touch on all these topics. For this project the maximum respondent burden will be 250 hours. A burden table (Table 1) for the entire project is presented at the end of this letter.

Sub-Project 2: Health Insurance and Other Related Questions Cognitive Interviewing Study

Background Information about Cognitive Testing of Questionnaires: The methodological design of this proposed study is consistent with the design of typical cognitive testing research. As you know, the purpose of cognitive testing is to obtain information about the processes people use to answer survey questions as well as to identify any potential problems in the questions. The analysis will be qualitative.

Specific Plans for the Proposed Study: The staff of the NCHS Questionnaire Design Research Laboratory is requesting approval to conduct fifty total cognitive interviews. As the overall purpose of this project is to design a series of questions that will capture ACA-related topics on the NHIS, the proposed project will follow an iterative design. In this design, the initial rounds of interviews will be more open-ended and attempt to understand how respondents understand and communicate about these ACA-related topics. Analysis of these interviews, combined with the data from the above NHIS web interviews, will allow staff members of the QDRL and NHIS to propose ACA-related questions. These questions will then be tested, revised, and re-tested as necessary. The ultimate goal of this combined project is to develop a solid set of ACA-related questions that can be added to the January NHIS production survey.

As the QDRL portion of this project will start with open-ended qualitative interviews (instead of tightly-focused cognitive interviews about proposed questions), there is no set questionnaire at this time. However, Attachment 1b shows the basic open-ended interview protocol for the first iterative rounds that will lead to the initial design of the ACA-related questions. Topics include the respondent’s understanding of his or her healthcare coverage, health insurance participation, healthcare utilization, and their decisions on potentially participating in HIMs and other similar mechanisms for obtaining and changing their coverage.

We will recruit up to 50 respondents to participate in this sub-study. The respondents will be recruited from the Washington, DC metropolitan area, or other major metropolitan areas that cross state boundaries. Because HIMs are state-based, recruiting a sample that crosses jurisdictional boundaries will allow the QDRL to examine how this program-based variation will impact the validity and interpretations of the proposed questions. The up-to 50 respondents will participate in face-to-face interviews, either in the Questionnaire Design Research Laboratory at NCHS in Hyattsville, MD or at a mutually-agreed upon offsite location, such as a library or an office. These interviews will follow the procedures laid out in the QDRL’s generic ICR (0920-0222).

Recruiting and Informed Consent Procedures for the Cognitive Interviewing Study: We propose to recruit the up-to 50 adult respondents (Age 18 and older) through newspaper advertisements, flyers, special interests groups, word-of-mouth, QDRL Respondent Database, etc. The newspaper advertisement/flyer used to recruit respondents is shown in Attachment 2. The screener used to determine eligibility of individuals responding to the newspaper advertisement/flyer, etc. is shown in Attachment 3a. The screener used to determine eligibility of individuals from the QDRL Respondent Database is shown in Attachment 3b. Within these constraints, we hope to recruit participants with some demographic (particularly in terms of gender, education, race/ethnicity, and income) and geographic (as explained above) variety.

Interviews will be conducted by QDRL staff members with English speaking respondents for up to 60 minutes per interview (including screener). All interviews conducted in the Questionnaire Design Research Laboratory will be video and audio recorded to allow researchers to review the behaviors and body language of the respondents. Interviews conducted offsite will only be audio recorded. These recordings will allow researchers to insure the quality of their interview notes.

After respondents have been briefed on the purpose of the study and the procedures that QDRL routinely takes to protect human subjects, respondents will be asked to read and sign an Informed Consent document.

The interviewer will then ask the respondent to confirm that he/she understands the information in the Informed Consent, and then state that we would like to record the interview. The recorder will be turned on once it is clear that the procedures are understood and agreed upon. The interviewer will then orient the respondent to the cognitive interview with the following introduction:

Lauren may have told you that we will be working on some questions that may be on, or will eventually be added to, national and international health surveys. Before that happens, we like to test them out on people like you. Most of the questions in this study are about your health and health insurance and coverage. We are interested in your answers, but also how you go about making them. I may also ask you questions about the questions—whether they make sense, what you think about when you hear certain words, and so on.

I will read each question to you, and I’d like you to answer as best you can. Please try to tell me what you are thinking as you figure out how to answer. Also, please tell me if:

• there are words you don’t understand,

• the question doesn’t make sense to you,

• you could interpret it more than one way,

• it seems out of order,

• the answer you are looking for is not provided.

The more you can tell us, the more useful it will be to us as we try to develop better questions. Okay? Do you have any questions before we start?

The probing in this sub-study will follow the normal semi-structured procedures laid out in the QDRL’s generic package. Each interview in this sub-study will be followed by a semi-structured debriefing session with the respondent which will allow the interviewer to discuss any inconsistencies that arose as well as additional questions that may have arisen as part of the scripted portion of the interview.

After the interview, respondents will be given the thank-you letter signed by Charles J. Rothwell, Acting Director of NCHS, a copy of the informed consent document, and $40. Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets.

We propose paying participants $40, which is our standard payment. In total, for this sub-study, the maximum respondent burden will be 50 hours of interviewing.

Overall Project Burden

A burden table for the entire project is shown below in Table 1:

Table 1: Project Burden

Sub-Study	Number of participants	Number of responses per participant	Average hours per response	Response burden
NHIS Web Interview	1,500	1	10/60	250 hours
Cognitive Interviews	50	1	1	50 hours
Total	1,550			300 hours

Attachments (5)

cc:

Mary Moien

Verita Buie

Tony Richardson

DHHS RCO

5