2018 Nhis Ssb 122717

7

Supporting Statement B

Revision Request for Clearance

NATIONAL HEALTH INTERVIEW SURVEY

OMB No. 0920-0214

(Expiration Date 12/31/2019)

Contact Information:

Stephen Blumberg

Division of Health Interview Statistics

National Center for Health Statistics/CDC

3311 Toledo Road

Hyattsville, MD 20782

301.458.4107 (voice)

301.458.4035 (fax)

[email protected]

December 27, 2017

Table of Contents

1. Respondent Universe and Sampling Methods……………………………………………………………………3

2. Procedures for the collection of Information…………………………………………………………………….3

3. Methods to Maximize Response Rates and Deal with Nonresponse………………………………….4

4. Test of Procedures or Methods to be Undertaken…………………………………………………………….6

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing

Data………………………………………………………………………………………………………………………………….6

List of Attachments

Attachment 1 Legislative Mandates

Attachment 2a 60-Day Federal Register Notice

Attachment 2b Public Comment

Attachment 3a Redesigned Questionnaire Modules and Periodicity

Attachment 3b Main and Redesigned Questionnaire Structures

Attachment 3c Need for Core Modules in Main Questionnaire

Attachment 3d Need for Core Modules in Redesigned Questionnaire

Attachment 4a Main Questionnaire Household Composition and Family Section

Attachment 4b Main Questionnaire Adult Section

Attachment 4c Main Questionnaire Child Section

Attachment 4d Main Questionnaire Supplement Questions

Attachment 4e Main Questionnaire Supplement Descriptions

Attachment 4f Main Questionnaire Flashcards

Attachment 5a Redesigned Questionnaire Household Roster Section

Attachment 5b Redesigned Questionnaire Adult Section

Attachment 5c Redesigned Questionnaire Child Section

Attachment 6a Main Reinterview Questionnaire

Attachment 6b Redesigned Reinterview Questionnaire

Attachment 7 Methodological Projects Description

Attachment 8 Data Collection Procedures

Attachment 9 Redesign Consultations

Attachment 10a Advance Letter

Attachment 10b NHIS Brochure

Attachment 10c Supplemental Mailings

Attachment 10d Thank You Letter

Attachment 11 Research Ethics Review Board Approval Notice

Attachment 12 Listing of Proposed items for 2019 and 2020

B. Collection of Information Employing Statistical Methods

1.Respondent Universe and Sampling Methods

The NHIS is a cross‑sectional household interview survey. The respondent universe is the civilian, noninstitutionalized population of the US. The sampling plan follows a multistage probability design. Approximately every ten years, the NHIS sampling plan is revised following the decennial census of the population. From 2006-2015, the sampling plan was based on the 2000 decennial census. In 2016, a new sampling plan was implemented to keep the sample current with population distribution changes over the decade.

The 2016 NHIS sample design is structured with a nationally-focused design as its core, and contains large reserve samples that can be used to achieve state or minority estimation objectives. The sample design takes into account demographic shifts in the U.S. civilian, noninstitutionalized population, and allows for additions and contractions in the sample size to reflect funding availability and to meet estimation goals. The base sample consists of approximately 35,000 completed household interviews annually. To balance the precision of national and state-level estimates, most of the sample (approximately 25,000 completed interviews) is allocated proportionally to state populations to maximize the precision of national-level estimates. A smaller portion of the sample (approximately 10,000 completed interviews) is shifted to increase sample in the 10 least populous states, enabling state-level estimates of key variables to be produced for all 50 states and DC by pooling 3 years of data. This flexibility reflects the increasing demand for state-level health outcomes, in particular support of the focus on state-based health care.

While the sampling frame for the NHIS had traditionally used field listing by the Census Bureau, in order to contain costs, the frame used from 2016 onward employs a commercial list that covers addresses within all 50 states and the District of Columbia. Supplementary field listing is undertaken to improve coverage in rural areas with poorly defined addresses and in high density areas with addresses that are too general (such as drop boxes for apartment buildings).

The sample design implemented in 2016 has not been found to affect estimates generated using NHIS data compared to previous years. To monitor the design’s performance, NHIS analysts perform monthly checks as part of routine data quality reviews. In addition to comparing the unweighted and weighted frequencies, the input and output specifications are reviewed, and the flowcharts are compared to the skip instructions and universes for each question. If a difference is found, steps are taken to determine whether the change is legitimate or whether there is a factor other than the programming of the questionnaire such as the location or context of the question in the questionnaire. If a difference persists, the paradata are reviewed to determine whether there are changes in the mean or median time spent on that question, whether interviewers had a high rate of backing up to return to that question, and whether other questions in that battery were similarly affected. Persistent differences are examined to determine whether there is any other interviewer effect, such as differences between newly-hired and experienced interviewers or newly-added primary sampling units compared to continuing primary sampling units. In addition, national estimates on the key set of indicators that are released in a quarterly report as part of the Early Release program are monitored by NHIS analysts.

2.Procedures for the Collection of Information

The U.S. Bureau of the Census is responsible for drawing the final sample and for performing the necessary field procedures related to data collection and initial processing. Specifications for the field operations are provided by the Division of Health Interview Statistics (DHIS) staff at NCHS.

DHIS staff provide to the Census Bureau specifications for the sample design, specific content of the questionnaire, detailed instructions for the administration of the interview, and procedures to carry out quality control measures, such as reinterview and paradata analysis. The Census Bureau, in addition to drawing the sample, performs supervisor and interviewer training and conducts the field operations. These operations include first contacting all households via an advance letter (Attachment 10a), followed by a personal visit. In some cases, contact via telephone is also used to follow up on respondents who were unable to be contacted in person or to complete the interview during a personal visit. Additional details about data collection procedures are provided in Attachment 8.

DHIS staff monitor the field activities through observation and communication with Census during all phases of data collection and through the analysis of paradata (such as audit trails, contact history, and item timing). Frequent status meetings are held to assess progress toward data collection goals.

All data are weighted to produce national estimates using the following four components: 1) The reciprocal of the probability of selection; 2) a household nonresponse adjustment within segment; 3) a first-stage ratio adjustment; and 4) a second stage ratio (or post stratification) adjustment to the U.S. population by age, sex, and race-ethnicity.

Standard errors may be calculated using a Taylor linearization approach as applied in SUDAAN variance software. (See: Research Triangle Institute. SUDAAN Language Manual; Release 11.0. Research Triangle Park, NC: Research Triangle Institute. 2012.)

A small sample of respondents is reinterviewed by the Census Bureau to ensure that interviewers are not submitting falsified information. NHIS reinterviews are conducted primarily by telephone, by staff at one of the Census Bureau’s centralized call centers. The reinterview is very brief and verifies that the original interview was completed. Typically, the NHIS reinterview is conducted within two to three weeks of the main survey with the same respondent who originally participated in the NHIS. The reinterview questionnaire is shown in Attachment 6.

Additional technical details on routine survey execution can be found in the National Center for Health Statistics (2016) Survey Description Document available at ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHIS/2016/srvydesc.pdf.

A sample of adult respondents that is not part of the sample set aside for MEPS (OMB# 0935-0118, expires 12/31/2018) may be selected to participate in follow-back surveys and special methodological projects. For instance, in 2018, NHIS respondents may be included as participants in methodological projects and cognitive testing that will inform the development of new rotating and supplemental content using web and/or mail survey tools. Additional details about these methodological activities are provided in Attachment 7.

3.Methods to Maximize Response Rates and Deal with Nonresponse

As 2017 NHIS data collection is still underway, the latest year of available data is from the 2016 survey. In 2016, the final household response rate was 67.9 percent. The family component was completed in 98.9% of participating households, for a final response rate of 67.1%. The sample adult component was completed in 80.9 percent of participating households in 2016, for a final response rate of 54.3 percent. The sample child component was completed in 92.3 percent of participating households in 2016, for overall final response rate of 61.9 percent.

The NHIS, like most surveys, has witnessed steadily declining response rates. Over the past ten years, response rates have fallen by more than fifteen percentage points, from 86.5% in 2005 to 70.1% to the latest available in 2015. Reasons for declining response rates are unclear but may include increased survey length, general mistrust of the government, growing time constraints, improvements in privacy screening technology for telephones, and other reasons.

To provide respondents with advance notification of the interview in an attempt to maximize response rates, an advance letter is sent to all sampled addresses prior to the interviewer's arrival (Attachment 10a). The letter legitimizes and justifies the survey, increasing the probability that the respondent will cooperate. It references the authorizing legislation of the survey, a statement of confidentiality and an explanation of how the data will be used, as well as the voluntary nature of the survey and other elements for informed consent. The letter further explains the purpose of and need for the survey and tells the respondent that there is some chance that they may be con­tacted more than once. If at the time of the initial contact the interviewer is told that the letter was not received, another letter is provided prior to the interview and time is allowed for the person to read it before proceeding. Additional written materials have been developed in recent years to supplement the advance letter, most recently the new informational brochure that was launched in 2016 to visually brand the survey and engage respondents in fulfilling the important mission of the NHIS (Attachment 10b). In addition, targeted interviewer training modules on improving respondent cooperation (such as gaining cooperation, accessing respondents through gatekeepers, and averting refusals) are presented at initial training for new interviewers and at least once a year during their annual refresher training.

If the time of contact is inconvenient for a respondent, interviewers offer to schedule an appointment for a more convenient time. If the respondent declines the interview with one interviewer, the field work supervisor often reassigns the case to an interviewer with more experience at converting reluctant respondents. Although face-to-face interviews are preferred, interviewers are allowed to substitute telephone interviews if attempts to get a face-to-face interview are not successful.

To explore the utility of employing monetary incentives in the NHIS, an incentive experiment was conducted in the summer of 2015. Fielded from May through July in the states included in three Census Regional Offices (New York, Philadelphia, and Denver), the experiment was designed to assess the impact of respondent incentives on response rates, survey costs, and data quality. Preliminary results from this experiment indicated, however, that offering incentives across the board to all respondents has limited potential to improve survey outcomes, and that incentives may be better suited for application in a targeted manner when they are deemed to be beneficial to the study design. Further evaluation of the use and impact of incentives could be part of future NHIS data collection years.

In the third quarter of 2016, an adaptive design experiment (OMB # 0920-0214, approved 6/9/2016;) was carried out to test the impact of adaptive case prioritization on sample representativeness and nonresponse bias, while maintaining survey costs and minimizing any possible negative effect on the overall response rate. Initial analysis suggests that the last two criteria were met: cost neutrality and no reductions in response rates. However, the primary goal of the experiment, reduction of nonresponse bias, does not appear to have been met. Although R-indicator values indicated greater sample representativeness in the treatment group, compared to the control group, for the first month of the experiment, corresponding values for the last two months indicated either no differences or less sample representativeness in the treatment group. In addition, and using past nonresponse bias analyses as a guide, comparisons of key health estimates between the treatment and control groups provided no evidence that the treatment estimates represented an improvement (i.e., reduction of nonresponse bias) compared to the control estimates. In sum, initial results were not indicative of an overall improvement in data quality due to the adaptive design protocol. Therefore, the decision was made for 2017 to return to data collection procedures in place prior to the experiment. Pending further evaluation of the results, a revised case prioritization protocol may be tested and/or implemented during 2018 NHIS data collection. Case prioritization would affect only interviewer activities, and would neither affect nor alter public burden from conducting the NHIS.

4.Tests of Procedures or Methods to be Undertaken

As described in greater detail in Supporting Statement A, a redesigned NHIS questionnaire, which differs from the current design in both content and structure, is scheduled to begin in 2019. To ensure the redesigned instrument is functioning properly, a dress rehearsal and systems test will take place in the last quarter of 2018 using a nationally representative sample of households. To this end, a random half of the sample for the final quarter of the year will receive the redesigned questionnaire that will be launched on the entire sample beginning in 2019, complete with the newly-designed content structure, respondent selection, and sponsored material. The other half of the sample will continue to receive the main 2018 NHIS questionnaire. This will be done by splitting in half the total NHIS sample for that quarter, which is subdivided into four separate panels and each panel is a representative sample of the U.S. population. All Census field Interviewers will be fully trained to administer both versions of the NHIS instrument. Hence, this dress rehearsal and systems test will help to identify any technical, logistical, or training issues that may be problematic in a full-scale implementation of the redesigned questionnaire in 2019. Our initial power analyses indicate that the split sample in the last quarter of 2018 would be large enough to detect differences of around 2% (with 80% power and type 1 error rate of .05) for a percentage around 9%, which is the current estimate of uninsurance, for instance. We would be able to detect differences around 4% for estimates around 50%. We would prefer to be able to detect smaller differences; however, we feel that the sample size is adequate given the multiple objectives of the field test.

For 2018-2020, a series of small-scale projects is planned to evaluate and inform future content for the redesigned questionnaire, building on and extending the findings from prior follow-back surveys and methodological experiments described in Supporting Statement A and Attachment 7. These projects will serve to inform the development of new rotating and supplemental content, by testing new and updated questionnaire items, evaluating the impact of different response options on response frequencies, and measuring respondents’ comprehension of health care-related terms and concepts.

Other developmental work related to the NHIS questionnaire is conducted by the NCHS Center for Questionnaire Design and Evaluation Research (CQDER) under their clearance (OMB No. 0920-0222, expires 07/31/18).

5.Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following person was consulted in the statistical aspects of the design and collection of the NHIS:

Van L. Parsons, Ph.D.

Statistical Research and Survey Design Staff

Office of Research and Methodology

National Center for Health Statistics

(301)458-4421

[email protected]

The following person is responsible for collection of the data:

Anne Furnia, Ph.D.

Survey Director, National Health Interview Survey

Demographic Surveys Division

U.S. Bureau of the Census

Suitland, MD

(301)763-6780

[email protected]

The following person is responsible for analysis of the NHIS data:

Stephen Blumberg, Ph.D.

Associate Director for Science

Division of Health Interview Statistics

National Center for Health Statistics

(301)458-4107

[email protected]

7