OMB Memo

Generic Information Collection Request

Request: The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We propose to conduct an iterative cognitive evaluation of the next National Health Interview Survey-Teen (NHIS-Teen) online questionnaire. The anticipated fielding is 2025, but the timeline is dependent on funding As a part of this current submission, we are seeking approval for cognitive interviews conducted by Census Bureau staff.

Purpose: The cognitive testing goal is to test the National Health Interview Survey for Teens. For the next version of the NHIS-Teen survey, the National Center for Health Statistics (NCHS) is adding a series of approximately 25-30 new questions to the NHIS-Teen questionnaire. NCHS has asked the Center for Behavioral Science Methods (CBSM) to conduct recruitment and cognitive interviews designed to evaluate new and revised questionnaire items. The purpose of the cognitive pretesting is to determine if teenagers are able to comprehend the questions and answer reliably and accurately.

Topic areas for pretesting include:

• Self-esteem and well-being

• Social isolation and loneliness

• Mental health treatment

• Resiliency

• Stressful life events

• Social connectivity

Population of Interest: To test the revised NHIS-Teen instrument, we will recruit a total of 35 students ages 12-17. We will strive to recruit a sample with diverse demographic characteristics, geographic diversity, and school levels (middle school and high school).

Timeline: Testing will be conducted iteratively from January 2024 through April 2024. Recruiting will start early January 2024.

Language: Testing will be conducted in English only.

Method: We will conduct remote user sessions using video conferencing software (Microsoft Teams) with up to 35 English-speaking participants. Instructions on how to install Microsoft Teams will be sent as an attachment to the confirmation email once an interview has been scheduled (see Attachment I). Interviewers will use the scripted protocol seen in Attachment II. Since all participants are under the age of 18 a parent or guardian will be sent a consent form (see Attachment I). Participants will also give oral consent at the start of the interview.

Each interview will be 60 minutes in length and will allow time to complete the NHIS-Teen survey and administer a set of in-depth retrospective probes about a subset of questions. From prior experience, we anticipate needing to contact approximately 95 individuals to yield the 35 desired interviews to account for ineligible participants and cancellations.

Sample: Participants will be teenagers from ages 12-17. We plan on testing approximately 35 teenage participants. In addition to covering a spectrum of demographic characteristics, we aim to recruit a small subset of students with learning disabilities and students who engage in recreational physical activity or organized sports. There will be three rounds of testing with 10 participants in the first round, 15 participants in the second round, and 10 participants in the final confirmation round. Recruiting will begin one week prior to starting interviews and will last the duration of the round. Interview rounds will last two to three weeks.

Recruitment: Cognitive interview participants will be recruited using CBSM partnerships with student organizations all over the U.S.; through Craigslist; through listservs; social media; and through personal networks. See Attachment I for the language to be used in recruitment advertisements and for the eligibility screener. Participants will be recruited through their parents who will respond to the screener. The screener will be available as a self-response internet instrument using Qualtrics, and a recruiter administered instrument should parents prefer to call in. Participants and their parents will receive materials by email to remind them of their interview time and any other directions or information they need.

Protocol:

Researchers from the Center for Behavioral Science Methods (CBSM) at the U.S. Census Bureau will carry out the cognitive interviews remotely. Between three and four staff members will be trained to conduct the interviews to allow flexibility in interview scheduling. All interviewers will be required to demonstrate a strong understanding of the interview protocol before beginning interviews. With participant permission, interviewers will record the screen containing the virtual session. This will allow interviews to review participants’ responses to the self-administered survey as well as verbal responses to probes when writing summaries and c conducting analysis. The CBSM project leader will conduct interviewer training, supervise staff, and monitor data quality throughout the data collection period.

During each interview, regardless of round, participants will be instructed to self-respond to the online survey in its entirety (see Attachment III for full NHIS-Teen questionnaire). Participants will be instructed to think aloud as they respond to the survey, offering insight into their thought process and allowing interviewers to observe whether any questions are potentially problematic. The NHIS-Teen instrument will be programmed in Qualtrics. The interviewer will observe the participant as they answer the survey items, noting any questions or problems the participant has with regards to particular items. After completing the survey, participants will be asked a set of probing questions about some of the survey items, and about any questions or problems the participant had when answering the main survey questions. After all probing questions, interviewers will ask a set of debriefing questions. The survey questions will be iteratively tested within each of the three rounds, with the protocol and question wording being adapted based on the results from the previous round. See Attachment II of this submission for the full cognitive interview protocol.

Cognitive interviewing techniques allow researchers to evaluate potential response error and to assure that the survey provides valid data. In general, the goal of cognitive testing is to assess the participants’ comprehension of the questionnaire items, including question intent and the meaning of specific words and phrases in the item. Data from cognitive interviews can identify potentially problematic questions that are not understood as intended. This testing can also examine the participants’ retrieval of relevant information from memory, decision processes involved with answering a question, and questions that are difficult to answer due to cognitive burden.

The cognitive interviews will assess issues such as:

The subject’s understanding of terms in the survey

How confident the subject is in their response

How they remembered the information they provided in factual questions

Whether they found a response choice that fit their answer

How easy or difficult it is to answer a question

Issues with sensitive questions

Consistency of answers within the questionnaire and in comparison to the expected range of answers

Use of Incentive: To ensure that we can recruit participants from all desired populations, and to thank them for completing the interview, each student will be offered a $50 incentive. A standard envelope containing $50 cash will be mailed via USPS in a single priority mail flat rate envelope. Since some survey questions discuss sensitive topics, the envelopes will also include a small handout with resources for teens (see Attachment I, Section 9). The priority mail flat rate envelopes have tracking and delivery confirmation. Tracking numbers will be sent to participants once the shipping labels are generated.

Estimate of Hourly Burden

Exhibit 1 summarizes the number of interviews to be conducted for all three rounds, as well as the recruitment burden. Each interview will be 60 minutes in length and will allow time to complete the NHIS-Teen survey and administer a set of in-depth retrospective probes about a subset of questions. From prior experience, we anticipate needing to contact approximately 95 individuals to yield the 35 desired interviews to account for ineligible participant and cancellations.

Exhibit 1. Estimated Participants Burden of Cognitive Interviews

Response Type	Number of Participants	Burden Hours per Participant	Total Burden hours*
Recruitment Screener	95	.17	16
Consent Procedure/ Installation of Software	35	.17	6
Cognitive Interview	35	1	35
Total Burden Requested	95		57

*Some numbers have been rounded.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Betsarí Otero Class

Center for Behavioral Science Methods

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-2235

[email protected]

3