OMB Memo

Generic Information Collection Request

Cognitive Testing of National Survey of Children’s Health Mail Materials

Request: The U.S. Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We propose to conduct cognitive interviews to pretest and vet possible enhancements for the 2020 National Survey of Children’s Health (NSCH) letters and postcards. We are seeking approval for this project.

Background: The NSCH is conducted by the Census Bureau and sponsored by the Health Resources and Services Administration in the Department of Health and Human Services. The NSCH is conducted using self-administered paper and web modes. The survey first employs a screener instrument, which asks whether children 17 or younger live or stay at the household, along with some questions about those children (NSCH-S1). Then, if there are children in the households, a child is randomly selected to be the focus of a main topical questionnaire (Child and Adolescent Health Measurement Initiative, 2017). The three topical questionnaires collect detailed information about children who are aged 0-5 (NSCH-T1), 6-11 (NSCH-T2), and 12-17 (NSCH-T3).

Several changes to NSCH mail materials are being considered for 2020. Beginning in 2020, the NSCH will utilize updated mail materials for outreach with sampled addresses - letters and postcards for the screener surveys and topical questionnaires. The Census Bureau’s Demographic Statistical Methods Division (DSMD) Survey Methodology (SM) team was tasked with enhancing these materials in preparation for the 2020 implementation. The SM team updated the letters based on an expert review of those that are currently used along with adding new visual elements to enhance the reader experience and highlight key messaging related to the survey. The next step is to conduct cognitive interviews to evaluate how well the revised materials work and to identify any ways they may be improved.

Purpose: The purpose of this research is to assess mail materials under consideration for the 2020 NSCH. Employing cognitive interviewing, we will assess the clarity and perception of possible options for the visual design and messaging, with the goal of producing materials that maximize the likelihood that sample cases would respond to the survey. We will interview adults from the general U.S. population to assess their understanding of the messages and information presented in the proposed materials. The results of the cognitive interviews will influence decisions about the final content and format of NSCH letters and postcards for the 2020 survey.

Population of Interest: Participants will be screened during the study effort to ensure that all participants are adults that typically open their mail in their home. An attempt will be made to recruit a mix of individuals with and without children 17 years old or younger that usually live or stay with them. This will allow us to gauge the perception and clarity of the materials among those households eligible and ineligible for the full NSCH survey.

Language: While all materials will contain both English and Spanish messaging, cognitive interviews will be conducted in English only.

Timeline: Cognitive interviews will be conducted from August 21, 2019 to October 25, 2019.

Project scope: The cognitive testing of the proposed revisions to the NSCH mail materials will be carried out according to the schedule described in Table 1.

Table 1. Estimated Project Schedule

Project Activity	Dates
Recruiting for Round 1 (Ongoing)	Tuesday, 8/13/19 – Friday, 9/6/19
Conduct Round 1 Interviews	Wednesday, 8/21/19 – Friday, 9/6/19
Data Analysis – Round 1 Preliminary Results	Tuesday, 9/17/19
Prepare Materials for Round 2	Friday, 9/27/19 – Monday, 10/7/19
Recruiting for Round 2 (Ongoing)	Monday, 9/30/19 – Friday, 10/25/19
Conduct Round 2 Interviews	Tuesday, 10/8/19 – Friday, 10/25/19
Data Analysis – Round 2 Preliminary Results	Tuesday, 11/5/19
Develop Final Content Recommendations	Monday, 11/18/19 – Friday, 11/29/19
Deliver Report	Friday, 11/29/19

General Protocol: Participants will be asked to review proposed mail materials for the 2020 NSCH. Participants will be asked to use the “think aloud” method to explain their feedback on the materials. We will use concurrent and retrospective probing to get a full understanding of the participant’s thought processes and interpretations of different variations of the screener card and associated mailings. After completing the task, each participant will be asked debriefing questions. All interviews will be audio-recorded to facilitate a summary of the results. Participants will be asked to sign consent forms for permission to be recorded.

Sample: A total of 30 participants will be interviewed over two rounds (15 per round) in the Washington, D.C., Baltimore, MD., and Flagstaff, Arizona metropolitan areas. We will screen participants during the study recruitment phase to ensure that at least half in each round have a child 0-17 years old that usually lives or stays in their household. This will help us understand how those who are ineligible and eligible for the NSCH would respond to the mailings.

Recruitment: We will use a multi-pronged strategy to recruit participants, such as posting ads on Craigslist, distributing flyers, and using personal networks. While not nationally representative, an attempt will be made to recruit participants with varying demographic characteristics (e.g., sex, education, age, race and ethnicity).

Use of Incentive: Due to the length and complexity of the task as well as the necessity to travel to test locations, we plan to offer a $40 incentive to offset the costs of participation, such as travel and parking.

Below is a list of materials to be used in this research.

Letters:

1. NSCH11wa_1

2. NSCH11wa_2

3. NSCH11wa_3

4. NSCH12wa(SC)_1

5. NSCH12wa(SC)_2

6. NSCH12wa(SC)_3

7. NSCH13_1

8. NSCH13_2

9. NSCH13_3

10. NSCH21b_1

11. NSCH21b_2

12. NSCH21b_3

13. NSCH22b_1

14. NSCH22b_2

15. NSCH22b_3

Postcards:

1. NSCH-PCW_1

2. NSCH-PCW_2

3. NSCH-PCW_3

Length of interview: Based on past experience with similar pretesting efforts, it requires up to four screener conversations to recruit and schedule one participant. Each screener conversation lasts approximately three minutes. We estimate it will take 6 hours to screen and recruit 30 participants. We estimate it will take 60 minutes per participant to complete the cognitive interview. For 30 participants, the estimated burden for the interviews is therefore 30 hours, bringing the total burden to no more than 36 hours.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

M. Christopher Stringer

Demographic Statistical Methods Division

U.S. Census Bureau

Washington, D.C. 20233

[email protected]

3