OMB Letter

Generic Information Collection Request

Cognitive Testing of National Survey of Children’s Health Questionnaire

Request: The U.S. Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We propose to conduct cognitive interviews to pretest the revised screener survey and topical questionnaires for the National Survey of Children’s Health (NSCH). We are seeking approval for this project.

Background: The National Survey of Children’s Health (NSCH), is conducted by the Census Bureau and sponsored by the Maternal and Child Health Bureau )MCHB) of the Health Resources and Services Administration. The NSCH is conducted using self-administered paper and web modes. The survey first employs a screener instrument (NSCH-S1) and then one child is selected to be the focus of a main topical questionnaire. The three topical questionnaires collect detailed information about children who are aged 0-5 (NSCH-T1), 6-11 (NSCH-T2), and 12-17 (NSCH-T3). The Census Bureau’s Demographic Statistical Methods Division (DSMD) Survey Methodology (SM) team was tasked with evaluating new content for the topical questionnaires in preparation for the upcoming survey cycle. The SM team made recommendations to enhance the proposed new items and updated the topical questionnaires to include the new enhanced content requested by MCHB. There are a total of 19 new items proposed for cognitive testing and possible inclusion on the topical questionnaires in 2021. These include four items on vision screening, five items on nutrition and outdoor play in young children, one item related to unfair treatment due to a disability or health condition, and nine items on COVID-19.

Good nutrition, physical activity, and optimal sleep are essential during the first 5 years of life for children’s growth, health, and development. Conversely, poor nutrition, inactivity, poor sleep, and excessive screen time can lead to childhood obesity, poor motor skills, and a lack of school readiness.^1-3 Evidence-based strategies to promote healthy eating behaviors and encourage physical activity (or active play) often focus on key daily behaviors: encouraging fruit and vegetable consumption, avoiding sugar sweetened beverages, limiting non-educational screen time, and encouraging physical activity. Data at the state level on these behaviors are unavailable for children birth to 5 years of age, and national data require a combination of multiple survey years to have adequate sample sizes. These factors limit the ability to monitor critical practices and behaviors and attenuate federal and state level efforts to prioritize, plan, and evaluate current efforts to support optimal early child health and growth. To address these gaps, five new items on nutrition and outdoor play have been proposed for inclusion in the National Survey of Children’s Health.

The question identification and development process for these five new items was based on a review of existing questionnaires, input from federal partners (Federal Data Consortium on Birth to 24 Months; April 14, 2020), and input from subject matter experts. The proposed dietary constructs are intended to capture frequency of intake of sugar-sweetened beverages, vegetables, and fruit among children under the age of 5. Similar questions assessing these constructs with proxy reports from parents and caregivers have been validated with the population of interest.^{4, 5} Assessing dietary intake during the past week – as opposed to the past month – was based on evidence suggesting that estimates of dietary intake are more accurate for shorter and more recent time frames.⁶ Additionally, using the past week as the reference period aligns with existing National Survey of Children’s Health questions. Although these questions do not permit quantification of intake and are subject to the same limitations (e.g., recall bias, social desirability bias, and proxy reporting) as many other dietary assessment methods, these types of food frequency questions are appropriate for surveillance when the goal is to assess a limited number of dietary factors, or food groups, while also minimizing respondent burden.⁷ The proposed active play questions focus on one aspect of active play – specifically, outdoor play among children 3 to 5 years of age. Since there are no national or state-level estimates of outdoor play for this age group, this measure can fill a critical gap in the literature and is of interest within the context of other existing survey items such as neighborhood and household factors. Outdoor play questions were based on the Burdette proxy report, a two-item survey instrument that assesses parental or caregiver report of outdoor playtime, which has been validated with direct measures of physical activity among preschool-aged children.^8,9 Framing the question within the context of an average weekday and an average weekend day aligns with the language in the original Burdette proxy report and with existing National Survey of Children’s Health questions. Overall, state level data are needed to monitor a constellation of key health behaviors and practices to support optimal early child health and growth. Without these data, national, state, and local-level programs will be unable to effectively prioritize resources to help US children build a strong and healthy foundation for their health and wellbeing.

The next step is to conduct cognitive interviews to evaluate how well the revised materials work and to identify any ways they may be improved. We will also probe on one existing item on the T1 questionnaire and six existing items in the T2 and T3 questionnaires and test the use of gender-neutral (e.g. they, their) pronouns in three questions on the screener instrument. Gender-neutral pronouns were tested in the topical questionnaires during cognitive interviewing for the 2020 NSCH and MCHB requested they now be tested in the screener survey.

The full set of items being tested are enumerated below in tables 1 (new content) and 2 (existing/revised content)

Table 1. New NSCH Questionnaire Content

Question Number/Survey	Question Summary
C12, C12a, C13, C13a/ T1,T2,T3	New items on ever receiving a vision screening, recommendation for additional eye care, seeing an eye doctor, and care received from an eye doctor (replaced existing items)
H7/T1	New item on frequency of sugary drinks
H8/T1	New item on frequency of eating vegetables
H9/T1	New item on frequency of eating fruit
H10/T1	New item on time spent playing outdoors - weekdays
H11/T1	New item on time spent playing outdoors - weekends
I9/T1, I10/T2,T3	New leaf item on treated or judged unfairly because of their disability or health condition
I11/T1, I12/T2,T3	New COVID-19 item – doctor told anyone in household they had/likely had COVID-19
I12/T1, I13/T2,T3	New COVID-19 item – anyone in household tried to get tested for COVID-19
I13/T1, I14/T2,T3	New COVID-19 item – anyone in household been tested for COVID-19
I14/T1, I15/T2,T3	New COVID-19 item – child had health care visits via video/phone because of pandemic
I15/T1, I16/T2,T3	New COVID-19 item – child miss preventive check-ups because of pandemic
I16/T1, I17/T2,T3	New COVID-19 item – reasons contributing to child missing preventive check-ups
I17/T1, I18/T2,T3	New COVID-19 item – events that happened in household as a result of pandemic
I18/T1, I19/T2,T3	New COVID-19 item – school, daycare, other child care arrangement unavailable as a result of pandemic
I19/T1, I20/T2,T3	New COVID-19 item – child separated from a parent/adult caregiver as a result of pandemic

Table 2. Existing/Revised NSCH Questionnaire Content

Question Number/Survey	Question Summary
9/Screener	Child limited in ability to do things -Testing use of non-gendered pronoun within question text
11/Screener	Child needs treatment or counseling -Testing use of non-gendered pronoun within question text
11a/Screener	Whether problem lasts 12 months or longer -Testing use of non-gendered pronoun within question text
H6/T2,T3	Amount of screen time
H8/T2,T3	Handling demands of raising children
H19/T2,T3	Emotional support for parenting
I9/T2,T3	Adult child can rely on
I9/T1 I10/T2,T3	Leaf item on treated or judged unfairly because of their race or ethnic group
I11/T2,T3	Behaviors when family faces problems

Purpose: The purpose of this research is to interview adults from the general U.S. population who have a child or children within the age ranges covered by the three topical questionnaires. The research will assess their understanding of the questions and response options as presented in the proposed revised NSCH survey instruments. In the cognitive interviews, we will assess the clarity and potential effectiveness of the new items and identify respondent issues with comprehension or interpretation, with the goal of producing valid survey items that minimize the potential for measurement error attributable to the questions themselves. The results of the cognitive interviews will influence decisions about final content and wording of the topical questionnaires for the next NSCH survey cycle.

Population of Interest: Participants will be screened during the study recruitment effort to ensure they have a child within the target age ranges of the topical questionnaires. This will allow testing of the NSCH-T1 questionnaire with participants whose reference child is aged 0-5, the NSCH-T2 questionnaire with a reference child aged 6-11 and the NSCH-T3 questionnaire with a reference child aged 12-17. To assess the clarity of items regarding COVID-19 we will attempt to recruit some participants who have been tested for the virus or its antibodies.

Language: The interviews will be conducted in English.

Timeline: Cognitive interviews are planned from August 2020 through October 2020.

Project scope: The cognitive testing of the proposed revisions to the NSCH screener and topical questionnaires will be carried out according to the schedule described in Table 3.

Table 3. Estimated Project Schedule

Recruiting (Ongoing)	Monday 8/18 - Tuesday 10/20
Conduct Cognitive Interviews	Thursday, 8/20 - Friday 10/23
Data Analysis – Preliminary Results	Friday 9/11 - Monday 11/2
Develop Final Content Recommendations	Tuesday 11/10 - Thursday 11/19
Deliver Report	Thursday 11/19

General Protocol: While it is ideal to conduct cognitive interviews in person, due to COVID-19 restrictions, DSMD will conduct this research remotely, via Skype for Business. Skype for Business is a Census Bureau approved platform and allows for secure video conferencing capabilities and screen-sharing to maximize both the respondent and interviewer experience. Skype for Business interviews will be conducted in accordance with guidelines and policies outlined in the internal Census Bureau policy memo Use of Skype for Business for Conducting Title 13 Qualitative Research.

At the start of the interview, the participant will electronically sign an online consent form via Qualtrics that references the OMB approval for the study, the confidentiality of the session, the voluntary nature of the study, and acknowledges that the session will be recorded.

Once the consent form has been signed, any participant questions will be answered and audio recording will begin. The participant will begin the interview by opening a PDF version of the NSCH screener survey from a website provided by the researcher. The participant will see the questions on their screen and will provide their responses and feedback verbally to the interviewer. Upon completing the screener survey, the participant will open a PDF of the appropriate topical questionnaire (based on the age of the reference child). The researcher will walk the participant through the screener and topical questionnaires using a think aloud technique, verbally expressing their thoughts to allow researchers to gain insight into how they interpret and respond to questions. The cognitive interviewer will administer both concurrent and retrospective probes to gain a full understanding of respondent perceptions of specific elements of the survey. After completing the task, if time allows, each participant will be asked a set of debriefing questions.

Sample: A total of 40 participants will be interviewed over two rounds (20 per round) throughout the U.S. Participants will be screened during the study recruitment to ensure they have children within the target age ranges of the topical questionnaires. DSMD SM researchers will conduct up to 10 interviews per round using the NSCH-T1 questionnaire, up to 5 interviews per round using the NSCH-T2 questionnaire, and up to 5 interviews per round using the NSCH-T3 questionnaire.

Recruitment: We will use a multi-pronged strategy to recruit participants, such as posting ads on internet and social media platforms (e.g. Facebook, Craigslist, NextDoor) parenting group pages, and using personal networks. While not intended to be nationally representative, an attempt will be made to recruit participants with varying demographic characteristics (e.g., sex, education, age, race and ethnicity). We will also attempt to recruit participants from areas that have been heavily impacted by the COVID-19 pandemic, in an effort to efficiently test those questions.

Use of Incentive: Due to the difficulty of recruiting this population (parents of young children) during the COVID-19 pandemic, we will offer an incentive of $40 per participant to cover the cost of childcare and the associated burden around childcare at this time.

Below is a list of materials to be used in this research.

1. NSCH 2021 Screener Survey_Cognitive Interviewing Version

2. NSCH 2021 T1_Cognitive Interviewing Version

3. NSCH 2021 T2_Cognitive Interviewing Version

4. NSCH 2021 T3_Cognitive Interviewing Version

Length of interview: Based on past experience with similar pretesting efforts it requires four screener conversations to recruit one participant. Each screener conversation lasts approximately three minutes. We estimate it will take 8 hours to screen and recruit 40 participants. We estimate it will take 60 minutes per participant to complete the cognitive interview. For 40 participants, the estimated burden for the interviews is therefore 40 hours, bringing the total burden to no more than 48 hours with recruitment.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

M. Christopher Stringer

Demographic Statistical Methods Division

U.S. Census Bureau

Washington, D.C. 20233

[email protected]

References:

1. Piercy KL, Troiano RP, Ballard RM, et al. The Physical Activity Guidelines for Americans. JAMA. 2018;320:2020-2028.

2. US Department of Health and Human Services, US Department of Agriculture. 2015-2020 Dietary Guidelines for Americans. 8th ed2015.

3. Woo Baidal JA, Locks LM, Cheng ER, Blake-Lamb TL, Perkins ME, Taveras EM. Risk Factors for Childhood Obesity in the First 1,000 Days: A Systematic Review. Am J Prev Med. 2016;50:761-779.

4. Koleilat M, Whaley SE. Reliability and Validity of Food Frequency Questions to Assess Beverage and Food Group Intakes among Low-Income 2- to 4-Year-Old Children. J Acad Nutr Diet. 2016;116:931-939.

5. Randall Simpson JA, Keller HH, Rysdale LA, Beyers JE. Nutrition Screening Tool for Every Preschooler (NutriSTEP): validation and test-retest reliability of a parent-administered questionnaire assessing nutrition risk of preschoolers. Eur J Clin Nutr. 2008;62:770-780.

6. Smyth JD. Designing Questions and Questionnaires. The SAGE Handbook of Survey Methodology. Vol SAGE Publications Ltd. Los Angeles2016:218-235.

7. Thompson FE, Midthune D, Kahle L, Dodd KW. Development and Evaluation of the National Cancer Institute's Dietary Screener Questionnaire Scoring Algorithms. J Nutr. 2017;147:1226-1233.

8. Burdette HL, Whitaker RC, Daniels SR. Parental report of outdoor playtime as a measure of physical activity in preschool-aged children. Arch Pediatr Adolesc Med. 2004;158:353-357.

9. Rice KR, Joschtel B, Trost SG. Validity of family child care providers' proxy reports on children's physical activity. Child Obes. 2013;9:393-398.

6