SSB-Childhood Experiences 4.28.23

Supporting Statement B for Request for Clearance:

COLLABORATING CENTER FOR QUESTIONNAIRE DESIGN AND EVALUATION RESEARCH

Childhood Experiences

OMB No. 0920-0222

Expiration Date: 01/31/2026

Contact Information:

Amanda Titus

Laboratory Manager, Collaborating Center for Questionnaire Design and Evaluation Research

Division of Research and Methodology

National Center for Health Statistics/CDC

3311 Toledo Road, Room 54515448

Hyattsville, MD 20782

301-458-4579

[email protected]

April 28, 2023

Table of Contents

B. Collections of Information Employing Statistical Methods

B.1. Response Universe and Sampling Methods 3

B.2. Procedures for the Collection of Information 3

B.3. Methods to Maximize Response Rates and Deal with Nonresponse 7

B.4. Tests of Procedures or Methods to be Undertaken 8

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting

and/or Analyzing Data 8

LIST OF ATTACHMENTS

Attachment 1: Questions- Childhood Experiences

Attachment 2a-2c: Advertisement

Attachment 3a-3c: Screening script

Attachment 4a-4f: Informed Consent

Attachment 5a-5b: Data collection sheet

Attachment 6: Thank You Letter

Attachment 7:  Data Retention Policy

Attachment 8: Distress Guide

Attachment 9: Nondisclosure Affidavit

B. STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

This proposed collection will use qualitative methodologies.

Qualitative Collections. While survey research employs a deductive, quantitative methodology and relies on a relatively large population-based probability sample to support statistical inference and representativeness, methods such as cognitive interviewing employs an inductive, qualitative methodology and generally relies upon a relatively small sample. Unlike survey research, the primary objective of the qualitative methods the National Center for Health Statistics (NCHS) and the Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) employ is not to produce statistical data that can be generalized to an entire population. Rather, their objective is to provide an in-depth exploration of particular concepts, processes and/or patterns of interpretation. Samples used for qualitative research generally do not achieve full inclusivity of all social and demographic groups. As a general rule, sample definitions are based upon the content of the survey, as well as the purpose and objectives of the particular study.

2. Procedures for the Collection of Information

The methodological aim of this study is consistent with the design of most NCHS/CCQDER cognitive interviewing studies, that is, to understand the construct captured by each question, identify patterns of interpretation across respondent groups, and explore potential sources of response error.

The testing procedure conforms to the cognitive interviewing techniques that have been described in CCQDER’s generic clearance package (OMB No. 0920-0222, exp. 01/31/2026) Analysis will be conducted using the constant comparative qualitative method and will focus on the constructs captured by each question, consistency of patterns across respondent groups, and potential causes of response error. Findings (like all CCQDER studies) will be documented in a final report and made publicly accessible on a searchable website at https://wwwn.cdc.gov/QBank.

Recruitment. We propose to recruit up to 90 English speaking respondents including adults (aged 18 and over) who may or may not be the parent or guardian of a child 0-17 years old and adolescents ages 14-17. In addition to those criteria, demographic diversity and intersectionality is also a priority, with the goal of achieving a purposive sample that includes a mix of genders, age, race, and educational attainment.

Cognitive interviews will be one-on-one between a single interviewer and a respondent and will be no longer than 60 minutes. The 60 minutes includes 55 minutes for completing the interview and 5 minutes for completing the data collection sheet (Attachment 5a-5b). Interviews will be conducted by CCQDER staff. Given the social distancing requirements of COVID-19, interviews will take place via video conference through Zoom. These platforms have been approved by CDC and NCHS Information Security Offices.

We aim to recruit respondents of varying age, race, educational attainment, and socio-economic status (SES) – using household income as a proxy for SES. The initial goal is to recruit groups in equal proportion, to the extent possible – that is, within the constraints of those willing to participate and the inclusion criteria of the study. Nevertheless, recognizing that respondents with different backgrounds and life experiences may interpret the questions differently, we also aim to recruit a sample that enhances our understanding of those with multiple social positions. We aim to recruit respondents with a mix of demographics but with a focus on those with lower SES and minority households. The Principal Investigators will monitor recruitment in collaboration with the CCQDER Operations Team and continuously update recruitment goals to ensure adequate recruitment of populations that may have differential comprehension of the questions evaluated. Furthermore, because qualitative sampling is based on theoretical relevance, it should be noted that on-going analysis may reveal the need to recruit more from one group than others. Recruitment will be carried out through a combination of advertisements, flyers, word-of-mouth, and the CCQDER Respondent Database. The advertisement/flyer used to recruit respondents is shown in Attachment 2a-2c.

Screening and scheduling procedures. The first contact with potential respondents will occur in response to the flyers or advertisements. Interested people will leave contact information (name and telephone number) on the CCQDER voice mail system or in an email. A CCQDER Recruiter will then call the potential respondent and give a brief description of the nature of the study, video conference and recording procedures, and monetary remuneration. The CCQDER Recruiter will determine through a brief series of questions whether the potential respondent possesses experiences salient to the research and characteristics necessary to achieve sample diversity. The 5-minute screener used to determine eligibility of individuals responding to the advertisements/flyers is shown in Attachment 3a-3c. The 5-minute screener used to determine eligibility of individuals from the CCQDER Respondent Database is shown in attachment 5a-5b. Note that wording of the templates has been approved and is contained within our umbrella package. Only project specific information has been added to the documents. It is anticipated that as many as 150 individuals may need to be screened in order to recruit 90 participants.

If the person does possess the desired research characteristics and would like to participate, they will be scheduled for an interview. Otherwise, the volunteer will be asked whether they would be interested in participating in future cognitive interviews. Telephone numbers and the minimal demographic information listed earlier will be obtained for all scheduled volunteers and for those who would like to be contacted in the future. For those callers who are ineligible for the study and do not want to be contacted in the future, only demographic characteristics will be maintained for future analysis of successful recruitment efforts.

Recruiters will also assess respondents’ video conferencing capabilities and will work with them to set up the video conference application (Zoom). After respondents are scheduled, a confirmation email will be sent which will include information about the date and time of interview, instructions for using the video conferencing application and information about informed consent. If the video conferencing application fails to work properly, the recruiter will use this script to cancel the interview.  Script: Unfortunately, we cannot move forward if your [video conference application/Zoom] is not working properly because these interviews will be conducted over video chat. I’m sorry, we have to cancel this interview.  Would it be okay if I added your name, telephone number, age, educational level, and race to our database so that I can contact you about other studies coming up in the future?  If yes, add to database.  If no: OK, thank you for your time.  Your name and any information you gave me will not be added to our database.

Questionnaire Design Research Laboratory (QDRL) Interview Methods.

Interview methodology:

In-person: After respondents have been briefed on the purpose of the study and the procedures that CCQDER routinely takes to protect human subjects, respondents will be asked to read and sign an Informed Consent document (Attachment 4a-4f). Respondents will also be asked to fill in their demographic characteristics on the Respondent Data Collection Sheet (Attachment 5a-5b).

Virtual: For virtual cognitive interviews, respondents and interviewers will see and hear each other through the video conference software from their own computer, tablet, or cellphone from their respective locations. A recruiter will meet the respondent on the scheduled video conferencing appointment (through Zoom). The recruiter will make sure the respondent is prepared for the interview and that the video conferencing application is working properly. The recruiter will go over the informed consent information that was previously emailed to the respondent (Attachment 4a-4f) and remind the respondent of the remuneration procedure. The recruiter will collect the information from the Respondent Data Collection Sheet (Attachment 5a-5b). At this point, the interviewer will join the video conference the recruiter will leave the meeting.

The interviewer will then ask the respondent to confirm that he/she understands the information in the Informed Consent (Attachment 4a-4f), and then state that we would like to record the interview. The recorder will be turned on once it is clear that the procedures are understood and agreed upon.

Once the recording has started, the interviewer will confirm that the respondent has agreed to be recorded. The interviewer will confirm that the respondent has agreed to be recorded by asking, “Do you agree that the interview will be video recorded? Yes, I agree or No, I don’t agree.”

The interviewer will then orient the respondent to the cognitive interview with the following introduction:

[Fill staff name] may have told you that we will be working on some questions that will eventually be added to national surveys. Before that happens, we like to test them out on a variety of people. The questions we are testing today are about [fill].

We are interested in your answers, but also in how you go about making them. I may also ask you questions about the questions—whether they make sense, what you think about when you read certain words, and so on.

For self-administered: I will give you the questionnaire [if virtual: by sharing my screen with you], and you may fill it out as if you received the survey in the mail at home.

For interviewer-administered: I will read each question to you.

I’d like you to answer the questions as best you can. Please tell me if:

there are words you don’t understand,

the question doesn’t make sense to you,

you could interpret it more than one way,

or if the answer you are looking for is not provided.

Afterwards, we will talk about your answers, and I will ask what you were thinking as you figured out how to answer. The more you can tell us, the more useful it will be to us as we try to develop better questions. Okay? Do you have any questions before we start? [If yes, answer questions. If not,] let’s get started.

If virtual and self-administered: I will now share my screen with you. Please go ahead and fill out the form, scrolling down as necessary. When the respondent has finished filling out the form, the interviewer will end the screen-sharing mode and begin discussing respondent answers.

Once the discussion has ended, the interviewer will thank the respondent and either hand them the thank you note (Attachment 5a-5b) and renumeration amount or, in the case of virtual interviews, remind them of the renumeration procedure (via FedEx delivery). Finally, the interviewer will ask: Would it be okay if I added your name, telephone number, age, educational level, and race to our database so that I can contact you about other studies coming up in the future? If yes, the information will be added to the database. If no, the interviewer will reply: OK, thank you for your time. Your name and any information you provided to us will not be added to our database. The interview will then be complete.

Video/Audio Recording:

Video or audio recording is required for this project. These recordings will allow researchers to review the behaviors and body language of the respondents and to ensure the quality of their interview notes. Recordings will be used and retained in accordance with the CCQDER Data Storage and Access Policy (Attachment 7). Researchers from CCQDER and RSS contractors will use the recordings for research purposes. RSS contractors viewing/listening to recordings in the QDRL under CCQDER supervision have read and signed a non-disclosure affidavit and completed confidentiality training.

In-person: All interviews conducted in the QDRL or off-site will be video and/or audio recorded. In rare cases, a study participant may initially agree to be video recorded during the telephone screening but change their mind at the time of the interview. If the respondent changes their mind in the case of an in-person interview, the participant will be asked if they agree to be audio recorded.  If they decline to be audio recorded the interview will proceed without recording.  In this case the interviewer will depend on their handwritten notes when conducting analysis.

Virtual: CCQDER will request permission to video record all virtual interviews conducted via Zoom. If the respondent consents to being video recorded and then changes their mind after the interview has begun, the interview will proceed with audio-only recording. In this case, the interviewer will instruct the respondent to turn off their camera and explain that only audio will be recorded as long as the respondent’s camera remains off. In the event that the respondent’s camera is turned back on, the interviewer will remind the respondent to turn the camera off if they do not want to be recorded. If the respondent does not want to be recorded but is unable to turn their camera off, the interview will be terminated.

NCHS government issued encrypted laptops will be used to video and audio record the interviews conducted off-site or virtually by both CCQDER researchers and RSS contractors. An NCHS government issued encrypted flash drive will be used to transfer the recordings from the interviewer’s laptop to the CCQDER’s secure LAN. The encrypted flash drive is FIPS 140-2 compliant and approved for use by OCISO.

Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets. Once the video and audio recordings are transferred to the QDRL Network, the recordings will be deleted from encrypted flash drive. Once deleted, the files are no longer available for use.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Our experience has shown that paid advertisements on social media, newspapers and flyers attract a large pool of potential laboratory research respondents. These recruitment mechanisms have been productive in the past for obtaining a diverse group of respondents to help us determine potential sources of error in survey questions. For those questionnaires that target specif­ic subgroups, special recruitment procedures will be developed to identify respondents. Direct contact to solicit support from church groups, employers, and/or social or service organizations will be explored as possible recruitment methods. Also, the offer of incentives to help cover out of pocket expenses has been a proven motivation for volunteers to participate in the study.

After laboratory volunteers have been recruited, the probability of the respondent failing to show is minimized by making reminder phone calls to volunteers.

4. Tests of Procedures or Methods to be Undertaken

The purpose of questionnaire evaluation is not to obtain survey data, but rather to obtain information about the processes people use to answer questions as well as to identify any potential problems in the questions. This work has been effective for enhancing the quality of data of CDC, NCHS, and other Federal surveys cognitively tested by the CCQDER for over 30 years.

5. Individual Consulted on Statistical Aspects and Individuals and/or Analyzing Data

The person with overall responsibility for the methodological and technical aspects of the described activities is:

Kristen Miller, Ph.D.

Director, Collaborating Center for Questionnaire Design and Evaluation Research

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland

(301) 458-4625

[email protected]

9