Note to Reviewer - NLSY27 Consent Testing Redlined

5/9/2024

NOTE TO THE REVIEWER OF: OMB CLEARANCE #1220-0141

“Cognitive and Psychological Research”

FROM: Robin Kaplan

Research Psychologist

Office of Survey Methods Research

SUBJECT: Submission of Materials for Cognitive Testing of the Youth Assent and Parental Consent forms for NLSY27 New Cohort

This package was modified on 7.10.2024. Changes are tracked and highlighted.

Please accept the enclosed materials for approval under the OMB clearance package 1220-0141, “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the research and the materials to be used in the research.

The maximum number of burden hours is estimated to be 157.

If there are any questions regarding this project, please contact Robin Kaplan at [email protected].

Introduction and Purpose

The National Longitudinal Survey of Youth 1979 (NLSY79) and 1997 (NLSY97) is a longitudinal project that follows a sample of American youth born between 1957 and 1964 with 12,686 respondents aged 14 to 22 for the NLSY79, and born between 1980 and 1984 with8,984 respondents aged 12-17 for the NLSY97. The NLSY79 and NLSY97 collect extensive information on respondents’ labor market behavior and educational experiences. The survey also includes data on the youths’ family and community backgrounds to help researchers assess the impact of schooling and other environmental factors on these labor market entrants. Data from the NLSY79 and NLSY97 also aid in determining how youths’ experiences relate to establishing careers, participating in government programs, and forming families (https://www.bls.gov/nls/nlsy79.htm; https://www.bls.gov/nls/nlsy97.htm).

The NLSY program is in the process of planning for a new youth cohort – the National Longitudinal Survey of Youth New Cohort (NLSY). The NLSY New Cohort (NLSY27) will enable researchers to examine how the lessons learned using previous NLS cohorts may have changed over time, as well as permit the study of new factors affecting the lives and labor market experiences of a new generation. As in previous NLS cohorts, BLS plans to ask the new NLSY cohort members a core set of questions that provide extensive information on employment, training, education, income, assets, marital status, fertility, health, and occupational and geographical mobility, as well as administering cognitive and non-cognitive assessments. BLS is also considering alternative (non-survey) data sources that may potentially be incorporated into the new NLSY cohort to improve accuracy, increase granularity, provide information on new topics, and/or reduce respondent burden. In addition, BLS is considering how innovations in interview techniques can enhance respondent experience and data collection.

As part of this process, the NLS program office is revising their parental consent and youth assent forms to make them clearer and easier to understand. These revised forms should facilitate participation in the NLSY by making the goals of the survey clear and provide key information to youth participants and their parents, including why they were sampled for the survey, the survey topics and procedures, how long the survey takes, incentives, confidentiality, the parents’ role in the data collection process, and information about linking data to other sources. The revised forms also use section sub-headers to make this content easier to follow.

Currently, at the first interview, interviewers walk participants through the consent form in-person during production interviews to obtain consent. Parents sign a form where they consent to their children (ages 11-17) to participate in the NLSY (see Attachment A for the full form language). In addition, youth are provided an assent form to agree to participate in the study (see Attachment B for the full form language). The purpose of this research is to assess the following aspects of the informed consent process with youth and parents of youth eligible for the NLSY:

• Do youth and parents understand the information in the assent and consent forms?

• Do parents understand the concept of consenting to link data to other sources? Are they willing to provide that consent? What concerns do they have about linking data, if any?

• Do younger youth (11-14) understand the assent form differently than older youth (15-17)?

• Do youth and parents think they would be willing to participate in the NLSY? What concerns do they have, if any?

• How can the forms be improved? (e.g., adding more information, reordering the information, etc.)

Methodology, Participants, and Recruitment

To collect feedback on the youth assent and parental consent forms, a combination of online web surveys, and moderated cognitive interviews with parent/child dyads, and intercept testing with youth will be conducted. Iterative testing will be used between the different methodologies so that feedback from each test will be used to inform the next. As this is pretesting, we expect that modifications may be made during the study based on initial results. Although the goals of the testing, and overall design, will remain the same, findings from each round may be used to improve the forms in subsequent rounds. Modifications are likely to be changes to wording on the forms and reordering of the information in the forms and to the debriefing questions to reflect any changes.

Round	Data collection type	Sample size	Participant group
1	Web survey (pretesting)	400	Parents of children 11-14 (n=200) and 15-17 (n=200)
2	Moderated interviews	14 dyads	Parent/child dyads 11-14 (n=7) and 15-17 (n=7)
2	Intercept testing	40	Children 11-14 (n=20) and 15-17 (n=20)
3	Web survey (post-testing)	400	Parents of children 11-14 (n=200) and 15-17 (n=200)

Round 1:

Round 1 of testing will focus on gathering preliminary data from parents about their expectations for consent forms for their children to participate in federal surveys, and initial feedback on the language used in the form.

Web survey pretesting

To gather preliminary feedback on the information that parents expect or want to see in a parental consent form for a federal survey, the first phase of testing will recruit parents with youth in the target age groups (11-14 and 15-17). Parents will complete a web survey where they imagine that their child(ren) have been sampled for the NLSY with a brief description of the survey. They will be asked debriefing questions to better understand what they would want to know about the survey, how they would expect the consent process to work (e.g., mode, level of interviewer interaction), and the top 3 topics they want the parental consent form to address.

Participants will be recruited from CloudResearch, an online marketplace where individuals sign up to participate in online research tasks. They will be provided a link to an instrument hosted by Qualtrics. Participants with children in the age groups of interest (11-14 and 15-17) will be invited to participate in the study. Participants will be informed that they may be invited to participate in a follow-up survey later (see Round 3 of testing). The protocol for the Round 1 web survey is shown in Attachment C.

Because BLS is interested in determining the most effective messaging for future cohort members and parents to link their data to other sources, participants will be randomly assigned to read one of two messages regarding data linkage consent. Half of the participants (n=200) will receive a message framed around the benefits of consenting to link data, while the other half of participants (n=200) will not receive this message.

• Benefit appeal:

In addition, with your permission, your survey responses can be linked to data from public records or other government agencies who authorize it. By linking your survey responses, your data will be more useful to researchers studying the experiences of your child’s generation.

In addition, to assess participants’ comprehension of consent to link records, we will include a select all with eight different agencies or contractors. Participants will be asked to select the agencies that would be able to access the NLSY27 data. This will provide insight into participants’ understanding of the content of the informed consent materials, as well as assess which agencies they believe would gain access to the data if they consented to link their data. One fictional agency was included as an attention check for data quality.

Round 2:

Round 2 of testing will use the revised parental consent form based on feedback from Round 1, and additionally, collect feedback from children ages 11-17 on the youth assent form.

Moderated remote cognitive interviews with parent/youth dyads

Using feedback from the first round of testing, parent/child dyads will participate in remote one-on-one moderated cognitive interviews via Microsoft Teams. Each session will last 60 minutes and will be moderated by OSMR staff. We will recruit equal numbers of parents of children from the younger age cohort (11-14) and older aged cohort (15-17) to assess if there are differences by age group. At the beginning of the session, we will obtain consent from the parent for their child to participate in the interview assessing the youth assent form, as well as the parent’s consent to participate in their own interview assessing the parental consent form. We will either interview the parent about the parental consent form for the first 30 minutes, followed by an interview with the youth for the second 30 minutes; alternatively, we will use Microsoft Teams breakout rooms to simultaneously interview the parents and child (this will depend on the participants’ home setup, including whether two or more devices are available to participate in the interviews simultaneously).

During the session, participants will read the appropriate consent from (parental consent or youth assent). They will provide open-ended feedback on their overall impression of the form. Afterward, they will answer debriefing questions about specific language used in each of the forms, and answer questions to assess comprehension of the form language, ideas to improve the forms, and any questions or concerns they would have consenting or assenting to participate in the NLSY.

To find eligible parent/child dyads for remote one-on-one interviews, we will use the following recruitment methods:

• Advertisements on bulletin boards (on the internet, online marketplaces, and/or in public spaces)

• Snowball sampling, by asking participants to forward the study advertisement to others who may meet the eligibility criteria

• Flyers on bulletin boards at grocery stores, libraries, and other community locations

• Emails to community listservs

Interested people will be asked to contact OSMR staff to learn more about the study and see if they are eligible. The study recruiter will administer screening questions to determine eligibility. The advertisements and screening questions are in Attachment G.

Intercept testing with youth

Youth will participate in a very brief (2-5 minute) session asking for feedback on the youth consent form. Youth ages 11-17 will be approached in public places, such as shopping centers, libraries, and coffee shops. The youth will be presented with a brief description of the NLSY study, and asked 2-3 follow-up questions such as their comprehension of the survey, what else they would want to know about the study before agreeing to participate, and if they think they would be willing to participate in the study. Youth will receive a non-monetary token incentive of their choice (e.g., pens or fidget toys). See Attachment E for the intercept testing protocol.

Round 3:

Round 3 of testing will use the feedback from previous rounds to test a near-final version of the parental consent form.

Web survey with parents

Using the feedback from the previous rounds of testing to revise the forms, the final phase of testing will invite parents who previously participated in the web survey from Round 1 via CloudResearch, and will be provided a link to an instrument hosted by Qualtrics. If not at all 400 participants from Round 1 choose to participate in the follow-up web survey, we will invite new qualifying participants to complete the Round 3 web survey. Parents will read the revised parental consent form NLSY. Then they will be asked debriefing questions about the final version of the form to assess comprehension and areas that could be improved. Participants will be recruited from CloudResearch, an online marketplace where individuals sign up to participate in online research tasks. Responses to the first round of testing will be compared to this round to assess revisions to the parental consent form. The protocol for the Round 3 web survey is shown in Attachment F.

Burden Hours

This study will use up to 157 burden hours.

Web Surveys (Round 1 and 3)

For the web surveys, up to 400 participants per testing round (for a total of 800) with children aged 11-17 will self-select into the study posting on the CloudResearch platform. The web surveys in Rounds 1 and 3 will take no more than 10 minutes to complete.

Moderated interviews (Round 2)

For the remote, moderated cognitive interviews with parent/child dyads, we anticipate making contact with up to 30 people to conduct eligibility screening in order to schedule interviews with 14 parent/child dyads. Screening is expected to take no more than 5 minutes. The interview sessions will last no more than 60 minutes.

Intercept testing (Round 2)

For intercept testing, we expect to contact up to 120 parents and children in public settings to obtain up to 40 participants. Initial contact to gauge interest in the study should last no more than 30 seconds, and the interview questions will last no more than 5 minutes.

Estimated Burden Hours, by Round and Method

Method	Participants contacted	Recruitment hours	Recruitment total hours	Participants completed	Session hours	Session total hours	Total collection burden
Round 1 web survey	400	0	0	400	0.17	68	68
Round 2 moderated interviews	30	0.08	2.4	14 dyads	1	14	16.4
Round 2 intercept testing	120	.008	1	40	0.08	3.2	4.2
Round 3 web surveys	400	0	0	400	0.17	68	68
Totals	950			854			157

Payment to Participants

Web survey participants (in Rounds 1 and 3) will receive $2.50 for participating in the 10-minute survey, a typical rate for this type of task. Participants who complete the moderated interviews (in Round 2) will receive $50 for their participation. This payment is for participant’s costs of participation, including internet connection and child care. The incentive will be issued to the parent only. Youth who participate in intercept testing (in Round 2) will receive a non-monetary token incentive (e.g., pens or fidget toys) valued at no more than $3.00.

Data confidentiality

Web Surveys

Statement for participants completing web surveys

Online survey participants will be informed of the OMB number and the voluntary nature of the study.

This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141 (Expiration Date: July 31, 2027). Without this currently approved number, we could not conduct this survey. This survey will take approximately 10 minutes to complete. If you have any comments regarding this estimate or any other aspect of this study, send them to [email protected]. The BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response. This survey is being administered by Qualtrics and resides on a server outside of the BLS Domain. Your participation is voluntary, and you have the right to stop at any time.

Moderated remote interviews:

Statement for moderated interview participants

Remote interview participants will be read this information (if being interviewed over the phone) or shown the following information through screen sharing on Microsoft Teams, and asked for their verbal consent prior to beginning the study. Data from remote interviews will be recorded in a spreadsheet stored on BLS servers, without participant identifying information.

• We estimate the session will last up to 60 minutes. If you have any comments regarding this estimate or any other aspect of this study, send them to [email protected].

• Your participation in this research project is voluntary, and you have the right to stop at any time.
  
  

• The Bureau of Labor Statistics is conducting this voluntary study under OMB No. 1220-0141, which expires on July 31, 2027. Without this currently approved number, we could not conduct this research. Your responses are also protected by law:

The Bureau of Labor Statistics, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act (44 U.S.C. 3572) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. During this research you may be video recorded or you may be observed. Per the Cybersecurity Enhancement Act of 2015, Federal information systems are protected from malicious activities through cybersecurity screening of transmitted data.

In accordance with the Privacy Act of 1974 as amended (5 U.S.C. 552a), this study is being conducted by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under the authority of 29 U.S.C.2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The information will only be used by and disclosed to BLS personnel and contractors who need the information for activities related to improving BLS information collection. Information on routine uses can be found in the system of records notice, DOL/BLS – 14, BLS Behavioral Science Research Laboratory Project Files (81 FR 47418). 

Do you have any questions before we proceed?

(For parent) Do you agree to you and your child participating?

(For child) Do you agree to participate?

Intercept testing:

Statement for youth intercept testing

Please read the following silently to yourself:

• The Bureau of Labor Statistics, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law.

• In accordance with the Confidential Information Protection and Statistical Efficiency Act (44 U.S.C. 3572) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

• Per the Cybersecurity Enhancement Act of 2015, Federal information systems are protected from malicious activities through cybersecurity screening of transmitted data.

• In accordance with the Privacy Act of 1974 as amended (5 U.S.C. 552a), this study is being conducted by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under the authority of 29 U.S.C.2.

• Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections.

• The information will only be used by and disclosed to BLS personnel and contractors who need the information for activities related to improving BLS information collection. Information on routine uses can be found in the system of records notice, DOL/BLS – 14, BLS Behavioral Science Research Laboratory Project Files (81 FR 47418). 

Do you agree to participate?

Attachments

Attachment A: Parental consent form

Attachment B: Youth assent form

Attachment C: Round 1 web survey instrument with parents

Attachment D: Round 2 moderated interview protocol with parent/child dyads

Attachment E: Round 2 intercept testing protocol with youth

Attachment F: Round 3 web survey instrument with parents

Attachment G: Advertisements and screening questions for moderated interviews with parent/child dyads

10