Youth Empowerment Information, Data Collection, and Exploration on Avoidance of Sex (IDEAS)
OMB Information Collection Request
0970 – New Collection
Supporting Statement
Part A
August 2020
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families (ACF)
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, DC 20201
Project Officer:
Caryn Blitz
Part A
Type of Request: This Information Collection Request (ICR) is for a new descriptive study. We are requesting three years of approval.
Description of Request: The Administration for Children and Families (ACF) has designed the Youth Empowerment Information, Data Collection, and Exploration on Avoidance of Sex (IDEAS) project as an exploratory study from a nationally drawn sample to identify messages and approaches to empower youth to make healthy and informed decisions related to sexual risk avoidance, teen pregnancy prevention, and their well-being. ACF is seeking approval to collect survey data from a target population of youth ages 14–18, young adults ages 19–24, and parents1 of youth ages 14–18. The sample will be drawn from an online research panel formed through a scientific and systematic process. The data collected from the survey instruments will be used to provide ACF with information to help refine and guide program development in the area of adolescent pregnancy prevention. We do not intend for this information to be used as the principal basis for public policy decisions.
ACF at the U.S. Department of Health and Human Services (HHS) seeks approval for a survey of youth, young adults, and parents from across the country as a part of the Youth Empowerment Information, Data Collection, and Exploration on Avoidance of Sex (IDEAS) project. The Youth Empowerment IDEAS project is a descriptive study designed to identify strategies and approaches that resonate with youth and empower them to make healthy and informed decisions related to sexual risk avoidance, teen pregnancy prevention, and their well-being. Data for the project will be collected through four survey instruments: (1) a Parent Survey (Instrument 1, for parents of youth ages 14 to 18), (2) a Youth Survey (Instruments 2 and 3, divided into two, 20-minute parts, for ages 14 to 18), and (3) a Young Adult Survey (Instrument 4, for ages 19 to 24). Data collected from these surveys will provide ACF with up-to-date information from youth and parents on topics such as communication between youth and parents, attitudes and beliefs about life event sequencing, sexual behavior, and education on topics such as sexual behaviors and health and well-being. The data collected from the Youth Empowerment IDEAS project will provide a unique opportunity to understand youth, young adult, and parents’ thoughts and opinions on sex education and related topics. If the surveys are not implemented, ACF will lack timely data about current appropriate strategies, topics, messages, and themes likely to resonate with youth, young adults, and their parents for use in designing educational programs and policies to address risky sexual behaviors and adolescent health.
There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.
The surveys will collect exploratory data about current youth, young adult, and parent attitudes, opinions, knowledge, and experiences on topics such as communication between youth and parents, attitudes and beliefs about life event sequencing, sex, and sex education. The data will inform current and future Adolescent Pregnancy Prevention and Sexual Risk Avoidance Education (SRAE) programs, both within and outside ACF. Findings will also be communicated to ACF program providers, including current SRAE grantees, to inform programming.
The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker and is not expected to meet the threshold of influential or highly influential scientific information. The data are not intended to draw national prevalence rates on these topics.
ACF proposes to examine the following research questions through the Parent, Youth, and Young Adult Survey data:
What are the Parent, Youth, and Young Adult Survey participants’ attitudes and beliefs related to youth sexual behavior?
What are the Parent, Youth, and Young Adult Survey participants’ knowledge regarding the prevalence of youth sexual risk behaviors?
What are the types of sexual risk behaviors responding youth and young adults engage in?
What topics do parents and youth discuss, and to what extent do parents monitor their child’s behaviors?
What topics are covered in sexual education classes among respondents, and what are the opinions of parents, youth, and young adults regarding what sexual education topics should be covered?
What topics do participating youth and young adults want more information on in sex education?
What are the participating parent, youth, and young adult attitudes and beliefs about life event sequencing?
What are the similarities and differences in responses between parent and child dyad respondents?
For all of these questions, ACF is also interested in how the responses vary across subgroups within the study sample, such as by demographics, geographic location, and (among youth respondents) sexual activity. Attachment A: Research Questions, Sources, and Item Crosswalk provides a crosswalk of the research questions and data collection instruments.
Study Design
The Youth Empowerment IDEAS project is exploratory in nature and aims to gather high quality data from respondents across the country. This study is a one-time, cross-sectional design, with data collection occurring over a 4-month period. The study is not designed to be representative of the target populations or subgroups, as discussed further in Supporting Statement Part B, Section B6. Data for the study will be collected through four survey instruments administered via web with telephone follow-up for nonresponse. Table A2 provides a summary of the study design.
Table A2. Study design summary
Instruments |
Respondent |
Content, Purpose of Collection |
Duration |
||
1. Parent Survey |
Parents of youth age 14-18 |
Content: Focal child’s demographics, parent demographics, parent relationship and communication with the focal child, child’s exposure to sexual education and parental beliefs about sexual education, attitudes and beliefs about youth sexual behavior, and knowledge of youth sexual risk behaviors and the focal child’s behaviors. The survey will also include an item asking for consent for their child to participate in the Youth Survey Purpose: Collect information on parent attitudes, opinions, knowledge, and experiences on topics such as communication between youth and parents, attitudes and beliefs about life event sequencing, sex, and sex education. Conduct dyad analysis with youth. |
20 minutes |
||
2. Part 1 Youth Survey |
Youth age 14-18 |
Content: Demographics, relationship and communication with parents, aspirations, and attitudes and beliefs about youth sexual behavior. Purpose: Collect information on youth attitudes, opinions, knowledge, and experiences on topics such as communication between youth and parents, and attitudes and beliefs about life event sequencing. Conduct dyad analysis with parent. |
20 minutes |
||
3. Part 2 Youth Survey |
Youth age 14-18 |
Content: Knowledge of prevalence of risk behaviors and risk prevention, exposure to and beliefs about sexual education, and sexual risk behaviors. For those respondents who did not already complete Part 1, Part 2 will also contain a set of demographic questions. Purpose: Collect information on youth attitudes, opinions, knowledge, and experiences on topics such sex, and sex education. Conduct dyad analysis with parent. |
20 minutes |
||
4. Young Adult Survey |
Young adults ages 19-24 |
Content: Demographics, relationship and communication with parents when the young adult was in high school, aspirations, and attitudes and beliefs about youth sexual behavior, knowledge of prevalence of risk behaviors and risk prevention, exposure to and beliefs about sexual education while in middle and high school, and sexual risk behaviors. Purpose: Collect information on young adult attitudes, opinions, knowledge, and experiences on topics such as communication between youth and parents, attitudes and beliefs about life event sequencing, sex, and sex education. |
35 minutes |
||
|
|
|
|
||
Data collection will begin with invitation emails sent to the parent and young adult samples (Attachment E: Survey Invitation Emails).2 During the parent survey, the parent will be asked to list all youth in the household between the ages of 14 to 18. A sampling algorithm programmed into the parent survey will be run to randomly sample a focal child, if the parent has more than one child living in the household and meeting the age criteria. At the end of the parent survey, we will obtain the parent’s permission to contact their sampled youth by email, thereby triggering the release of the Youth Survey. Youth Surveys will be conducted on a flow basis throughout the field period as parental consent is obtained through parent completes. Supporting Statement Part B, Section B2, provides details on the study sampling procedures.
The study design intentionally strives to advance both the quality of data collected and the timeliness of data reporting. To achieve the latter goal, ACF seeks to complete all reporting from this effort by September 2021. To allow enough time for analysis and reporting following data collection, the data collection schedule must be succinct, yet also include various nonresponse follow-up strategies to address survey response rate. Non-response is a challenge facing survey research in general, but is exacerbated in youth surveys which require the added step of obtaining parental consent. To address these challenges while keeping to a 4-month data collection schedule, we propose identifying parent, youth, and young adult respondents through an existing, probability-based research panel. The AmeriSpeak research panel, operated by the National Opinion Research Center (NORC), is recruited using a two-stage, address-based sample design and has previously been used on the OMB-approved study for the CDC, “Test Predictability of Falls Screening Tools” (OMB Control #0920-1220). Using an existing panel saves time and improves the survey completion rate as respondents have already been recruited by NORC and have agreed to participate in web-based surveys. The panel is also critical for the purpose of surveying youth on a sensitive topic. As parents hold trust in the panel and understand that the surveys through the panel are research-based, there is an increased likelihood that parents will provide consent for their youth to participate, when compared to other modes of sampling and data collection. The use of the panel supports and preserves the schedule of a 4-month data collection period with analysis and reporting completed by September 2021.
Due to the focus of the study, the surveys include some sensitive questions (see Section A11). Therefore, data will be collected through a web survey, an ideal mode to reduce social desirability bias when respondents are posed sensitive questions (Kreuter et al. 2009).
Two potential limitations to the study design arise from using the AmeriSpeak panel. Panel members may become conditioned to completing surveys, and thus answer questions differently from their non-panel peers, or they may become fatigued from receiving multiple survey requests. Panel conditioning has been associated with responses to knowledge questions, but the effect was attributed to respondents having been exposed previously to knowledge questions about the same topic (Toepoel, Das, and van Soest 2009). It is unlikely that knowledge questions about the prevalence of youth sexual risk behaviors have appeared in many previous AmeriSpeak surveys. In addition, NORC minimizes panel fatigue by controlling the number of invitations panel members receive (see Supporting Statement Part B, Section B5).
Supporting Statement Part B, Section B2 discusses the data collection procedures in more detail.
Development of the surveys began with a review of 11 national surveys and 8 previously used research and evaluation instruments conducted by the contractor for this study, Mathematica. Additional details on the source instruments can be found in Attachment A: Research Questions, Sources, and Item Crosswalk. To achieve the specific goals of the research questions, we used items from these instruments where possible, but it was necessary to adapt numerous items and develop new items to meet the specific research goals of this study. Attachment A indicates the source for the question if applicable and identifies any new questions the agency has developed and tested. A comprehensive pre-test of the survey instruments was conducted under generic ICR 0970 – 0355.
Other Data Sources and Uses of Information
No other data sources or prior data collections are associated with this request.
ACF is using technology to collect and process data to reduce respondent burden and make data processing and reporting timelier and more efficient.
The contractor will program and administer the Parent, Youth, and Young Adult Surveys with Confirmit, a state-of-the-art survey software platform that the contractor uses to build and launch multimode surveys. Confirmit offers several advantages for respondents, including enabling web respondents to participate on their own time and using their preferred electronic device (smartphone, tablet, laptop, or desktop computers). The software has built-in mobile formatting to ensure that the display adjusts for device screen size. If needed, respondents can pause and restart the survey, with their responses saved. Confirmit includes tailored skip patterns and text fills. These features allow respondents to move through the questions more easily and automatically skip questions that do not apply to them, thus minimizing respondent burden.
ACF has carefully reviewed the information collection requirements to avoid duplication with existing studies or other ongoing federal teen pregnancy prevention evaluations. We believe that this requested data collection complements, rather than duplicates, the existing literature and the other ongoing federal teen pregnancy prevention data collection efforts. To construct the surveys for the Youth IDEAS Empowerment project, we identified seven topic areas that addressed our research questions and plans for subgroup analyses:
Demographics
Relationship and communication between parent and child
Attitudes and opinions about work, education, and relationships
Attitudes and beliefs about youth sexual behaviors
Knowledge of youth sexual risk behaviors
Exposure and opinions about sex education in school
Youth sexual risk behaviors (Youth and Young Adult Surveys only)
Although several large federal studies, such as the National Survey of Family Growth and the Youth Risk Behavior Survey, include some topics that will be covered in the surveys for the Youth Empowerment IDEAS project, none of the existing surveys cover the scope and depth of topics related to the research questions of interest for this study, nor do the existing studies attempt to obtain data from parent and youth dyads. Two other studies—the nonfederal National Longitudinal Study of Adolescent to Adult Health (Add Health) and the ACF-sponsored National Survey of Adolescents and Parents—reached parents and youth, but neither covers the scope of topics as exhaustive as the Youth Empowerment IDEAS project. In addition, the data are no longer timely, as earlier waves of Add Health that included youth and young adults (Waves I-III) are now almost 20 years old (Harris et al., 2009), and the National Survey of Adolescents and Parents was conducted in the mid-2000s.
As discussed in Section A2, to the extent possible and where appropriate, questions from existing federal and national surveys of adolescents, young adults, and parents were used or adapted to construct the surveys.
A5. Impact on Small Businesses
No small businesses are affected by the information collection.
This is a one-time data collection.
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and OMB regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. A first notice was published on February 8, 2019, Volume 84, Number 27, page 2874, and provided a 60-day period for public comment. A copy of this notice is included in Attachment B: 60-Day Federal Register Notices. During the notice and comment period, 22 comments were received. Most comments pertained to the data collection in general, however, three were related specifically to the survey instruments and resulted in revisions to the instruments. Table A8.1 provides a summary of these three comments, by individual survey-related comments and actions taken.
Table A8.1 Public comments and responses, related to the survey instruments, from notice published February 8, 2019
Public Comment |
Response/Updates Made |
The Proposed Data Collection replicates several questions taken from more than 15 different surveys nationwide. Replicated questions do not merely relate to topics such as demographics or personal information. This duplication contravenes the goal of the Paperwork Reduction Act of 1995 by imposing unnecessary paperwork burdens to individuals completing the survey. |
As part of our review of instruments currently in use, none covered the aims of the Youth Empowerment IDEAS project, which is looking for current attitudes and beliefs of parents and children. Where applicable, we used previously tested item from these instruments. When developing the survey, items from the NSAYA (Olsho et al., 2009) survey were considered, but data from this survey were not considered relevant, as it is already over a decade old. Additionally, since this public comment was received, the surveys have undergone another round of revisions with feedback from pretests and ACF leadership review. It is no longer the case that eight out of ten questions in this section are taken from other surveys. |
The survey uses language that reinforces bias by taking the heterosexual orientation of their readers for granted. For instance, Section G of the Youth Survey Draft addressing “sexual risk behaviors” provides several definitions of sex that must include a male penis. This definition overlooks the LGBTQ community, especially lesbians who under these definitions would be incapable of having sex. |
We have updated the introductory text at the start of Section G to clarify that we are asking about 3 types of sexual behaviors, based on a recommendation received during the public comment period. Additionally, the survey language has been revised, as noted below, to be more inclusive throughout. |
Youth Survey Question A6. The current question does not appropriately address transgender and gender non-conforming youth. Recommendation made to remove “Something else” option and replacing it with: “Transgender man,” “Transgender woman,” “Genderqueer or gender non-conforming,” and “Another gender identity.” |
This item has been re-written to include “Transgender man,” “Transgender woman,” and “Another gender identity” in place of “Something else” category. |
Youth Survey Questions B1-B3, B5-B7, F5, F9; Parent Survey Questions C1, C2, C4. To improve inclusivity and reduce heteronormative language, recommendation made to add three additional items for each of these questions: Benefits of masturbation and solo sexual pleasure; Diverse gender identities and sexual orientations; Experiences of guilt and shame from masturbation or sex. |
Questions in these sections allow for an open-ended response so that participants may write-in their own response to address any topics we may not have covered. Open-ended responses will be reviewed and coded during analysis. |
Youth Survey Questions C16- 21. The phrasing of these questions presupposes a binary view of gender (either “male” or “female”). Recommendation made to use more expansive, inclusive language, such as “romantic partner” in place of “boyfriend or girlfriend.” |
The survey instruments have been updated to include “romantic partner.” |
Youth Survey Questions F5 and F9. The survey question says, “Female and male reproductive systems,” and language assumes a common form of female and male reproductive systems that is not inclusive of transgender and intersex youth. Recommendation to use more expansive and descriptive language. |
The survey has been updated with this more inclusive response language. |
Youth Survey Section G – Sexual Risk Behavior. The definition of “sex” in this section is very narrow - limited to vaginal, oral, and anal sex. The definition might not match up with youth expectations or definitions of sex and could send a stigmatizing message that other forms of sexual behaviors are unnatural or invalid. Recommendation to change the introductory language at the beginning of Section G. |
The survey has been updated with more inclusive introductory language at the start of this section. |
In addition to the changes to address public comments, the survey underwent revisions based on the pretest, described in Supporting Statement Part B, Section B3.
Due to these changes, a second notice was published on April 2, 2020, Volume 85, Number 65, pages 18575-18576, also providing a 60-day period for public comment. A copy of this second notice is included in Attachment B: 60-Day Federal Register Notices. During the second comment period, ACF received a total of 1,111 comments. Of those, 1,107 were from individuals submitting nearly identical comments through grassrootmessages.com. Attachment G: Individuals Submitting Public Comments Through Grassroots Messages provides a list of those individuals. Of the remaining comments received, one comment was non-germane, and the three others were from Planned Parenthood Federation of America, American Atheists, and Equality North Carolina. The 1,110 germane comments were similar in nature, expressing disagreement with the proposed data collection. Concerns centered around duplication of other survey efforts, stigmatization of LGBTQ populations, failure to include important constructs with regard to comprehensive sex education (e.g., gender roles, sexual orientation, and gender identity), sufficient established research in this topical area, and asking parents unnecessary questions related to political and religious views. Table A8.2 provides a summary of these comments, by individual survey-related comments and actions taken.
Table A8.2 Public comments and responses, related to the survey instruments, from notice published April 2, 2020
Public Comment |
Response/Updates Made |
The Proposed Data Collection instruments are not inclusive of the LGBTQ community. |
Throughout the survey instruments we have attempted to use inclusive, non-heteronormative language and have sought to provide a wider range of response options. For example, when defining the term “sex” we use neutral language to describe and distinguish between oral, vaginal, and anal sex such that they are not described using biological body parts. Throughout the instruments, we use gender neutral language to describe relationships. For example, when describing relationships, we use the phrasing “romantic partner, boyfriend, or girlfriend” to be fully inclusive. This wording was pretested extensively with youth, young adults, and parents and we received positive feedback on the wording as being inclusive. In addition, throughout the instrument questions allow for an open-ended response so that participants may write-in their own response. Open-ended responses will be reviewed and coded during analysis. |
The survey instruments fail to assess the inclusion of several core components of comprehensive sex education, such as teaching about gender roles, sexual orientation, and gender identity. |
We have sought to collect important variables of gender and sexual orientation on the surveys. For example, as context, the survey instruments include an item asking participants to self-identify their gender and another item that asks youth and young adults to self-identify their sexual orientation/attraction. The section on Exposure and Opinions about Sex Education contains a number of survey questions that seek to measure a broad range of topics that students may receive in a comprehensive sex education course, including topics such as reproductive biology, learning how pregnancy occurs and how to avoid it, learning about sexually transmitted diseases and infections and ways to prevent transmission, and learning about specific forms of birth control (condoms, birth control pills, the patch, the shot, the ring, an IUD, or an implant. In addition, questions in this section allow for an open-ended response so that participants may write-in their own response. Open-ended responses will be reviewed and coded during analysis. |
The proposed survey asks children and parents invasive and unnecessary questions about their political and religious beliefs. This has no place in a data collection about the effectiveness of sex education. |
Understanding how religious belief might moderate respondents’ behaviors and attitudes will be important when interpreting survey findings and translating them to actionable steps in improving programming. Participants will be able to skip past any questions they do not wish to answer. |
The proposed data collection instruments are unneeded as data about the efficacy of sex education programs is already collected through the Youth Risk Behavior Survey, the Teen Pregnancy Prevention Program, and other federal government mechanisms. |
One of the unique goals of the proposed data collection is to obtain direct feedback from parents, young adults, and youth on what they believe should be taught in sexual education programs. The survey instruments do not attempt to contribute to the literature on sexual education program effectiveness. |
Concerns with anticipated number of completes for a web survey, annualizing burden over three years and that the time per participant is too low.
|
The number of anticipated completes included in the burden estimates for the data collection are based on similar efforts NORC has conducted with the Youth Empowerment IDEAS project target population. We understand the concern of more people responding to the survey than anticipated. As a precaution, the study team will monitor response rates daily and will release cases for the project in short waves to ensure that the number of completes does not go over the anticipated burden estimates. It is the OMB standard procedure to calculate burden estimates over a three-year period. The time per participant included in the burden estimates are calculated based on the average length of time participants took to complete the survey during pretests. Pretests were conducted with a range of ages for each instrument, including 14-year olds. We calculated the start and finish time for each participant and averaged them across all participants, by instrument. Additionally, while we recognize that some of the topics in the survey are sensitive, the pretest did not uncover issues with reluctance to answer items or inability to answer items by the younger participants. Participants will be able to skip past any questions they do not wish to answer. |
The survey instruments do not ask about the subject of abortion, education about abortion, or emergency contraceptive methods such as the morning-after pill. The instruments do not ask for information regarding the effectiveness or thoroughness of sex education instruction, and they actually suggest to respondents that instruction regarding long-term birth control methods might “make it more likely for teens to have sex.” The omission of questions related to select sex education topics and the suggestive phrasing of questions related to topics avoided by SRA programs are scientifically unsound and bias the collection instruments and their potential results towards SRA programs.
|
We did not include questions on the effectiveness of sex education received within the survey. The survey is not focused on evaluating the efficacy of the programs that youth participated in, but rather understanding the topics that are covered and the topics they think are important to cover. The item referenced is asking for youth’s opinion on instruction of long-acting methods and during pretests we did not receive feedback that youth or parents found the item problematic. We do have several items in Youth Survey, item F5 on different topics related to teaching birth control methods that aim to understand the thoroughness of instruction in this area. Regarding the omission of questions related to topics avoided by SRA programs, during the survey pretest, youth were consistently asked if any topics or messages were missing from the lists. Based on this feedback, we added several items to the list of topics covered in sex education and in the final round of pretesting there were no remaining suggestions of topics. Throughout the instrument questions allow for an open-ended response so that participants may write-in their own response. Open-ended responses will be reviewed and coded during analysis. |
The proposed collection instruments are also biased in support of opt-in provisions to school sex education programs. Several questions ask whether respondents believe that parents should decide whether or not their children participate in sex education classes, but the instruments do not ask whether respondents believe such decisions should involve opt-in or opt-out provisions. |
The question is designed to get a sense of youth attitudes/opinions on parent’s role in sex education. While we understand that opt-in and opt-out provisions are very different, the goal of this item was not to understand how parents determine participation, but to understand if youth think parents should have any role in that decision making. The item was added to the survey based on feedback from youth in the pretest who felt this was important to capture. |
The proposed collection instruments suggest a false equivalency between various methods of birth control. Specifically, the collection instruments ask respondents about their use of condoms and birth control pills, as well as other contraceptive devices, alongside far less effective methods such as withdrawal and natural family planning. |
The referenced questions are asking about methods used at first intercourse and most recent intercourse. The list was adapted from a question used in YRBS, with the addition of natural family planning. While not common among youth, this addition was included because we are surveying youth/young adults up to through age 24. While we understand that the methods noted are less effective, it is important to understand if youth are regularly using them so program messaging can be directed towards teaching on the effectiveness. |
The proposed collection instruments are highly flawed in their methodology for achieving their stated purpose. ACF states that one major goal of the proposed collection activity is to “identify messages and themes that are most likely to resonate with youth.” However, the proposed collection instruments do not ask about what messages and themes would resonate with teen and young adult respondents. Rather, the instruments inquire as to some opinions held by parents, teens, and young adult respondents regarding some of the messages and themes that ACF unilaterally developed. |
One of the unique goals of the proposed data collection is to obtain direct feedback from parents, young adults, and youth on what they believe should be taught in sexual education programs. The survey instruments contain a number of items developed and extensively tested with parents, youth, and young adults that cover a wide variety of messages and topics most often taught in sexual education programs and were reviewed by a number of experts (see Table A8.3). The survey pretesting period provided a time to test the messages and themes in the instruments and to identify others. During the survey pretest, youth and young adults in three different geographical regions of the U.S. were consistently asked if any topics, themes, or messages were missing from the survey overall or specifically from the items with lists of topics and messages. Based on their feedback, we revised the wording of some questions and added several items to the list of topics covered in sex education. In the final round of pretesting youth and young adults did not have further additional suggestions of messages or topics and frequently indicated that the instruments had covered the topics well. |
To develop the Parent, Youth, and Young Adult Surveys, ACF consulted with national subject matter experts in adolescent health, sexual education, and instrument development. Table A8.3 displays the experts consulted and their affiliation.
Table A8.3. Experts Consulted During Survey Development of the Youth Empowerment IDEAS Project
Name |
Affiliation |
Brooke Hempell |
Senior Vice President of Research, Barna Research |
John Vessey |
Associate Professor of Psychology, Wheaton College |
Lisa Rue |
Senior Advisor and Strategic Partnerships at Cliexa |
Peter Scales |
Senior Fellow, Search Institute |
Scott Gardner |
Professor, Brigham Young University – Idaho |
Tim Rarick |
Professor, Brigham Young University – Idaho |
Wade Horn |
Director, Deloitte Consulting LLP’s Public Sector Practice; Former Assistant Secretary, ACF |
OMB guidance identifies a number of factors to be considered in the decision to offer tokens of appreciation; the proposed tokens in this study address a number of them: data quality, complex study design, improved coverage of minority populations, and past experience (Office of Management and Budget 2016).
Participants in the Youth Empowerment IDEAS project are participating in the AmeriSpeak panel. AmeriSpeak does not offer cash remuneration, and instead provides survey choice points to redeem for prizes such as movie tickets and gift cards.
We plan to offer Parent and Young Adult Survey respondents AmeriSpeak points (AmeriPoints) equivalent to $20. For the Part 1 Youth Survey, eligible sample members will be offered AmeriPoints equivalent to $15. We expect that the Part 2 survey will have a lower response rate than Part 1, thus, as a token of appreciation for completing the Part 2 Youth Survey, eligible sample members will be offered AmeriPoints equivalent to $20. Because the youth sample is derived from the parents, it is possible that some youth selected are not AmeriSpeak panel members (see Supporting Statement Part B, Section B.2 for more information on sample formation). Non-panel member youth who complete the survey will be provided with gift cards in the equivalent amounts, $15 for Part 1 and $20 for Part 2, for their participation.
The incentives are intended to improve data quality, particularly by improving coverage among minority populations and increasing participation among both parties in the parent–youth dyads. The teen birth rate varies widely across racial/ethnic groups (Martin et al. 2017). Rates of diagnosed sexually transmitted infections (STIs) are also higher among racial or ethnic minorities (CDC 2017). The groups more likely to experience teen pregnancy and STIs also tend to be underrepresented in research studies. Incentives have been shown to increase participation among minorities and other hard-to-reach groups, including individuals with lower incomes or education levels (Singer and Kulka 2002). In addition, Berlin et al. (1992) evaluated differences in survey completion rates between no incentive, $20, and $35 and found significantly higher response rates among minority and low education populations when respondents received either the $20 or $35 financial incentive. By encouraging sample members who are otherwise reluctant to respond to the survey, the offer of incentives reduces the risk of the study producing inaccurate or unreliable estimates and improves representativeness. This is especially important so that the views of parents and youth in groups disproportionately impacted by teen pregnancy and STIs can be collected to inform the topics and strategies incorporated into sexual risk avoidance programming. Completing data collection with parent–youth dyads also addresses the research objectives so that the similarities and differences in perspectives between parents and their children can be explored.
The proposed incentive amounts are similar to the $25 post-pay incentive that OMB approved for respondents who completed the web-based Parent Survey for the Project LAUNCH Cross-Site Evaluation (OMB #0970-0373). After data collection started, it was found that early sample members were not responding. Completion rates and representativeness both improved following the addition of the incentives (Lafauve et al. 2018).
Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier. During recruitment for the AmeriSpeak research panel (unrelated to this study), NORC collects individual respondent’s PII (for example, name, address, email address, and the names and ages of household members) solely for purposes of conducting its research business, which includes prequalifying members or households for surveys and communicating with panel members. After selecting panel members, NORC uses periodic surveys to collect a range of demographic, social, health, attitudinal, and other information about panel members and their households. PII collected by NORC for conducting its research business is maintained in a separate database from completed questionnaires and computerized data files used for analysis. The contractor, Mathematica, will not have access to this PII. Mathematica will provide NORC with links to the web survey that NORC will communicate to selected panel members. PII collected in the Parent Survey for the purposes of contacting youth will be stored separately from all survey data. The survey response data are identified only by an incremented ID number.
Assurances of Privacy
Information provided during this data collection will be kept private to the extent permitted by law. During the informed consent process and assent process for youth under 18, respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law (see Section A11 for Institutional Review Board [IRB] details). As specified in the contract, the contractor will comply with all federal and departmental regulations for private information.
Every AmeriSpeak panelist is provided a Privacy Statement (Attachment C: AmeriSpeak Privacy Statement) that outlines the information that will be collected and how the information will be used. Because each panel member is asked to provide key demographic data such as age, gender, race/ethnicity, state of residence, household income, and more, the Privacy Statement also tells panel members how they can verify the accuracy of their PII and how they can request that the information be deleted or updated. The AmeriSpeak Privacy Statement includes the following:
A promise to treat all AmeriSpeak panelists and their information with respect.
The assurance that participation in any AmeriSpeak study is completely voluntary and that panel members may choose not to answer any questions that they do not wish to answer. Furthermore, panel members may withdraw their participation in AmeriSpeak at any time.
AmeriSpeak will not share the PII with any clients unless panel members have given explicit permission to do so. Only survey responses will be shared with clients.
AmeriSpeak has established security measures to protect the security and confidentiality of its panel members.
Panel members control their personal information and have the right to view their personal information or ask AmeriSpeak to delete it.
Due to the sensitive nature of this research, the evaluation will obtain a Certificate of Confidentiality. The study team will apply for this certificate and provide it to OMB once it is received. The Certificate of Confidentiality helps to assure participants that their information will be kept private to the fullest extent permitted by law.
Data Security and Monitoring
As specified in the contract, the contractor will protect respondents’ privacy to the extent permitted by law and comply with all federal and departmental regulations for private information. The contractor has developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ PII. The contractor will ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor who perform work under this contract or subcontract are trained on data privacy issues and comply with the above requirements. All study team staff involved in the project will receive training on (1) limitations of disclosure; (2) safeguarding the physical work environment; and (3) storing, transmitting, and destroying data securely. All Mathematica staff sign the Mathematica Confidentiality Agreement, complete online security awareness training when they are hired, and receive annual refresher training thereafter.
As specified in the evaluator’s contract, the contractor will use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The contractor will securely generate and manage encryption keys to prevent unauthorized decryption of information in accordance with the Federal Processing Standard. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology requirements and other applicable federal and departmental regulations. In addition, the contractor has a plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or personally identifiable information that ensures secure storage and limits on access.
The goal of the Youth Empowerment IDEAS project is to identify age-appropriate strategies, skills, messages, and themes likely to resonate with youth on topics of sexual risk avoidance and teen pregnancy prevention education. Questions pertaining to sexual attitudes and the behaviors of youth and sexual attitudes of their parents can be sensitive in nature, but the questions are necessary to meet the main study objectives. Table A11 indicates the survey topics and questions deemed sensitive on the Parent, Youth, and Young Adult surveys. IRB review of the data collection protocol, consent, and assent forms will be initiated upon OMB approval of the study.
All survey participants will be informed about the sensitive nature of the questions and that they can refuse to answer any questions that they are uncomfortable answering at the start of the survey. The same applies to youth younger than 18 years old during the parental consent process.
Although high levels of nonresponse are not expected on these items, asking sensitive questions can increase item nonresponse. Using a web survey helps to mitigate this as participants are more likely to disclose sensitive information when an interviewer is not present (Tourangeau et al. 2000).
Table A11. Summary of Sensitive Questions to Be Included on the Parent, Youth, and Young Adult Surveys and Their Justification
Topic and Survey Questions |
Justification |
Gender identity and sexual orientation Youth A4a, A4b, A5 Young Adult Survey A4a, A4b, A5 Parent H22a, H22b |
ACF has a strong interest in improving programming that serves lesbian, gay, bisexual, transgender, and questioning youth and is sensitive and inclusive of all youth. To better understand these subgroups, questions on gender identity and sexual orientation are included in the survey. |
Religion Youth A10 Young Adult Survey A10 Parent Survey H17 |
Research has shown that religion or religiosity has a protective effect on youth sexual behaviors (Vesely et al. 2004). Understanding how religious belief might moderate respondents’ behaviors and attitudes will be important when interpreting survey findings and translating them to actionable steps in improving programming. |
Attitudes and beliefs related to sexual activity Youth Section D Young Adult Survey Section D Parent Survey Section F |
To understand what adolescent pregnancy prevention messages connect with youth and their parents, it is necessary to understand their attitudes and beliefs related to sexual activity. Responses to questions in these sections can guide the direction of future programming, ensuring it addresses common attitudes and beliefs among youth and parents. |
Sexual activity (including intentions) and contraceptive use Youth Section G Young Adult Survey Section G Parent Survey G3 |
The surveys are designed to measure knowledge, attitudes, behaviors, communication, education, and expectations related to teen sexual behaviors. These questions address the behavioral component and are necessary to provide context to other topical areas of the survey, such as attitudes and expectations. The majority of these questions are asked only of youth and young adults who report being sexually active. For respondents who do not report sexual activity, we ask their intentions regarding sexual activity. Intentions for engaging in sex and other risk-taking behaviors are extremely strong predictors of subsequent behavior. Intentions will be an important mediator predicting behavior change. |
Explanation of Burden Estimates
Table A12 shows the estimated respondent burden for the data collection instruments under the Youth Empowerment IDEAS project.
Parent Survey. Email invitations will be sent to 4,586 eligible parent online research panelists, and we estimate that 1,550 will complete the Parent Survey. Based on pre-test timings, we estimate it will take 20 minutes to complete this survey over the web, for an estimated annual burden of 172 burden hours (1,550/3 * 0.333).
Youth Survey. The Youth Survey will be split into two 20-minute surveys (Part 1 and Part 2) to improve response rates and reduce breakoffs, as explained in Section A9 and in Supporting Statement B, Sections B4 and B5. We estimate receiving parental consent from 80 percent of the 1,550 parents who complete the Parent Survey. Based on this, we estimate that we will send survey invitations to 1,240 youth. Of those, we estimate that 675 will complete Part 1 and 590 will complete Part 2, for a total annual burden of 75 hours (675/3 * 0.333) and 66 hours (590/3 * 0.333), respectively.
Young Adult Survey. Email invitations will be sent to 3,183 young adults for the Young Adult Survey. Of those, we estimate that 775 will complete the survey. The survey will take approximately 35 minutes to complete, for a total annual burden of 151 hours (775/3 * 0.583).
Table A12. Total Burden Requested Under this Information Collection
Instrument |
Total Number of Respondents |
Annual Number of Respondents |
Number of Responses per Respondent |
Average Burden Hours per Response |
Annual Burden Hours |
Average Hourly Wagea |
Total Annual Cost |
|
1. Parent Survey |
1,550 |
517 |
1 |
.333 |
172 |
$25.72 |
$4,423.84 |
|
2. Part 1 Youth Survey |
675 |
225 |
1 |
.333 |
75 |
-- |
-- |
|
3. Part 2 Youth Survey |
590 |
197 |
1 |
.333 |
66 |
-- |
-- |
|
4. Young Adult Survey |
775 |
258 |
1 |
.583 |
151 |
$25.72 |
$3,858.00 |
|
Estimated Annual Burden Total |
464 |
|
$8,281.84 |
|||||
a Average hourly wage for parents and young adults is the “All Occupations” Wage from the May 2019 Bureau of Labor Statistics.4
Estimated Annualized Cost to Respondents
The estimated hourly wage for the parent and young adult respondents is $25.72, the mean hourly wage of “All Occupations” from the Bureau of Labor Statistics (May 2019). The estimated annual cost to respondents (based on burden hours for parent and young adult respondents) for the proposed data collection is $8,307.56 (323 hours * $25.72).
There are no additional costs to respondents.
The total cost for the data collection activities (see Table A14) under this current request will be $1,722,660. Annual costs to the federal government will be $574,220 for the proposed data collection.
Table A14. Estimated Total Cost by Category
Cost Category |
Estimated Costs |
Instrument Development and OMB Clearance |
$788,935 |
Field Work-Online Research Panel |
$533,725 |
Analysis/Reporting |
$400,000 |
Total costs over the request period |
$1,722,660 |
Annual costs |
$574,220 |
This is a new data collection.
Nearly all survey data will be categorical or dichotomous (for example, have parents ever discussed various topics with their child). We will calculate percentages for these variables and the mean for any continuous variables (for example, respondent age). We will tabulate these estimates separately for subgroups of interest, such as younger and older youth, and compare responses between the parent, youth, and young adult samples. We will also analyze parent–youth dyads to better understand how parents and their youth have similar or different opinions, experiences, and perspectives.
Data collection and reporting will be completed according to the following schedule (pending OMB approval):
October 2020-February 2021: Collect data from parents, young adults, and youth
March–April 2021: Prepare survey data for analysis
May–September 2021: Conduct analyses and prepare reports on findings
No exceptions are necessary for this information collection.
Attachments
Attachment A Research Questions, Sources, and Item Crosswalk
Attachment B 60-Day Federal Register Notices
Attachment C AmeriSpeak Privacy Statement
Attachment D Technical Overview of the AmeriSpeak Panel
Attachment E Survey Invitation Emails
Attachment F Survey Nonresponse Reminders
Attachment G Individuals Submitting Public Comments Through Grassroots Messages
Instruments
Instrument 1 IDEAS Parent Survey
Instrument 2 IDEAS Youth Survey Part 1
Instrument 3 IDEAS Youth Survey Part 2
Instrument 4 IDEAS Young Adult Survey
Berlin, M., L. Mohadjer, J. Waksberg, A. Kolstad, I. Kirsch, D. Rock, and K. Yamamoto. “An Experiment in Monetary Incentives.” In Proceedings of the Survey Research Methods Section, American Statistical Association. Washington, DC: American Statistical Association, 1992, pp. 393–398.
Centers for Disease Control and Prevention. “Sexually Transmitted Disease Surveillance 2016.” Atlanta, GA: U.S. Department of Health and Human Services, 2017.
Commendador, K. A. “Parental Influences on Adolescent Decision Making and Contraceptive Use.” Pediatric Nursing, vol. 36, no. 3, 2010, p. 147.
Harris, K.M., C.T. Halpern, E. Whitsel, J. Hussey, J. Tabor, P. Entzel, and J.R. Udry. 2009. The National Longitudinal Study of Adolescent to Adult Health: Research Design [WWW document]. URL: http://www.cpc.unc.edu/projects/addhealth/design.
Kreuter, F., S. Presser, and R. Tourangeau. “Social Desirability Bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity. Public Opinion Quarterly, vol. 72, no. 5, December 2009, pp. 847–865.
Lafauve, K., K. Rowan, K. Koepp, and G. Lawrence. “Effect of Incentives on Reducing Response Bias in a Web Survey of Parents.” Presented at the American Association of Public Opinion Research Annual Conference: Denver, CO, May 16–19, 2018.
Martin, J. A., B. E. Hamilton, M. J. K. Osterman, A.K. Driscoll, and T. J. Matthews. “Births: Final Data for 2015.” National Vital Statistics Report, vol. 66, no. 1. Hyattsville, MD: National Center for Health Statistics, 2017.
Office of Management and Budget. “Questions and Answers When
Designing Surveys for Information Collections.” Office of
Information and Regulatory Affairs, Office of Management and Budget,
2016. Available at:
https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/assets/OMB/inforeg/
pmc_survey_guidance_2006.pdf.
Olsho, L., J. Cohen, D. Walker, A. Johnson, and G. Locke. “National Survey of Adolescents and Their Parents: Attitudes and Opinions about Sex and Abstinence—Final Report.” Cambridge, MA: Abt Associates, February 2009.
Singer, E., and R. A. Kulka. “Paying Respondents for Survey Participation.” In Studies of Welfare Populations: Data Collection and Research Issues. Panel on Data and Methods for Measuring the Effects —of Changes in Social Welfare Programs, edited by Michele Ver Ploeg, Robert A. Moffitt, and Constance F. Citro. Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press, 2002.
Toepoel, V., M. Das, and A. van Soest. “Relating Question Type to Panel Conditioning: Comparing Trained and Fresh Respondents.” Survey Research Methods, vol. 3, no. 2, 2009, pp. 73-80.
Tourangeau, R., L. J. Rips, and K. Rasinski. The Psychology of Survey Response. New York: Cambridge University Press, 2000.
Vesely, S. K., V. H. Wyatt, R. F. Oman, C. B. Aspy, M. C. Kegler, S. Rodine, L. Marshall, and K. R. McLeroy. “The Potential Protective Effects of Youth Assets from Adolescent Sexual Risk Behaviors.” Journal of Adolescent Health, vol. 34, no. 5, 2004, pp. 356–365.
Wight, D., and D. Fullerton. “A Review of Interventions with Parents to Promote the Sexual Health of Their Children.” Journal of Adolescent Health, vol. 52, no. 1, 2013, pp. 4–27.
1 Parents are defined as parents or guardians with biological children or children they are responsible for, such as stepchildren, adopted children, and foster children.
2 Panel sample members have previously consented to participate in the AmeriSpeak panel. The AmeriSpeak panel is operated by the National Opinion Research Center (NORC) and is not affiliated with ACF.
3 Examples of sensitive topics include (but not limited to) Social Security number; sex behavior and attitudes; illegal, antisocial, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships (e.g., family, pupil-teacher, employee-supervisor); mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities that indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment, WIC, or SNAP benefits); immigration/citizenship status.
4
May 2019 National Occupational Employment and Wage Estimates United
States https://www.bls.gov/oes/
current/oes_nat.htm#00-0000.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Mathematica Standard Report Template |
| Author | Sclark |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |