SRAE PAS_ SSA_080519_clean_092519

Sexual Risk Avoidance Education Program Performance Analysis Study

(SRAE PAS)

OMB Information Collection Request

0970-New Collection

Supporting Statement

Part A

August 2019

Submitted By:

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer:

Caryn Blitz

CONTENTS

Overview 1

A1. Necessity for the Data Collection 2

A2. Purpose of Survey and Data Collection Procedures 3

A3. Improved Technology to Reduce Burden 9

A4. Efforts to Identify Duplication 9

A5. Involvement of Small Organizations 9

A6. Consequences of Less Frequent Data Collection 9

A7. Special Circumstances 10

A8. Federal Register Notice and Consultation 10

A9. Incentives for Respondents 11

A10. Privacy of Respondents 11

A11. Sensitive Questions 13

A12. Estimation of Information Collection Burden 14

A13. Cost Burden to Respondents or Record Keepers 17

A14. Estimate of Cost to the Federal Government 17

A15. Change in Burden 17

A16. Plan and Time Schedule for Information Collection, Tabulation, and Publication 17

A17. Reasons to Not Display OMB Expiration Date 18

A18. Exceptions to Certification for Paperwork Reduction Act Submissions 18

Supporting references for inclusion of sensitive questions or groups of questions 18

TABLES

A2.1. Collection Frequency for Performance Measures Data 7

A2.2 SRAE PAS 60-Day Federal Register Notice: Response to Comments 10

A11.1. Summary of Sensitive Questions to be Included on the Participant Entry and Exit Surveys, and their Justification 13

A12.1. Total Burden Requested Under this Information Collection (by Grantee Type) Participants 16

FIGURES

1. Levels of Performance Measures Data 4

INSTRUMENTS

Instrument #1 – Participant Entry Survey

Instrument #2 – Participant Exit Survey

Instrument #3 – Performance Reporting System Data Entry Form

Instrument #4 – Sub-awardee Data Collection and Reporting Form

APPENDICES

Appendix A. 60-Day Federal Register Notice

Appendix B. 60-Day Federal Register Notice -- Comments

Overview

• Status of study: This is a new information collection entitled “Sexual Risk Avoidance Education Performance Analysis Study (SRAE PAS)”. This request is for three years.

• What is being evaluated (program and context) and measured: The Administration for Children and Families (ACF) is seeking approval to collect survey data from SRAE grantees and their program participants ages 14-18. The goal of the SRAE PAS is to collect performance measures data on: characteristics of youth involved in programming; youth sexual behavior, sexual behavior intentions, and behaviors relevant to the success sequence¹; range of services youth receive; program structure, cost, and support for implementation; program attendance, reach, and dosage; how programs addressed the A-H topics²; and how programs influenced youth participants’ knowledge, attitudes, and behaviors at program exit.

• Type of study: Performance measure data collection for use in program monitoring and improvement.

• Utility of the information collection: The performance measures information collected through the SRAE PAS surveys will allow both the program office and grantees to monitor and report on progress in implementing SRAE programs. It will also inform program technical assistance and help grantees manage and improve their programs. We do not intend for this information to be used as the principal basis for public policy decisions.

• Time Sensitivities: ACF requests approval by early fall 2019 in order to start data collection in January 2020. Implementation of many SRAE programs occurs in schools, and new cohorts of youth often begin receiving programming at the start of the spring semester. If we are not able to begin data collection in January, we will miss the ability to collect entry survey data from youth who are participants in programs that begin at the start of the spring semester. Early fall approval provides time to train SRAE grantees on the final performance measures and for grantees to obtain any needed approvals from their school districts to administer the performance measures.

A1. Necessity for the Data Collection

The Sexual Risk Avoidance Education Program Performance Analysis Study (SRAE PAS) seeks to collect performance measures data from SRAE program participants and providers that will allow both the program office and grantees to monitor and report on progress in implementing SRAE programs.

With this new information collection request (ICR), the Family and Youth Services Bureau (FYSB) and Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF) at the Department of Health and Human Services (HHS) seek approval for collecting data from SRAE program participants and providers. These data will provide ACF with up-to-date information on youth participant characteristics, service receipt, perceived influences of the programs, and on program infrastructure, reach and scope, and program content. In addition, the data will help ACF to better understand grantees’ technical assistance needs and inform grantees of program operations and participants’ perception of the program. This information will facilitate continuous quality improvement of SRAE programming, ultimately benefiting the youth who participate. This is the first federal effort to collect performance measures data on sexual risk avoidance programming.

We plan to collect data via a set of instruments developed for the collection of SRAE performance measures:

Instrument #1: Participant entry survey

Instrument #2: Participant exit survey

Instrument #3: Performance Reporting System Data Entry Form

Instrument #4: Sub-awardee Data Collection and Reporting Form

The collection and analysis of performance measures (PM) play a unique role in the mix of current federal evaluation efforts to expand the evidence base on teen pregnancy prevention programs. The objective of the SRAE PM effort is to document the operationalization of SRAE-funded programs in the field and to assess participants’ perceptions of the programs’ influences. The PM effort will include information on implementation of evidence-based and evidence-informed programs.

Study Background

The consequences of adolescent sexual activity remain a critical social and economic issue in the United States, shaping the lives of thousands of teens and their families every year. Despite declining births to teen mothers over the past 25 years, the teen birthrate in the United States remains higher than in other industrialized countries and varies widely across geographic regions and racial/ethnic groups (Martin et al., 2017). Further, adolescents and young adults account for half of all new sexually transmitted infection (STI) cases each year (Centers for Disease Control and Prevention, 2017). Sexual activity of youth is also related to engaging in other risk behaviors, such as alcohol and substance use.

In FY2018, Congress updated the Title V, Section 510 legislation, renaming the former Abstinence Education program “Sexual Risk Avoidance Education.”

The updated legislation enables grantees to “implement education exclusively on sexual risk avoidance (meaning voluntarily refraining from non-marital sexual activity).” Education for sexual risk avoidance must include messages to youth that are:

• focused on normalizing the optimal health behavior of avoiding non-marital sexual activity;

• medically accurate and complete;

• age-appropriate;

• based on adolescent learning and developmental theories; and

• culturally appropriate, recognizing the experiences of youth from diverse communities, backgrounds, and experiences.

The prior A-H criteria were replaced with A-F criteria³ to ensure youth receive a variety of information to improve their future health and well-being. The new legislation also permits including medically accurate information on contraception that ensures program participants understand that contraception offers physical risk reduction and not risk elimination. Demonstrations, simulations, or distribution of contraceptive devices are not permitted in SRAE programs.

Legal or Administrative Requirements that Necessitate the Collection

The SRAE program is funded generally under the authority of Title V, Section 510 of the Social Security Act, 42 U.S.C. § 1310, and specifically by the appropriation for General Departmental Management for the Office of the Secretary under Division H, Title II of the Consolidated Appropriations Act, 2016, Pub. L. 114-113.

The collection of performance measures supports compliance with the GPRA Modernization Act of 2010 (Public Law 111-352).

A2. Purpose of Survey and Data Collection Procedures

Overview of Purpose and Approach

This is a data collection project focused on program performance measures for program management. The data will be collected through participant surveys and grantee and sub-awardee reporting forms. The data will be used by the program office and grantees to (1) monitor and report on progress in implementing SRAE programs, and (2) implement continuous quality improvement of SRAE programs. The information collected is meant to contribute to the body of knowledge on ACF/FYSB programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.  

The plan for collecting and reporting the performance measures data reflects the multiple layers that grantees are using to support program delivery. For example, some grantees may directly implement the programs. In other arrangements, grantees may deliver programs through other providers (e.g., sub-awardees). Ultimately, the grantees will be responsible for ensuring that all performance measures are reported to ACF. The data that the grantees report to ACF will originate from three levels: the grantee, grantees’ sub-awardees, and the youth participants completing entry and exit surveys (see Figure 1, below). For some performance measures, grantees will provide data about activities or decisions that they undertake directly at the grantee level. For other measures, data will come from the sub-awardees of the grantee because sub-awardees implement the SRAE curricula and data collection efforts. In addition, some data will come from entry and exit surveys completed by youth participants.⁴ The efforts expected to be undertaken at each level and the estimated level of burden are further explained in Section A.12.

Figure 1. Levels of Performance Measures Data

The performance measures data will be reported by grantees to the SRAE data warehouse. The contractors will maintain the warehouse and make any necessary adjustments or updates as needed.

Data Analysis Questions

The performance measures will provide insight on questions related to the participant and program characteristics, program inputs and outputs, and participants’ perceptions of the programs’ influences on them, including:

• Questions related to States:

• How many States awarded sub-awards each year?

• How many States directed sub-awardees on how to design key elements of their programs – e.g. emphasizing abstinence and medically accurate contraception – versus leaving the decisions to sub-awardees?

• How did grantees determine which adolescent development theories would be most appropriate?

• Questions related to sub-awardees:

• What proportion of sub-awardees provided services through schools, versus community-based settings, versus other settings?

• What proportion of sub-awardees addressed each of the A-H topics?

• What was the sub-awardee program budget? How was the budget allocated?

• Questions related to other grantees and community-based organizations/providers:

• How many implementation partners are there?

• How many implementation sites are there?

• How many facilitators and educators are directly employed by the grantee? How much funding is allocated towards educators and facilitators employed by the grantee?

• Have adaptations been made to evidence-based interventions (EBPs)? If so, were these:

• Planned with a clear rationale, e.g.:

• Cultural (changes to make lessons culturally relevant to the target population)?

• Change in the delivery setting?

• Changes to/omissions of lessons?

• Increases or decreases to the length of time dedicated to the lesson(s)?

• Other?

• Questions related to program instructors:

• What proportion of program instructors received training in the programs they implemented? Additionally, how much and what type of training did instructors receive?

• For school-based interventions, how were program topics offered, i.e., within the school day in traditional subjects, in clubs, or as other after school extended learning opportunities?

• What proportion of program instructors kept logs of how they implemented program lessons, i.e. maintained fidelity to the curriculum?

• Questions related to program participants:

• How many youth were served by SRAE programs?

• What were the characteristics (e.g. demographics) of the youth served? What behaviors did youth engage in before participation in SRAE (reported at program entry)?

• At program exit, what influences did youth participants perceive the program had on their knowledge, attitudes, and behavioral intentions related to the topics covered in SRAE programs? For example, what percentage of youth participants reported at program exit that participating in SRAE made them more likely to intend to delay having sex?

• What trends in the perceived influences of the program can be identified over time? For example, do youth who receive SRAE programming in one year perceive that the program influenced their knowledge, attitudes, and behaviors differently than youth who receive SRAE programming in a later year?

A major objective of the performance measures analysis will be to construct, for grantees and ACF, a picture of SRAE implementation in the form of a basic set of statistics across all grantees. These statistics, for example, will answer questions for the overall SRAE programs, such as:

• What programs were implemented, and for how many youth?

• What are the characteristics of the populations served?

• To what extent were members of vulnerable populations served?

• How many youth participated in most program sessions or activities?

• How many entities are involved at the sub-awardee level in delivering programs?

• How do grantees allocate their resources?

• How do participants feel about the programs, and how do they perceive its influences on them?

• What challenges do providers experience in implementing SRAE programs?

Data Collection Plan

The study involves collection of surveys from youth participants (Instruments 1 and 2), and grantees (Instrument 3) and their sub-awardees (Instrument 4). The frequency with which performance data will be collected from grantees is summarized in Table A2.1.

Universe of Data Collection Efforts

Youth will complete the Participant Entry and Exit Surveys (Instruments 1 & 2) to provide self-report data on youth demographics and special population status (e.g., foster care, homelessness, adjudicated, etc.). Since the data will not be linked across surveys, demographic questions will be included in both entry and exit surveys. The participant entry survey will also collect information on sexual behavior, other behaviors (such as drug and alcohol usage), knowledge, and attitudes about adhering to the success sequence and A-H topics. The participant exit survey will also collect information on sexual behavior and other behavior intentions (e.g., participant intention to avoid sex) and participants’ perceptions that their participation in the program had an influence on their knowledge and attitudes relating to the success sequence and other A-H topics.

Grantees and subawardees will use the program-level data collection forms to submit their data on SRAE program structure and delivery

• SRAE program structure refers to how grant funds are used, the program models selected, the ways in which grantees and sub-awardees support program implementation, and the characteristics of the youth served, including attendance, reach, and dosage.

• SRAE program delivery refers to the extent to which the intended program dosage is delivered, youths’ attendance, youths’ perceptions of program influences and their experiences in the programs, and challenges experienced implementing the programs.

To understand program structure, grantees will provide the: amount of the grant allocated for various activities; approach to staffing SRAE at the grantee level; grantee provision of training, technical assistance, and program monitoring; number of sub-awardees, their funding, program models, populations, and settings; number of program facilitators, their training on the program model, and the extent to which they are monitored to ensure program quality; and characteristics of the youth entering the programs. Grantees will provide this information using Instrument 3, and sub-awardees will provide information using Instrument 4. Sub-awardees will submit their data to grantees, who will then compile this information and submit it to ACF once a year (Instrument 3).

To understand program delivery, grantees will be asked to provide the number of completed program hours for each cohort; number of youth who ever attended an SRAE program, and by subpopulations, such as youth in foster care or the juvenile justice system; youths’ attendance; youths’ perceptions of program influences and program experiences; and challenges providers face implementing their programs. This information will be collected from sub-awardees (Instrument 4) and submitted to ACF by the grantees twice a year (Instrument 3).

Table A2.1. Collection Frequency for Performance Measures Data

Instrument/Category	Frequency of Collectiona	Frequency of Reporting to ACF
#1 Participant Entry Survey
Demographics; Sexual behaviors and intentions; Pregnancy history	Program Entry	Twice a year
#2 Participant Exit Survey
Demographics; Participant perceptions of program influences on sexual intentions and, for SRAE participants, on other behaviors; Participant assessments of program experiences	Program Exit	Twice a year
#3 Grantee Performance Reporting Data Entry Form	Ongoing, as programs are implemented
Total respondent counts of measures in the Entry and Exit surveys Total respondent counts by measures of attendance, reach, and dosage Program completion by cohort Implementation challenges and technical assistance needs Administrative data on program features and structure, cost, and support for program implementation	Program Entry and Exit At cohort completion At cohort completion Twice a year Once a year Once a year	Twice a year At cohort completion At cohort completion Twice a year Once a year Once a year
#4 Sub-awardee Data Collection and Reporting Form	Ongoing, as programs are implemented
Fidelity to evidence-based program models (e.g., intended program delivery hours, target populations) Staff perceptions of implementation challenges and technical assistance needs	Once a year Twice a year	Once a year Twice a year

^a “Collection frequency” refers to when grantees, their sub-awardees, and program staff collect the data that will later be compiled and reported to ACF. Grantees will be reporting the data twice a year to ACF in order to inform continuous quality improvement.

Intended Use of Information

ACF will use the performance measures data to (1) track how grantees are allocating their funds; (2) assess whether SRAE objectives are being met (e.g., in terms of the populations served); and (3) provide technical assistance to help drive programs toward continuous improvement of service delivery. In addition, ACF will use the information to fulfill reporting requirements to Congress and the Office of Management and Budget concerning the SRAE initiative. ACF also intends to share grantee and sub-awardee level findings with each grantee to inform their own program improvement efforts.

The analyses will be descriptive and will include computing means and sums of continuous numeric measures (such as the number of participants served), producing frequency distributions of categorical and string variables (such as the program models implemented), and using cross-tabulations to explore potential relationships between variables. Grantee-level data will be used to create an overview of SRAE grantees—by funding stream—including staffing, funding, and other supports offered to program providers (such as training, technical assistance, and observations of program delivery). Program provider data will inform an aggregate profile of SRAE programs, including their program models, staffing, settings, and numbers of participants. Data from youth entry surveys will provide a picture of the aggregate characteristics of youth entering SRAE programs, including their past behaviors related to topics covered in the programs, to help assess the level of risk of the population served. Data from youth exit surveys will connect demographic characteristics related to youths’ perceptions of program influences. Survey data will equip grantees to make better programmatic decisions by giving them a better understanding of the participants being served and how those participants perceive the program. For example, if younger youth report on the exit survey that they did not feel the material was presented clearly, the grantee may consider modifying language or examples used in the program to better meet the needs of that population. Grantees can also use the data to communicate to partners and their communities about the SRAE programs. ACF will use data from the surveys to better understand and report trends in SRAE participant populations, such as the percentage of youth served who are from vulnerable populations. The agency will also be better able to monitor grantees’ performance for certain domains, including to what extent a grantee is serving youth from their priority target populations.

However, these data are not designed to evaluate the impact of the program, and this limitation will be clearly stated in all written and digital products associated with the information collection. In addition, the entry and exit survey data will not be linked, so the analyses will not include any cross-tabulations of youths’ past behaviors with their perceptions of the program’s influences on future behaviors.

In addition to annual written memos and reports to ACF, displays of performance measures data will be available to ACF staff and SRAE grantees through an interactive data dashboard. The dashboard will allow users to drill down below the aggregate level to examine a measure for a particular grantee, program provider, or program model. Grantees will only have access to their own grantee-level data and to the provider- and program-level data for each of their program providers. Grantees will also have access to aggregate information across grantees within their funding stream for comparison. If program providers access the dashboard, they will only have access to their own provider- and program-level data and will not have access to other providers’ data.

The Participant Entry Survey (Instrument #1), Participant Exit Survey (Instrument #2), the Performance Reporting System Data Entry Form (Instrument #3), and the Sub-awardee Data Collection and Reporting Form (Instrument #4) are attached.

A3. Improved Technology to Reduce Burden

To comply with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and to reduce grantee burden, ACF is (1) providing common data element definitions across SRAE grantees and program models, (2) collecting these data in a uniform manner through the SRAE data warehouse, (3) using the SRAE data warehouse to calculate common performance measures across grantees and program models, and (4) developing an SRAE Performance Measures Dashboard that is interoperable with the SRAE data warehouse to provide near-real-time data reporting for SRAE grantees, FYSB project officers, and other ACF staff. Using the SRAE data warehouse reduces reporting burden and minimizes grantee and sub-awardee costs related to implementing the reporting requirements. Implementing the Dashboard reduces data analysis and report production time so that grantees can receive near-real-time data through an interactive data dashboard.

A4. Efforts to Identify Duplication

ACF has carefully reviewed the information collection requirements to avoid duplication with existing studies and believes that this requested data collection complements, rather than duplicates, the existing literature and other ongoing evaluations and projects. While performance measures data have been collected for other teen pregnancy prevention programs, this is the first time performance measures data will be collected for the SRAE programs.

A5. Involvement of Small Organizations

Programs in some sites may be operated by community-based organizations. ACF and its contractor teams will provide thorough training and technical assistance throughout the entire data collection effort, from the planning period all the way through data analysis. This training and technical assistance should help to minimize the burden on small organizations.

A6. Consequences of Less Frequent Data Collection

The Government Performance and Results Act (GPRA) requires federal agencies to report annually on measures of program performance. Therefore, it is essential that grantees report the performance data described in this ICR to ACF. Failure to collect performance measures across all grantees will inhibit ACF from carrying out its reporting requirements to Congress and OMB. Further, failure to collect data will inhibit grantees and ACF from reporting to other key stakeholders on SRAE program design, implementation, and perceived influences of the program on participants’ knowledge, attitudes, and behaviors.

A7. Special Circumstances

There are no special circumstances for the proposed data collection efforts.

A8. Federal Register Notice and Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13 and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995)), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on January 31, 2019 and provided a 60-day period for public comment. A copy of this notice is included as Appendix A.

During the notice and comment period, 6 sets of comments were received, which are included as Appendix B. Comments included: 1) concerns about sensitive questions and their possible impact on youth; 2) suggested wording changes of existing questions; 3) suggested additional questions; 4) suggested data collection process changes, 5) concerns that the questions are insufficient to meet the program office’s needs, and 6) comments in support of the questions. Responses to categories 1-5 are listed below.

Table A2.2 SRAE PAS 60-Day Federal Register Notice: Responses to Comments

Type of Comment	Response
Sensitive questions and their possible impact on youth	ACF has experience conducting surveys with similar populations using the same sensitive questions. For most programs and participants, the questions do not pose a problem. In limited cases, ACF grants waivers to programs that wish to be exempt from asking certain questions on the participant surveys. FYSB reviews waiver requests and approves them on a case-by-case basis. Youth may skip questions they do not wish to answer. Surveys responses are kept private and do not include identifying information.
Suggested wording changes	We considered wording changes where possible.
Suggested additional questions	We considered additional questions where possible.
Suggested data collection process changes	We considered data collection process changes where possible.
5. Questions are insufficient to meet the program office’s needs	We consulted with the program office to develop the performance measures tailored to their needs. The SRAE PAS focuses on performance measurement, not evaluation. The goal of performance measurement focuses on accountability and planning, and continuous quality improvement. The primary focus of performance management is on inputs, outputs, and participants’ perceived influence of the program on their knowledge, attitudes, and behavioral intentions. Unlike evaluation, performance measurement does not focus on impacts. The survey items are intentionally not identical at program entry and program exit because the surveys are not intended to provide pre- and post-test data. There is no control group, and comparisons of any identical entry and exit survey items could be misconstrued as program effects, when they could actually be due to other changes over time unrelated to the program. (In addition, the set of respondents present at entry and exit could differ, and the entry and exit survey data will not be linked.) Instead, we measure sexual and other behaviors at program entry, and sexual and other behavior intentions at program exit.

Consultation with Experts

To develop the SRAE performance measures, ACF consulted with staff of the contractor (Susan Zief, Lara Hulsey, and Lauren Murphy – expertise in adolescent pregnancy prevention performance measures). ACF also consulted internal staff from FYSB and OPRE and select SRAE grantees, as well as FYSB and ACF leadership.

A9. Incentives for Respondents

No incentives are proposed for the information collection.

A10. Privacy of Respondents

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. At no time will ACF have access to or receive potentially identifiable information. At no time will this information be linked or linkable to survey information.

Participant-level data. Grantees and subawardees will collect the participant-level data required for PM reporting. Grantees will then enter this information into the SRAE data warehouse. Grantees and sub-awardees will be responsible for ensuring privacy of participant-level data and securing institutional review board (IRB) approvals to collect these items, as necessary. Some of the grantees may need IRB approval based upon their local jurisdiction mandates. During trainings on the performance measures and through follow up technical assistance activities, we will provide guidance to grantees on determining whether they need IRB approval and following the proper procedures of their locality to obtain IRB approval or exemptions, as needed. Grantees will be required to inform participants of the measures taken to protect the privacy of their responses.

The contractor will provide training and technical assistance to grantees and sub-awardees on data privacy and security. This training will include reminders of the particularly sensitive items in the entry and exit surveys (see Table A11.1 below) and the importance of keeping such data secure. We will establish procedures for ensuring the protection of sensitive information. For example, grantees and sub-awardees will be instructed to: submit only de-identified data, keep any identifiable information in secure locations (such as digital files with restricted access or locked filing cabinets) separate from survey data and accessible only to authorized persons; and destroy any lists linking ID numbers to participant names or other identifying information after the data are submitted. Grantees will also be required to use secure procedures when transmitting data, such as from a sub-awardee or site. These procedures will involve encrypted e-mail or a CD sent using secure shipping procedures (marked confidential, packaged separately from any identifying information, and using a tracking number to follow up if data are not received). All grantee staff with access to data will be required to sign privacy agreements. We will advise grantees and sub-awardees against linking entry survey and exit survey data to compare pre-program responses to post-program responses, in the absence of a comparison or control group. We will also discuss the risk of identification due to small cell sizes and set standards for the minimum number of participants reported for any analysis.

Grantee-level data. Grantees will enter all data into the SRAE data warehouse. The contractor will design the SRAE data warehouse to ensure the security of the data maintained there. Electronic data from the SRAE grantees will be stored in a location within the contractor. The data warehouse is a research cloud-based system that provides the appropriate level of security (e.g. granting users different levels of access depending on their role) based on the sensitivity or identifiability of the data. Personal identifiers will not be submitted to ACF. Data generated by the SRAE data warehouse will be in aggregate form only.

The contractor will create and house a SRAE Performance Dashboard to provide authorized stakeholders with self-service access to various views of a subset of the SRAE performance measures. The Dashboard will display data at the grantee and national levels and will allow users to drill down along dimensions of interest such as time, geographic region, or curriculum. As needed, the contractor will enforce security roles to prohibit grantees from accessing others’ data.

The Dashboard will be interoperable with the SRAE data warehouse. It will have a near-real-time interface with the SRAE data warehouse so it can display the status of data submissions and help monitor agencies’ compliance with reporting requirements.

Consent Forms. Grantees will receive guidance for active or passive consent. The following information is also included on the first page of Instrument 1 and Instrument 2:

• The purpose of the information collection and how the information is planned to be used to further the proper performance of the functions of the agency;

• An estimate of the time to complete the instrument;

• That the collection of information is voluntary; and

• That responses will be kept private to the extent permitted by law.

• That an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is XXXX-XXXX and the expiration date is XX/XX/XXXX.

For all information collected, grantees will submit aggregate or de-identified data, so no personally identifiable information is collected by ACF and its contractors. ACF is working with the ACF and HHS Offices of the Chief Information Officer (OCIO) to complete any necessary requirements related to privacy and security, as needed. Information will not be maintained in a paper or electronic system from which they are actually or directly retrieved by an individual’s personal identifier.

A11. Sensitive Questions

A key objective of SRAE programs is to prevent teen pregnancy through a decrease in non-marital sexual activity. We understand that issues pertaining to the sexual behavior of youth and young adults can be very sensitive in nature; however, the data analysis questions pertaining to the performance measures are necessary to understanding program functioning.

Table A11.1 provides a list of sensitive questions that will be asked on the participant entry and exit surveys and the justification for their inclusion.

Table A11.1. Summary of Sensitive Questions to be Included on the Participant Entry and Exit Surveys, and their Justification

Topic	Justification
Participant Entry Survey (Instrument #1)
Sexual orientation (Question 6; also on the Exit Survey)	This question will allow us to document the extent to which SRAE programs serve lesbian, gay, bisexual, transgender and questioning youth.
Vulnerable populations (Question 8; also on the Exit Survey)	This question will allow us to document the extent to which SRAE programs serve youth in foster care, homeless, and adjudicated youth.
Use of alcohol, tobacco, and other substances (Question 9; also on the Exit Survey)	Collection of this information will allow us to document the characteristics of the population served by SRAE programs and the degree to which they engage in risky behavior.
Sexual activity and incidence of pregnancy (Questions 14-18)	Level of sexual activity and incidence of pregnancy are central to the goals of the programs. Collection of this information will allow us to document the characteristics of the population served by SRAE programs and the degree to which they engage in risky behavior.
Participant Exit Survey (Instrument #2)
Participants’ perceptions of SRAE’s influence on their sexual activity (Questions 11-13)	Reducing intentions to engage in sexual activity is among the central goals of SRAE-funded programs. Examination of whether participating youth consider the programs to have accomplished SRAE goals is an important element of gauging the success of the overall SRAE program.

The contractor will provide training and technical assistance via webinars, data collection tools and guidance documents, and a help desk, through which reporting agencies can submit requests. Training and technical assistance activities will include content related to understanding the measures, administering the surveys, and submitting the data. Grantees will be encouraged to collect data on all questions, if possible. Grantees will inform program participants that their participation is voluntary and they may refuse to answer any or all of the questions in the entry and exit surveys. All grantees will have the opportunity to opt out of asking sensitive questions if necessary.

A12. Estimation of Information Collection Burden

Table A12.1 provides the estimated annual burden calculations for the performance measures reporting.

1. Annual Performance Measures Burden for Youth Participants

We estimate the number of participants completing these surveys based on data collected with similar grantees involved in performance measures data collection. The amount of time it will take for youth to complete the entry and exit surveys is estimated based on previous experience administering similar surveys to youth participants, including the pretest of the entry and exit surveys (See SSB for additional information). We estimated the cost of this burden by assuming that 10 percent of the youth served by SRAE grantees will be age 18 or older, and then assigning a value to their time of $7.25 per hour, the federal minimum wage. The estimate of the proportion of youth served by SRAE programs that will be 18 or older is based on a review of SRAE grant applications and the anticipated number of youth 18 or older served by SRAE grantees.

Participant entry survey. We expect the SRAE grantees to serve approximately 1,367,874 participants over the three-year OMB clearance period, for an average of about 455,958 new participants per year (numbers not shown in table). Once we apply a 95 percent response rate to the participants, we anticipate 433,160 respondents to the entry survey each year (455,958 x 0.95 = 433,160). Based on previous experience with similar instruments and the pretest, we estimate the participant entry survey to take 8 minutes (0.1333 hour) to complete. We estimate the total annual burden for this data collection to be 433,160 x .1333 = 57,740 hours.

Participant exit survey. We estimate that about 20 percent of the participants will drop out of the program prior to completion, leaving approximately 364,766 participants at the end of the program annually (455,958 x .8 = 364,766; numbers not shown in table).⁵ Of those, we expect 95 percent, or approximately 346,528 participants, will complete the participant exit survey each year (364,766 x .95 = 346,528). Based on previous experience with similar surveys and the pretest, we estimate the exit survey is estimatedto take youth 16 minutes (0.2667 hours) to complete. We estimate the total annual burden for this data collection to be 346,528 x .2667 hours = 92,419 hours.

2. Annual Performance Measures Burden for Grantees and Sub-awardees.

At least twice a year, all 137 grantees will be required to submit all of the required performance measures into the national data warehouse. They will gather this information with the assistance of their sub-awardees (estimated to be 2,330 across all grantees).⁶ Time for a designated grantee administrator to aggregate the data across each of the grantee’s sub-awardees and submit all of the required data into the warehouse is included in the burden estimates, along with time to collect information at the grantee-level that pertain to grantee structure, cost, and support for program implementation. The grantee and sub-awardee data collection efforts described below are record-keeping tasks.

Performance Reporting System Data Entry Form. This form includes all of these required data elements that the grantee will collect, aggregate, and submit into the national warehouse (see Instrument 3). We estimate the time for these activities to be 16 hours per response per grantee. We estimate the total annual burden for these activities to be 4,384 hours (137 x 2 x 16 hours).

Sub-awardee Data Collection and Reporting Form. The sub-awardees will conduct multiple activities to support the Performance Analysis Study each year (see Instrument 4). They will aggregate data on participant level entry and exit surveys (see Instruments 1 and 2) and on attendance and program session hours, report to the grantee on implementation challenges and needs for technical assistance, and report to the grantee on sub-awardee structure, cost, and support for program implementation. The total estimated annual burden for this data collection activity is estimated to be 60,502 hours (2,327 x 2 x 13 hours).

Table A12.1. Total Burden Requested Under this Information Collection -- By Grantee Type

Instrument	Type of Respondent	Total Number of Respondents	Annual Number of Respondents	Number of Responses Per Respondent	Average Burden Hours Per Response	Annual Burden Hours	Average Hourly Wage7	Total Annual Cost
1.Participant Entry Survey	DSRAE	161,916	53,972	1	0.1333	7195	$7.25	$5,220
	SSRAE	1,108,456	369,485	1	0.1333	49,252	$7.25	$35,706
	CSRAE	29,108	9,703	1	0.1333	1,293	$7.25	$935
Totals for Participant Entry Survey		1,299,480	433,160	1	0.1333	57,740	$7.25	$41,861
2.Participant Exit Survey	DSRAE	129,948	43,316	1	0.2667	11,552	$7.25	$8,374
	SSRAE	886,768	295,589	1	0.2667	78,834	$7.25	$57,152
	CSRAE	22,871	7,624	1	0.2667	2033	$7.25	$1,472
Totals for Participant Exit Survey		1,039,584	346,528	1	0.2667	92,419	$7.25	$66,998
3.Performance Reporting Data Entry Form (Record-keeping)	DSRAE	150	50	2	16	1,600	$23.69	$37,904
	SSRAE	117	39	2	16	1,248	$23.69	$29,565
	CSRAE	144	48	2	16	1,536	$23.69	$36,388
Totals for Performance Reporting Data Entry Form		411	137	2	16	4,384	$23.69	$103,857
4.Sub-awardee Data Collection and Reporting Form (Record-keeping)	DSRAE	3,450	1,150	2	13	29,900	$23.69	$708,331
	SSRAE	2,700	900	2	13	23,400	$23.69	$554,346
	CSRAE	831	277	2	13	7,202	$23.69	$170,615
Totals for Sub-awardee Data Collection and Reporting Form		6,981	2,327	2	13	60,502	$23.69	$1,433,292
Estimated Annual Burden Total						215,045		$1,646,008

Total Annual Cost for Youth Participants

The estimated average hourly wage for the youth sample that is over 18 is $7.25, the federal minimum wage⁸. We estimate that 10 percent of the youth sample will be over 18, for a total annual cost of $108,859 for both the participant entry and exit surveys.

Total Annual Cost for Grantees and Sub-Awardees

The annual cost for grantees is estimated to be $103,857 (4,384 hours x $23.69). The hourly wage rate represents the mean hourly wage rate for community and social service occupations ($23.69) (National Occupational Employment and Wage Estimates, Bureau of Labor Statistics, Department of Labor, May 2018).

The total annual cost for subawardees is estimated to be $1,433,292 (60,502 hours x $23.69).

Total Annual Burden and Cost Estimates

A total annual burden of 215,045 hours (and cost of $1,646,001) is requested in this ICR. This includes time and cost for performance measures data collection associated with participants, grantees and sub-awardees.

A13. Cost Burden to Respondents or Record Keepers

There are no additional costs to respondents.

A14. Estimate of Cost to the Federal Government

The estimated cost for development, collection, and analysis of the SRAE performance measures is $1,743,729 over the three years for requested clearance. The annual cost to the federal government is estimated to be $581,243.

A15. Change in Burden

This is a new data collection effort.

A16. Plan and Time Schedule for Information Collection, Tabulation, and Publication

Analysis Plan

Nearly all of the survey data will be categorical or dichotomous (for example, how often youth discuss various topics with a parent or adult). We will calculate percentages for these variables. We will calculate the mean for continuous variables (for example, respondent age). We will tabulate these estimates separately for subgroups of interest, such as younger and older youth, and compare and contrast responses between grantee cohorts.

Time Schedule and Publication

The Contractor shall retrieve program implementation data annually and youth participant data biannually from the SRAE Performance Measures data warehouse, analyze them, and report on them to ACF. These reports shall address ACF performance management data needs, e.g. performance measures in ACF’s performance budget, and meeting performance benchmarks. In particular, FYSB provides information on a subset of performance measures to OMB.

After the first year of each cohort, the annual reports shall discuss performance trends across the SRAE programs, as well as by subgroups of DSRAE, SSRAE, and CSRAE grantees and sub-awardees. We expect the annual draft reports for each grant cohort (DSRAE, SSRAE, and CSRAE) in approximately February 2021 and February 2022 and the end-of-cohort reports and briefs on the performance data for each grant cohort (DSRAE, SSRAE, and CSRAE) in approximately February 2022.

A17. Reasons to Not Display OMB Expiration Date

All instruments, consent and assent forms and letters will display the OMB Control Number and expiration date.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

Supporting references for inclusion of sensitive questions or groups of questions

Blake, Susan M., Rebecca Ledsky, Thomas Lehman, Carol Goodenow, Richard Sawyer, and Tim Hack. “Preventing Sexual Risk Behaviors among Gay, Lesbian, and Bisexual Adolescents: The Benefits of Gay-Sensitive HIV Instruction in Schools.” American Journal of Public Health, vol. 91, no. 6, 2001, pp. 940-46.

Boyer, Cherrie B., Jeanne M. Tschann, and Mary-Ann Shafer. “Predictors of Risk for Sexually Transmitted Diseases in Ninth Grade Urban High School Students.” Journal of Adolescent Research, vol. 14, no. 4, 1999, pp. 448-65.

Buhi, Eric R. and Patricia Goodson. “Predictors of Adolescent Sexual Behavior and Intention: A Theory-Guided Systematic Review.” Journal of Adolescent Health: Official Publication of the Society for Adolescent Medicine, vol. 40, no. 1, 2007, pp. 4.

Davis, E. C., and Friel, L. V. “Adolescent Sexuality: Disentangling the Effects of Family Structure and Family Context.” Journal of Marriage & Family, vol. 63, no. 3, 2001, pp. 669-681.

Dermen, K. H., M. L. Cooper, and V. B. Agocha. “Sex-Related Alcohol Expectancies as Moderators of the Relationship between Alcohol use and Risky Sex in Adolescents.” Journal of Studies on Alcohol, vol. 59, no. 1, 1998, pp. 71.

Fergusson, David M. and Michael T. Lynskey. “Alcohol Misuse and Adolescent Sexual Behaviors and Risk Taking.” Pediatrics, vol. 98, no. 1, 1996, pp. 91.

Goodenow, C., J. Netherland, and L. Szalacha. “AIDS-Related Risk among Adolescent Males Who have Sex with Males, Females, or both: Evidence from a Statewide Survey.” American Journal of Public Health, vol. 92, 2002, pp. 203-210.

Li, Xiaoming, Bonita Stanton, Lesley Cottrell, James Burns, Robert Pack, and Linda Kaljee. “Patterns of Initiation of Sex and Drug-Related Activities among Urban Low-Income African-American Adolescents.” Journal of Adolescent Health: Official Publication of the Society for Adolescent Medicine, vol. 28, no. 1, 2001, pp. 46.

Magura, S., J. L. Shapiro, and S. -. Kang. “Condom use among Criminally-Involved Adolescents.” AIDS Care, vol. 6, no. 5, 1994, pp. 595.

Raj, Anita, Jay G. Silverman, and Hortensia Amaro. “The Relationship between Sexual Abuse and Sexual Risk among High School Students: Findings from the 1997 Massachusetts Youth Risk Behavior Survey.” Maternal and Child Health Journal, vol. 4, no. 2, 2000, pp. 125‑134.

Resnick, M. D., P. S. Bearman, R. W. Blum, K. E. Bauman, K. M. Harris, J. Jones, J. Tabor, T. Beuhring, R. Sieving, M. Shew, L. H. Bearinger, and J. R. Udry. “Protecting Adolescents from Harm: Findings from the National Longitudinal Study on Adolescent Health.” JAMA: The Journal of the American Medical Association, vol. 278, no. 10, 1997, pp. 823.

Santelli, John S., Leah Robin, Nancy D. Brener, and Richard Lowry. “Timing of Alcohol and Other Drug use and Sexual Risk Behaviors among Unmarried Adolescents and Young Adults.” Family Planning Perspectives, vol. 33, no. 5, 2001.

Sen, Bisakha. “Does Alcohol-use Increase the Risk of Sexual Intercourse among Adolescents? Evidence from the NLSY97.” Journal of Health Economics, vol. 21, no. 6, 2002, pp. 1085.

Tapert, Susan F., Gregory A. Aarons, Georganna R. Sedlar, and Sandra A. Brown. “Adolescent Substance use and Sexual Risk-Taking Behavior.” Journal of Adolescent Health: Official Publication of the Society for Adolescent Medicine, vol. 28, n3, 2001, pp.181.

Upchurch, DM. and Kusunoki Y. “Associations Between Forced Sex, Sexual and Protective Practices, and STDs Among a National Sample of Adolescent Girls.” Women's Health Issues, vol. 14, no. 3, 2004, pp.75-84.

1 The “success sequence” refers to a series of milestones in life that predict adult self-sufficiency and well-being. Although definitions vary, researchers commonly describe the success sequence to include achieving at least a high school education, finding and maintaining a full time job, and getting married before having children.

2 (A) has as its exclusive purpose, teaching the social, psychological, and health gains to be realized by abstaining from sexual activity; (B) teaches abstinence from sexual activity outside marriage as the expected standard for all school age children; (C) teaches that abstinence from sexual activity is the only certain way to avoid out-of-wedlock pregnancy, sexually transmitted diseases, and other associated health problems; (D) teaches that a mutually faithful monogamous relationship in context of marriage is the expected standard of human sexual activity; (E) teaches that sexual activity outside of the context of marriage is likely to have harmful psychological and physical effects; (F) teaches that bearing children out-of-wedlock is likely to have harmful consequences for the child, the child’s parents, and society; (G) teaches young people how to reject sexual advances and how alcohol and drug use increases vulnerability to sexual advances; and (H) teaches the importance of attaining self-sufficiency before engaging in sexual activity. Retrieved online at: https://www.ssa.gov/OP_Home/ssact/title05/0510.htm.

3 A. The holistic individual and societal benefits associated with personal responsibility, self-regulation, goal setting, healthy decision-making, and a focus on the future. B. The advantage of refraining from non-marital sexual activity in order to improve the future prospects, and physical and emotional health of youth. C. The increased likelihood of avoiding poverty when youth attain self-sufficiency and emotional maturity before engaging in sexual activity. D. The foundational components of healthy relationships and their impact on the formation of healthy marriages and safe and stable families. E. How other youth risk behaviors, such as drug and alcohol usage, increase the risk for teen sex. F. How to resist and avoid, and receive help regarding sexual coercion and dating violence, recognizing that even with consent teen sex remains a youth risk behavior.

4 Although both entry and exit surveys will be administered to most youth participants, the entry survey respondents and the exit survey respondents will be considered separate samples for analysis. As discussed below, the entry and exit surveys are not part of a pre/post design, and individual participants’ responses will not be linked across surveys.

5 Based on our experience with data collection efforts with similar populations, we anticipate a program completion rate of 80 percent.

6 The estimated number of sub-awardees is based on information provided by the SRAE grantees through their grant applications.

7 Calculations are based on the 10% of respondents over 18 years. Average Hourly Wage for Wage for Youth over 18 is the Federal Minimum Wage.

8 Federal Minimum Wage, US Department of Labor: https://www.bls.gov/cps/cpsaat44.pdf