SSB - Feedback on Current PREP Information Collections

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Feedback on Current Personal Responsibility Education Program (PREP) Information Collections

Formative Data Collections for ACF Research

0970 - 0356

Supporting Statement

Part B

September 2024

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Caryn Blitz, Tia Brown

Part B

As part of the effort to improve performance measures collected by adolescent pregnancy prevention programs, the Administration for Children and Families (ACF) is beginning a process to revise the Personal Responsibility Education Program (PREP) youth surveys (approved under OMB #: 0970-0497) and surveys approved for the Sexual Risk Avoidance Education Program Performance Analysis Study (SRAE PAS) (OMB # 0970-0356).

B1. Objectives

Study Objectives

The purpose of this collection is to gather feedback from grant recipients about current performance measures included in Personal Responsibility Education Program (PREP) youth surveys (approved under OMB #: 0970-0497) and surveys approved for the Sexual Risk Avoidance Education Program Performance Analysis Study (SRAE PAS) (OMB # 0970-0356). The information from these focus groups will (1) inform potential improvements to the performance measures in the PREP program, and (2) contribute to an agency-wide initiative to select sexual orientation and gender identity (SOGI) items for use with youth in Administration for Children and Families (ACF) programs. The agency-wide effort will also be informed by pretesting activities, which will be submitted for review and approval under the Pretesting of ACF Data Collection Activities umbrella generic (0970-0355).

Generalizability of Results

The proposed grant recipient focus groups are intended to inform instrument development for ACF data collections (including the PREP participant surveys) by obtaining feedback from PREP grant recipients on some of the currently approved information collections for PREP¹. It is not designed to promote statistical generalization to other sites or service populations.

Appropriateness of Study Design and Methods for Planned Uses

Data collected under this generic clearance will be used to inform the revision of PREP participant surveys and the selection of SOGI measures for youth in ACF programs, which will further ACF’s learning agenda. The focus groups with PREP grant recipients will provide information about their past experiences administering the PREP entry and exit surveys to youth participants and their suggestions for improvements to the questions.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.  

B2. Methods and Design

Target Population

The target populations for this generic information collection request are PREP grant recipient staff for the focus groups. There are currently four PREP funding streams for a total of 97 PREP grant recipients.

Sampling and Site Selection

The study team will email all grant recipients from each of the four PREP funding streams inviting them to participate in the PREP grant recipient staff focus groups. Participation will be limited to 20 individuals per session, and the first 20 from each funding stream to register will be included. Consequently, they may not be representative of the universe of PREP grant recipients.

B3. Design of Data Collection Instruments

Development of Data Collection Instrument

The focus group protocols were developed to solicit feedback on the current information collections items and focus on grant recipient experience with administering and reporting the data. The development of the protocol was informed by information concerning youth having difficulty answering particular survey questions from PREP grant recipients, Family and Youth Services Bureau (FYSB) federal project officers, other FYSB evaluation and research staff, and Office of Planning ,Research, and Evaluation (OPRE) staff.

B4. Collection of Data and Quality Control

ACF is contracting with Mathematica for this data collection and Mathematica will oversee all data collection and ensure quality control measures are in place and followed for the data collection activity. To ensure an efficient and standardized data collection process, Mathematica study staff will participate in a project-specific data collection training. The training will cover the expectations and best practices for collecting high-quality data and procedures around data privacy and security.

Mathematica will collect data by moderating all focus group discussions, following a data collection protocol (Instrument 1). Discussions will be held virtually on a platform such as WebEx or Zoom, and all participants, including the experienced focus group interviewers, will log into the platform. The groups will include up to 20 participants each, for a total of up to 80 participants. During the focus group discussions, the participants will review each survey question, followed by a guided discussion using the relevant protocol. A notetaker from Mathematica will take live notes. Following each discussion group, the moderator and notetaker will debrief on the session together and review the notes. The notetaker will finalize the notes, which will be reviewed by the moderator to ensure accuracy and completeness.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

Response rates will not be calculated or reported.

NonResponse

As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated.

B6. Production of Estimates and Projections

The data will not be used to generate population estimates, either for internal use or for dissemination. Policy decisions will not be made based on the data collected for this formative data collection, as the data are not representative.

B7. Data Handling and Analysis

Data Handling

We will recruit participants using an existing mailing list maintained by Mathematica for contacting PREP grant recipients. The grant recipient focus groups will not be audio recorded.

All notes will be stored on Mathematica’s secure network. No one outside the study team will have access to the data. Only Mathematica staff working directly on this project have access to the project folder on the network where notes will be saved. All notes will be destroyed per contract requirements.

Data Analysis

This project will not employ complex data analytic techniques. The study team will analyze the focus group discussion data qualitatively and will use the information to assess and improve the clarity and comprehension of the participant entry and exit survey questions and response options. Information on SOGI questions will be used in conjunction with feedback received during future pre-testing efforts. More information about those activities and analysis plans will be provided in a planned submission under the Pretesting of ACF Data Collection Activities umbrella generic (0970-0355).

Data Use

ACF is currently engaged in efforts to improve the PREP participant entry and exit survey questions, and to select questions on SOGI for use across ACF youth programs to better represent how youth participants self-identify, as well as to align with EO 14075. Findings from this effort in conjunction with pretests to be completed will inform the development of the final SOGI measures for the PREP youth participant entry and exit surveys and to support the agency-wide initiative at ACF to select SOGI items for youth. After conducting the focus group discussions, the study team will develop an internal ACF memorandum describing the results, including proposed changes to the measures, and justification for the proposed changes. These findings will be used with results from SOGI pretesting efforts to propose final updates.

B8. Contact Persons

Table B.2 lists the federal and contract staff responsible for the study, including their affiliation and email address.

Table B.2. Staff responsible for study

Name	Affiliation	Email address
Caryn Blitz	Office of Planning, Research, and Evaluation Administration for Children and Families U.S. Department of Health and Human Services	[email protected]
Tia Brown	Office of Planning, Research, and Evaluation Administration for Children and Families U.S. Department of Health and Human Services	[email protected]
Brian Goesling	Mathematica	[email protected]
Lara Hulsey	Mathematica	[email protected]
Melissa Thomas	Mathematica	[email protected]
Maya Reid	Mathematica	[email protected]

Attachments

Appendix A: Email recruitment and reminders

Instrument 1: Grant Recipient Focus Group Discussion Protocol

1 The PREP Performance Measures (OMB #: 0970-0497).

6