SSB_PREP PYP_Final Aug 2017

Justification for the Collection of Information for Local Evaluations as part of the Personal Responsibility Education Program (PREP): Promising Youth Programs (PYP)

OMB Information Collection Request

New Collection

Supporting Statement

Part B

August 2017

Submitted By:

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4th Floor Mary Switzer Building

330 C St, SW

Washington, D.C. 20201

Federal Project Officer:

Seth Chamberlain (COR)

Project Monitor:

Kathleen McCoy

B1. Respondent Universe and Sampling Methods

For the local evaluation information collection, all 29 grantees will provide the information requested of their evaluation, which will depend on their study design (described in more detail in Supporting Statement A, Section A2). All grantees will complete the following two instruments: (1) evaluation abstract template and (2) CONSORT diagram template. Grantees conducting local evaluations with an impact design will also complete the baseline equivalence template. All of this information is necessary to provide Technical Assistance supporting rigorous local evaluations by the grantees.

For the interviews/focus groups, 64 youth representing target populations currently underserved by existing sexual health curricula will participate in focus groups or interviews. The project team is working with ACF leadership to identify up to four underserved populations that will be the focus of the adapted sexual health curricula. Decisions on populations will be based on overall need and ACF priorities. Populations under consideration include youth who have experienced trauma, trafficked youth, transient youth, cognitively impaired or disabled youth, immigrants and refugees, transgender youth, and parents and caregivers of youth in foster care. Focus groups or individual interviews will be conducted with youth in the selected target populations. The format of the data collection will vary depending on the population and whether the project team and ACF determine focus groups or interviews are the most effective data collection method. The project team will identify PREP programs or other organizations that serve the target populations and recruit those programs or organizations to serve as recruitment partners for the interviews/focus groups. We will then work with these recruitment partners to identify a convenience sample of youth served by the organization who are members of the target populations and in the desired age range. The goal of these interviews is to gather information on from youth in the selected target populations to help inform development of a theory of change to support curriculum development. Once identified, we will recruit and consent youth in the target population to participate in an interview or focus group (Attachment A). As needed for the selected target populations, we will obtain consent from parents (Attachments B and C).

B2. Procedures for Collection of Information

The local evaluation information collection activities included in this ICR are part of grantees’ cooperative agreements. They will report data with varying periodicity, depending on the instrument. The following information will be submitted electronically:

• Evaluation abstract template (Instrument 1): Each grantee will complete the abstract template each year, including an initial draft/revision, and then a second version that reflects comments received from the TA contractor, OPRE, and FYSB.

• CONSORT diagram (Instrument 2) and baseline equivalence templates (Instrument 3): Grantees will complete one or both templates (depending on their proposed design) semi-annually, beginning after they have begun sample enrollment and baseline data collection and continuing through the end of data collection in the fourth year of the grant.

The interview/focus group information data collection activities (Instrument 4) will occur once during 2017 and 2018 and each individual will participate in one 90-minute focus group or interview. The discussion will focus on: (1) youths’ experiences receiving sexual health education programs; (2) preferences regarding the content and delivery of sexual health education programs; (3) how youths make decisions regarding relationships, sexual activity, and use of condoms and other birth control methods; and (4) youths’ access to and sources of sexual health information. To accommodate individual participant needs and reduce the burden of participating as much as possible, youth may be asked to participate in an individual interview held in-person or by telephone videoconference, or a focus group held using virtual technology or a secure online discussion board platform.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Expected Response Rates

The local evaluation information collection activities included in this ICR are part of grantees’ cooperative agreements. Therefore, we anticipate that all grantees will provide the requested information on the schedule indicated in Supporting Statement A, Section A16.

For the interviews/focus groups with youth, participation will be voluntary for recruited youth; however, using the strategies described in the “maximizing response rate section” below, we anticipate that at least 80 percent of youth approached for an interview will respond.

Dealing with Nonresponse

For the local evaluation information, if an individual grantee is not responsive to a request for information, the contractor will work with staff in ACF’s OPRE and FYSB to ensure they provide the requested information as soon as possible. Grantees must provide this information, as discussed in the funding opportunity announcement and in Grantees’ cooperative agreements.

For the interviews/focus groups with youth, if youth are not interested in participating in a discussion, we will continue to identify additional youth until we complete the intended number of interviews.

Maximizing Response Rates

Local evaluation information collection. We anticipate that all grantees will provide the information requested. Our past experience indicates that more than 95 percent of grantees comply with the request at each time point, with a handful not able to complete a request in a timely manner due to extenuating circumstances. In ACF’s and our contractor’s past experience, Teen Pregnancy Prevention grantees (Cohorts 1 and 2) compiled similar documents under both the Evaluation Technical Assistance and Training Contract and Evaluation Training and Technical Assistance in Program Evaluation projects funded by the Office of Adolescent Health and ACF. Grantees complied with requests with limited exceptions due to rare events such as staff illness, need for extra data processing time, or lack of new data since prior report. To ensure that response rates are maximized, the contractor will:

• Use consistent templates for each wave of data collection. To improve consistency and reliability of the data, the grantees will use the same template at each reporting period. This will minimize grantee burden for reporting periods once they are familiar with the request and have any necessary statistical programs developed.

• Work closely with the grantees’ project officer. If there are any difficulties with receiving a response from a particular grantee, the evaluation team will work closely with all of the project officers to ensure the vast majority, if not all, grantees complete the required templates.

Interview/focus groups with youth. To maximize response rates for the youth discussions, we will take the following steps:

• Use multiple methods for recruiting and scheduling interviews. An email/letter will be sent to all youth that are selected to participate in an interview to inform them of the interview (Attachment D). Subsequently, trained staff will contact youth to schedule the interview one week in advance. The staff member who will be conducting the interview will contact the participant four hours prior to the interview for confirmation.

• Schedule interviews at a convenient time for respondents. A small team of interviewers will conduct interviewers at a time (and place, if in-person) convenient for the youth. For in-person interviews, conducting interviews in public places in the participants’ own neighborhoods often enhances participants’ level of comfort with the interview process which may improve the quality of their responses, particularly regarding sensitive topics.

• Use trained interviewers. Qualitative interviewers will receive intensive and comprehensive training in how to conduct the discussions with youth. During training, interviewers will learn about the goals of the youth discussions and its relation to the larger curriculum development tasks, and the research questions that the interviews are intended to address. They will also be trained on the interview guide and how it will work for both one-on-one discussions and focus groups held virtually or through secure online discussion boards, and how to skillfully probe via follow-up questions, regardless of discussion mode. To maximize response rates, interviewers will be trained to encourage youth to participate as fully as possible, but to also allow youth to opt out of questions they do not want to answer.

• Open-ended questions with discussion guide. There is no script that interviewers must follow verbatim, but they will use a discussion guide (Instrument 4) to help ensure that interviewers systematically cover each topic of interest while still preserving the freedom for the discussions to be primarily respondent-led. The absence of a structured script helps develop rapport between the interviewer and respondent, which increases the completeness of the data and the response rate.

• Provide youth with $20 as a token of appreciation for participation. After participating in an interview/focus group youth will be provided with $20 as a token of appreciation for their participation. We will inform youth during recruitment that they will receive this token after they participate in an interview/focus group.

B4. Tests of Procedures or Methods to be Undertaken

All local evaluation information collection instruments build on existing materials and previous experience from similar evaluation TA efforts by the Contractor. Consequently, pretesting of these templates has not been planned.

Because the youth discussion guide builds on previous experience from similar efforts, drawing from topics that have been explored in prior studies, it will not be pretested.

B5. Individual(s) Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The grantees’ local evaluators will collect data for the CONSORT diagrams and baseline equivalence assessments. Grantees will report aggregate data to ACF and it will be reviewed by Mathematica and its subcontractors.

The data collection tools for this project were developed in partnership with staff in ACF’s Office of Planning, Research, and Evaluation and in consultation with staff in ACF’s Office of Family Assistance.

4