TPP23 Performance Measures__T1&2

OMB# 0937-XXXX

Expiration Date: XX/XX/20XX

TPP Performance Measures for

FY2023

Tier 1 and Tier 2 Rigorous Impact Evaluation

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0937-XXXX. The time required to complete this information collection is estimated to average 9 hours per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Health & Human Services, OS/OCIO/PRA, 200 Independence Ave., S.W., Suite 336-E, Washington D.C. 20201, Attention: PRA Reports Clearance Officer

Grantee-Level Questions: Dissemination

Grantees report project-level summary responses for each of the following during each 6-month reporting period.

Dissemination

How many manuscripts have you had accepted for publication in the past year (including both articles that were published and those that have been accepted but not yet published)? Do not include manuscripts previously reported as published. _____

Please list the references for any published manuscripts published in the past year.

How many of each unique communication channel/medium (website, blog, social media) does your organization use to share information about teen pregnancy prevention and the TPP-funded grant project? (Select all that apply)

_____Blog (outside of grantee’s website)

_____Newspaper/articles

_____Social Media (such as Facebook, Twitter, Instagram, YouTube, etc.)

_____Website

_____Peer Reviewed Publication (include box to require grantee to enter citation)

During the reporting period, where was information about the project presented? Write the number of times each presentation occurred.

_____National Conference/Event (include box to require grantee to enter citation)

_____Statewide Conference/Event (include box to require grantee to enter citation)

_____Local Meeting/Event

Partners

Indicate the number of partners involved in implementing the grant-funded project during the reporting period. Partners are external organizations/agencies with whom the grantee has a written agreement (such as signed MOU, contract, or Letter of Commitment) or who are integral to the implementation, monitoring, and evaluation of the grant-funded project. Examples of partners may include program/intervention implementers (such as those organizations that provide sites, staffing, or both for TPP programming), partners who provide the supportive services to Tier 1 program participants, organizations that recruit TPP program participants, and/or organizations that provide ongoing strategic support to the project.

Total Number of Partners (unduplicated, report as of the end of the 6-month reporting period) ______

Sustainability (to be reported annually)

How many partners have firm plans in place to continue the project activities (program implementation, training, research, etc.) after the end of OPA grant funding? ________

How much funding have you secured to continue project activities beyond the end of the project period? (Enter an amount)

How many different sources of funding do you have in place to support the grant project beyond the end of the project period? _______

Training

Trainings would include professional development activities or technical assistance relevant to the implementation of project activities and provided to anyone responsible for implementing any aspect of the TPP grant project. Trainings may be for staff (from grantee and partner agencies) or community members (for example, youth trained as peer educators, community members serving on advisory groups.) Stakeholders who receive the TPP intervention as the end user or target population of the TPP intervention/program proven effective should be included under the reach section and not under training.

In the reporting period, how many TPP program trainings (e.g. training of facilitators on the content of and/or how to implement TPP programs proven effective [Tier 1] or TPP intervention [Tier 2]) have been provided through the TPP grant project? ________

In the reporting period, how many individuals affiliated with the TPP grant project (such as partner agencies, community members, stakeholders, project staff, youth who work with the project) have you or one of your partners trained on the TPP program (i.e. programs proven effective or TPP intervention) via the grant funding? ________

In the reporting period, how many other trainings (professional development or technical assistance activities relevant to the project) related to the execution of the TPP project (e.g., classroom management, community engagement, adolescent development, etc.) have been provided through the TPP grant project? ________

In the reporting period, how many individuals affiliated with the TPP grant project (such as partner agencies, community members, stakeholders, project staff, youth who work with the project) have you or one of your partners trained (professional development or technical assistance activities relevant to the project) on other topics related to the execution of the TPP project (e.g. classroom management, community engagement, adolescent development, etc.) via the grant funding?________

Questions at Group (Program or Intervention) Level

These items are for Tier 1s and Tier 2 Rigorous Impact Evaluation (RI) Grantees.

For each group (this can be a group, section, unit, or class) receiving the Teen Pregnancy Prevention (TPP) program proven effective (Tier 1) /promising intervention (Tier 2) together), grantees report the following.

Name of the TPP Program (Tier 1) or Intervention (Tier 2) being delivered:

Tier 1 grantees would report the name of the effective program (i.e., program proven effective).

Tier 2 RI grantees would report the TPP intervention that is being rigorously evaluated and would only report performance data for the participants in the intervention (treatment) arm of the rigorous study.

State/Territory where implemented:

Setting of Implementation: select one or more of the following that best describes where the majority of sessions in the group took place

In-school (Programs that take place primarily or exclusively during a school day on a school campus. This category may include public or private schools, traditional or alternative schools, of any grade level).

Clinic-based (to include school-based health centers)

Faith-based

Settings specifically serving youth experiencing homelessness such as drop-in centers and shelters (such as drop in shelter/centers, other)

Settings specifically serving youth in foster care (such as the child welfare system/foster care, group homes, residential centers)

Juvenile justice (such as detention centers, residential centers –serving uniquely juvenile justice youth, camps)

Other out-of-school time/community (programs that primarily take place outside of school hours, and may be located within a community organization not listed above or on a school campus before or after the school day)

Technology-based (includes programs that do not take place in a physical location, such as virtual programs, text messaging, apps, internet-based programs, etc.)

Urbanicity of Implementation Site: urban, rural, suburban

Reach and Demographics of TPP Participants

For each section (class or group) of TPP effective programs or promising interventions implemented with youth, how many youth participated in your program for at least one activity in the reporting period? Report total numbers per group and numbers by each demographic category below:

Age – 10 or younger, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 or older, Not reported

Grade – 6 or less, 7, 8, 9, 10, 11, 12, GED program, Technical/vocational training/college, Ungraded, Not currently in school, Not reported

Ethnicity – Hispanic or Latinx, Not Hispanic or Latinx, Not reported

Race – American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or Other Pacific Islander, White, More than one race, Not reported

Sex – Male, Female, Not reported

Gender Identity – Male; Female; Female-to-Male (FTM)/Transgender Male/Trans Male; Male-to-female (MTF)/Transgender Female/Trans Woman; Identifies as neither exclusively male nor female; Something Else; Not reported

Sexual Orientation – Straight or heterosexual; Bisexual; Lesbian, gay, or homosexual;

Something Else; Have not Decided; Not reported

Total

For each section (class or group) of the effective program (Tier 1)/promising intervention (Tier 2) implemented with non-youth participants, how many non-youth participants attended at least one activity of your effective program (Tier 1)/promising intervention (Tier 2) in the reporting period? Indicate the unduplicated total number in each category and report numbers by each demographic category below

Caregivers (such as parents, legal guardians, siblings, extended family; foster parents; “chosen” family members of adolescents): _____

Ethnicity – Hispanic or Latinx, Not Hispanic or Latinx, Not reported

Race – American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or Other Pacific Islander, White, More than one race, Not reported

[Note: this count should only include caregivers who receive a TPP effective program (Tier 1) or a promising intervention to be rigorously tested (Tier 2). Caregivers who are engaged in other aspects of the grant project should be included elsewhere in the stakeholder engagement item.)

Youth-serving professionals (such as social workers, health care providers, teachers, juvenile justice staff, court staff): ____________

Ethnicity – Hispanic or Latinx, Not Hispanic or Latinx, Not reported

Race – American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or Other Pacific Islander, White, More than one race, Not reported

[Note: this count should only include youth-serving professionals for whom a TPP effective program (Tier 1) or promising intervention to be rigorously tested (Tier 2) are the target population. Professionals who are trained to provide services to youth should be included in the partner’s item. Professionals who are involved in the systems approaches should be included within the stakeholder engagement item]

Dosage of TPP effective programs/promising interventions

These items track the amount of programming received by participants in each group of the TPP effective program (Tier 1)/promising intervention (Tier 2). Program participants are anyone (youth, caregivers, youth-serving professionals) who are the target population for an effective program (Tier 1) or a promising intervention being rigorously evaluated (Tier 2).

What is the average (mean) attendance for program participants in each group? (determined by the percentage of sessions attended by each participant in the section) _______

How many participants in each group received at least 75% of the programming? _________

Observational Fidelity and Quality

All TPP Grantees are expected to observe at least 5% of the sessions of the TPP effective program (Tier 1) or promising intervention being rigorously evaluated (Tier 2). Each session should be observed for fidelity (adherence) to planned activities and overall quality. TPP grantees are expected to develop schedules and implementation plans to ensure that the minimum number of sessions are observed. An observer ideally should be independent from the implementation, familiar with the program model, and may be an internal or external evaluator, supervisor (program director, program coordinator), or a program partner.

In order to track completions, grantees are asked to report general information about sessions implemented. A session is generally a unit of the program delivered within a meeting day of the section. A session may correspond to a full lesson (or module) from the curriculum. In instances where multiple lessons are being implemented in a single day, a grantee may choose to define a session as the entire day’s programming, provided that by doing so, the grantee is still able to report observed sessions as a whole number.

Session Information:

Note: these must be reported as whole numbers

Number of sessions (lessons) planned ________

Number of sessions (lessons) completed ________

Number of sessions (lessons) observed ____________

Observer reported fidelity

Using the fidelity monitoring tool from the program/intervention developer, report the adherence (%) for observed sessions within each section.

For each effective program/promising intervention session (meeting or lesson) that was observed during the section, what is the percent adherence to the number of activities planned? (Grantees who observe more than one session per section report the average (mean) adherence percentage for the session)

Adherence = number of activities completed/number of activities planned

__________

Observer reported quality (Based on the TPP observation form).

Rate the overall quality of the session observed on scale of 1 (poor) – 5 (excellent).

Fidelity Process Form (see the TPP Fidelity Process form)

What is the overall total score on the TPP fidelity process form (Scale of 0 – 26).

Stakeholder Engagement Measures

The stakeholder engagement items are designed to track individuals who were engaged within the overall grant project. Engagement could involve providing input on any stage of the overall grant project’s implementation, such as, but not limited to, serving on an advisory group, providing feedback on the development of program materials, participating in the continuous quality improvement processes for the project, helping to plan participant recruitment strategy, rating the youth-friendliness of supportive services, etc. The participants served by an actual TPP program (effective program in Tier 1 or promising innovative intervention in Tier 2) should be counted under the reach section and not under stakeholder engagement.

Project stakeholder engagement: How many stakeholders (such as youth, youth-serving professionals, caregivers, or other community members) were engaged within the grant project during the reporting period? Report the number for each category below.

Youth _________

Caregivers __________ (such as parents, guardians, foster parents of youth, etc.)

Youth-serving professionals ______ (teachers, educators, social workers, clinical providers, other healthcare workers, juvenile justice officers, etc.)

Community members ______ (such as faith leaders, business leaders, and any other members of the community that may not fit into categories above)