Supporting Justification for the OMB Revision of Clearance #0990-0438:
Teen Pregnancy Prevention Performance Measures Collection, Revised for FY2020
Part A: Justification for the Collection of the Data
May 2020
Submitted by:
Office of Population Affairs
U.S. Department of Health and Human Services
1101 Wootton Parkway, Suite 200
Rockville, MD 20852
Contact Person: Tara Rice, [email protected], 240-453-8123
Background
The Office of Population Affairs (OPA) is requesting a revision of the existing approval for the collection of performance measures data from Teen Pregnancy Prevention (TPP) Program grantees (OMB # 0990-0438, expiration 10/31/2021). The proposed revision updates the performance measures to reflect the priorities of the new FY2020 funding announcements. The request revises the respondent numbers based on the anticipated number of new grants.
The TPP Program was initiated in FY 2010 as one of six major evidence-based policy initiatives across the Federal government. OPA supports two types of grants through the TPP program (1) projects that replicate TPP program models that have been shown to be effective through rigorous evaluation (Tier 1) and (2) research and demonstration projects that develop and test additional models and innovative strategies to prevent teen pregnancy (Tier 2). Funding for the TPP Program is authorized under the authority of Division A, Title II of the Further Consolidated Appropriations Act, 2020 (Public Law No. 116-94). The Act provides $100 million annually for making competitive awards to public and private entities to fund programs that reduce teen pregnancy. In FY 2020, OPA anticipates awarding new 3-year cooperative agreements under three funding announcements, which would use an estimated $78 million of the appropriated funding for the TPP program (Exhibit 1).
Performance measures have consistently been a requirement of the TPP grant program since its inception. The majority of the measures that are included in this information collection request previously received OMB clearance (0990-0392 and 0990-0438) for the FY2010 and FY2015 cohorts of TPP grantees; these measures have been successfully used over the past 10 years with the initial 2 cohorts of TPP grantees. Based on data collection experiences over the last eight years, the proposed measures have been modified slightly from those submitted for the FY2015 grantees. OPA has eliminated questions with poor response rates (cost measures). In addition, OPA has revised the dissemination and partnership items, added new stakeholder engagement items, and added an item specific to the Tier 2 Innovation network announcement that would track the number of projects by stage of development supported by the grants. The service referral item will only be required for Tier 1 Grants, as it is most relevant to the FY2020 Tier 1 FOA. The new items will be important for tracking the progress of the new FY2020 grants.
Exhibit 1: TPP Projects to be Funded in FY 2020
New TPP Project Funding Opportunity in FY 2020 |
Anticipated Total Annual Funding |
Estimated # of Awards |
Estimated Annual Award Amount |
Rigorous Evaluation |
Optimally Changing the Map for Teen Pregnancy Prevention (Tier 1) |
$54.725 million |
54-65 |
$750,000 - $1,000,000 |
No |
Tier 2 Innovation and Impact Network Grants: Achieving Optimal Health and Preventing Teen Pregnancy in Key Priority Areas |
$18.275 million |
9-15 |
$1,000,000 -$2,000,000 |
No |
Tier 2, Phase II Rigorous Evaluation of Promising Interventions (Tier 2 Phase II) |
$5 million |
5-10 |
$500,000 - $1,000,000 |
Yes |
A1. Need and Legal Basis
The TPP program is authorized under Division A, Title II of the Further Consolidated Appropriations Act, 2020 (P.L. 116-84). The performance measure collection is critical to OPA because it provides the agency with data to both effectively manage the TPP program, and to comply with accountability and federal performance requirements described in the 1993 Government Performance and Results Act (P.L. 103-62), and updated in the 2010 GPRA Modernization Act (P.L. 111–352). Moreover, collecting and reporting on data for performance measures are funding requirements for the grants, as stated in the funding opportunity announcements.
The performance measures to be reported to OPA are summarized in Exhibit 2. The specific questions that grantees will answer to address these measures are found in the information collection forms.
Exhibit 2: Performance Measures for the FY2020 TPP Cohort
Performance Measure Constructs |
Data source |
Dissemination |
Grantee/Sub-awardee Administrative Records |
|
|
|
|
Number of Partners |
|
|
Grantee/Sub-awardee Administrative Records |
|
|
Training |
Grantee/Sub-awardee Administrative Records |
|
|
|
|
|
|
Sustainability |
|
|
Grantee/Sub-awardee Administrative Records |
|
|
|
Grantee/Sub-awardee Administrative Records |
Dosage |
|
|
Grantee/Sub-awardee Attendance Records
|
|
|
Fidelity and Quality |
|
|
Administrative Records |
|
Fidelity Monitoring Logs |
|
TPP Observation Form Fidelity Process Form
|
Referrals and Linkages to Supportive Services (Tier 1 only) |
|
|
Tier 1 Grantee/Sub-awardee Administrative Records |
Innovation Network Specific Measure (Tier 2 Innovation) (New)
|
Tier 2 Network Grantee Administrative Records
Administrative Records |
Engagement Measure (New)
|
|
|
|
A.2 Information Users
The data collection activities will provide information to OPA leadership and federal project officers to help them to more effectively manage the TPP program. We anticipate that the aggregate data will be made available to Congress (reach, dosage, training and partners), and the public at large (all measures) to assess program performance. Use of these data is vital for ensuring on-going improvement of the TPP program and through dissemination efforts, broader understanding and support of programs designed to prevent teen pregnancy.
A.3. Use of Information Technology and Burden Reduction
Grantees will enter performance measure data into a multi-use, cloud-based reporting system, either by using point and click entry. The cloud-based system can reduce burden for respondents by programming specific items for each cohort of grantees, so that grantees only have to look at questions or upload data that are relevant for them. Programming automatically performs necessary calculations for respondents, and will validate responses. For point and click entry, a branching mode of presentation allows respondents to go directly to the sections they need, without having to go through the system in a linear progression. The system also produces a data set of measures across all grantees using relevant filters (e.g., for Tier 1 grantees only), which saves time on preparation of the data for analysis. Data are also available for grantees to export into Excel files to custom design reports. Screenshots of the web-based reporting system are included in the supplemental materials.
A.4. Efforts to Identify Duplication and Use of Similar Information
The FY2020 TPP performance measures data collection will be the only data collection that will provide information on the performance of the TPP program for grantees funded in FY2020. Most of the performance measures are information that would be collected by grantees as part of their routine administrative records (e.g., numbers of publications, numbers of participants served, and attendance at program sessions). Please see Exhibit 2 for data sources for each performance measure.
The majority of the constructs in the collection have been previously approved by OMB through collection OS 0990-0392, “Office of Adolescent Health and Administration for Children, Youth and Families Teen Pregnancy Prevention Performance Measure Collection” and OS 0990-0438 “FY2015 Performance Measures Teen Pregnancy Prevention Grantees. One new construct is specific to the Innovation Network (Tier 2) grants. (see the forms for the complete list of measures). The Tier 1 Supportive Services item is a modification of the previously approved healthcare linkages item from the current approved collection and will only be collected from Tier 1 grantees. The stakeholder engagement item is new to the FY2020 collection, and reflects priorities specified within the FY2020 Tier 1 and FY2020 Tier 2 Innovation network funding announcements.
A.5. Impact on Small Businesses
The only respondents are the TPP grantees; thus, no small businesses will be involved in the collection of data in this study.
A.6. Consequences of Not Collecting the Information/Collecting Less Frequently
GPRAMA requires that government agencies report on their performance measures annually. Therefore, it is essential that grantees report on these performance measures at least annually to OAH. In addition, collection and reporting of performance measure data is a requirement of all TPP grantees as stated in the Funding Opportunity Announcement.
Since the inception of the TPP program, the program office has collected TPP performance measures semi-annually, and OMB has previously approved this frequency of data collection (OMB control numbers 0990-0392 and 0990-0438). OPA, to better monitor the complex TPP grant projects collects the data twice each year. The semi-annual data collection allows federal staff monitoring grants to provide data-driven feedback to grantees. In addition, the mid-year aggregate data allows staff to assess the performance of the grant program as a whole and to plan for technical assistance and training needs of grantees.
A.7. Special Circumstances
There are no special circumstances that occur when collecting this information.
A.8. Federal Register Comments and Persons Consulted Outside the Office of Population Affairs
A 60-day notice was published in the Federal Register on February 26, 2020, in Volume, Vol. 85, No. 38, page 11095-6, and provided a 60-day period for public comments. No public comments were received (See Supplemental Materials, 60 Day FRN).
In 2010 and 2011, OAH (now OPA) consulted with staff of RTI International, which was the contractor responsible for helping develop the TPP performance measures and performance measure reporting system, and a panel consisting of experts in the fields of performance measurement, teen pregnancy prevention, and evidence-based practice. In addition, OAH (now OPA) presented information on the performance measures to TPP grantees and their evaluators at two conferences, and solicited their input. OAH (now OPA) also consulted and received feedback from other Federal staff working in the area of teen pregnancy prevention from ASPE, ACF, and CDC. The bulk of the performance measures that are part of this clearance package have been successfully used during the past 8 years in the data collection for the “Office of Adolescent Health and Administration for Children, Youth and Families Teen Pregnancy Prevention Performance Measure Collection.” Additional measures in 2015 and 2020 were developed in consultation with staff within what is now OPA.
A list of individuals in the expert panel in 2010 who provided input regarding the process evaluation is found in Exhibit 3.
Exhibit 3. Persons Consulted Outside the Agency for Performance Measure Collection for FY 2010 Grantees |
||||||||||||||||||||
Expert Work Group |
||||||||||||||||||||
|
|
|||||||||||||||||||
|
|
|||||||||||||||||||
|
|
|||||||||||||||||||
|
|
|||||||||||||||||||
A.9. Payments to Respondents
There are no payments to staff of grantee organizations completing the performance measure reporting forms.
A.10. Assurance of Confidentiality
Data will be kept private to the extent allowed by law. All participant performance measures data are reported to OPA by grantees de-identified and aggregated to the level of a section (group or cohort receiving programming together). No personal identifiers will be used in the reporting of any data to OPA.
The Web-based reporting system has been designed to ensure the security of the data obtained. Individual users designated by the grantees will be assigned user names and passwords that will grant them access to the performance measures website. There, users will have the opportunity to provide data that will be stored in a secure Microsoft Azure database (FedRAMP approved, Authority to Operate (ATO) granted on May 21, 2020), utilizing a relational table structure, facilitating expedient data retrieval and analysis. The grant-specific data users submit on the cloud-based database, will be accessible only to the grantee user, federal staff, and administrators assigned to this project. Screenshots are included in the supplemental materials.
The TPP performance measures have consistently been waived from Institutional Review Board (IRB); a copy of the waiver letter may be found in the supplemental materials.
A.11. Sensitive Questions
Grantees report on the demographics, such as race/ethnicity, of the program participants served; collecting race and ethnicity is usual practice for HHS data collections. These data will be reported to OPA at the group level. No personal identifying information will be collected from participants; no questions about attitudes or beliefs are collected in the performance measures.
A.12 Burden Estimate (Total Hours & Wages)
A.12A Estimated Annualized Burden Hours
The total annual burden is estimated to be 1155 hours for grantees to collect, summarize, and report the data for the performance measures (Exhibit 4). Estimates were based on data collection timing for similar measures collected from FY2015 TPP grantees, as the majority of items in this collection are either unchanged from the original approved version, or have minor changes.
Exhibit 4: Annualized Burden Hour Table
Forms (If necessary) |
Respondents (If necessary) |
Number of Respondents |
Number of Responses per Respondents |
Average Burden per Response |
Total Burden Hours |
Grantees (partners and sustainability) |
All TPP grantees |
90 |
2 |
15/60 |
45 |
Grantees (training) |
All TPP Grantees |
90 |
2 |
15/60 |
45 |
Grantees (dissemination) |
All TPP Grantees |
90 |
2 |
30/60 |
90 |
Grantees (Stakeholder Engagement) |
All TPP Grantees |
90 |
2 |
15/60 |
45 |
Grantees ( Reach and Demographics) |
Tier 1 and Tier 2 Phase II Grantees |
64 |
2 |
3 |
384
|
Grantees (Dosage) |
Tier 1 and Tier 2 Phase II Grantees |
64 |
2 |
2 |
256 |
Grantees (Fidelity and Quality) |
Tier 1 and Tier 2 Phase II Grantees |
64 |
2 |
2 |
256 |
Grantees (Tier 2 Innovation Network) |
Tier 2 Innovation Network Grantees |
14 |
2 |
15/60 |
7 |
Grantees (Supportive Services) |
Tier 1 Supportive Services |
54 |
2 |
15/60 |
27 |
Total |
|
|
2 |
|
1155 |
Average burden hours for grantees or sub-awardees
All of the data will be reported by the grantees and their implementing sub-awardees twice a year (see Exhibit 6 for schedule). Grantees will collect most of these data for their own administrative purposes and are expected have their own systems in place to track the data. Thus, the only additional burden to grantees for reporting the performance measures is the time it takes them to assemble the necessary data and enter it into the reporting forms. Respondents can either upload spreadsheets into the web-based system or enter the data directly into the system using a point and click method. The burden may be significantly less for grantees who manage their data using contractor-designed spreadsheets. There is no burden to program participants.
The estimated 1-year annualized cost to respondents is $34,650.00 as shown in Exhibit 5. Salaries of the grantee staff collecting data, entering data, and summarizing and reporting data may vary across agencies. We estimate an average hourly rate of $30.
Exhibit 5. Estimated 1-Year Annualized Cost to Respondents
Forms |
Type of Respondent |
Estimated Number of Respondents |
Total Burden Hours |
Hourly Wage Rate |
Total Respondent Costs |
Partners & Sustainability |
All Grantees |
90 |
45 |
$30.00 |
$1,350.00 |
Trainings |
All Grantees |
90 |
45 |
$30.00 |
$1,350.00 |
Dissemination |
All Grantees |
90 |
90 |
$30.00 |
$2,700.00 |
Stakeholder Engagement |
All Grantees |
90 |
45 |
$30.00 |
$1,350 |
Reach & Demographics |
Grantee (Tier 1 and Tier 2 Phase II Only) |
64 |
384 |
$30.00 |
$11,520.00 |
Dosage |
Grantee (Tier 1 and Tier 2 Phase II Only) |
64 |
256 |
$30.00 |
$7,680.00 |
Fidelity and Quality |
Grantee (Tier 1 and Tier 2 Phase II Only) |
64 |
256 |
$30.00 |
$7,680.00 |
Innovation Network Tier 2 Measure |
Grantee (Tier 2 Innovation Network Only) |
14 |
7 |
$30.00 |
$240 |
Supportive Services Tier 1 |
Grantee (Tier 1 Only) |
54 |
27 |
$30.00 |
$810 |
Total |
|
|
1155 |
|
$34,650.00 |
A.13 Capital Costs (Maintenance of Capital Costs)
There are no capital costs associated with this study.
A.14 Cost to Federal Government
The cost to the federal government is $1,500,000 over 3 years (time remaining on contract) to complete the contract. The cost includes the estimated cost of coordination with OPA, IRB and OMB applications, revision of the data reporting system for the new cohort of grantees, on-going maintenance of the data reporting system, training and technical assistance to the grantees and federal staff in the use of the data reporting system, and data analysis and reporting by the contractor staff. Annual cost to the government is approximately $500,000 per year.
A.15 Program or Burden Changes
This is a revised information collection. Some changes in burden within this submission reflect an update in the anticipated number of respondents, which are based upon the estimated maximum number of awards in the 3 FY2020 FOAs.
Some of the grantee-level items as discussed above were revised and reworded for clarity and are program changes designed to reflect the new funding priorities contained in the new FOAs. The dissemination item includes more specific information about social media, to reflect the FY2020 FOAs focus on development of dissemination plans; burden increased per response to reflect these changes.
The service referral question is essentially the same as the healthcare linkages item in the original approved submission. The referral item is now only to be asked of Tier 1 grantees, rather than all TPP grantees; the change reflects different priorities contained within the FY2020 FOAs.
There are new forms in this revision for stakeholder engagement, which is a key expectation within the three FY2020 FOAs and Tier 2 Innovation Network projects. Finally, the cost items which were collected previously for FY2015 grantees were deleted from this revision, resulting in a reduction in the burden.
The participant-level items (participant reach & demographics, dosage, fidelity & quality) are essentially the same as in the approved FY2015 collection; the key change is a reduction in burden because OPA anticipates we will only be collecting these items from Tier 1 and Tier 2 Phase II grantees, and not from Tier 2 Innovation Network grantees. The Tier 2 Innovation Network grantees are expected to work with innovators and early stage program developers and are not expected to work directly with program participants. In the FY2015 cohort, the Tier 2A projects served a similar target population (that is, innovators) and did not have participant-level data to report. For this reason, OPA developed a measure to track projects funded by stage from the Tier 2 Innovation Network grantees.
16. Tabulation of Data and Schedule
Data collection with the revised measures would begin as soon as OMB approval was granted. Assuming that OMB approval for this collection is granted by the fall of 2020, the FY 2020 cohort of TPP grantees will first collect and report data in Winter 2021. The anticipated schedule is included below (Exhibit 6); no data collection using the FY2020 revised measures would begin prior to OMB approval). The FY2020 grantee cohort will continue to collect and report data according the anticipated schedule below until summer 2023 at the semi-annual reporting periods (30 days after each reporting period, anticipated to be January 31 and July 30). The performance measures contractor will then analyze the data and prepare a written report, summarizing findings. Data will be aggregated across all projects, as well as broken down by type of grantee (i.e., Tier 1 and Tier 2, and by FOA (Tier 2 Phase II and Tier 2 Innovation Network). The annual reports would be published on the OPA website using aggregate-level data.
The anticipated schedule for these FY2020 grants assumes a July 2020 award date for all three TPP FOAs;
Grant Year:
Reporting period:
6 month period #1: July 1 – December 31, Deadline: January 31
6 month period #2: January 1 – June 30; Deadline July 30
Exhibit 6: Anticipated Reporting Schedule for TPP FY2020 Performance Measures
Grant Year |
Semi-Annual Period |
Data Reported to OPA |
FY2020 (year 1) |
July 1 – December 31. 2020 |
January 31, 2021 |
|
January 1 – June 30, 2021 |
July 30, 2021 |
FY2021 (year 2) |
July 1 – December 31, 2021 |
January 31, 2022 |
|
January 1 – June 30, 2022 |
July 30, 2022 |
FY2022 (year 3) |
July 1 – December 31, 2022 |
January 31, 2023 |
|
January 1 – June 30, 2023 |
July 30, 2023 |
A.17. Display of Expiration Date for OMB Approval
The expiration date for OMB will be displayed on all data collection instruments.
A.18. Exceptions to Certification Statement.
There are no exceptions to the certification statement.
List of Supplemental Materials:
Federal Register Notice to the Public
Institutional Review Board (IRB) Exemption Letter
Program Observation (Quality) Form (Referenced in Fidelity and Quality Form)
Sample Program Fidelity Monitoring Tool (Referenced in Fidelity and Quality Form
TPP Fidelity Process Form (Referenced in Fidelity and Quality Form)
TPP Database Web-Shots of Data Entry Screens
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement for |
| Author | wallace |
| File Modified | 0000-00-00 |
| File Created | 2021-01-14 |