Performance Measures_Part_A_Supporting_Statement_draft12020_cleanversion2182020_rev09900438 (002)

Performance Measures_Part_A_Supporting_Statement_draft12020_cleanversion2182020_rev09900438 (002).docx

Teen Pregnancy Prevention (TPP) Performance Measures for FY2020

OMB: 0990-0438

Document [docx]
Download: docx | pdf


Supporting Justification for the OMB Revision of Clearance #0990-0438:

Teen Pregnancy Prevention Performance Measures Collection, Revised for FY2020



Part A: Justification for the Collection of the Data





May 2020





Submitted by:

Office of Population Affairs

U.S. Department of Health and Human Services

1101 Wootton Parkway, Suite 200

Rockville, MD 20852





Contact Person: Tara Rice, [email protected], 240-453-8123


Background



The Office of Population Affairs (OPA) is requesting a revision of the existing approval for the collection of performance measures data from Teen Pregnancy Prevention (TPP) Program grantees (OMB # 0990-0438, expiration 10/31/2021). The proposed revision updates the performance measures to reflect the priorities of the new FY2020 funding announcements. The request revises the respondent numbers based on the anticipated number of new grants.



The TPP Program was initiated in FY 2010 as one of six major evidence-based policy initiatives across the Federal government. OPA supports two types of grants through the TPP program (1) projects that replicate TPP program models that have been shown to be effective through rigorous evaluation (Tier 1) and (2) research and demonstration projects that develop and test additional models and innovative strategies to prevent teen pregnancy (Tier 2). Funding for the TPP Program is authorized under the authority of Division A, Title II of the Further Consolidated Appropriations Act, 2020 (Public Law No. 116-94). The Act provides $100 million annually for making competitive awards to public and private entities to fund programs that reduce teen pregnancy. In FY 2020, OPA anticipates awarding new 3-year cooperative agreements under three funding announcements, which would use an estimated $78 million of the appropriated funding for the TPP program (Exhibit 1).



Performance measures have consistently been a requirement of the TPP grant program since its inception. The majority of the measures that are included in this information collection request previously received OMB clearance (0990-0392 and 0990-0438) for the FY2010 and FY2015 cohorts of TPP grantees; these measures have been successfully used over the past 10 years with the initial 2 cohorts of TPP grantees. Based on data collection experiences over the last eight years, the proposed measures have been modified slightly from those submitted for the FY2015 grantees. OPA has eliminated questions with poor response rates (cost measures). In addition, OPA has revised the dissemination and partnership items, added new stakeholder engagement items, and added an item specific to the Tier 2 Innovation network announcement that would track the number of projects by stage of development supported by the grants. The service referral item will only be required for Tier 1 Grants, as it is most relevant to the FY2020 Tier 1 FOA. The new items will be important for tracking the progress of the new FY2020 grants.





















Exhibit 1: TPP Projects to be Funded in FY 2020

 

New TPP Project Funding Opportunity in FY 2020

Anticipated Total Annual Funding

Estimated # of Awards

Estimated Annual Award Amount

Rigorous Evaluation

Optimally Changing the Map for Teen Pregnancy Prevention (Tier 1)

$54.725 million

54-65

$750,000 - $1,000,000

No

Tier 2 Innovation and Impact Network Grants: Achieving Optimal Health and Preventing Teen Pregnancy in Key Priority Areas

$18.275 million

9-15

$1,000,000 -$2,000,000

No

Tier 2, Phase II Rigorous Evaluation of Promising Interventions (Tier 2 Phase II)

$5 million

5-10

$500,000 - $1,000,000

Yes



A1. Need and Legal Basis

The TPP program is authorized under Division A, Title II of the Further Consolidated Appropriations Act, 2020 (P.L. 116-84). The performance measure collection is critical to OPA because it provides the agency with data to both effectively manage the TPP program, and to comply with accountability and federal performance requirements described in the 1993 Government Performance and Results Act (P.L. 103-62), and updated in the 2010 GPRA Modernization Act (P.L. 111–352). Moreover, collecting and reporting on data for performance measures are funding requirements for the grants, as stated in the funding opportunity announcements.

The performance measures to be reported to OPA are summarized in Exhibit 2. The specific questions that grantees will answer to address these measures are found in the information collection forms.

Exhibit 2: Performance Measures for the FY2020 TPP Cohort

Performance Measure Constructs

Data source

Dissemination

Grantee/Sub-awardee Administrative Records

  • # of publications by type

  • # of presentations by type

Number of Partners


  • # of formal partners

Grantee/Sub-awardee Administrative Records

  • Retention of partners

Training

Grantee/Sub-awardee Administrative Records

  • # of staff trained

  • # of trainings provided



Sustainability


  • Amount of new funding received to support the program and services

  • # partners planning to continue the programming after the grant ends

Grantee/Sub-awardee Administrative Records



  • General Information: program model, setting, urbanicity, state

  • # of youth served, by characteristics (e.g., age, gender, race/ethnicity, grade)

  • # of parents/other clients served

  • #youth-serving professionals served


Grantee/Sub-awardee Administrative Records

Dosage


  • Mean percent of total intended program services received by youth

Grantee/Sub-awardee Attendance Records


  • Youth who received > 75% of the program


Fidelity and Quality


  • Number of program-specified sessions planned and implemented

  • Number of program sessions observed

Administrative Records

  • Adherence (%) to program-specified activities, based on observation

Fidelity Monitoring Logs

  • Overall Quality of implementation, based on observations

  • Fidelity Process Score

TPP Observation Form

Fidelity Process Form




Referrals and Linkages to Supportive Services (Tier 1 only)

  • # of referrals made by program staff to youth-friendly off-site providers, by type of services

Tier 1 Grantee/Sub-awardee Administrative Records

Innovation Network Specific Measure (Tier 2 Innovation) (New)

  • # of projects by stage of development

Tier 2 Network Grantee Administrative Records


Administrative Records


Engagement Measure (New)

  • # of stakeholders engaged in the project



A.2 Information Users



The data collection activities will provide information to OPA leadership and federal project officers to help them to more effectively manage the TPP program. We anticipate that the aggregate data will be made available to Congress (reach, dosage, training and partners), and the public at large (all measures) to assess program performance. Use of these data is vital for ensuring on-going improvement of the TPP program and through dissemination efforts, broader understanding and support of programs designed to prevent teen pregnancy.



A.3. Use of Information Technology and Burden Reduction



Grantees will enter performance measure data into a multi-use, cloud-based reporting system, either by using point and click entry. The cloud-based system can reduce burden for respondents by programming specific items for each cohort of grantees, so that grantees only have to look at questions or upload data that are relevant for them. Programming automatically performs necessary calculations for respondents, and will validate responses. For point and click entry, a branching mode of presentation allows respondents to go directly to the sections they need, without having to go through the system in a linear progression. The system also produces a data set of measures across all grantees using relevant filters (e.g., for Tier 1 grantees only), which saves time on preparation of the data for analysis. Data are also available for grantees to export into Excel files to custom design reports. Screenshots of the web-based reporting system are included in the supplemental materials.



A.4. Efforts to Identify Duplication and Use of Similar Information



The FY2020 TPP performance measures data collection will be the only data collection that will provide information on the performance of the TPP program for grantees funded in FY2020. Most of the performance measures are information that would be collected by grantees as part of their routine administrative records (e.g., numbers of publications, numbers of participants served, and attendance at program sessions). Please see Exhibit 2 for data sources for each performance measure.



The majority of the constructs in the collection have been previously approved by OMB through collection OS 0990-0392, “Office of Adolescent Health and Administration for Children, Youth and Families Teen Pregnancy Prevention Performance Measure Collection” and OS 0990-0438 “FY2015 Performance Measures Teen Pregnancy Prevention Grantees. One new construct is specific to the Innovation Network (Tier 2) grants. (see the forms for the complete list of measures). The Tier 1 Supportive Services item is a modification of the previously approved healthcare linkages item from the current approved collection and will only be collected from Tier 1 grantees. The stakeholder engagement item is new to the FY2020 collection, and reflects priorities specified within the FY2020 Tier 1 and FY2020 Tier 2 Innovation network funding announcements.





A.5. Impact on Small Businesses



The only respondents are the TPP grantees; thus, no small businesses will be involved in the collection of data in this study.



A.6. Consequences of Not Collecting the Information/Collecting Less Frequently



GPRAMA requires that government agencies report on their performance measures annually. Therefore, it is essential that grantees report on these performance measures at least annually to OAH. In addition, collection and reporting of performance measure data is a requirement of all TPP grantees as stated in the Funding Opportunity Announcement.



Since the inception of the TPP program, the program office has collected TPP performance measures semi-annually, and OMB has previously approved this frequency of data collection (OMB control numbers 0990-0392 and 0990-0438). OPA, to better monitor the complex TPP grant projects collects the data twice each year. The semi-annual data collection allows federal staff monitoring grants to provide data-driven feedback to grantees. In addition, the mid-year aggregate data allows staff to assess the performance of the grant program as a whole and to plan for technical assistance and training needs of grantees.



A.7. Special Circumstances



There are no special circumstances that occur when collecting this information.



A.8. Federal Register Comments and Persons Consulted Outside the Office of Population Affairs



A 60-day notice was published in the Federal Register on February 26, 2020, in Volume, Vol. 85, No. 38, page 11095-6, and provided a 60-day period for public comments. No public comments were received (See Supplemental Materials, 60 Day FRN).



In 2010 and 2011, OAH (now OPA) consulted with staff of RTI International, which was the contractor responsible for helping develop the TPP performance measures and performance measure reporting system, and a panel consisting of experts in the fields of performance measurement, teen pregnancy prevention, and evidence-based practice. In addition, OAH (now OPA) presented information on the performance measures to TPP grantees and their evaluators at two conferences, and solicited their input. OAH (now OPA) also consulted and received feedback from other Federal staff working in the area of teen pregnancy prevention from ASPE, ACF, and CDC. The bulk of the performance measures that are part of this clearance package have been successfully used during the past 8 years in the data collection for the “Office of Adolescent Health and Administration for Children, Youth and Families Teen Pregnancy Prevention Performance Measure Collection.” Additional measures in 2015 and 2020 were developed in consultation with staff within what is now OPA.



A list of individuals in the expert panel in 2010 who provided input regarding the process evaluation is found in Exhibit 3.

Exhibit 3. Persons Consulted Outside the Agency for Performance Measure Collection for FY 2010 Grantees

Expert Work Group

Donald Moynihan

[email protected]

University of Wisconsin

Lafollette School of Public Affairs

305 Observatory Hill Office Building

1225 Observatory Dr

Madison, WI 53706

(608) 263-6633


Kathryn Newcomer

[email protected]

George Washington

SPPPA

MPA Bldg 601

805 21st St NW

Washington, DC 20052

(202) 994-3959


Katherine Suellentrop


[email protected]

The National Campaign to Prevent Teen and Unplanned Pregnancy

1776 Massachusetts Ave, NW, suite 200

Washington, DC 20036

(202) 478-8515


Edward Mullen

[email protected]

Columbia University

School of Social Work

1255 Amsterdam Ave Room 1102

New York, NY 10027

(212) 851 2413


Douglas Kirby (deceased)

[email protected]

ETR Associates

4 Carbonero Way

Scotts Valley, CA 95066

(831) 438-4060


Forrest Alton

[email protected]

SC Campaign to Prevent Teen Pregnancy

1331 Elmwood Avenue, Suite 140

Columbia, SC 29201

(803) 771-7700


Emily Ball

[email protected]

Administration for Children and Families

26 Federal Plaza

New York, NY 10278

(212) 264-2890 x273





A.9. Payments to Respondents

There are no payments to staff of grantee organizations completing the performance measure reporting forms.

A.10. Assurance of Confidentiality

Data will be kept private to the extent allowed by law. All participant performance measures data are reported to OPA by grantees de-identified and aggregated to the level of a section (group or cohort receiving programming together). No personal identifiers will be used in the reporting of any data to OPA.

The Web-based reporting system has been designed to ensure the security of the data obtained. Individual users designated by the grantees will be assigned user names and passwords that will grant them access to the performance measures website. There, users will have the opportunity to provide data that will be stored in a secure Microsoft Azure database (FedRAMP approved, Authority to Operate (ATO) granted on May 21, 2020), utilizing a relational table structure, facilitating expedient data retrieval and analysis. The grant-specific data users submit on the cloud-based database, will be accessible only to the grantee user, federal staff, and administrators assigned to this project. Screenshots are included in the supplemental materials.

The TPP performance measures have consistently been waived from Institutional Review Board (IRB); a copy of the waiver letter may be found in the supplemental materials.

A.11. Sensitive Questions



Grantees report on the demographics, such as race/ethnicity, of the program participants served; collecting race and ethnicity is usual practice for HHS data collections. These data will be reported to OPA at the group level. No personal identifying information will be collected from participants; no questions about attitudes or beliefs are collected in the performance measures.



A.12 Burden Estimate (Total Hours & Wages)



A.12A Estimated Annualized Burden Hours



The total annual burden is estimated to be 1155 hours for grantees to collect, summarize, and report the data for the performance measures (Exhibit 4). Estimates were based on data collection timing for similar measures collected from FY2015 TPP grantees, as the majority of items in this collection are either unchanged from the original approved version, or have minor changes.













Exhibit 4: Annualized Burden Hour Table

Forms

(If necessary)

Respondents

(If necessary)

Number of Respondents

Number of Responses per Respondents

Average Burden per Response

Total Burden Hours

Grantees (partners and sustainability)

All TPP grantees

90

2

15/60

45

Grantees (training)

All TPP Grantees

90

2

15/60

45

Grantees (dissemination)

All TPP Grantees

90

2

30/60

90

Grantees (Stakeholder Engagement)

All TPP Grantees

90

2

15/60

45

Grantees

( Reach and Demographics)

Tier 1 and Tier 2 Phase II Grantees

64

2

3

384


Grantees (Dosage)

Tier 1 and Tier 2 Phase II Grantees

64

2

2

256

Grantees (Fidelity and Quality)

Tier 1 and Tier 2 Phase II Grantees

64

2

2

256

Grantees (Tier 2 Innovation Network)

Tier 2 Innovation Network Grantees

14

2

15/60

7

Grantees (Supportive Services)

Tier 1 Supportive Services

54

2

15/60

27

Total



2


1155







Average burden hours for grantees or sub-awardees



All of the data will be reported by the grantees and their implementing sub-awardees twice a year (see Exhibit 6 for schedule). Grantees will collect most of these data for their own administrative purposes and are expected have their own systems in place to track the data. Thus, the only additional burden to grantees for reporting the performance measures is the time it takes them to assemble the necessary data and enter it into the reporting forms. Respondents can either upload spreadsheets into the web-based system or enter the data directly into the system using a point and click method. The burden may be significantly less for grantees who manage their data using contractor-designed spreadsheets. There is no burden to program participants.





A.12B Estimated Annualized Cost to Respondents



The estimated 1-year annualized cost to respondents is $34,650.00 as shown in Exhibit 5. Salaries of the grantee staff collecting data, entering data, and summarizing and reporting data may vary across agencies. We estimate an average hourly rate of $30.


Exhibit 5. Estimated 1-Year Annualized Cost to Respondents

Forms

Type of Respondent

Estimated Number of Respondents

Total Burden Hours

Hourly Wage Rate

Total Respondent Costs

Partners & Sustainability 

All Grantees

90

45

$30.00

$1,350.00

Trainings

All Grantees

90

45

$30.00

$1,350.00

Dissemination

All Grantees

90

90

$30.00

$2,700.00

Stakeholder Engagement

All Grantees

90 

45

$30.00

$1,350

Reach & Demographics

Grantee (Tier 1 and Tier 2 Phase II Only)

64

384

$30.00

$11,520.00

Dosage

Grantee (Tier 1 and Tier 2 Phase II Only)

64

256

$30.00

$7,680.00

Fidelity and Quality  

Grantee (Tier 1 and Tier 2 Phase II Only)

64

256

$30.00

$7,680.00

Innovation Network Tier 2 Measure

Grantee (Tier 2 Innovation Network Only)

14

7

$30.00

$240

Supportive Services Tier 1

Grantee (Tier 1 Only)

54

27

$30.00

$810

Total

 

 

 1155

 

$34,650.00



A.13 Capital Costs (Maintenance of Capital Costs)

There are no capital costs associated with this study.

A.14 Cost to Federal Government

The cost to the federal government is $1,500,000 over 3 years (time remaining on contract) to complete the contract. The cost includes the estimated cost of coordination with OPA, IRB and OMB applications, revision of the data reporting system for the new cohort of grantees, on-going maintenance of the data reporting system, training and technical assistance to the grantees and federal staff in the use of the data reporting system, and data analysis and reporting by the contractor staff. Annual cost to the government is approximately $500,000 per year.

A.15 Program or Burden Changes

This is a revised information collection. Some changes in burden within this submission reflect an update in the anticipated number of respondents, which are based upon the estimated maximum number of awards in the 3 FY2020 FOAs.

Some of the grantee-level items as discussed above were revised and reworded for clarity and are program changes designed to reflect the new funding priorities contained in the new FOAs. The dissemination item includes more specific information about social media, to reflect the FY2020 FOAs focus on development of dissemination plans; burden increased per response to reflect these changes.

The service referral question is essentially the same as the healthcare linkages item in the original approved submission. The referral item is now only to be asked of Tier 1 grantees, rather than all TPP grantees; the change reflects different priorities contained within the FY2020 FOAs.

There are new forms in this revision for stakeholder engagement, which is a key expectation within the three FY2020 FOAs and Tier 2 Innovation Network projects. Finally, the cost items which were collected previously for FY2015 grantees were deleted from this revision, resulting in a reduction in the burden.

The participant-level items (participant reach & demographics, dosage, fidelity & quality) are essentially the same as in the approved FY2015 collection; the key change is a reduction in burden because OPA anticipates we will only be collecting these items from Tier 1 and Tier 2 Phase II grantees, and not from Tier 2 Innovation Network grantees. The Tier 2 Innovation Network grantees are expected to work with innovators and early stage program developers and are not expected to work directly with program participants. In the FY2015 cohort, the Tier 2A projects served a similar target population (that is, innovators) and did not have participant-level data to report. For this reason, OPA developed a measure to track projects funded by stage from the Tier 2 Innovation Network grantees.

16. Tabulation of Data and Schedule

Data collection with the revised measures would begin as soon as OMB approval was granted. Assuming that OMB approval for this collection is granted by the fall of 2020, the FY 2020 cohort of TPP grantees will first collect and report data in Winter 2021. The anticipated schedule is included below (Exhibit 6); no data collection using the FY2020 revised measures would begin prior to OMB approval). The FY2020 grantee cohort will continue to collect and report data according the anticipated schedule below until summer 2023 at the semi-annual reporting periods (30 days after each reporting period, anticipated to be January 31 and July 30). The performance measures contractor will then analyze the data and prepare a written report, summarizing findings. Data will be aggregated across all projects, as well as broken down by type of grantee (i.e., Tier 1 and Tier 2, and by FOA (Tier 2 Phase II and Tier 2 Innovation Network). The annual reports would be published on the OPA website using aggregate-level data.

The anticipated schedule for these FY2020 grants assumes a July 2020 award date for all three TPP FOAs;

  • Grant Year:

    • Reporting period:

      • 6 month period #1: July 1 – December 31, Deadline: January 31

      • 6 month period #2: January 1 – June 30; Deadline July 30







Exhibit 6: Anticipated Reporting Schedule for TPP FY2020 Performance Measures

Grant Year

Semi-Annual Period

Data Reported to OPA

FY2020 (year 1)

July 1 – December 31. 2020

January 31, 2021


January 1 – June 30, 2021

July 30, 2021

FY2021 (year 2)

July 1 – December 31, 2021

January 31, 2022


January 1 – June 30, 2022

July 30, 2022

FY2022 (year 3)

July 1 – December 31, 2022

January 31, 2023


January 1 – June 30, 2023

July 30, 2023



A.17. Display of Expiration Date for OMB Approval

The expiration date for OMB will be displayed on all data collection instruments.

A.18. Exceptions to Certification Statement.

There are no exceptions to the certification statement.


List of Supplemental Materials:


Federal Register Notice to the Public


Institutional Review Board (IRB) Exemption Letter

Program Observation (Quality) Form (Referenced in Fidelity and Quality Form)

Sample Program Fidelity Monitoring Tool (Referenced in Fidelity and Quality Form

TPP Fidelity Process Form (Referenced in Fidelity and Quality Form)

TPP Database Web-Shots of Data Entry Screens



12


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File TitleSupporting Statement for
Authorwallace
File Modified0000-00-00
File Created2021-01-14

© 2024 OMB.report | Privacy Policy