OAHNon-Substantive change Request

DEPARTMENT OF HEALTH AND HUMAN SERVICES Office of the Secretary

Washington, DC 20201

TO: Sherette Funn, OS/OCIO, PRA Reports Clearance Officer

THROUGH: Darius Taylor, HHS-OCIO, PRA Report Clearance Officer

FROM: Amy Feldman Farb, Office of Adolescent Health

CC: Evelyn Kappeler, Office of Adolescent Health;

Tara Rice, Office of Adolescent Health

SUBJECT: Non-Substantive Change Request to 0990-0438 (Office of Adolescent Health Teen Pregnancy Prevention, FY 2015-2020 Performance Measure)

DATE: April 19, 2017

­­­­­­­­­­­­­

On October 26, 2015 OMB approved the Office of Adolescent Health Teen Pregnancy Prevention, FY 2015-2020 Performance Measure information collection package (0990-0438).  Part of this data collection proposal included measures referenced as the “Cost Performance Measures”.  The cost performance measures questions were designed to track the costs to grantees and all partner agencies of implementing Teen Pregnancy Prevention (TPP) program models and providing relevant program services. The five cost performance measures are listed below:

Cost Performance Measures

1. Personnel Costs

Indicate the total personnel costs, including salary, payroll taxes, and benefits, paid during this reporting year by each organization and implementation partner supported by the grant:

<<<name of grantee organization>>> $ ___________

<<<name of partner organization #1>>> $ ___________

<<<name of partner organization #2>>> $ ___________

<<<name of partner organization #3>>> $ ___________

For each organization or implementation partner supported by the grant, estimate the proportion of personnel costs used to support each of the following activities during this reporting year:

General Admin.

Participant recruitment or retention

Staff training & technical assistance

Providing program services

Fidelity monitoring

Evaluation

<<grantee>>

%

%

%

%

%

%

<<partner #1>>

%

%

%

%

%

%

<<partner #2>>

%

%

%

%

%

%

2. Office Space and Facilities

Indicate the category that best describes the cost of office space and facilities used by the grantee and any implementation partners (check one):

	All office space and facilities are used free of charge to the grantee and implementation partners
	Some office space and facilities are used free of charge and others require payment
	All office space and facilities require payment

If some or all of the office space and facilities require payment, indicate the total amount paid by the grantee and any implementation partners during this reporting year: $ ________________

3. Financial Diversification and Sustainability

For each of the following resource categories, indicate the amount of funding outside of the grant received during this reporting year to assist with ongoing and future program activities:

Fund raising or cash (donations, fee for service, etc.)		$
Other grants		$
Internal agency funding		$
In-kind contributions (estimate value)		$
Other specify:		$

4. Payments to Program Developer or Distributor

Indicate the total amount of any payments made to a program developer or distributor during this reporting year by either the grantee or any implementation partners:

$

What types of materials, supports, and/or services were covered by these payments during this reporting year (check all that apply)?

Curriculum or other program materials and supplies
Licensing fees
Fidelity monitoring or quality improvement services
Evaluation
Other specify:

5. Other Direct Costs To Support Program Implementation

Apart from any payments made to a program developer or distributor, indicate the total amount paid during this reporting year by the grantee or any implementation partner for each of the following:

Other program materials and supplies from an outside provider		$
Monetary incentives, including gift cards, for program enrollment or participation		$
Non-monetary incentives for program enrollment or participation (t-shirts, etc.)		$
Program supports (meals, transportation, etc.)		$
Program recruitment materials		$
Media campaigns		$

The intent of collecting the cost performance measures was to obtain ongoing data on the costs incurred by grantees and their partners when implementing TPP program models. To date, there has been scant documentation of the cost of implementing various Teen Pregnancy Prevention programs; the current research literature also says very little about the relative cost of various programmatic approaches, the drivers of program costs, or how program costs compare to the effectiveness of the programs. OAH deemed that collecting cost information would be important for its own planning purposes.

During the 2010-2015 cohort of the TPP grant program, OAH implemented a cost study (OMB # 0990- 0425) with 28 Tier 1 TPP grantees using the cost measures questions to assess the program implementation costs; this study served as a pilot test of these measures prior to our information collections request. The majority (26 out of 28) of the grantees participating in the cost study implemented and collected cost data on a single TPP program model implemented in a single setting.

At this time, we have collected and analyzed the first round of performance measures data from the 2015-2020 cohort of TPP grantees, and we note several concerns with the cost questions. The personnel cost items, require grantees to report personnel costs for their agency as well as each partner organization. The TPP grants are structured very differently in the cohort funded for the period 2015-2020 (see Table 1 on the next page). The 50 TPP Tier 1B grantees are replicating program models to scale; each grantee is working with an average of 7 partner agencies (range of 1 to 132 partners per grant), and implementing an average of 4 program models per grantee. Because grantees (in particular the TPP Tier 1Bs) have more partner agencies to report personnel cost than OAH anticipated, the personnel cost items have been very burdensome for grantees to collect, leading to delays in data submission and issues with data quality for the cost measures. In addition, grantees have had considerable difficulty interpreting the definitions of the categories within the personnel costs, and have been inconsistent in assigning activities to the various categories. In particular, the 2015-2020 cohort grantees are expected to implement TPP program models and undertake other activities such as community mobilization and health care referrals. Although OAH has instructed grantees to focus only on program model costs, the majority of Tier 1 grantees have had difficulty isolating these costs from other grant activities. This has resulted in background noise within the data that we have not been able to resolve. At least 15% of the participants provided at least one response within the personnel costs that did not make sense based on their implementation plans, and at least 8% of grantees were not able to provide responses to the personnel cost items.

We have also identified noise in the Office Space and Facilities Cost Item; the primary concern is that this section should cover both grantee and implementing partners. It is unclear whether grantees have been consistently including their partners. It is also unclear whether grantees have used both indirect and direct costs to determine these figures.

The Other Direct Costs to Support Program Implementation item has not provided us with relevant information. A review of the performance measures technical assistance emails related to cost measures notes the following as common issues. The 21 Tier 2B grantees reported considerable challenges in distinguishing between monetary incentives for program enrollment versus incentives given for evaluation enrollment. The 58 Tier 1 grantees reported challenges defining who would count as an outside provider of program models.

Because we now know collecting the aforementioned items is burdensome to the participants and the benefit to the collection of the information is now lower, OAH plans to halt collection of three cost questions: personnel costs, office space and facilities, and other direct costs to support program implementation. OAH will continue to collect cost data from the TPP grantees on financial diversification and sustainability and payments to the program developer. OAH will also continue to collect the grantee-level TPP performance measures (training of facilitators, partnerships, program monitoring, health care linkages) and the program-level measures (participant reach, fidelity, dosage).

Therefore, we're asking for a non-substantive change and change in burden from 1,252 total burden hours to 1,220 total burden hours. 

Table 1: Summary of TPP grants in FY2015

TPP Funding Stream	Agency	Total Annual Funding	Award Size	Independent rigorous evaluation	# of grants
Capacity Building to Support Replication of Evidence-Based TPP Programs (Tier 1A)	OAH	$5 million	$400,000 - $750,000	No	8
Replicating Evidence-Based TPP Programs to Scale in Communities with the Greatest Need (Tier 1B)	OAH	$60 million	$500,000 - $2,000,000	No	50
Supporting and Enabling Early Innovation to Advance Adolescent Health and Prevent Teen Pregnancy (Tier 2A)	OAH	$3 million	$1,000,000 -$1,500,000	No	2
Rigorous Evaluation of New or Innovative Approaches to Prevent Teen Pregnancy (Tier 2B)	OAH	$18 million	$400,000 - $1,000,000	Yes	21
Effectiveness of TPP Programs Designed Specifically for Young Males (Tier 2C)	CDC DRH	$2 million	$600,000 - $1,000,000	Yes	3
TOTAL					84

Table 2. Estimated Annualized Burden Hours

Forms (If necessary)	Type of Respondent	Estimated Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Total Burden Hours
Grantee- Level Measures
Dissemination	Grantee staff	84	2	.25	42
Number of partners	Grantee staff	84	2	.25	42
Number of facilitators trained	Grantee staff	84	2	.25	42
Health-care linkages	Grantee staff	84	2	.25	42
Program-Level Measures
Participant reach	Grantee/Sub-awardee staff	84	2	2.8	470.4
Dosage	Grantee/Sub-awardee staff	84	2	1.7	285.6
Fidelity	Grantee/Sub-awardee staff	84	2	1.7	285.6
Cost	Grantee/Sub-awardee staff	84	2	.25	42
Total					1,251.6

Table 3 describes the proposed burden changes. OAH proposes eliminating the following cost measures: personnel costs, office space and facilities, and other direct costs of program implementation. OAH would collect the remaining cost performance measures (financial sustainability and costs to program developers) from grantees once per year rather than twice per year. This results in a decrease in the total burden hours from 1,252 hours to 1,220 hours. In the data collections we have performance thus far we have revised the burden hours to reflect the real time they are taking minus the deleted items, so it isn’t a perfect original total minus original items being deleted. All proposed burden changes are highlighted in yellow in Table 3.

Table 3. Revised Estimated Annualized Burden Hours

Forms (If necessary)	Type of Respondent	Estimated Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Total Burden Hours
Grantee- Level Measures
Dissemination	Grantee staff	84	2	.25	42
Number of partners	Grantee staff	84	2	.25	42
Number of facilitators trained	Grantee staff	84	2	.25	42
Health-care linkages	Grantee staff	84	2	.25	42
Program-Level Measures
Participant reach	Grantee/Sub-awardee staff	84	2	2.8	470.4
Dosage	Grantee/Sub-awardee staff	84	2	1.7	285.6
Fidelity	Grantee/Sub-awardee staff	84	2	1.7	285.6
Cost	Grantee/Sub-awardee staff	84	1	.12	10
Total					1,219.6