Non-material Change Request

_MEMORANDUM

TO:	OMB, Office of Information and Regulatory Affairs
FROM:	Department of Labor	DATE: 01/25/2018
SUBJECT:	PRA Non-Material Change to National Evaluation of the Performance Partnership Pilots for Disconnected Youth (P3) Program (OMB Control No. 1290-0013)

This memo requests a non-material change to the Paperwork Reduction Act (PRA) package previously approved by the Office of Management and Budget (OMB Control No. 1290-0013) for data collection from April 2017-2020. The purpose for this request is to include six additional sites to the approved data collection activities.

Reason for non-material change

The Performance Partnership Pilots for Disconnected Youth (P3) was originally authorized in the Consolidated Appropriations Act of 2014 (Sec. 526), with continuing authority appropriated in the three subsequent years (2015-2017). The resulting re-authorizations of P3 generated six additional P3 pilots (known as the cohort 2/3 pilots) being awarded in 2017. With the goal of producing a comprehensive study of P3’s implementation and systems work, the U.S. Department of Labor (DOL) modified the P3 evaluation contract with Mathematica Policy Research to include the six cohort 2/3 pilots. The research study seeks to extend the data collection instruments already approved for the cohort 1 pilot to conduct one round of site visits to the six cohort 2/3 sites (OMB Control No. 1290-0013). No other changes to the implementation and systems study are planned.

Thus, the inclusion of the six additional visits reflects an increase in burden but a non-

substantive change in the design of the P3 evaluation’s implementation and systems study.

Data collection activities

The six cohort 2/3 site visits are in addition to the originally planned two visits to each of the nine cohort 1 grantees. The study design, as described in the original PRA package, includes four data collection activities: (1) site visit interviews, (2) focus group discussions with program participants, (3) a survey of partner managers, and (4) a survey of partners.

Site visit master protocol. Site visits to the nine cohort 1 and six cohort 2/3 pilots will provide information on P3 implementation and systems change. The study team will conduct interviews with pilot administrators and staff, partner leaders and managers, and frontline staff. We will conduct one site visit to each cohort 2/3 pilot in addition to two site visits to each cohort

1 pilot (the first of which was conducted in 2017). As the first visit to cohort 1 pilots, the only visit to each cohort 2/3 pilot will be a two-person visit, led by a member of the evaluation

leadership team. The second visit to cohort 1 pilots, planned for around spring 2018, will be one- person visits. Depending on the scope of the pilot, the visits will be from two to three days.

Interviews with P3 administrators/managers and frontline staff will last, on average, 75 minutes. Most will be one-on-one interviews, but we anticipate that some of the frontline-staff interviews will be with small groups of two to three staff. We estimate the maximum total hours for P3 data collection at the 15 sites, including two visits to nine cohort 1 sites and one visit to six cohort 2/3 sites, will be 450 (24 visits × 15 respondents × 75 minutes per interview).

Youth focus group discussion guide. Focus group discussions with P3 youth participants will provide important information about youths’ program experiences, views on whether the program is meeting their needs, challenges that interfere with their participation, and suggestions for program improvements. To collect this information, the study team will conduct an average

of three focus groups at each of the nine cohort 1 pilots and one focus group at each of the six cohort 2/3 pilots. The study team anticipates including an average of eight youth in each focus group, and that each interview will last about one hour. We anticipate that half of the focus group participants will be younger than 18, thus requiring their parent’s or guardian’s consent to participate.

Thus, we estimate the total maximum reporting burden for the youth focus groups will be

264 hours (33 focus groups × 8 respondents in each group × 60 minutes per group discussion).

Partner manager survey. The survey of partner managers will provide important systematic information about how the partner managers view collaboration within their communities. Although site visit interviews will provide important information on the P3 system and how the P3 partners have worked together since the initial application design through the implementation of the P3 pilot, the survey provides an opportunity to assess key aspects of the partnership using quantitative data.

We expect to conduct the survey with approximately 10 partner managers on each visit. The survey will take an average of 5 minutes for the partner manager to complete. We will administer the survey in each round of site visits. Thus, we estimate the total maximum reporting burden for the partner manager survey will be 20 hours (10 cohort 1 partner managers × 9 pilots × 2 visits ×

5 minutes) + (10 cohort 2/3 partner managers × 6 pilots × 1 visit × 5 minutes).

Partner network survey. A survey of partners will provide systematic information about the relationship across providers. The study team will administer a short survey (about 10 minutes)

to direct service staff of each pilot partner. We anticipate conducting this survey once with up to

10 provider staff at each pilot. The survey will use social network analysis questions to focus on

partners’ interactions with one another. The study team will field the survey in each pilot shortly

after completing its 2018 site visit.

We expect to conduct the survey with approximately 150 partners (15 P3 pilots with up to 10 partner staff per site). We expect the survey will take about 10 minutes to complete, on average, per respondent. We will stagger survey administration, enabling us to test administration procedures in one pilot before administering to all sites. We expect the total maximum reporting burden for the partner network survey to be 25 hours (15 pilots × 10 partner staff × 10 minutes).

Burden

The total estimated maximum hours of burden for the data collection included in this non- substantive change request, including the original 18 (two visits to the 9 cohort 1 pilots) and additional site visits to the six new pilots, is 759 hours (see Table 1). This equals the sum of the estimated burden hours for the site visit interviews (450), youth focus groups (264), partner manager survey (20) and the partner network survey (25).

This burden estimate reflects an increase of 175.5 hours resulting from the inclusion of the six cohort 2/3 pilots to the data collection for the implementation and system study.

_{Table 1. Annual burden estimates for data collection activities, over 36 months}

Respondents	Total number of respondents over evaluation	Annual number of responses	Number of responses per respondent	Average burden time per response (minutes)		Total burden hours over evaluation	Annual burden hours
Site visit interviews		Total number of respondents over evaluation
Administrators and staff (Cohort 1)	135	90	2	75		337.5	112.5
Administrators and staff (Cohort 2/3)	90	30	1	75		112.5	37.5
Focus group discussions
With Youth	264	88	1	60		264	88
Partner manager survey									90
Partner managers (Cohort 1)	90	60	2	5		15	5.0
Partner managers (Cohort 2/3)	60	20	1	5		5	1.7
Partner network survey									150
Frontline staff	150	50	1	10		25	8.3
Total	889	338	--		--	759	253

The total monetized burden hours over 36 months is estimated to be $14,301 (Table 2). This reflects an increase of $3,517 as a result of the inclusion of the six cohort 2/3 pilots.

_{Table 2. Monetized burden hours, over 36 months}

_Respondents

Total maximum burden (hours)

Type of respondent

Estimated hourly wages

Total indirect cost burden

Annual monetized burden hours

Semi-structured interviews

Grantee and partner managers (Cohort 1) 135 Frontline staff (Cohort 1) 202.50 Grantee and partner managers (Cohorts 2/3) 45 Frontline staff (Cohorts 2/3) 67.5		Manager	$33.38	$4,506	$1,502
		Frontline staff	$19.36	$3,920	$1,307
		Manager	$33.38	$1,502	$501
		Frontline staff	$19.36	$1,307	$436
Subtotal	450	--	--	$11,235	$3,746
Youth focus groups
Youth	264	--	$7.25	$1,914	$638
Partner manager survey
Partner managers (Cohort 1)	15	Partner manager	$33.38	$501	$167
Partner managers (Cohorts 2/3)	5	Partner manager	$33.38	$167	$56
Subtotal	20	--	--	$668	$223
Partner network survey
Frontline staff	25	--	$19.36	$484	$161
Total	759	--	--	$14,301	$4,768