1291-0NEW PART A_LEAP_Implementation Study OMB_083116

Part A: Justification for the Collection of Data
for the Linking to Employment Activities Pre-Release (LEAP) Grant Programs

August, 2016

Submitted to:

Office of Management and Budget

Submitted by:

Chief Evaluation Office

Office of the Assistant Secretary for Policy

United States Department of Labor

200 Constitution Avenue, NW

Washington, DC 20210

This page has been left blank for double-sided copying.

OFFICE OF MANAGEMENT AND BUDGET SUPPORTING STATEMENT PART A: JUSTIFICATION FOR THE linking to employment activities pre-release (LEAP) Evaluation

The Chief Evaluation Office (CEO) of the U.S. Department of Labor (DOL) has commissioned an evaluation of the Linking to Employment Activities Pre-Release (LEAP) program. LEAP is an ambitious effort to establish satellite American Job Centers (AJCs) in local jails to improve the employment and recidivism outcomes of participants after their release from incarceration. The LEAP evaluation offers a unique opportunity to study how linking pre- and post-release employment services supports participants’ successful reentry into the community. To accomplish these aims, Mathematica Policy Research (Mathematica) and its subcontractor Social Policy Research Associates (SPR) have been contracted to conduct a comprehensive implementation study. This package requests clearance for the following three implementation-study data collection activities:

1. Semi-structured interviews with administrators and managers;

2. Focus groups with LEAP frontline staff, representatives from LEAP partner organizations, and program participants receiving pre- and post-release services; and

3. Respondent information forms (RIFs) from focus group participants.

A.1. Circumstances making the collection of information necessary

More than 9 million people are released from U.S. jails every year, and at least half are rearrested within three years. The rising number of individuals exiting incarceration facilities and the persistently high rates of recidivism among those formerly incarcerated has prompted increased attention to offender reentry as a policy issue with profound reach and impact. Although many experts believe that finding stable employment can be a critical factor in the successful transition from prison or jail to the community, several studies document the challenges ex-offenders face in finding and maintaining employment upon release (Apel and Sweeten 2010; Raphael 2008; Western et al. 2001). Existing research suggests that coordinating pre-release and post-release employment services might increase the likelihood that offenders access employment-related services after release, which might, in turn, improve their employment and recidivism outcomes (Nelson and Trone 2000; Wilson et al. 2005).

It is within this context that DOL provided $10 million in two-year LEAP grants to 20 local workforce investment board (LWIB) grantees. LEAP grants, awarded in June 2015, aim to strengthen ties between the public workforce system and local correctional facilities and improve the employment and recidivism outcomes of formerly incarcerated individuals. The LEAP grant program and the subsequent evaluation are authorized by Section 171 of the Workforce Investment Act which states that “the Secretary shall, through grants or contracts, carry out demonstration and pilot projects for the purpose of developing and implementing techniques and approaches, and demonstrating the effectiveness of specialized methods, in addressing employment and training needs. Such projects shall include the provision of direct services to individuals to enhance employment opportunities and an evaluation component” (Sec.171(b) 1998). Section 171 also states that “the Secretary shall, through grants or contracts, carry out research projects that will contribute to the solution of employment and training problems in the United States”, including multistate projects that “address the specialized employment and training needs of particular service populations” (Sec.171(c) 1998).

1. LEAP program model

LEAP programs are designed to provide a continuum of services, starting with pre-release career services while participants are still incarcerated, to post-release services after they return to the community. As illustrated in Figure A.1, pre-release services are provided by staff within the jail facility and include skills and career interest assessments, preparation of an independent development plan (IDP), comprehensive case management, resume and job search workshops, job readiness training, and industry-specific vocational training. Staff then support a direct “hand-off” of participants from pre-release services in the jail-based AJC to post-release services in the community-based AJC. Post-release services, provided by community AJCs, include job search and placement assistance, vocational training and certification, transitional jobs, retention support, wraparound services, and links to other community resources. The combination of pre- and post- release services, as well as close coordination between staff providing these services to participants, has the potential to improve participants’ work readiness, attitudes about employment, knowledge of AJC and partner services, connections to employers, and education and training with the ultimate goal of positively impacting their employment, job retention, earnings, and recidivism.

Figure A.1. Conceptual framework of LEAP program model

2. Overview of the implementation evaluation

The LEAP evaluation is designed to explore how employment services in pre-release correctional settings that are directly linked with post-release community-based AJC workforce services can support the successful re-entry of ex-offenders into the community and improve their employment and recidivism outcomes. Through this proposed evaluation, the study team will document how each grantee provides and links participants with pre- and post-release services and the approaches used to increase participants’ knowledge of, access to, and use of AJCs as workforce partners and brokers of relationships with employers and other community-based resources. Additionally, this information will be used to identify program model typologies that indicate programs and features associated with potentially promising outcomes.

The LEAP implementation evaluation will take place over three years (2015–2018) and will address the following five research questions:

1. How and why do LEAP grantees vary in structure, partnerships, and implementation of the LEAP grants?

2. What are the key program elements that are common to successful models of jail-based AJCs? What implementation practices appear promising to ensure the model’s sustainability or replicability, and what steps have been taken to ensure sustainability after grant funding?

3. How is LEAP implemented and how are outcomes, including those proposed in grant applications, tracked? How do programs achieve proposed outcomes, such as job readiness and reduced recidivism, and what challenges and strengths influenced implementation?

4. How does LEAP address the unique challenges of transitioning offenders and related subgroups of this population?

5. What are the perceptions of participants, facility-based staff and stakeholders, community-based AJC staff members, and community-based partners regarding the LEAP program?

3. Overview of the data collection

Understanding the effectiveness of the LEAP program requires data collection from multiple sources. To collect these data, the study team will conduct two rounds of site visits to each of the 20 grantees, with the first in spring/summer 2016 and the second in spring 2017. These visits will last two days and will include in-person visits to the local area served by each LWIB grantee, including both the jail-based and community-based AJCs. After obtaining assent from program staff and partners and written consent from program participants, the study team will collect a rich set of qualitative data from LEAP administrators, frontline staff, program partners (staff from community-based nonprofits or business or employer partners), and program participants. The data covered by this clearance include semi-structured interview data, focus group data, and demographic and contextual data from focus group participants through Respondent Information Forms (RIFs). Next, each of these data collection activities are described in detail.

Semi-structured administrator and manager interviews. During each visit, two members of the study team will conduct semi-structured, in-depth interviews with administrative staff responsible for the LEAP program, including the (1) LWIB grantee, (2) the LEAP program manager, (3) jail administrators, (4) jail-based AJC managers, and (5) community-based AJC managers. The data collected using the semi-structured interview protocols will focus on management practices and challenges, the pre- and post-release services offered and linkages between them, staff roles and responsibilities, participants’ needs and expectations, the service take-up rate, the mix of partners and their roles, strategies for engaging LEAP participants upon release, plans for sustainability, and challenges associated with establishing an AJC in a jail. The master protocol will be used to create individual discussion guides based on the respondent type before the site visits. Table A.1 below displays the topics that will be addressed with each respondent type.

Table A.1. Site visit topics by respondent

	Site Visit Interview and Focus Group Topics	Semi-structured Interviews					Focus Groups
		LEAP Project Manager	Jail Administrator	Jail-AJC Manager	Community-AJC Manager	LWIB Representative	Participant Focus Groups	Frontline Staff Focus Groups	Partner Staff Focus Groups
	Grantee/Partner Information
A	Context
A1	Administrative structure, supervision, management	X		X	X	X		X	X
A2	Grantee background	X				X
A3	Local context	X	X		X	X		X
A4	Experience with transitioning offenders	X	X	X	X	X		X	X
B	Partnerships
B1	Partners and partner roles	X	X	X	X	X		X	X
B2	Partner arrangements/contracts	X							X
B3	Partner goals	X	X						X
B4	Partner experience with offenders and transitioning offenders	X							X
B5	Partnerships with specialized service providers	X				X			X
B6	Relationship with employers	X			X	X		X	X
B7	Relationship with jail	X	X	X		X		X	X
C	Sustainability (for second round of visits)
C1	Identification of funding opportunities	X				X
C2	Attitude toward post-grant sustainability	X	X	X	X	X
	Program Implementation and Outcomes
D	Pre-release jail-based services
D1	Participant recruitment, intake, and enrollment	X	X	X			X	X
D2	Assessment and service planning		X	X			X	X	X
D3	Case management and supportive services			X	X		X	X	X
D4	Job development, placement, and retention services			X	X		X	X	X
D5	Staffing (staff roles and qualifications)	X	X	X	X			X	X
D6	Participant flow	X	X	X	X		X	X	X
E	Transitional and post-release services
E1	Handoff to external AJC services	X		X	X			X
E2	Post-release follow-up with ex-offenders and providers	X		X	X		X	X
E3	Post-release use of community-based services	X		X	X		X	X	X
F	Outcome tracking
F1	Service and outcome data collection capacity	X	X	X	X			X	X
F2	Post-release employment services	X			X			X
F3	Post-release supportive services	X			X			X
F4	Post-release employment outcomes	X				X		X
F5	Post-release arrest/incarceration incidence	X	X			X		X
	Approach to Transitioning Offenders
G	Program features
G1	Ex-offender specific employment services			X	X
G2	Connections to housing providers			X	X				X
G3	Connections to substance abuse treatment			X	X				X
G4	Connections to mental health services			X	X				X
G5	Tailoring to specific sub-populations	X		X	X	X
H	Stakeholder Perceptions
H1	Implementing changes in response to feedback	X		X				X
H2	Program attitudes and opinions	X	X	X	X	X	X	X	X

Focus groups. During each implementation study visit, the study team will conduct three focus groups per site. In the first implementation study visit, separate focus groups will be conducted with LEAP frontline staff, representatives from LEAP partner organizations, and jail-based LEAP participants receiving pre-release services. During the second implementation study visit, focus groups will be conducted with LEAP frontline staff, jail-based LEAP participants receiving pre-release services, and community-based LEAP participants receiving post-release services. Focus groups that take place outside of the jail facilities will be digitally recorded for internal use only by the study team. The team will seek approval from the jail facilities in the study to allow recording of site visits conducted in the jails but anticipate that it will not be allowed in most of the sites. Table A.2 provides additional details on the structure of the focus groups and general topic areas covered, by respondent type. Table A.1 lists the specific topics that will be addressed with each focus group respondent.

Table A.2. Focus group data collection details, by respondent type

Focus group respondent	When	Focus group topic areas
Frontline staff	Site visits 1 & 2	Each group will include three to seven staff members who provide direct services to LEAP participants and will last about 90 minutes. The groups will be held in a location that is convenient for staff, which could be either the jail or the community-based AJC. These sessions will capture the perceptions and experiences of the frontline staff working with LEAP program participants.
Community partners	Site visit 1 only	Each group will include up to seven individuals from participating partner agencies or community-based institutions such as local businesses. These groups will likely take place at a community-based AJC and will last about 60 minutes. These sessions will capture the perceptions and experiences of the community-based agencies working with the AJCs and LEAP program participants.
LEAP participants	Pre-release: site visits 1 & 2 Post-release: site visit 2 only	The pre-release participant focus groups will involve three to five individuals (depending on the restrictions about the number of people allowed in group settings within the jail) and will cover topics such as participants’ perceptions about LEAP recruitment, intake, and enrollment, as well as their opinions about and experiences receiving pre-release services. The post-release participant focus group will involve five to seven individuals who have been released from jail. The post-release focus group will be held in a community-based AJC and will capture information about the transition to post-release services as well as participant successes and challenges seeking employment after release.

Consent to participate in the research study will be obtained from all focus group participants. To fully ensure informed consent, and because LEAP program participants might be characterized as a special population by the Institutional Review Board (IRB), the study team will collect written consent from all LEAP program participants (pre- and post-release) at the start of each focus group. Written consent forms will describe the purpose of the study; outline the information that will be collected; explain the risks, benefits, and voluntary nature of participation; and collect participants’ consent to participate in the focus groups. In accordance with similar approved data collections of program staff, this same information will be communicated to LEAP frontline staff and partners when obtaining their verbal consent at the start of each focus group.

Respondent information forms. At the start of each focus group, the study team will hand out a paper RIF to each participant. This brief, two-page questionnaire will gather demographic characteristics and other relevant information including education and work experience for all participants, arrest history for LEAP participants, program responsibilities for LEAP staff, and organizational affiliation and role in LEAP program for community AJC partners. The questionnaires will not ask for personally identifiable information, and group facilitators will place a linked participant identification number on each information form to keep the data separate from personal identifiers.

Collectively, these data will enable the team to answer the previously mentioned research questions for the implementation study. Qualitative analysis will result in an implementation study report about grantees’ early implementation experiences (2017). Additionally, the final report, anticipated for release by 2019, will summarize findings from all evaluation components, and the team will hold three annual briefings with DOL over the three-year evaluation.

A.2. Purposes and use of the information

The data collected through the activities summarized in this request will be used by DOL to comprehensively and systematically describe LEAP programs, including their organization, administration and management, services, delivery structures and processes; participants served; and common implementation successes and challenges. These data and the study team’s qualitative and descriptive analyses will provide DOL and other policymakers with important information to guide management decisions and future planning efforts regarding such grant programs. Without conducting the three data collection activities described in this request, a comprehensive evaluation of the LEAP grant programs cannot occur, and thus stakeholders would not be aware of the context in which jail-based and community-based AJCs operate, any operational challenges that AJCs or their partner organizations face, or staff and participant perceptions about the LEAP program.

Instruments used to collect data for the implementation study and their proposed uses are described in Table A.3 and are located at the end of this Justification Statement.

Table A.3. Data collection instruments included in the request

Data collection instrument	How study team will use the data
1. Semi-structured interview master protocol	The study team will conduct qualitative analysis of the interview data to comprehensively describe LEAP programs, including their organization, management, services, delivery structures, participants served, and common implementation successes and challenges.
2. Focus group protocols Frontline staff focus group protocol (round 1) Frontline staff focus group protocol (round 2) Partner staff focus group protocol Pre-release participant focus group protocol Post-release participant focus group protocol	The study team will use qualitative techniques to analyze the data gathered through focus groups to understand the perspectives of key stakeholders implementing and participating in LEAP programs.
3. Respondent information forms (RIFs) Frontline and partner staff RIF Pre-release participant RIF Post-release participant RIF	Descriptive analyses of the RIF data will be used to contextualize the information gathered through the focus groups.

Through qualitative analysis of interview and focus group data, along with descriptive analysis of the data collected from the RIFs, the study team will produce an implementation study report in spring 2017 about grantees’ early implementation experiences and a final report in 2018 that will summarize findings from all evaluation components. Further explanation of how the study team will analyze and report on the data collected is outlined in Section A.16, “Plans for tabulation and publication of results.”

A.3. Use of technology to reduce burden

The data collection efforts will use advanced technology to reduce burden on program participants and on staff at participating agencies whenever possible. Although the semi-structured interviews will not require the use of technology, the community-based focus groups will be digitally recorded to minimize the time spent with respondents, enabling study team members to take shorthand notes with the audio recording as back-up. Wherever possible, the study team will also digitally record the in-jail focus groups; however, the team anticipates that many jails will not permit this. In these instances, the study team members will need to rely solely on their notes. Again, because many jails prohibit visitors and researchers from using technology within the facility (for example, in the Evaluation of HealthLink and the RExO Evaluation, cell phones, tablets, and laptops were banned within the jail), the study team will collect written consent from LEAP participants and data from RIFs using self-administered pencil-and-paper forms. This approach was chosen because even if some jail settings can accommodate the use of technology, electronic collection of the consent and RIF data would not significantly reduce burden on participants (particularly given the high cost of programming such data collection instruments).Consequently, a member of the study team will collect and securely transport the paper copies of the consent forms and RIFs to Mathematica, where a trained research assistant programmer will perform data entry.

A.4. Efforts to avoid duplication

The site visit and focus group data being collected for the LEAP evaluation is not otherwise available from existing sources. Before visiting sites, the study team will gather pertinent data from AJC and LWIB websites and other publically available sources, and will confirm this information during pre-visit communication with grantee staff. The study team will only request new information during the site visit interviews. Similarly, respondents will not be asked for the same information more than once. Respondents for the semi-structured interviews during site visits will not participate in the focus groups during the site visits. Further, the information that will be requested from focus group participants in the RIFs will not be a part of the focus group discussion.

A.5. Methods to minimize burden on small entities

Employer partners might participate in the 60-minute partner focus group in the implementation study data collection effort. It is expected that, at each of the 20 grantee sites, at least one employer will participate, yielding a total of 20 employers across all grantees and site visits. It is possible that some of these employers might be from small businesses. To minimize burden on any small businesses that participate in the focus group, decisions about the timing and locations of the focus groups will consider employers’ schedules. Only employers who are willing to participate will be asked to attend the focus group and, as with all data collection activities, participants will be reminded that their participation is completely voluntary. Finally, partner focus groups will only take place during the first round of site visits, limiting the number of collections that could potentially impact small businesses entities.

A.6. Consequences of not collecting data

The federal investment of resources into LEAP programs requires the systematic collection of comprehensive implementation data. If the information is not collected, DOL and other program stakeholders will not have the evidence necessary to identify promising jail-based and community-based AJC practices, nor will they understand the implementation challenges and successes of LEAP programs. Furthermore, without these data, federal policymakers will not be able to describe the full range of LEAP institutional features, including the breadth of services offered, partnership relationships and arrangements, services to target populations, and sustainability plans, all of which are of particular interest to DOL.

A.7. Special circumstances

No special circumstances apply to this data collection. In all respects, the data will be collected in a manner consistent with federal guidelines. Respondents will not be required to report information more often than quarterly, to submit more than one original and two copies of any document, to retain records, or to submit proprietary trade secrets.

A.8. Federal Register announcement and consultation

1. Federal Register announcement

The 60-day notice to solicit public comments was published in the Federal Register on Wednesday, December 23, 2015 (80 FR 79936). No comments were received.

A.9. Payments or gifts

The evaluation’s data collection plan includes respondent payments for post-release program participants who take part in focus groups. Post-release focus group members are those who have been released from jail and have received LEAP services in the community. To encourage LEAP program participants to attend the focus group, the study team will offer them a $25 gift card. Although this is a nominal amount that is not large enough to be coercive for program participants, the payment serves two purposes: (1) to facilitate recruitment by increasing the likelihood that a LEAP participant will volunteer to participate in the focus group, and (2) to gain study participants’ cooperation in data collection activities, as the modest payment also indicates to participants that their time is valuable and their participation is appreciated. In a seminal meta-analysis, Singer et al. (1999) found that incentives in face-to-face and telephone surveys were effective at increasing response rates, with a $10 increase in incentive resulting in approximately a 3.3 percentage point increase in response rate, on average. They, as well as others, have found some evidence that incentives are also useful in boosting response rates among underrepresented demographic groups, such as low-income and minority individuals (Beebe et al. 2004; Berlin et al. 1992; de Heer and de Leeuw 2002; Singer and Kulka 2000). This is a significant consideration for LEAP, as it is well documented that minorities and otherwise disadvantaged populations are overrepresented in the criminal justice system.

A.10. Assurances of privacy

The study team will take a number of measures to safeguard the data that are part of this clearance request, including those collected through the site visits and focus groups. The team will apply for a Certificate of Confidentiality (CoC) from the National Institutes of Health, which is a widely-recognized resource for federal and non-governmental research to ensure protection of participants’ privacy against compulsory legal demands. Site visitors will assure respondents of the privacy of their responses to the extent permitted by law and will tell them about the CoC in written consent forms, in RIFs, and at the start of interviews and focus groups. Individual respondents will be informed that their participation is voluntary, that they can decline to answer any questions that they prefer not to answer, and that all responses will be used for research purposes only. Further, they will be assured that no individual respondents will be identified, unless required by law, and all data will be securely stored.

Site visit interviews and focus groups will be conducted in private areas, such as offices or conference rooms. For focus groups involving pre-release participants, jail security staff may need to be present during the focus group. At the start of each interview and focus group, the study team will read a statement to assure respondents of privacy and ask for their verbal consent to participate in the interview. This statement can be found at the top of the LEAP Site Visit Master Protocol and the focus group protocols. LEAP participants will also be asked to sign a physical consent form, which will be read aloud to ensure participants understand the form. Additionally, any data elements used for recruitment of focus group participants, such as name and telephone number, will be destroyed after completion of the focus groups. Interview transcripts and resultant reports from qualitative coding of the data will not identify respondents by name. RIFs completed at the beginning of the focus groups will be tagged with study-specific identifiers and will not contain any names or personal contact information. Pre-release participants will not be asked any questions relating to their experience in the jail outside of LEAP activities.

The contractor complies with DOL data security requirements by implementing security controls for processes that it routinely uses in projects that involve sensitive data. Further, the contractor secures personally identifiable information and other sensitive project information and strictly controls access on a need-to-know basis.

A.11. Justification for sensitive questions

The RIFs will collect background information from individuals who have consented to participate in this evaluation and will contain some questions that respondents might find sensitive, such as those about demographic characteristics (for example, race and ethnicity), or, for LEAP pre- and post-release participants, questions about prior involvement in delinquent activities and previous contacts with the justice system (number of arrests, number of convictions, types of convictions, and length of incarceration). In accordance with IRB requirements, no data collection activities will ask about participants’ experiences in jail regarding the conditions of confinement.

Information about the demographic characteristics of staff, partner, and participant focus group participants will provide additional information for interpreting the focus group data and for comparing data across LEAP sites. Similarly, the collection of information on prior involvement with the criminal justice system will be important to understand differences in participant focus group data by subgroups (for example, by severity or frequency of offense histories). Past evaluations have included similar questions without any evidence of significant harm. As described earlier, all sample members will be assured of the privacy of their responses before being asked to fill out the form and will be informed that they can skip any questions they do not wish to answer. All data will be held in the strictest confidence and reported in aggregate, summary format only, eliminating the possibility of individual identification.

A.12. Estimates of burden hours

1. Hours by activity

Table A.5 provides the total burden estimates for each of the three data collection activities for which this package requests clearance. All of the activities are anticipated to take place within a three-year period. To calculate the estimated cost burden for respondents, average hourly wages from the U.S. Bureau of Labor Statistics, National, State, Metropolitan, and Nonmetropolitan Area Occupational Employment and Wage Estimates for 2014 were multiplied by the number of hours per respondent type (see Table A.5). The following summarizes the annual burden estimates for each of the three data collection activities as well as the monetized burden estimates per respondent type and data collection effort:

Semi-structured interviews. While on site, the study team will conduct interviews with (1) LWIB representatives and grant managers, (2) the LEAP program manager, (3) jail administrators, (4) jail-based AJC managers, and (5) community-based AJC managers. Interviews will last, on average, 1.5 hours. The estimated total maximum reporting burden for the interviews is a total of 303 burden hours across the 20 sites (67 respondents per year * 3 years* 1.5 hours).

Focus groups. The study team expects to conduct an average of two 90-minute focus groups with frontline staff, one 60-minute focus group with partners, and three 60-minute focus groups with program participants in each of the 20 sites. The total maximum burden for the focus groups is estimated to be 906 hours ([282 frontline staff * 1.5 hours] + [141 LEAP partners * 1 hour] + [201 LEAP pre-release participants + 141 post-release participants = 342 participants * 1 hour] = 906).

Respondent information forms. The study team expects to collect a total of 765 RIFs across the 20 sites and four groups of focus group respondents, for a total of approximately 51 burden hours ([282 frontline staff + 141 LEAP partners) = 423 staff and partners * 3 minutes per staff/ partner RIF= 1,269 minutes] + [201 LEAP pre-release participants + 141 post-release participants= 342 participants * 5 minutes participant RIF= 1,710 minutes] = 765 RIFs and 148 minutes per site = 2,979 minutes, or 51 burden hours).

2. Total estimated burden hours and monetized costs

The total estimated maximum burden hours for the data collection included in this request for clearance is 1,260 hours (see Table A.5), which equals the sum of the estimated burden for interviews with the LEAP program administrators and managers and focus groups and the collection of RIFs from (1) frontline staff, (2) partners, and (3) LEAP program participants (303 + 906 + 51 =1,259 burden hours).

To calculate the estimated cost burden for respondents, average hourly wages from the U.S. Bureau of Labor Statistics, National, State, Metropolitan, and Nonmetropolitan Area Occupational Employment and Wage Estimates, for 2014 were multiplied by the number of hours per respondent type (see Table A.5). The following summarizes the monetized burden estimates per respondent type:

• Program administrator—semi-structured interviews. To estimate the total estimated cost associated with conducting semi-structured interviews with program administrators, the average wages for “social and community service managers” ($32.56) were multiplied by the time per interview (90 minutes) by the number of respondents (201), resulting in a total estimated cost of $9,816.

• LEAP frontline staff. To estimate the burden costs for LEAP frontline staff, average wages for “miscellaneous community and social service specialists” ($21.03) were multiplied by the time per focus group (90 minutes) by the number of respondents (282) to determine the focus group burden of $8,895. The RIF burden resulted in a zero cost, given the burden per individual was monetized to near zero.

• LEAP partners. To estimate the cost burden for LEAP partners, wages were weighted based on the predicted distribution of respondents, assuming that for every seven partners interviewed, there would be six representatives from social service agencies (termed here as service partners) and one representative from the business community (termed here as employer partners). First, for the service partners, average wages for “social and community service managers” ($32.56) were used, and the burden for employer partners was based on the average wages for “managers of companies and enterprises” ($60.33). These wages were then proportionally multiplied by the total number of burden hours (141 hours for focus groups + 7 hours of RIF burden= 148 hours) resulting in a total of $5,397 ([$32.56*148]*.86 + [$60.33*148]*.14).

• LEAP participants. It was assumed that there is no opportunity cost associated with inmates’ time while under custody, and thus the monetized burden for pre-release participants for both the focus groups and the RIFs is $0. To calculate the cost associated with collecting data through focus groups for post-release participants, the federal minimum wage of $7.25 was used. This wage was multiplied by 141 hours of focus group burden to yield a total monetized burden for post-release participants of $1,022 ($7.25 * 141 hours for post-release participants = $1,022). The RIF burden resulted in a zero cost, given the burden per individual was monetized to near zero.

This results in a total, rounded monetized burden cost of $25,131 ($9,816 for program administrators + $8,895 for frontline staff + $5,397 for LEAP partners + $1,022 for LEAP participants = $25,131).

Table A.5. Monetized burden hours for implementation study data collection

Respondents		Number of respondents	Annual number of respondents (rounded)a		Number of responses per respondent	Average burden time per response	Total burden hours	Annual burden hours (rounded)a	Time valueb		Monetized burden hours (rounded)	Annual monetized burden hours (rounded)
				Semi-structured interviewsc
Round 1
LEAP program administrators		100	33		1	90 minutes	151	50	$32.56		$4,884	$1628
Round 2
LEAP program administrators		101	34		1	90 minutes	152	51	$32.56		$4,932	$1644
				Focus groups and respondent information forms (RIFs)d
Round 1
LEAP frontline staff
RIF		141	47		1	3 minutes	7	2	$21.03		$0	$0
Focus group		141	47		1	90 minutes	211	71	$21.03		$4,448	$1483
LEAP pre-release participants
RIF		100	33		1	5 minutes	9	3	$0.00		$0	$0
Focus group		100	33		1	60 minutes	100	33	$0.00		$0	$0
Partner staffe
RIF		141	47		1	3 minutes	7	2	$32.56 / $60.33		$258	$86
Focus group		141	47		1	60 minutes	141	47	$32.56 / $60.33		$5,139	$1713
Round 2
LEAP frontline staff
RIF		141	47		1	3 minutes	7	2	$21.03		$0	$86
Focus group		141	47		1	90 minutes	211	71	$21.03		$4,448	$1,483
LEAP pre-release participants
RIF		101	34		1	5 minutes	9	3	$0.00		$0	$0
Focus group		101	34		1	60 minutes	101	34	$0.00		$0	$0
LEAP post-release participants
RIF		141	47		1	5 minutes	12	4	$0		$0	$0
Focus group		141	47		1	60 minutes	141	47	$7.25		$1,022	$341
				Subtotals
Semi-structured interviews		201	67		1	90 minutes	303	101	--		$9,816	$3,272
RIFs		765	255		1	3 minutes/ 5 minutes	51	17	--		$258	$86
Focus groups		765	255		1	90minutes/ 60 minutes	906	302	--		$15,057	$5,019
Total		1,731	577		1	--	1,260	420	--		$25,131	$8,377

^a The figures for the annual number of responses, annual burden hours, and annual monetized burden hours are based on three years of data collection.

^bThe average hourly wages were obtained from the U.S. Bureau of Labor Statistics, National, State, Metropolitan, and Nonmetropolitan Area Occupational Employment and Wage Estimates, 2014. Estimates of program administrators and managers wages are based on the average wages for “social and community service managers” ($32.56). Estimates of wages for frontline staff are based on the average wages for “miscellaneous community and social service specialists” ($21.03). Estimates of wages for partners are primarily based on the average wages for “miscellaneous community and social service specialists” ($32.56), but approximately one participant per partner focus group will be an employer partner. Employer partner wages are estimated based on the average wages for “managers of companies and enterprises” ($60.33). Monetized estimates for pre-release participants were assumed to be $0.00, as there are no opportunity costs associated with pre-release participants’ burden; the federal minimum wage of $7.25 was used for post-release participants.

^c Assumes interviews with five program and jail administrators per program per round.

^d Assumes that round 1 visits will include the following per program: one focus group with up to seven frontline staff from the facility based and community-based AJCs, one focus group with up to seven participants from partner organizations or entities, and one focus group per program with up to five pre-release participants. Assumes that round 2 of site visits includes the following for each program: one focus group with up to seven LEAP frontline staff, one focus group per program with up to five pre-release participants, and one focus group per program with up to seven post-release participants.

^e The monetized burden for focus groups with partners assumes that six partners will participate at an estimated average hourly wage of $32.56 and one employer partner will attend at an estimated average hourly rate of $60.33.

A.13. Estimates of cost burden to respondents

There will be no direct costs to respondents for the LEAP implementation study. The proposed information collection plan will not require respondents to purchase equipment or services or to establish new data retrieval mechanisms.

A.14. Annualized costs to the federal government

DOL, as with most other federal agencies, contracts with firms that have proven experience with program evaluation to conduct all evaluation activities. Federal employees rely on contract staff to perform the majority of the work described in this package and have no direct role in conducting site discussions or focus groups, developing study protocols or designs, collecting data directly using these instruments, or analyzing or producing reports using these data. The role of federal staff is almost entirely restricted to managing these projects. The costs that contractors incur to perform these activities are essentially direct federal contract costs associated with conducting site visits, discussions, and focus groups.

This estimate of federal costs is a combination of (1) direct contract costs for planning and conducting this research and evaluation project, including any necessary information collection and (2) salary associated with federal oversight and project management.

1. Estimates of direct contract costs

Three categories of direct costs to the federal government are associated with conducting this project. These costs are routine and typical for studies such as this. The first category is design and planning, including external review of the design by a technical working group of outside subject matter experts. This work is estimated to cost $202,249. The second category is data collection, which will occur through the project period, and is estimated to cost $655,029. The final category is for analysis and reporting. This category includes synthesizing the findings into conclusions and producing deliverables, such as reports. This work is estimated to cost $435,176. The total estimated direct costs are:

$202,249 (design) + $655,029 (data collection) + $435,176 (reporting) = $1,292,454

Although this project is expected to have a duration of three years, an accurate estimate of the annualized direct contract cost will vary considerably from year to year because each of the three categories of costs is focused on a specific period in the project life cycle. The design and planning costs are obviously front-loaded, the data collection costs will be incurred throughout most of the project, and the analysis and reporting costs will occur close to the end of the project. As a very basic estimate, the total estimated direct costs can be divided by the three years of the study to produce an estimate of the average annualized cost:

$1,292,454 ÷ 3 years of study = $430,818 per year in estimated direct contract costs.

2. Estimates of federal oversight and project management costs

CEO staff have regular duties and responsibilities for initiating, overseeing, and administering contracts to perform research and evaluation on behalf of agency programs and offices. To carry out the implementation study, federal staff will need to perform certain functions that, although clearly part of their normal duties, are directly attributable to this specific research and evaluation project. For purposes of calculating federal salary costs, DOL assumes:

1. A Senior Evaluation Specialist GS-14, step 4, based in the CEO in Washington, D.C., who would earn $56.57 per hour to perform this work, and would spend approximately 200 hours per year on this project. To account for fringe benefits and other overhead costs, the agency has applied a multiplication factor of 1.6 (200 hours X $56.57 x 1.6=$18,102). Total estimated federal costs are $54,306.

Wages are drawn from the most current available estimates of wages and salaries available at https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/15Tables/html/DCB_h.aspx. A summary of estimated federal costs for the LEAP evaluation is presented in Table A.6.

Table. A.6. Summary table of estimated federal costs for the LEAP evaluation

Estimates of direct contract costs		Total costs	Annualized Costs (rounded)
Design and planning for the study		$202,249	$67,416
Data collection		$655,029	$218,343
Analysis and reporting		$430,818	$143,606
Subtotal for direct contract costs		$1,292,454	$430,818
Estimates of direct federal staff costs
1 GS-14		$54,306	$18,102
Sum of costs		$1,345,760	$448,920

Note: Federal staff costs are based on Salary Table 2015-DCB (Step 2, incorporating the 1.5 percent general schedule increase and a locality payment of 24.22 percent for the locality pay area of Washington-Baltimore-Northern Virginia, DC-VA-WV-PA); DOL grade ranges are as of October 2015 (https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/15Tables/html/DCB_h.aspx).

A.15. Reasons for program changes or adjustments

This is a new information collection.

A.16. Plans for tabulation and publication of results

The analysis of data collected from the site visits, administrator interviews, focus groups, service provision observations, and case file reviews will not require statistical methodology or estimation. The qualitative data collected will be analyzed using qualitative coding software such as NVivo or ATLAS.ti. Because the implementation study is examining program implementation, study findings will apply only to the LEAP grantees and will not be more broadly generalizable. Although the data collected from focus group participants through the RIFs will be analyzed using descriptive methods (for example, analysis of frequencies, means, and ranges on various data elements), this information will only be used to contextualize findings from the qualitative data, not to make inferences about the broader LEAP population.

In spring 2017, the team will produce a report about grantees’ early implementation experiences. A final report in 2018 will summarize findings from all evaluation components, including the findings from the implementation study. Additionally, the study team plans to hold three annual briefings with DOL on topics of interest to the overall evaluation.

A.17. Approval not to display the expiration date for OMB approval

The OMB approval number and expiration date will be displayed or cited on all forms completed as part of the data collection.

A.18. Explanation of exceptions

No exceptions are necessary for this information collection.