Supporting Statement Part A

OMB Control Number: 1290-0043

Cost Benefit Analysis Study of Apprenticeship Evidence-Building Portfolio

OMB Expiration Date: 10/31/25

This is a generic Information Collection to add to the umbrella Information Collection Request (ICR) for OMB Control Number 1290 – 0043.

Cost Benefit Analysis Study of Apprenticeship Evidence-Building Portfolio

1290-0043

Supporting Statement

Part A

January 2024

Submitted By:

Chief Evaluation Office
U.S. Department of Labor
200 Constitution Ave NW
Room S-4307
Washington, DC 20210

Project Officer:

Janet Javar

A1. Necessity for the Data Collection

The Chief Evaluation Office (CEO) at the U.S. Department of Labor (DOL) seeks approval for conducting interviews with employers partnering with Scaling Apprenticeship and Closing the Skills Gap grantees by phone or in a virtual interview for a Cost-Benefit Analysis Study for the Apprenticeship Evidence-Building Portfolio (AEP) project. The data collected as part of this Cost-Benefit Analysis Study of the AEP will be used to understand whether the net benefits of apprenticeship, including both labor and market and non-labor market benefits and all costs, are positive for society. Past cost-benefit analyses have focused on the perspective of employers only and have not focused on a particular industry. This study will produce separate estimates focused on healthcare and information technology, and it will estimate full social net benefits rather than just net benefits for employers. These employer interviews are new data collection to support the Cost-Benefit Analysis Study.

This proposed information collection meets the following goals of DOL’s generic clearance for formative data collections (1290-0043) and will:

• Inform the development of DOL research.

• Maintain a research agenda that is rigorous and relevant.

• Ensure that research products are as current as possible.

Study Background

The Cost-Benefit Analysis Study of the Scaling Apprenticeship and Closing the Skills Gap employer partners is a new study in a broader portfolio of apprenticeship research. This study background section reviews other studies in the portfolio and the broader purpose of the portfolio.

The Chief Evaluation Office (CEO) of the U.S. Department of Labor (DOL) intends to design and conduct evaluations of DOL-funded apprenticeship initiatives through the Apprenticeship Evidence-Building Portfolio. The portfolio of initiatives includes the Scaling Apprenticeship Through Sector-Based Strategies grants, Closing the Skills Gap grants, Youth Apprenticeship Readiness grants, and other DOL investments. The goal of this study is to build evidence on apprenticeship models, practices, and partnership strategies in high-growth occupations and industries.

The overall study is comprised of several components: (1) an implementation study of the Scaling Apprenticeship and Closing the Skills Gap grants to develop typologies of apprenticeship models and practices, identify promising strategies across the portfolio, and to better understand the implementation of models to help interpret impact evaluation findings; (2) an impact study of the Scaling Apprenticeship and Closing the Skills Gap grants to estimate the effect of program participation on earnings and employment; (3) a study of registered apprenticeship state systems and partnerships to assess their capacity to develop, design, modify, implement, replicate, sustain, expand/scale up, and evaluate apprenticeship strategies and models; (4) an implementation evaluation of the Youth Apprenticeship Readiness grant program to understand service delivery design and implementation, challenges, and promising practices. An additional Cost-Benefit Analysis Study of the Scaling Apprenticeship and Closing the Skills Gap grants has been added to the portfolio in 2023 to understand whether the net benefit of apprenticeship, including both labor market and non-labor market benefits and all costs, is positive for society. The remainder of Part A pertains specifically to the Cost-Benefit Analysis Study.

Legal or Administrative Requirements that Necessitate the Collection

There are no legal or administrative requirements that necessitate the collection. CEO is undertaking the collection at the discretion of the agency.

A2. Purpose of Survey and Data Collection Procedures

Overview of Purpose and Approach

DOL will use the data collected through the employer interviews summarized in this request to gain an understanding of the costs and benefits of their apprenticeship initiatives. The interview data and the study team’s analysis will provide DOL with a better understanding of the costs and benefits of employers’ engagement in apprenticeship initiatives. Employers will be those identified in consultation with grantees of the Department’s Scaling Apprenticeship Through Sector-Based Strategies grants and Closing the Skills Gap grants.

Research Questions

The Cost-Benefit Analysis Study of the AEP evaluation will address five research questions:

1. What monetary and non-monetary benefits do interviewed employers experience from participating in an apprenticeship program? How do interviewed employers value non-monetary benefits of participating?

2. What monetary and non-monetary costs do interviewed employers experience from participating in an apprenticeship program? How do interviewed employers value non-monetary costs of participating?

3. What are interviewed employers’ net benefits of participating in an apprenticeship program?

4. When interviewed employers’ net benefits are combined with the costs and benefits of other stakeholders, what are the social net benefits of an apprenticeship program?

5. How do these costs, benefits, and net benefits vary across interviewed employers and in different industries?

Study Design

To address the research questions for the Cost-Benefit Analysis Study, the study team will conduct employer interview by phone or videoconference with employers partnering with Scaling Apprenticeship Through Sector Based Strategies grantees and Closing the Skills Gap grantees starting in the winter of 2023. We will identify the employers for interview through a review of grant documents listing employer partners. We will also conduct email outreach and clarifying calls to the grantees to collect their feedback on the employers identified for interviews and to request their recommendations for additional employer partners that should be considered for an interview. In this outreach, the study team will also ask the Scaling Apprenticeship Through Sector Based Strategies grantees and Closing the Skills Gap grantees to provide contact information for any employer partners that are identified for the interviews.

The study team will conduct an hour phone or video interview with each participating employer, depending on the preferences of the employer. For the 60 employer interviews, we estimate a total of 60 respondents. This assumes that half of the interviews will include one respondent (30*1) from the Scaling Apprenticeship Through Sector Based Strategies grant program who are knowledgeable about the employers’ apprentices experiences with apprenticeship and the other half will include one respondent (30*1) from the Closing the Skills Gap grant program.

The most important weakness of the study design for the Cost-Benefit Analysis Study is that many categories of costs and benefits are difficult to measure, even for the employers experiencing them. For example, while the productivity attributable to an individual apprenticeship can be estimated by an employer familiar with the apprentice, there is no precise way to measure this in the same way that the employer can provide a price measurement of the apprentice’s wages or tuition. To overcome this limitation, our interview guide provides clear, simple directions for estimating apprentice productivity (and other difficult to measure categories). Interviewers will also participate in training to standardize their understanding of the questions and approach to the interview.

Universe of Data Collection Efforts

Respondents will include employers that are partnering or otherwise working with Scaling Apprenticeship Through Sector Based Strategies and Closing the Skills Gap grantees. The study team anticipates collecting from grantees the names and relevant background information for approximately 120 employers. From that list, the study team will select 80 employers to interview, with the aim of completing 60 total employer interviews (75 percent completion rate). When selecting which employers to interview, the study team will consider the following employer characteristics to ensure we capture diverse perspectives:

• Employer size and role. The study team aims to interview different sized employers representing a variety of roles and number of apprenticeship occupations.

• Industry representation. The study team will interview employers representing the healthcare and information technology industries, with a goal of 30 interviews in healthcare and 30 interviews in information technology.

• Grantee partnership. The study team intends to include employers working with a range of grantees, to ensure that different grantee models and strategies are represented.

As needed due to employer nonresponse or refusal to participate, we will identify additional employers to interview from the remaining identified employers. To the extent possible, we will select additional employers to maintain the diversity of the initial sample.

A3. Improved Information Technology to Reduce Burden

The study team will conduct interview by telephone or video conference, whichever mode is preferred by the employer. In advance of the interview, participating employers will be provided both a dial-in phone number to access the meeting by phone and a link to access the interview through a video conferencing platform (Teams or Zoom).

A4. Efforts to Identify Duplication

This project will not involve collecting information that is available from other sources. The study team will ensure that employers are only interviewed once for the phase of the AEP project covered under this generic clearance. Select employers may have been involved in past data collection efforts in fall of 2022 and the spring of 2023 carried out during AEP site visits, but the site visit interviews focused primarily on employer experiences with the implementation of the AEP grants and did not include discussions of costs and benefits of apprenticeship; therefore, the information collected will not be duplicative. For any employers that participate in both an interview in this phase and the earlier site visit, the study team will review the notes from the initial interview in advance and appropriately tailor the cost-benefit study interview protocol to avoid duplication.

A5. Involvement of Small Organizations

Employer partners will participate in phone or virtual interviews. Some of these employers may be small businesses. To minimize burden on any small businesses that participate, we will request only information required for the intended use of the data collected. We will also be flexible to employers’ schedules when scheduling the discussions. As with all data collection activities, we will remind participants that their participation is completely voluntary. 

A6. Consequences of Less Frequent Data Collection

This is a one-time data collection with employers for the phase of the AEP study covered under this generic clearance.

A7. Special Circumstances

There are no special circumstances for the proposed data collection efforts.

A8. Federal Register Notice and Consultation

No public comments are requested for this information collection.

Consultation with Experts Outside of the Study

Outside of DOL and the study team, we have consulted with the following experts:

• Robert Lerman, Institute Fellow, Urban Institute

• John Marotta, Senior Policy Associate, Urban Institute

A9. Incentives for Respondents

No incentives for respondents are proposed for this information collection.

A10. Privacy of Respondents

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law.

As specified in the contract, the Contractor will comply with all Federal and Departmental regulations for private information. The Contractor has developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ personally identifiable information and has been approved by our Institutional Review Board. The Contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements. All researchers involved in this data collection have signed DOL’s contractor confidentiality/non-disclosure agreement.

A11. Sensitive Questions

There are no sensitive questions in this data collection.

A12. Estimation of Information Collection Burden

Total Annual Cost

Table A.1. includes assumptions about the annual number of respondents expected; the number of responses per respondent; the average hours of burden per response; the annual burden hours estimated; the time value assumed for respondents; and the total annualized monetary burden hours for the employer interviews. The data collection activity this request covers takes place in one year, so the costs listed are the annual cost.

Table A.1. Total Burden Requested Under this Information Collection

Instrument	Total Number of Respondents	Annual Number of Respondents1	Number of Responses Per Respondent	Average Burden Hours Per Response	Annual Burden Hours	Average Hourly Wage	Total Annual Cost1
Employer interview	60	60	1	1 hours	60	70	$4,204
Estimated Annual Burden Total					60	--	$4,204

Note: Numbers are rounded to the nearest whole number for the columns “Average hourly wage” and “Total annual cost” columns. 

¹Hourly cost is the 2022 mean wage for Human Resources Managers ($70.07) from May 2022 National Occupational Employment and Wage Estimates (bls.gov).

We estimate that the study team will successfully interview 60 respondents (30 employer staff associated with Scaling Apprenticeship and 30 employer staff associated with Closing the Skills Gap grantees).

We estimate interviews will take about one hour to complete. The annualized burden is approximately 60 hours. Using the mean wage for Human Resources Managers of $70.07, we estimate the annual cost to be $4,204.

A13. Cost Burden to Respondents or Record Keepers

There are no additional costs to respondents.

A14. Estimate of Cost to the Federal Government

The total cost to the Federal government for the data collection activities under this current request will be $196, 364. Costs result from the following two categories:

1. The annualized cost to the federal government for the evaluation contractor, the Urban Institute and its partners Mathematica and Capital Research Corporation (Contract Number: DOL #1605DC-19-F-00312), to carry out this task of the evaluation is $190,000.

2. The annualized cost for federal technical staff to oversee the evaluation is $6,364. This is calculated by the following: an annual level of effort of 200 hours for one Washington, DC-based Federal GS-14 step 1 employee earning $63.64 per hour. (See Office of Personnel Management 2023 Hourly Salary Table at https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2023/DCB.pdf ). This study will take one-half of a year, therefore, the annualized cost is 100 hours X $63.64= $6,364.

The total annualized cost to the federal government is $196,364 ($190,000 + $6,364).

A15. Change in Burden

This is for an individual information collection under the umbrella formative generic clearance for DOL research (1290-0043).

A16. Plan and Time Schedule for Information Collection, Tabulation and Publication

The interviews with employers will begin in early 2024, followed by analysis of the data.

Analysis Plan

Analysis of employer interviews will involve standardized recording of reported costs and benefits across employers to produce an estimate of total costs, benefits, and net benefits. The study team will begin by writing up detailed notes from the interviews using a standardized summary template that is aligned with the research questions and topics of interest. To code the qualitative data for key themes and topics, a coding scheme that aligns with the summary template will be developed. These data will be used to explore key research questions for the Cost-Benefit Analysis Study and better understand themes that emerged during interviews.

Publications

Findings from the Formative Study of AEP will inform DOL about employer costs and benefits associated with apprenticeships and apprentices. These data will be combined with information from the AEP impact study on net benefits of apprenticeship for apprentices (a separate data collection effort), and information on costs to the government of operating an apprenticeship program to generate a social net benefit estimate. The findings from this data collection will be used to understand the costs and benefits of employers partnered with the Scaling Apprenticeship and Closing the Skills Gap grants, but are not meant to provide policy recommendations and are not meant to be generalizable. For this reason, the findings will likely be provided in public-facing briefs, though such publication is not the primary purpose of the data collection.

A17. Reasons Not to Display OMB Expiration Date

All instruments will display the expiration date for OMB approval.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

1 Annualized over the three-year period of the umbrella generic. Estimates are rounded to the nearest whole number.

8