Cost Benefit Analysis Study of Apprenticeship Evidence-Building Portfolio

OMB Control No. 1290-0043

OMB Expiration Date: 10/31/2024

Employer Interview Protocol for the Apprenticeship Evidence-Building Portfolio Evaluation

INTRODUCTION 

I am/we are researchers with The Urban Institute/Mathematica/Capital Research Corporation, private research organizations based in Washington, DC which conduct policy-related research on a variety of social welfare and economic issues. 

 

This project is being conducted under contract to the U.S. Department of Labor. Our discussion here today is part of the Apprenticeship Evidence-Building Portfolio: Scaling Apprenticeship Through Sector-Based Strategies Grants and Closing the Skills Gap Grants evaluation project, a national study sponsored by the U.S. Department of Labor. The study will learn how apprenticeship programs can help improve the skills and employment outcomes of American workers. A major aim of the study is to learn more about the different models of apprenticeship being implemented. In this conversation we will focus on the costs and benefits of apprenticeship that you experience as an employer.

Introductory Statement:   I/we know that you are busy, and we will be as focused as possible and will only ask questions that are relevant to your experience.  Your participation in this interview is voluntary and you may choose not to answer some or any of our questions. We will do everything we can to secure the privacy of the information you provide but ask that you consider using headphones or finding a private space to talk so that other people cannot hear or see this conversation.

We believe the risks of participating in this study are minimal. We believe there are no direct benefits to you, but we hope that the findings from this study will benefit the project by providing research insights on how apprenticeship programs can benefit workers.

My colleague and I will be taking notes in order to document what we hear during our discussion, and we may record this discussion. We do not share these notes with anyone outside of our research team, including Department of Labor, and we will destroy these notes after the end of our project. When we compile our reports, the names of individual respondents will not be included. If we choose to quote you, we will only identify your title or your role as an employer in a particular sector. you will not be quoted directly by name in any of our reports. While it is possible that you might be identified by your title, we will do our best to minimize the chance of that occurring. 

Finally, to help us accurately capture the information you share, we would like to record this interview. The recording is just a back-up for our notes and will be kept within our small research team. The interview recording will be deleted once we have developed a full set of notes from the interview.

 

Do you have any questions for me?  

[If we decide to record the interview] Are you okay with us recording the interview to improve the accuracy of our notes? Do I have your permission to begin the interview? [BEGIN INTERVIEW IF CONSENT GRANTED]

OMB Burden Statement: Public reporting burden for this interview is estimated to average one hour per response. The burden estimate includes the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and submitting the interview. This collection of information is voluntary. You are not required to respond to this collection of information unless it displays a valid OMB control number. Please send comments regarding the burden estimate or any other aspect of this collection of information to the Chief Evaluation Office, U.S. Department of Labor, 200 Constitution Ave NW, Room S-4307, Washington, DC 20210 or [email protected] and reference OMB control number 1290 – 0043. NOTE: Please do not send your completed interview to this address. 

1. Employer Background

I-1. Which of the following best describes your firm’s industry? (choose one)

• Accommodation and Food Services

• Administrative and Support and Waste Management and Remediation Services

• Agriculture, Forestry, Fishing, and Hunting

• Arts, Entertainment, and Recreation

• Construction

• Educational Services

• Finance and Insurance

• Health Care and Social Assistance

• Information

• Management of Companies and Enterprises

• Manufacturing

• Mining, Quarrying, and Oil and Gas Extraction

• Professional, Scientific, and Technical Services

• Public Administration

• Other Services (except Public Administration)

• Real Estate Rental and Leasing

• Retail Trade

• Transportation and Warehousing

• Utilities

• Wholesale Trade

• Other (please specify): __________________________

I-2. Approximately how many people are employed at the location where apprentices work? (in Full-Time Equivalents (FTEs)). If your company has apprentices working at multiple sites, please tell us how many are employed at the location where the apprentice selected for Section IV works.

• Less than 50 FTEs

• 50-99 FTEs

• 100-500 FTEs

• More than 500 FTEs

2. Recruitment

II-1. We are interested in learning more about the types of apprenticeship programs employers sponsor or hire from. These can be registered or unregistered programs. Please identify the largest healthcare or IT occupation for which your company sponsors apprenticeships or hires apprentices. Please enter the number of apprentices that have been hired during the last five years.

Occupation Number of Apprentices

1. (i) ________________ (ii) _____________ apprentices

II-2. Which of the following best characterizes the type of apprenticeship program your firm uses in [OCC – the Occupation identified in II-1]?

• Registered: Independent program, non joint

(single employer, not covered by collective bargaining agreement).

• Registered: Independent program, joint

(single employer, covered by a collective bargaining agreement.)

• Registered: Group program, non joint

(multi-employer association that is not covered by a collective bargaining agreement.)

• Registered: Group program, joint

(multi-employer association, covered by a collective bargaining agreement.)

• Unregistered apprenticeship program

DIRECTIONS: The next questions ask you to imagine how you would fill the position in your firm if you did not hire (an) apprentice(s) in [OCC].

II-3. If you did not have apprentices in [OCC], would you be more likely to fill this position by hiring a qualified individual(s) into [OCC] from the outside or by training or promoting one or more of your current employees into a qualified status in [OCC]? If you might use both, which would you expect to use most?

_____________ hiring from outside

(INTERVIEWER GO TO II-4)

_____________ training or promoting

(INTERVIEWER GO TO II-7)

II-4. What is your estimate of how long it would take to fill this position by hiring an individual into [OCC] from outside your company?

______________months

II-5. Relative to putting an apprentice into an apprenticeship in [OCC], what is your estimate of the costs of filling the position in [OCC] with an individual from outside your company?

• More than $3000 lower than an apprentice

• $2000 – $2999 lower

• $1000 – $1999 lower

• $500 – $999 lower

• Less than $50 lower

• About the same

• Less than $500 higher

• $500 – $999 higher

• $1000 – $1999 higher

• $2000 – $2999 higher

• More than $3000 higher than an apprentice

II-6. If you filled the position in [OCC] with another individual instead of an apprentice, what would be the gross hourly wage before taxes to the employer or worker of that worker when hired?

$ _ _ . _ _ per hour

(INTERVIEWER GO TO SECTION III)

II-7. What is your estimate of how long it would take to promote or train a current employee into a qualified position in [OCC]?

______________months

II-8. What is your estimate of the non-wage costs to your company associated with promoting or training a current employee into a position in [OCC] without putting them through the apprenticeship program? Non-wage costs may include additional training to move the employee into the new position or the cost of reviewing and interviewing internal applicants for the position.

• $0 - $499

• $500 - $999

• $1,000 - $2,999

• $3,000 – $4,999

• $5,000 - $6,999

• $7,000 - $9,999

• $10,000+

II-9. If you filled the position of a qualified individual in [OCC] with a current employee who you have promoted or trained, what would that employee’s hourly wage before taxes become when promoted?

$ _ _ . _ _ per hour

3. Apprenticeship Characteristics

DIRECTIONS: Now we want to ask about your actual experiences with apprenticeships.

III-1. a) How long does it take for a typical apprentice in [OCC] to successfully complete the apprenticeship at your company? If progression and completion are determined by mastery of competencies, please provide the typical length of time it takes to master all competencies.

______________ MONTHS

b) How many wage increases (or pay progressions) does an apprentice in [OCC] take to complete an apprenticeship? Please assume the completion wage is the final wage increase.

_______________ WAGE INCREASES

III-2. We are interested in the retention rates of apprentices. For the typical cohort of apprentices (that is, apprentices that started at the same time), please identify the number of apprentices persisting to each wage increase. The number of apprentices who persist to the final wage increase is the number of apprentices who complete the program:

[Interviewer fill in number of steps based on answer to III-1b.]

	Number of apprentices persisting to this wage increase
Starting size of typical cohort	_________ apprentices
Cohort persisting to first wage increase	_________ apprentices
Cohort persisting to second wage increase	_________ apprentices
Cohort persisting to third wage increase	_________ apprentices
Cohort persisting to fourth wage increase	_________ apprentices
Cohort persisting to fifth wage increase	_________ apprentices
Cohort persisting to sixth wage increase	_________ apprentices

4. Experience with a Specific Apprentice

DIRECTIONS: In order to better understand the employer’s costs and benefits from participating in apprenticeships, the study team would like to know more about apprentices’ wages and productivity at each wage step. Please think of a specific apprentice [APPRENTICE] whose performance you considered typical who has completed the program at least a year ago.

NOTE: This section and the following section may be most easily answered by [APPRENTICE]’s mentor or supervisor.

IV-1. We would like to know about [APPRENTICE]’s wages and productive value to the firm at each wage increase. Please provide (1) [APPRENTICE]’s hourly wage for each wage increase, (2) the number of hours [APPRENTICE] worked at that increase, (3) the value to your company of [APPRENTICE]’s productivity compared to a fully qualified worker which we are defining as a first year journeyworker or someone of equivalent skill in [OCC] (measured as a percentage of the value of a fully qualified worker’s productivity), and (4) the number of hours for which [APPRENTICE] was paid while in related technical instruction. If your wage progression is determined by mastery of competencies please provide the average or typical number of hours that an apprentice will work at this increase of the wage progression.

For example, if [APPRENTICE]’s job performance per week is valued at about half of what you would expect from a fully qualified worker in [OCC], then [APPRENTICE]’s relative performance would be 50%; if [APPRENTICE]’s job performance is valued at approximately the same as a fully qualified worker, then [APPRENTICE]’s productivity compared to a fully qualified worker would be 100%.

[Number of increases to be filled in based on answer to III-1-b.]

	Hourly wage (base, not including benefits, overhead, or indirect costs)	Total hours worked at this wage rate (not in related technical instruction)	% Productive compared to the value produced by a fully qualified worker	Total hours paid while in related technical instruction
First wage increase	$__ __.__ __	_____ hours	_____ percent	_____ hours
Second wage increase	$__ __.__ __	_____ hours	_____ percent	_____ hours
Third wage increase	$__ __.__ __	_____ hours	_____ percent	_____ hours
Fourth wage increase	$__ __.__ __	_____ hours	_____ percent	_____ hours
Fifth wage increase	$__ __.__ __	_____ hours	_____ percent	_____ hours
Sixth wage increase	$__ __.__ __	_____ hours	_____ percent	_____ hours

IV-2. What is the cost of [APPRENTICE]’s non-wage benefits, such as insurance premiums for health care benefits and workers’ compensation and employer’s share of payroll taxes, not including payments for related training instruction or supplies, for one full year: (You may answer with a percent of wages or dollar amount)

__________ dollars [__________ percent]

IV-3. How much did you pay for tuition, fees, books, and equipment for [APPRENTICE]’s related technical instruction? Do not include the tuition and fees or expenses paid by the apprentices themselves or by a partner organization that assists in hiring your apprentices.

[Number of increases to be filled in based on answer to III-1-b.]

	Tuition, fees, etc.
Through first wage increase	$__ __.__ __
Through second wage increase	$__ __.__ __
Through third wage increase	$__ __.__ __
Through fourth wage increase	$__ __.__ __
Through fifth wage increase	$__ __.__ __
Through sixth wage increase	$__ __.__ __

IV-4. In a typical week, how many total hours did [APPRENTICE]’s mentor(s) (denoted MENTORS) spend in training or supervising [APPRENTICE]? During that supervision time, what was/were MENTORS’ productivity as a percent of their normal productivity (if more than one mentor, approximate an average)? For example, if MENTOR was providing hands-on training to [APPRENTICE] and they were jointly producing output, then MENTOR’S relative productivity might be only 50% of their usual productivity. What was MENTORS’ [base, non-loaded] hourly wage (if more than one mentor, average weekly wage)?

[Number of increases to be filled in based on answer to III-1-b.]

	Total hours spent training or supervising APPRENTICE across all MENTORS (typical week)	MENTORS’ relative productivity while supervising compared to normal	MENTORS’ hourly wage (base, not including benefits, overhead, or indirect costs)
First wage increase	_____ hours	_____ percent	$__ __.__ __
Second wage increase	_____ hours	_____ percent	$__ __.__ __
Third wage increase	_____ hours	_____ percent	$__ __.__ __
Fourth wage increase	_____ hours	_____ percent	$__ __.__ __
Fifth wage increase	_____ hours	_____ percent	$__ __.__ __
Sixth wage increase	_____ hours	_____ percent	$__ __.__ __

IV-5. How much financial assistance or incentives did you receive from the state or local government to cover costs associated with APPRENTICE? Financial assistance may include subsidies for related technical instruction or mentoring.

­­­$__________________

5. Apprenticeship Completer’s Productivity

You indicated that [APPRENTICE] has completed successfully their apprenticeship.

V-1. Is [APPRENTICE] still at your company?

______ Yes

______ No longer at company (SKIP to VI-1)

V-2. How many months (including the apprenticeship) has [APPRENTICE] worked for your company since the beginning of their apprenticeship?

______ Months employed since the beginning of apprenticeship

V-3. a. What was [APPRENTICE]’s hourly before-tax wage rate when they completed their apprenticeship? $ _ _ . _ _/hour

b. Is that their current hourly before-tax wage rate?

______ Yes (SKIP to V-4)

______ No (If No, ask V-4c.)

c. What is their current rate? $__ . __/hour

V-4. How many hours per week does [APPRENTICE] usually work? ______ hours

DIRECTIONS: We are interested in how wages compare between apprentices and non-apprentice employees with the same tenure as APPRENTICE after the apprenticeship is over.

V-5. Have you hired or do you have a non-apprentice [OCC] employee with a similar tenure as [APPRENTICE]?

______ Yes

______ No (SKIP to V-9)

V-6. What is the non-apprentice [OCC] employee’s before-tax hourly wage? $_ _ ._ _/hour

V-7. How many hours per week does the non-apprentice [OCC] usually work? ______ hours

V-8. What is the non-apprentice [OCC] employee’s productivity relative to [APPRENTICE]’s current productivity? (If they are equally productive, then the relative productivity would be 100%. If [APPRENTICE] seems to be 20% more productive, then the relative productivity would be 80%.)

_____________% (May be more or less than 100%)

(GO TO SECTION VI)

V-9. DIRECTIONS: Back in Section II, we asked you to imagine how you would fill a position in [OCC] with a qualified worker. Now we would like you to imagine that you have a non-apprentice worker in [OCC] with similar tenure as APPRENTICE after the apprenticeship is over.

1. What would be the non-apprentice [OCC] employee’s before-tax hourly wage?

$_ _ ._ _/hour

2. How many hours per week would the non-apprentice [OCC] employee’s usually work?

­­­­______ hours

3. What is the non-apprentice [OCC] employee’s productivity relative to [APPRENTICE]’s current productivity? (If they are equally productive, then the relative productivity would be 100%. If [APPRENTICE] seems to be 20% more productive, then the relative productivity would be 80%.)

_____________% (May be more or less than 100%)

6. Other Costs and Benefits of the Apprenticeship Program

VI-1. If you registered your program within the last five years, what would you estimate was the total cost of registering your apprenticeship program in [OCC], including the cost of staff time spent in planning, setting up, and getting approval for the program and one-time payments or fees paid to partner organizations or contractors helping you set up the program? Do not include any costs that were covered or reimbursed by a partner organization that assisted you in developing the program:

__________ dollars

VI-2. Are you paying for any additional accommodations or supports for your apprentices? Probe: Accommodations or supports could include things like special technology for apprentices with disabilities or transportation assistance that is separate from your standard compensation package.

For each accommodation or support, please provide the type of accommodation or support, the number of apprentices who benefit from the accommodation or support in a typical year, and the amount of money you spend in total on the accommodation or support in a typical year.

	Type of accommodation or support (text description)	Number of apprentices who benefit from the accommodation or support in a typical year		Amount of money you spend in total on the accommodation or support in a typical year
First accommodation or support	_____	_____ apprentices	$__ __.__ __
Second accommodation or support	_____	_____ apprentices	$__ __.__ __
Third accommodation or support	_____	_____ apprentices	$__ __.__ __
Fourth accommodation or support	_____	_____ apprentices	$__ __.__ __
Fifth accommodation or support	_____	_____ apprentices	$__ __.__ __

VI-3. Has your company experienced any of the following additional benefits from hiring apprentices in [OCC] that have likely improved your company’s performance in the short-run or long-run?

Benefit	Has occurred, and likely improved firm’s profitability
Reduced turnover	Y N
Improved pipeline of skilled employees	Y N
Development of future managers	Y N
Improved productivity of co-workers	Y N
Improved firm culture	Y N
Product or process innovation	Y N
Employee engagement and loyalty	Y N
Reduced use of overtime or temporary workers	Y N
Reduced downtime	Y N
More on-time delivery	Y N

VI-4. On a scale from one to five, where one is not important and five is very important, how important were these additional benefits of the apprenticeship program, compared to the benefit of increasing the productivity of [APPRENTICE] when they started to their completion:

• 1 – not important compared to the benefit of increasing the productivity of apprentices

• 2 – somewhat valuable (50%)

• 3 – equally valuable (100%)

• 4 – more valuable (150%)

• 5 – very valuable compared to the benefit of increasing the productivity of apprentices (200%)