Workforce Board and Partners Interview Guide

OMB No. 1290-XXXX

Exp. Date XX/XX/2021

OMB Package

Evaluation of the American Apprenticeship Initiative (AAI)

Site Visit Interview Guide:

Workforce Development Board (WDB) Partners

Note: If the Workforce Development Board is the AAI grantee, use the Grant Director Guide; if the WDB is a partner to the AAI grant, use this guide.

Introduction

I am (we are) researchers from Abt Associates, a private research organization based in Cambridge, MA, which conducts policy-related research on social and economic issues. This project is being conducted by Abt Associates and Urban Institute under contract to the U.S. Department of Labor. The goal of our visit today is to help us understand the early implementation experiences of American Apprenticeship Initiative (AAI) grant-funded projects. The evaluation (DOL-ETA-16-F-00006) aims to identify lessons learned from your experiences in implementing the projects under AAI. As part of this evaluation, we are conducting visits to 10 grantees. In addition to our site visits, we are collecting and analyzing data on program participation, services, and outcomes provided by the U.S. Department of Labor as well as administering a survey to a subset of AAI participants.

During our site visits to each of the project sites, we are interviewing AAI grant project directors and staff, as well as partner organizations and employers. We are here to learn more about the role the WDB is playing in the AAI grant. Our aim is to learn from your experiences, not to audit or judge your programs.

Before beginning the interview, I (we) want to thank you for agreeing to participate in the study. I (we) know that you are busy, and we will try to be as focused as possible. We have many questions and are going to talk with many different people, so please do not feel as though we expect you to be able to answer every question. Your participation in this discussion is voluntary, and you may choose not to answer any question.

In addition, before we start, I want to let you know that though we take notes at these interviews, information is never repeated with the name of the respondent. When we write our reports and discuss our findings, we compile and present information from the full set of interviews without identifying any one person. We do present information at the organization level but in ways that do not identify particular individuals. Do you have any questions before we begin?

[Interviewer Instructions: If the interview is going to be recorded, inform the respondent and request permission to record the interview.]

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 1 hour per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is required to obtain or retain benefit (FOA-ETA-15-02). Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to [email protected] and reference the 0MB Control Number 1290-XXXX. Comments can also be mailed to:  U.S. Department of Labor, Chief Evaluation Office, 200 Constitution Ave., NW, S-2312, Washington, DC  20210.  Note: Please do not return the completed interview guide to the email or mailing address.

1. BACKGROUND ON THE ORGANIZATION AND ROLE IN AAI GRANT

1. What is your job title and what are your primary responsibilities?

2. How long have you been working in this/these position(s) of [title from Q1] at [name of local WIOA/Workforce Development Board]?

3. Please provide a brief description/overview of your organization’s involvement/role in the AAI grant. Some possible probes (note – for each probe, ask the interviewee to briefly highlight role his/her organization played):

• Serve as apprenticeship sponsor

• Helped with development of the AAI proposal

• Helped with design of the AAI grant intervention

• Provided leadership on understanding and analyzing the need for education and training in the local labor market

• Helped identify target industries/occupations/regional clusters/hiring needs of employers for apprenticeships

• Helped identify target populations

• Helped with bringing partners together for the AAI grant (check in especially with AJC, employers (economic development agency might help with engaging employers, do they work with EDAs?), RTI/educational partners)

• Helped with recruitment of employers (or other entities) to participate in and sponsor apprenticeship and/or pre-apprenticeship training under the AAI grant

• Helped with organizing and bringing together training providers to provide apprenticeship training under the AAI grant

• Helped identify and refer potential apprentices or pre-apprentices

• Provided additional support services

• Use WIOA funding to support apprenticeships or pre-apprenticeships

4. What were the key factors that initially motivated your organization to participate in the AAI grant? Possibilities:

• Wanted to begin or expand an apprenticeship/pre-apprenticeship program(s) in the workforce catchment area

• Concerned about existing or future shortage of qualified workers in target industries

• Interested in the availability of AAI funds to:

  • Support on-the-job learning (overhead costs, shadowing, mentoring)

  • Develop and deliver related technical instruction (RTI)

  • Support partnership building

  • Develop promotional materials

  • Support recruitment

  • Develop curriculum and standards

• Employers were advocating for need to recruit and train workers to enter high growth occupations

• Saw the need for better benchmarking of skills within certain occupations

• Wanted to help employers to recruit a diverse, qualified workforce

• Wanted to introduce national credential linked to registered apprenticeship

• Improvements to workforce indicators for WIOA participants who enter apprenticeships or pre-apprenticeship programs (employment placement, degree or certification receipt)

5. What is the geographic area covered by your WDB?

6. What are the general characteristics of the local labor market, particularly in regard to occupations with apprenticeship programs?

Probes:

1. How has the local labor market changed over the course of the grant period?

2. Which industries have a high demand for workers?

3. Which sectors, occupations, and mix of companies are expanding or adopting apprenticeships?

7. To what extent were apprenticeship and pre- apprenticeship programs in existence within your service area (and the area served by the AAI grant) prior to the start of the AAI grant? How, if at all, was the WDB involved in planning and implementing these apprenticeships? For example:

   1. To what degree have you promoted the use of apprenticeships or pre-apprenticeship programs as a core part of your career pathways or sector-based WDB initiatives?

   2. How central are pre-apprenticeship and apprenticeship initiatives to your youth workforce development programming?

   3. Have you collaborated with unions, employers, employer organizations or other entities on developing apprenticeship or pre-apprenticeship programs?

8. Please describe your organization’s experience with the AAI project during the early stages of the grant period.

   1. Did your organization provide input into the design of AAI either during the proposal process or once the AAI grant was awarded? How did you participate? [Probe for analyzed labor market data to discern need for education/training in local area; identified target industries/occupations/hiring needs; identified target population for apprenticeship; identified support services; convened partners; identified or recruited employers to offer apprenticeships; identified and referred potential apprenticeship participants]

   2. What other organizations or partners were most involved in the design and early implementation of the AAI grant? Please briefly describe which organizations were most involved and their role.

   3. From your organization’s perspective, were there any factors that facilitated or hindered the start-up or implementation of the AAI grant?

   4. Were there program components or services that were originally planned by you or other partners that were not implemented or were altered during implementation?

B. ROLE OF WDB IN AAI EMPLOYER RECRUITMENT

1. How, if at all, has the WDB recruited employers to participate in the AAI program?

Interviewer Instructions: Please make sure to collect the number and names of each participating employer. If no recruitment of employers has been conducted, skip the remainder of this section.

2. In what ways have your American Job Centers been involved in the AAI program? How else might they contribute?

3. If the WDB has been involved in recruitment of employers how did you identify potential employers? [E.g., labor market data, focus on certain industries]

4. What strategies have you used to contact employers in targeted industries? Which strategies do you believe have been most effective (and why)? Which strategies have been the least effective? Some possible recruitment methods include:

• Provide outreach on/Market apprenticeship expansions to current apprenticeship providers

• Direct contact of new employers through telephone or emails

• WDB staff outreach presentations with employers or business organizations

• Outreach campaigns using media (e.g., PSAs, TV, radio, newspaper, etc.)

5. How do you “sell” the apprenticeship idea to targeted employers?

6. In your experience, what factors motivate employers to start or expand apprenticeship programs? Possibilities include:

• Approach to provide skilled workforce to overcome an existing or future shortage of qualified workers

• Grant funds to support on-the-job learning, development and implementation of RTI, other activities (e.g., partnership-building activities, development of promotional and recruitment materials, development of curriculum and standards, other program activities)

• Ability to benchmark skills to high levels

• Screening process effective at identifying workers likely to succeed

• Help in reaching a diverse, qualified workforce

• National credential linked to registered apprenticeship

• Assistance in developing skill standards and registration

• Help registering apprenticeships

7. What are the key challenges in persuading employers to start or expand apprenticeships? Possibilities include:

• Lack of knowledge about apprenticeship

• Difficulty in identifying apprentice occupations

• Administrative hassle in registering the program

• Costs of the program

• Difficulty in finding related instruction provider that is a good fit

• Lack of trainers

8. How have you addressed these challenges?

C. ROLE OF WDB IN AAI PARTICIPANT RECRUITMENT

Next, I would like to discuss the apprentice recruitment.

1. Has your organization been involved in recruitment of apprentices for the AAI program?

Interviewer Instructions: If no recruitment of employers has been conducted, skip the remainder of this section.

1. If yes, for which specific employers?

2. How many apprentices to date has your organization recruited? How many of these recruits have been subsequently enrolled in AAI-funded apprenticeship programs?

3. If available, please provide enrollments by apprenticeship program.

2. What populations do you targeted for outreach and recruitment?

• Incumbent workers

• New entrants to the labor force

• Long-term unemployed

• Low-income

• Populations that may traditionally be underrepresented in apprenticeship, including:

• Women

• Racial and ethnic minorities

• Individuals with disabilities

• Low-skilled populations

• Veterans, including transitioning service members

• Out-of-school youth

• Formerly incarcerated

1. As the AAI initiative has progressed, have there been any changes in populations you are targeting? If so, what are they? Why?

3. How do you identify potential candidates for apprenticeships?

   1. American Job Centers (e.g., information sessions, flyers or other promotional materials, train staff to discuss with customers, other, please specify)

   2. Website

   3. Media (e.g., PSAs)

   4. Other, please specify

4. Does your organization have a recruitment goal as part of the AAI grant? If so, what is this goal and do you expect to meet it?

5. What does recruitment involve? [Probe for the WDB’s role—identifying and referring candidates to either the grantee or an apprenticeship sponsor; assessing or otherwise screening candidates before referring to a sponsor; conducting outreach and recruitment on behalf of specific employers; other, please specify]

6. Has your organization encountered recruitment challenges under the AAI grant? If so, what challenges have been encountered and how have they been addressed? Some possible challenges include:

• Have difficulty finding eligible participants

• Many who applied have insufficient basic skill levels

• Some applicants are not interested or motivated to participate in apprenticeship

• Apprenticeship period is too lengthy for some applicants

• Some of the outreach strategies haven’t resulted in many applicants

• Some applicants have difficulty getting to the program/employer location

• Didn’t have enough resources for recruitment

• Changing economic conditions in the areas has made recruitment more challenging

• Other, please specify

7. Overall, which recruitment or referral strategies of apprentices have been most effective? Do you feel any of these strategies are new, unique and/or innovative? Which strategies have been the least effective?

4. possible use of WIOA funds for apprenticeship

1. Do you use WIOA youth or adult funds to support the on-the-job training component of AAI apprenticeships?

2. Do you use WIOA youth or adult funds to pay for any of the costs of related classroom instruction for AAI apprentices or pre-apprentices, such as tuition, books, supplies, fees, uniforms, tools, and other required items. If so, specify which costs are covered with WIOA funds.

3. Do you use WIOA youth or adult funds to pay for basic skills and/or limited English proficiency training for AAI pre-apprentices?

4. Do you use WIOA youth or adult funds to pay for other AAI apprentice or pre-apprentice services such as tutoring, mentoring, and work experience (other than the apprenticeship OJT component, such as paid or unpaid internships)?

5. Do you view AAI apprentices and pre-apprentices as helping or hindering your WDB’s performance on such measures as employment, earnings, and credential attainment? Describe their impact.

4. AAI Grant Successes and Challenges

I would like to end our discussion today with your view on program successes and challenges, including lessons learned for the field.

1. Overall, what do you believe are the AAI initiative’s biggest strengths? Key weaknesses?

2. Are there ways in which the initiative could improve apprenticeship opportunities? If yes, how?

3. Are there ways in which the initiative could engage more employers? If yes, how?

4. Are there other approaches, strategies, or services that you believe would contribute to improved outcomes for apprentices?

5. Are there other ways that WDBs can be involved in the AAI Initiative? Are there other ways that AJCs can be involved in the AAI Initiative?

CHECKLIST OF ITEMS TO COLLECT FROM RESPONDENT (IF AVAILABLE)

• Brochures and other informational material about AAI and apprenticeship, in general

• Recruitment literature and forms for employers, participants

• Agreements or MOUs with AAI grantee or employers

10