OMB No. 1290-XXXX
Exp. Date XX/XX/2021
OMB Package
Evaluation of the American Apprenticeship Initiative (AAI)
Site Visit Interview Guide:
[Note to Interviewers: If Union is grantee, use the Grant Director guide]
I am (we are) researchers from Abt Associates, a private research organization based in Cambridge, MA, which conducts policy-related research on social and economic issues. This project is being conducted by Abt Associates and Urban Institute under contract to the U.S. Department of Labor. The goal of our visit today is to help us understand the early implementation experiences of American Apprenticeship Initiative (AAI) grant-funded projects. The evaluation (DOL-ETA-16-F-00006) aims to identify lessons learned from your experiences in implementing the projects under AAI. As part of this evaluation, we are conducting visits to 10 grantees. In addition to our site visits, we are collecting and analyzing data on program participation, services, and outcomes provided by the U.S. Department of Labor as well as administering a survey to a subset of AAI participants.
During our site visits to each of the project sites, we are interviewing union representatives as well as AAI grant project directors and staff, partner organizations, and employers. We are here to learn about your role in the service delivery model and understand how the model was implemented under the AAI grant. Our aim is to learn from your experiences, not to audit or judge your programs.
Before beginning the interview, I (we) want to thank you for agreeing to participate in the study. I (we) know that you are busy, and we will try to be as focused as possible. We have many questions and are going to talk with many different people, so please do not feel as though we expect you to be able to answer every question. Your participation in this discussion is voluntary, and you may choose not to answer any question.
In addition, before we start, I want to let you know that though we take notes at these interviews, information is never repeated with the name of the respondent. When we write our reports and discuss our findings, we compile and present information from the full set of interviews without identifying any one person. We do present information at the organization level but in ways that do not identify particular individuals. Do you have any questions before we begin?
[Note: If the interview is going to be recorded, inform the respondent and request permission to record the interview.]
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 1 hour per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is required to obtain or retain benefit (FOA-ETA-15-02). Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to [email protected] and reference the 0MB Control Number 1290-XXXX. Comments can also be mailed to: U.S. Department of Labor, Chief Evaluation Office, 200 Constitution Ave., NW, S-2312, Washington, DC 20210. Note: Please do not return the completed interview guide to the email or mailing address.
A. BASIC UNION INFORMATION AND AAI PROGRAM CONTEXT
Verify or complete with the respondent the following background information about the union and its relationship to the AAI grantee. (Note: Collect and pre-fill this information prior to the visit from the grant application, KD matrix, or other available documentation and verify with the respondent. For example, create a fact sheet based on the grant proposal and other available background material and send it in advance of the visit to the site for review and comment.)
Name of informant:
Name of union and local:
Union industries and occupations:
Approximate size of union local (number of members):
Union status in AAI program:
Identified as an institutional partner in the grant application
New participant recruited by AAI program
Date AAI operations began:
No prior apprenticeship program, initiated apprenticeship under AAI: (Y/N).
B. NATURE OF PARTNERSHIP
What is your role as a partner in the AAI program? Has your role changed since the program began? If so, in what ways?
Did the union participate in writing and developing the AAI proposal? If so, how?
As a partner of the AAI grant, what do you hope to achieve in being part of this initiative?
How do your union’s goals fit with the larger goals of the grant and initiative?
Please describe project start-up and early implementation. What was the union’s role?
How many total apprentices do you expect to serve as part of the AAI grant? By industries/occupations?
C. UNION ENGAGEMENT IN AAI
How did you become involved in the AAI initiative?
Already providing apprenticeships and was part of the grant application.
Recruited to provide apprenticeships. If you were recruited take us step-by-step through the process of planning, implementing and registering your apprenticeship program with the AAI program.
Recruited by grantee, WDB, AJC?
For your registered apprenticeship program please discuss the following:
Names of occupations registered
Expected duration of each apprenticeship by occupation
Goal for number of apprentices
Number of apprentices placed in program to date by occupation
Number of apprentices recruited from unrepresented groups (by underrepresented group) to date and target number over the course of the grant
Names of providers of related technical instruction (and type of training provided: basic skills, work readiness skills, occupational skills)
Linkages with or no linkages with degree programs (such as AAs)
If you have expanded an apprenticeship program through the AAI, please discuss the following for your apprenticeship program:
Names of any new occupations registered
Expected duration of each apprenticeship by occupation
Goal for number of additional apprentices under the expansion
Number of additional apprentices hired to date
Number of additional apprentices hired from unrepresented groups (by underrepresented group by occupation) to date and target number over the course of the grant.
Names of providers of related technical instruction and other instruction
Linkages with or no linkages with degree programs (such as AAs)
What is the organizational structure for your AAI apprenticeship program? Please describe apprenticeship program staffing, indicating the types of positions and the extent to which each position is full-time/part-time and the funding for each position. Please collect any materials that describe the structure – org chart, position descriptions, etc.
What funding do you receive from the AAI grantee? How are funds used?
What other assistance did you receive from the grantee to design and implement your program?
Did the WDB and/or AJC (and/or Economic Development Agency) provide assistance? If so, what kind?
What role did the national union play in designing and implementing the apprenticeship program?
D. TARGET POPULATION AND PARTICIPANT RECRUITMENT
How are apprentices recruited for your program? What role does the union play? The AAI grantee? Others?
Sponsor?
What methods are used to market or conduct outreach on the apprenticeship program? Possibilities include:
Union-related activities
Distribution of flyers, posters or other educational/informational
AAI program staff outreach presentations or orientations held in the area served (estimate number of presentation/orientations provided please)
Websites (union and/or grantee)
Toll-free informational hotlines
Outreach campaigns using media (e.g., PSAs, TV, radio, newspaper, ads on buses/bus shelters)
Social media (e.g., LinkedIn, Facebook, Twitter, Instagram, etc.) – please specify, which types of social media are used and how
Word-of-mouth
AAI partners
For your program, what populations are targeted for special outreach? Why? Some possible special target groups include:
Incumbent workers
New entrants to the labor force
Long-term unemployed
Low-income
Populations that may traditionally be underrepresented in apprenticeship, including:
Women
Black/African American and Hispanic men and women
Individuals with disabilities
Low-skilled populations
Veterans, including transitioning service members
Out-of-school youth
Formerly incarcerated
What has been the response of the targeted population(s) thus far?
Do you expect to meet your enrollment goals (Recall total goal and by occupation)? If not, why not? If yes, why?
Have there been recruitment challenges? If not, why not do you suppose? If so, what challenges have been encountered and how have they been addressed? Some possible challenges include:
Have difficulty finding eligible participants
Many who applied have insufficient basic skill levels
Some applicants are not interested or motivated to participate in apprenticeship
Apprenticeship period is too lengthy for some applicants
Partners that were supposed to provided referrals have not provided them in sufficient numbers
Didn’t have enough resources for recruitment
Changing economic conditions in the areas has made recruitment more challenging
Other, please specify
Overall, what recruitment or referral strategies being used under the AAI grant have been most successful? Do you feel any of these strategies are new, unique and/or innovative? Replicable?
E. MAIN TRAINING PROGRAM COMPONENTS/SERVICES – EXAMPLE(S) OF APPRENTICESHIP TRAINING FUNDED UNDER AAI
[Note: Cover these topics to the extent that they are not covered during the earlier discussion of employer involvement. It may be necessary to focus in on a particular employer (and/or sponsor) or apprenticeship program as an example of AAI-funded apprenticeship program. If a visit is conducted to a particular apprenticeship program or programs as part of the site visit, it may be appropriate to skip these questions and cover them at the employer/sponsor or training site where the apprenticeship training is being provided.]
What is your union’s role in apprenticeship training under the AAI grant? [Note to interviewer: If response is “no,” skip to section G]
Sponsor?
What are the key program components and activities for apprenticeships?
Are apprentices union members? If so, when do they become members?
Is the AAI-funded apprenticeship program time-based (e.g., completion of specific numbers of hours of training), competency-based or a combination of the two?
What is the usual duration of participation in the apprenticeship? Is there variation in how long it takes an AAI participant to complete the apprenticeship? If yes, what is the range of time (in weeks, months, or years) and why might it vary on a case-by-case basis??
If competency-based, how is it determined that a participant has achieved the necessary level of competence to complete the apprenticeship?
For each occupation, how many hours of RTI (i.e., classroom/web-based) are provided during the apprenticeship component? Over what period of time does RTI occur?
What is the typical schedule for an apprentice each week – e.g., 4 hours of formal classroom instruction and 36 hours of work)? Does this schedule change over the course of the apprenticeship?
Where and how is instruction provided (e.g., classroom, web-based, hybrid, laboratory/shop)?
Where is instruction provided (e.g., at a community college, at a union, at an employer site)?
Please provide a brief overview of the instruction provided (e.g., topics covered in classroom instruction). [Note: If applicable, request a syllabus for the coursework or a topic outline for the training.]
Are there innovative RTI methods being used as part of the apprenticeship initiative? Please briefly discuss. Possibilities might include:
Accelerated learning modules
Contextualized learning
Team teaching
Self-paced learning
Competency-based learning
Prior learning assessments
Technology-enabled learning, such as use of simulators, on-line teaching/learning
Integrating basic skills instruction with technical instruction
Has AAI funding been used for any of the following with respect to RTI, and if so, how:
Development of courses at post-secondary education level that are integrated into apprenticeship
Tuition and other educational fees
Delivery of instructional requirements (e.g., virtual learning technology, classroom instructors)
Cost of training facilities
Books, computers, etc.
How many hours of on-the-job instruction does an apprentice complete (i.e., each week, month, or year). Does this vary over time and, if so, how?
Where do apprentices work and in what types of jobs? For example, in what types of jobs do apprentices start and progress to during their involvement in the apprenticeship?
Who oversees and provide on-the-job instruction to the apprentice (i.e., while the apprentice is working)? Please briefly describe the nature of the workplace instruction provided to apprentices and the supervision.
What participation and performance benchmarks do supervisors use to assess apprentice performance and development?
What is the supervisor-to-apprentice ratio during the apprenticeship period (e.g., 1 supervisor to 3 apprentices)? Does it change over time?
With regard to on-the-job learning, have AAI funding been used to offset overhead costs associated with any of the following (and if so, how): providing training, job shadowing, mentoring, and additional supervision?
What is the schedule of wages and benefits for apprentices? Does it vary by type of job/occupation?
To what extent are non-training supports provided during the apprenticeship period (e.g., case management, career coaches/navigators counseling, and mentoring)? Who provides these supports?
To what extent are supportive services provided such as transportation assistance, work clothes/equipment, childcare, and needs-based payments? If provided, how are these supportive services structured and paid for? Who provides these services?
What, if any, post-completion services are provided to participants of the AAI-funded apprenticeship program? Who provides these services?
What percentage of apprentices complete apprenticeships? When is attrition most likely? Is attrition by occupation noted?
What percentage of completers remain with the union?
What aspects, if any, of the apprenticeship RTI or services do you think are most innovative? Effective?
F. LESSONS LEARNED TO DATE
To what extent do you think this program could and should be replicated by your union in other localities? In other industries or other unions? What features of this AAI program are most amenable to replication? What features of project are least amenable to replication? How does location, the target population served, or other distinctive features of your program make it either non-transferable or limit transferability?
To date, what do you consider to be the union’s most important accomplishments under the AAI grant?
To date, what do you believe to be the main lessons learned from your AAI grant for apprenticeship program design and operations?
What factors seem to explain success or lack of success for the apprenticeship program?
What innovations and lessons learned under the AAI grant can contribute to efforts to encourage more unions to adopt apprenticeships?
Are there recommendations for the WDB and/or AJCs to increase or enhance their engagement with unions? Federal and state OA? Economic Development Agency/Agencies?
CHECKLIST OF ITEMS TO COLLECT FROM RESPONDENT (IF AVAILABLE)
Copy of syllabus for the coursework or a topic outline for the training
SUPPLEMENTAL DISCUSSION GUIDE:
PRE-APPRENTICESHIP PROGRAM COMPONENT
[Note to interviewer: Omit this supplement if the AAI grant does not fund a pre-apprenticeship program component. If more than one pre-apprenticeship program component exists or training is provided for more than one type of occupation, where appropriate, indicate differences across programs/industry sectors/occupations on each of the questions. Ask this module only if interviewee is involved in the pre-apprenticeship program component.]
How many pre-apprentices do you expect to enroll over the course of the grant?
For which industries and occupations are pre-apprenticeship training provided? Why were these industries/occupations targeted?
What funding do you receive from the AAI grantee for the pre-apprenticeship program? How are funds used?
What other assistance did you receive from the grantee to design and implement your program? Did you receive assistance from the WDB and/or AJC related to this grant? OA rep?
What role did the national union play in designing and implementing the apprenticeship program? Are you a sponsor/sponsoring more than one apprenticeship program? Occupation(s)?
How do you market your program to prospective pre-apprenticeship participants? Possibilities include:
Distribution of flyers, posters
AAI program staff outreach presentations or orientations held in the area served (estimate how many provided please)
Informational websites
Toll-free informational hotlines
Outreach campaigns using media (e.g., PSAs, TV, radio, newspaper, ads on buses/bus shelters)
Social media (e.g., LinkedIn, Facebook, Twitter, Instagram, etc.) – please specify, which types of social media are used and how
Word-of-mouth
Primary partners
What populations are targeted for special outreach and why? Some possible special target groups include:
Incumbent workers
New entrants to the labor force
Long-term unemployed
Low Income
Populations that may traditionally be underrepresented in apprenticeship, including:
Women
Men and women of color
Individuals with disabilities
Low-skilled populations
Veterans, including transitioning service members
Out-of-School Youth
Formerly incarcerated
Do other organizations or grant partners make referrals to your pre-apprenticeship program? If so, describe. If not asked above, does the WDB and/or AJC make referrals to your pre-apprenticeship program?
Are you on pace to recruit the intended number of pre-apprenticeship participants (total and by occupation)? If not, why?
Have there been recruitment challenges? If so, what challenges have been encountered and how have they been addressed? If not, why? Some possible challenges include:
Have difficulty finding eligible participants
Many who applied have insufficient basic skill levels
Some applicants are not interested or motivated to participate in the program
Program is too lengthy for some applicants
Partners that were supposed to provided referrals have not provided them in sufficient numbers
Didn’t have enough resources for recruitment
Changing economic conditions in the areas has made recruitment more challenging
Other, please specify
What are the main steps involved in the AAI pre-apprenticeship intake process? Please provide an overview of the flow of new recruits through the intake process, including:
What are the eligibility requirements to enroll in the pre-apprenticeship component funded under the AAI grant? [Note: Request copies of AAI participant eligibility or intake form(s).] Possibilities include:
Be from a specific subpopulation
Reside within a specific geographic area
Meet income eligibility requirements (what are these requirements)
Meet education level requirements (e.g. high school diploma, GED/HSED) or basic skill levels or assessment
Who determines eligibility to participate in the AAI grant? What role does the associated RA (at Federal or state level) play in recruitment and intake?
During the intake process, are any assessments used (including formal tests such as the TABE, interest inventories, substance abuse screening)? [Note: Request copies of assessment forms, if relevant.]
Describe development of the curriculum for pre-apprentices? What part did the AAI grant staff play? Employers? RTI? WDB/AJCs? How is pre-apprenticeship training delivered, where and by whom?
Are there innovative instructional methods being used? By who for each discussed. Please briefly discuss. Possibilities might include:
Accelerated learning modules
Contextualized learning
Team teaching
Self-paced learning
Competency-based learning
Prior learning assessments
Technology-enabled learning, such as use of simulators, on-line teaching/learning
What is the nature of hands-on training? Is it volunteer work? Simulation lab? Who provides the training? What is the length of training?
What other services are provided to promote success in the pre-apprenticeship program and beyond? Does the AJC assist in any way? Examples include:
Job readiness classes
Career success workshops
Basic skills education (ESL or ABE)
Other
To what extent are non-training supports provided, such as case management, career coaches/navigators counseling, or mentoring? If provided, how are these supports structured? Who provides each? Do AJCs play a role? If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
To what extent are supportive services provided such as transportation assistance, childcare, work clothing/equipment, or needs-based payments? If provided, how are these supportive services structured and paid for? Who provides each? Do AJCs play a role? If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
Is the pre-apprenticeship training connected to higher education? If so, who and how? Do AAI pre-apprenticeship participants receive post-secondary (college) credits, credentials, and/or certificates? Please specify types of credits/credentials/certificates received. If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
How is the pre-apprenticeship connected to your registered apprenticeship program(s)? Are there apprenticeship slots available for all pre-apprentices who successfully complete?
What other post-completion services are provided to participants? Who provides these (for each described)? If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
What percentage of AAI participants complete pre-apprenticeship training (total and by occupation)? [Note: If available, collect this data from administrator sources prior to the visit or request data on attrition from the grantee.]
Is attrition greater than expected? If so when and why does attrition typically occur?
Are there certain types of participants that are more/less likely to complete pre-apprenticeship training?
If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
What percentage of AAI pre-apprenticeship completers become apprentices (total and by occupation) [Note: If available, collect this data from administrator sources prior to the visit or request data on attrition from the grantee.]
Are there factors that help explain the success or lack of success?
If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | ATTACHMENT 7 |
| Author | admined |
| File Modified | 0000-00-00 |
| File Created | 2021-01-21 |