OMB No. 1290-XXXX
Exp. Date XX/XX/2021
OMB Package
Evaluation of the American Apprenticeship Initiative (AAI)
Site Visit Interview Guide:
I am (we are) researchers from Abt Associates, a private research organization based in Cambridge, MA, which conducts policy-related research on social and economic issues. This project is being conducted by Abt Associates and Urban Institute under contract to the U.S. Department of Labor. The goal of our visit today is to help us understand the early implementation experiences of American Apprenticeship Initiative (AAI) grant-funded projects. The evaluation (DOL-ETA-16-F-00006) aims to identify lessons learned from your experiences in implementing the projects under AAI. As part of this evaluation, we are conducting visits to 10 grantees. In addition to our site visits, we are collecting and analyzing data on program participation, services, and outcomes provided by the U.S. Department of Labor as well as administering a survey to a subset of AAI participants.
During our site visits to each of the project sites, we are interviewing AAI grant project directors and staff, as well as partner organizations and employers. We are here to learn about your service delivery model and understand how it was implemented under the AAI grant. Our aim is to learn from your experiences, not to audit or judge your programs.
Before beginning the interview, I (we) want to thank you for agreeing to participate in the study. I (we) know that you are busy, and we will try to be as focused as possible. We have many questions and are going to talk with many different people, so please do not feel as though we expect you to be able to answer every question. Your participation in this discussion is voluntary, and you may choose not to answer any question.
In addition, before we start, I want to let you know that though we take notes at these interviews, information is never repeated with the name of the respondent. When we write our reports and discuss our findings, we compile and present information from the full set of interviews without identifying any one person. We do present information at the organization level but in ways that do not identify particular individuals. Do you have any questions before we begin?
[Interviewer Instructions: If the interview is going to be recorded, inform the respondent and request permission to record the interview.]
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 2 hours per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is required to obtain or retain benefit (FOA-ETA-15-02). Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to [email protected] and reference the 0MB Control Number 1290-XXXX. Comments can also be mailed to: U.S. Department of Labor, Chief Evaluation Office, 200 Constitution Ave., NW, S-2312, Washington, DC 20210. Note: Please do not return the completed interview guide to the email or mailing address.
A. BASIC GRANT INFORMATION AND PROGRAM CONTEXT
Verify with the respondent the following background information about the AAI grant. (Note: Collect and pre-fill this information prior to the visit from the grant application, KD matrix, or other available documentation and verify with the respondent. For example, create a fact sheet based on the grant proposal and other available background material and send it in advance of the visit to the site for review and comment.)
Original grant period: ___________ to ___________
Modified grant period (if applicable): ___________ to ___________
Does AAI grant fund apprenticeship and pre-apprenticeship program components?
___Apprenticeship-only
___Both apprenticeship and pre-apprenticeship
[Note: If respondent has pre-apprenticeship program component, after completing this discussion guide, go through the Supplemental Discussion Guide for Programs with Pre-Apprenticeship Programs with is attached to the end of this guide.]
Overall AAI grant:
Original AAI grant amount: $________________
Modified (and final) AAI grant amount (if applicable): $____________
For apprenticeship program component, please identify the following:
Industries targeted
Occupations targeted
What is your goal for the number of new apprenticeship positions to be developed under the grant? [Note: gather from grant application or other program documents and confirm with grantee.]
What geographic area does the AAI grant serve (e.g., identify specific counties or regions served])? [Note: collect information prior to the visit and verify during interview.]
What are the socioeconomic characteristics of the area served by the AAI grant? [Note: Collect these data from BLS prior to the site visit (for a, b, c), present them to informant and ask if they describe the immediate catchment area for the program in this way and whether there is anything to add that is particularly relevant to program structure, implementation, recruitment, or participant outcomes.]
Unemployment rate for area served (start of grant; at time of visit)
Availability of job openings and trends in area served, particularly within occupations for which apprenticeship/pre-apprenticeship training are being provided
Wage rates in area served for occupations for which apprenticeship/pre-apprenticeship training are being provided
Other local economic conditions that may affect the project’s ability to recruit participants or employers, as well as ability of apprentices to complete training and find employment (e.g., in- or out-migration of major employers, major layoffs, base closings, natural disasters)
Other distinctive demographic characteristics of the area served
Under the AAI initiative a “primary partnership” must include at least one Public and one Private Sector entity.” Please provide background on the two or more entities that formed the primary partnership.
Background on the Public Sector Entity or Entities Involved in Primary Partnership – [Note: Collect information below for each of the primary partnership entities, if there is more than one.]
Type of public sector entity
[Note: The Public Sector Entity must represent at least one of the following three types of entities:
The workforce investment system
Public or nonprofit education or training provider
DOL-recognized State Apprenticeship Agency]
Briefly describe the organization’s role in the AAI initiative
Is this organization the AAI grantee?
What amount of AAI funding does this organization receive?
If the organization is the grantee, what portion of the grant does it subcontract out to other organizations?
Briefly, what role does the organization play under the AAI grant
What is the extent of the organization’s prior experience with:
Pre-apprenticeship
Apprenticeship
WIOA or other workforce development programs/initiatives (if other, please explain)
Have there been any changes in public sector entities since the grant program was implemented? If so, why?
Background on the Private Sector Entity or Entities Involved in Primary Partnership – [Note: Collect information below for each of the primary partnership entities, if there is more than one.]
Type of private sector entity
[Note: The Private Sector Entity must represent at least one of the following types of private sector organizations:
Business
A consortium of businesses
A business-related nonprofit organization
A private organization functioning as a workforce intermediary for the express purpose of serving the needs of businesses]
Briefly describe the organization’s role in the AAI initiative
Is this organization the AAI grantee?
What amount of AAI funding does this organization receive?
If the organization is the grantee, what portion of the grant does it subcontract out to other organizations?
Briefly, what role does the organization play in AAI?
What is the extent of the organization’s prior experience with:
Pre-apprenticeship
Apprenticeship
WIOA or other workforce development programs/initiatives
Have there been any changes in private sector entities since grant program implemented? If so, what were they and why?
What are other relevant features about the AAI grantee or other organization(s) involved in the primary partnership that may affect AAI program design, implementation, or operations, such as organization size, source of funding, and recent changes in leadership/staffing.
What is the nature of local WIOA/Local Workforce Development Board involvement with apprenticeship training in the area served by the AAI grant? What, if any, is the involvement of the American Job Center(s)? Economic Development?
B. EMPLOYER RECRUITMENT AND ENGAGEMENT, AND ROLE IN THE AAI INITIATIVE
Note: Prior to the site visit, collect the following information from the grant application and draft out responses to each of the questions; if appropriate/feasible send draft to grantee prior to the visit for review and comment:
Who were the collaborating employers (and sponsors, who may/may not be employers) listed on the proposal?
[Provide a full list of employers.]
What methods did the proposal specify for reaching out to employers (and sponsors, if different/as applicable)?
What roles did the grantee propose to play in registering company apprenticeship programs?
In what ways did you as grantee expect to organize groups of employers (and sponsors, if different/as applicable)? By sector, by location?
If available in the proposal, which employers listed on the proposal already had at least one registered apprentice program?
Which employers originally listed in the proposal are and are not participating in the AAI project?
Why did employers who were expected to participate, not participate?
Are there additional employers (not part of the original proposal) that have been recruited to participate in the AAI project?
[Note: As a result of Q1 and Q2, make sure to collect the number and names of each participating employer (and sponsors, if different/as applicable).]
[If no new recruitment of employers has been conducted, skip to Q5.]
If new employers (and sponsors, if different/as applicable) have been recruited for the AAI grant, what methods have been used to reach out to and recruit employers? Which methods have proved more and less successful? Some possible recruitment methods include:
Direct contact through telephone or emails
Distribution of flyers
AAI program staff outreach presentations or orientations with employers or business organizations (approximately how many presentations)
Referrals from other organizations and employers (e.g., the Chamber of Commerce, other businesses organizations, employers and or other entities sponsoring apprenticeships, workforce development programs, etc.)
Outreach campaigns using media (e.g., PSAs, TV, radio, newspaper, etc.)
Websites/social media (e.g., LinkedIn, Facebook, etc.) – please specify, which types of social media are used and how
Word-of-mouth (participants, others?)
How have targeted employers responded to the outreach and recruitment effort? What about sponsors, if different/as applicable?
Has the AAI program been able to recruit the intended number of employers? Sponsors? Recall goal for employers (and sponsors)?
Has the pool of potential employers (and sponsors) for the program been sufficient?
Have there been more employers (and sponsors) interested in engaging with the AAI program than can be included (e.g., is there a waiting list for employers to join the program)?
What are the key barriers in persuading companies to start or expand apprenticeship programs? Or sponsor a program? Possibilities include:
Lack of knowledge about apprenticeship
Difficulty in identifying apprentice occupations
Administrative hassle in registering the program
Costs of the program
Difficulty in finding related instruction provider that is a good fit
Lack of trainers
What are the key factors that motivate employers (and sponsors) to start or expand apprenticeship programs? Possibilities include:
Existing or future shortage of qualified workers
Expected benefits during and soon after the apprenticeship
Subsidies to firms or for related instruction
Ability to benchmark skills to high levels
Screening process effective at identifying workers likely to succeed
Help in reaching a diverse, qualified workforce
National credential linked to registered apprenticeship
Assistance in developing skill standards and registration
For how many employers (including sponsors, especially if different than employer) has the grantee assisted in developing work processes (i.e., occupational standards) and in applying for registration?
From among the list of employers (and sponsors) involved in the initiative, please identify the employers (and sponsors) that have started new apprenticeship programs (i.e., that have become registered and have hired apprentices) as a result of participating in AAI. Please discuss the following for each employer:
Names of occupations registered.
Expected duration of each apprenticeship by occupation.
Number of apprentices hired to date and target number over the course of the grant.
Number of apprentices hired from unrepresented groups (by underrepresented group) to date and target number over the course of the grant.
Names of providers of related instruction.
Linkages with or no linkages with degree programs (such as AAs)
From among the list of employers involved in the initiative, please identify the employers (and sponsors) that have expanded existing registered apprenticeship programs as a result of participating in AAI (i.e., had registered apprenticeship with hired apprentices prior to the AAI grant). Please discuss the following for each employer:
Names of occupations registered.
Expected duration of each apprenticeship occupation.
Number of apprentices hired.
Number of apprentices hired from unrepresented groups (by underrepresented group)
Names of providers of related instruction.
Linkages with or no linkages with degree programs (such as AAs)
From among the list of employers involved in the initiative, please identify the employers (and sponsors) that developed apprenticeship programs but are waiting approval from the state or federal office of apprenticeship as a result of participating in AAI. Please discuss the following for each employer (and sponsor, if different/as applicable):
Names of apprentice occupations (to be approved).
For each occupation, whether the submission was sent to a state apprenticeship or to the federal office of apprenticeship.
Expected duration of each apprenticeship occupation (once approval is obtained).
Number of apprentices expected to be hired.
Number of apprentices expected to be hired from unrepresented groups (by underrepresented group)
Names of providers of related instruction (if known).
Linkages with or no linkages with degree programs (such as AAs), if known.
From among the list of employers (and sponsors) involved in the initiative, please identify the employers (and sponsors) who have expressed an intention of creating an apprenticeship program, but have not yet developed or submitted a proposal for registering the program. Please discuss the following for each employer (and sponsor):
Names of apprentice occupations.
For each occupation, whether the submission will be sent to a state apprenticeship or to the federal Office of Apprenticeship (OA).
Expected duration of each apprenticeship occupation (once approval is obtained).
Number of apprentices expected to be hired total and by occupation.
Number of apprentices expected to be hired from unrepresented groups (by underrepresented group and occupation)
Names of providers of related instruction (if known).
Linkages with or no linkages with degree programs (such as AAs), if known.
C. TARGET POPULATION AND PARTICIPANT RECRUITMENT
How do you market your apprenticeships to prospective participants? Possibilities include:
Distribution of flyers, posters or other educational/informational
AAI program staff outreach presentations or orientations held in the area served
Informational websites (estimated number of presentations given?)
Toll-free informational hotlines
Outreach campaigns using media (e.g., PSAs, TV, radio, newspaper, ads on buses/bus shelters)
Social media (e.g., LinkedIn, Facebook, Twitter, Instagram, etc.) – please specify, which types of social media are used and how
Word-of-mouth (participants, employers, other?)
Primary partners
What populations and/or subpopulations has the project targeted for special outreach? Why have these populations/subpopulations been selected? As the AAI initiative has progressed, have there been any changes in special populations targeted? If so, what are they? Why? Some possible special target groups include:
Incumbent workers
New entrants to the labor force
Long-term unemployed
Low income
Populations that may traditionally be underrepresented in apprenticeship, including:
Women
Black/African American and Hispanic Youth
Individuals with disabilities
Low-skilled populations
Veterans, including transitioning service members
Out-of-School Youth
Ex-offenders
Have employers (or sponsors) provided input on the target population or admission criteria? Please describe nature of input and specific criteria or requirements.
What organizations provide referrals of prospective AAI participants? Which organizations have provided the most referrals? Have any organizations fallen short of or exceeded original expectation for providing referrals (what was goal and what was met)? Possibilities include:
Employers
Unions
Workforce system – American Job Centers, Employment Service, Economic Development, and WIOA
Local Education Agencies (LEAs), such as the local public school system
Re-engagement centers (i.e., to link youth that have dropped out of school back to school or other types of training)
Community colleges and other educational institutions including proprietary schools
Courts/correctional system/juvenile justice/probation officers
Faith-based and community-based organizations
Other public and non-profit human service agencies
What has been the response of the targeted population(s) to the initiative?
Is the AAI-funded project expected to meet its enrollment goal (recall goal here)? If not, why not?
Has the pool of potential applicants for the program been sufficient?
Have there been more applicants than the program can serve (e.g., is there a waiting list to get into the program)?
How successful has the AAI grant been in recruiting individuals from groups that do not typically enroll in apprenticeship programs, such as women (e.g., for construction trades, men for nursing), Black/African American and Hispanic youth, individuals with disabilities, Veterans, out-of-school youth, etc.? Which groups have you been successful with and with which groups have you been unsuccessful? Any reasons for why?
Have there been recruitment challenges? If so, what challenges have been encountered and how have they been addressed? Some possible challenges include:
Have difficulty finding eligible participants
Many who applied have insufficient basic skill levels
Some applicants are not interested or motivated to participate in apprenticeship
Apprenticeship period is too lengthy for some applicants
Some of the outreach strategies haven’t resulted in many applicants
Partners that were supposed to provided referrals have not provided them in sufficient numbers
Some applicants have difficulty getting to the program/employer location
Didn’t have enough resources for recruitment
Changing economic conditions in the areas has made recruitment more challenging
Other, please specify
Overall, what recruitment or referral strategies being used under the AAI grant have been most successful? Do you feel any of these strategies are new, unique and/or innovative? Replicable?
D. AAI PARTICIPANT ELIGIBILITY, INTAKE, AND ASSESSMENT
What are the main steps involved in the AAI intake process? Please provide an overview of the flow of new recruits through the intake process.
What are the eligibility requirements to enroll in the apprenticeships funded under the AAI grant? [Note: Request copies of AAI participant eligibility or intake form(s).] Possibilities include:
Complete pre-apprenticeship program (i.e., for apprenticeship program component)
Be from a specific subpopulation
Reside within a specific geographic area
Meet income eligibility requirements (and what are these)
Meet education level requirements (e.g. high school diploma, GED/HSED) or basic skill levels
What process (if any) is involved in the selection/screening process before sending workers on employer interviews? Is the American Job Center or other (employer’s HR) involved in this process? Please describe this process including (if applicable):
How does the AAI program determine whether an applicant is the right fit?
About what percentage of applicants are rejected (e.g., screened out prior to being sent for employer interviews)?
What are the main reasons that applicants are screened out or fail to move through the intake process to be considered by employers?
What role do employers (and/or sponsors) play in the screening and selection process? Please describe this process including (if applicable):
How do employers (and/or sponsors) generally determine whether an applicant is the right fit for an apprenticeship program?
About what percentage of applicants are rejected (e.g., screened out during the employer interview process)? What are the major reasons employers (and/or sponsors) reject applicants?
What are the main reasons that applicants are screened out or fail to move through the intake process to be considered by employers (and/or sponsors)?
During the intake process, are any formal assessment tools/tests used by employers, sponsors, or the AAI grantee (including formal tests such as the TABE, interest inventories, substance abuse screening)? If yes, please identify these formal assessments. [Note: Request copies of assessment forms, if relevant.]
E. MAIN TRAINING PROGRAM COMPONENTS/SERVICES – EXAMPLE(S) OF APPRENTICESHIP TRAINING FUNDED UNDER AAI
[Note: Cover these topics to the extent that they are not covered during the earlier discussion of employer involvement. It may be necessary to focus in on a particular employer (and/or sponsor) or apprenticeship program as an example of a AAI-funded apprenticeship program. If a visit is conducted to a particular apprenticeship program or programs as part of the site visit, it may be appropriate to skip these questions and cover them at the employer (and/or sponsor) or training site where the apprenticeship training is being provided.]
Is the AAI-funded apprenticeship program time-based (e.g., completion of specific numbers of hours of training), competency-based or a combination of the two?
What is the usual duration of participation in the apprenticeship? Is there variation in how long it takes an AAI participant to complete the apprenticeship? If yes, what is the range of time (in weeks, months, or years) and why might it vary on a case-by-case basis??
If competency-based, how is it determined that a participant has achieved the necessary level of competence to complete the apprenticeship?
For each occupation, how many hours of RTI (i.e., classroom/web-based) are provided during the apprenticeship component? Over what period of time does RTI occur? Who provides RTI?
What is the typical schedule for an apprentice each week – e.g., 4 hours of formal classroom instruction and 36 hours of work)? Does this schedule change over the course of the apprenticeship?
Where and how is instruction provided (e.g., classroom, web-based, hybrid, laboratory/shop)?
Where is instruction provided (e.g., at a community college, at a union, at an employer site)?
Please provide a brief overview of the instruction provided (e.g., topics covered in classroom instruction). [Note: If applicable, request a syllabus for the coursework or a topic outline for the training.]
Are there innovative RTI methods being used as part of the apprenticeship initiative? Please briefly discuss (and include name of entity – community college, others as applicable). Possibilities might include:
Accelerated learning modules
Contextualized learning
Team teaching
Self-paced learning
Competency-based learning
Prior learning assessments
Technology-enabled learning, such as use of simulators, on-line teaching/learning
Integrating basic skills instruction with technical instruction
Has AAI funding been used for any of the following with respect to RTI, and if so, how:
Development of courses at post-secondary education level that are integrated into apprenticeship
Tuition and other educational fees
Delivery of instructional requirements (e.g., virtual learning technology, classroom instructors)
Cost of training facilities
Books, computers, etc.
While involved in RTI, how many hours of work at a job/employer site does an apprentice complete (i.e., each week, month, or year). Does this vary over time and, if so, how?
Where do apprentices work -- for which employers and in what types of jobs? For example, in what types of jobs do apprentices start and progress to during their involvement in the apprenticeship?
Who oversees and provide workplace/on-the-job instruction to the apprentice (i.e., while the apprentice is working)? Please briefly describe the nature of the workplace instruction provided to apprentices.
What is the supervisor-to-apprentice ratio during the apprenticeship period (e.g., 1 supervisor to 3 apprentices)? Does it change over time?
With regard to on-the-job learning, have AAI funding been used to offset overhead costs associated with any of the following (and if so, how): providing training, job shadowing, mentoring, and additional supervision?
Is the apprenticeship training connected to higher education? If so, which educational agency/institution(s) and how? Do AAI apprenticeship participants receive post-secondary (college) credits, credentials, or certificates? What are the credits, credentials, and/or certificates? What is the schedule for receipt of these credits, credentials, or certifications? Do apprentices need to pass assessments?
What is the schedule of wages and benefits for apprentices?
Are incentives or incentive payments used to influence better performance?
To what extent are non-training supports provided during the apprenticeship period (e.g., case management, career coaches/navigators counseling, and mentoring)? Who provides these services?
To what extent are supportive services provided such as transportation assistance, work clothes/equipment, childcare, and needs-based payments? If provided, how are these supportive services structured and paid for? Who provides these services?
What percentage of participants complete RTI? Are there certain types of apprentices that are more/less likely to complete RTI? By industry/occupation is there a notable difference?
What aspects, if any, of the apprenticeship RTI or services do you think are most innovative? Effective? Please explain.
F. AAI TRAINING PARTICIPATION LEVELS/CHARACTERISTICS AND EARLY OUTCOMES
[Note: Prior to site visit, collect available administrative data on participant enrollment levels, characteristics, and outcomes. If appropriate/feasible send data to grantee prior to the visit for review and comment.]
How many participants to date have been enrolled in training under the AAI grant? By industry/occupation, if applicable. [Note: Get information from QPR as appropriate.]
Do you anticipate achieving the AAI enrollment goal by the end of the grant (total and by industry/occupation)? If not, why not?
What are the main demographic and socio-economic characteristics of participants? What data, if any, is collected on apprentices other than what is entered in the QPR and how is it used? [Note: Get information from QPR as appropriate. If the grantee has its own pre-formatted report on participant characteristics, request a copy.]
4. How many participants enrolled in training have, to date, completed their apprenticeships? Have apprenticeships been completed within the expected apprenticeship period? Which industries/occupations? If applicable, is the program on track to achieve its overall goal for training completions? By industries/occupations? If not, why not? [Note: If available, collect any statistical report or tally of participant outcomes to date that the grantee produces.]
What number or proportion of apprentices cancelled? Have there been any distinguishing characteristics of those who cancelled? When does cancellation usually occur and why? In which industries/occupation(s) were they enrolled?
We would like to gather your views and any supporting data that you might have on the following outcomes for participants served to date. [Note: If available, prior to the visit, collect available administrative data on these measures; during the visit collect additional statistical reports or tallies of participant outcomes to date that the grantee produces.]
What kinds of credentials do pre-apprentices (if applicable) and apprentices obtain? For example:
Degrees (Associates, undergraduate)
Interim credentials (describe)
Adult basic education or GED (pre-apprentice only)
Occupational certificates
What occupations are apprenticeship completers entering? What are the average wage levels? What is the promotion potential? Are these occupations part of an established career ladder/pathway? Please obtain copy of ladder/pathway design.
Do completers stay with the same employer? Possible reasons do/don’t?
Has your program helped establish new pipelines of workers for the targeted industry/industries (including pipelines for youth)?
Are there ways in which the program has so far fallen short of its goals for training or assisting AAI participants? If yes, how? Why?
Are there ways in which the program has so far fallen short of its goals for engaging and serving employers (and/or sponsors)? If yes, how? Why?
Are there other approaches, strategies, or services that you believe would contribute to better outcomes for program participants?
G. AAI GRANT STAFFING AND COST/EXPENDITURES
What is the organizational structure for the AAI grant? Please describe AAI project staffing and management [Note: Request organizational chart and position descriptions for AAI grant.]
If available, number of AAI grant-funded staff by organization and position (i.e., number of full-time equivalent at the grantee and other subcontracted agencies)
What are the major ongoing costs/expenditures for the AAI program?
[Note: If available, collect line item budget and line item expenditure report, e.g., breaking down total expenditures under the AAI grant for items such as project staff, rent, equipment purchase or rental, subcontracts, etc.]
What has been the pattern of your grant expenditures? Have expenditures been higher during certain phases of the project? Has the project been fully implemented and reached a “steady-state” level of expenditure?
H. LESSONS LEARNED TO DATE
Have you developed sustainability plans for the apprenticeships implemented under the AAI grant? If so, please describe these plans. What sources of funding (including braided/blended, in-kind, etc. funding) will be used to sustain the apprenticeship activities implemented under the grant? Do you feel that your sustainability plan and leveraged resources are sufficient to sustain the activities of the grant after the completion of the grant?
To what extent do you think your program could and should be replicated in other localities? What features of your AAI program are most amenable to replication? What features of project are least amenable to replication? How does location, industry/occupation targeted, the target population served, or other distinctive features of your program make it either non-transferable or limit transferability?
To date, what do you consider to be your most important accomplishments under the AAI grant?
To date, what do you believe to be the main lessons learned from your AAI grant for apprenticeship program design and operations?
What innovations and lessons learned under the AAI grant can contribute to efforts to encourage more employers (and/or sponsors) to adopt apprenticeships?
What factors seem to explain success or lack of success for the apprenticeship program?
Any suggestions for increasing/enhancing engagement of/with the WDBs and/or AJCs? Federal and state OA? Economic Development Agency/Agencies?
CHECKLIST OF ITEMS TO COLLECT FROM SITE (IF AVAILABLE)
Background information about the grantee organization
Additional documentation detailing key components (e.g., RTI, on-the-job training or mentoring) Additional documentation detailing key services and components, such as course syllabi, course or training topics and competencies, college and career navigation, etc., and credits and credentials included in design, career pathway maps
Evaluation reports that may have been prepared on the program
Blank set of intake forms
Diagram showing how participants flow through the program
Organizational chart for the program
Occupational standards?
Participant recruitment materials
Employer recruitment materials
SUPPLEMENTAL DISCUSSION GUIDE:
PRE-APPRENTICESHIP TRAINING PROGRAM COMPONENT
[Note: Omit this supplement if the AAI grant does not fund a pre-apprenticeship program component. If more than one pre-apprenticeship program component exists or training is provided for more than one type of occupation, where appropriate, indicate differences across programs/industry sectors/occupations on each of the questions.]
If not already discussed, for which industries and occupations are pre-apprenticeship training provided? Why were these industries/occupations targeted?
What is your goal for the number pre-apprentices enrolled under the AAI grant (total and by industry/occupation)? [Note: gather from grant application or other program documents and confirm with grantee.]
How do you market your program to prospective pre-apprenticeship participants? Possibilities include:
Distribution of flyers, posters
AAI program staff outreach presentations or orientations held in the area served (estimate number of presentations/orientations provided)
Informational websites
Toll-free informational hotlines
Outreach campaigns using media (e.g., PSAs, TV, radio, newspaper, ads on buses/bus shelters)
Social media (e.g., LinkedIn, Facebook, Twitter, Instagram, etc.) – please specify, which types of social media are used and how
Word-of-mouth
Primary partners
What populations and/or subpopulations has the project targeted for special outreach to the pre-apprenticeship program? Why have these populations/subpopulations been selected? As the AAI initiative has progressed, have there been any changes in special populations targeted? If so, what were they? Why? Some possible special target groups include:
Incumbent workers
New entrants to the labor force
Long-term unemployed
Low income
Populations that may traditionally be underrepresented in apprenticeship, including:
Women
Young men and women of color
Individuals with disabilities
Low-skilled populations
Veterans, including transitioning service members
Out-of-School Youth
Ex-offenders
What organizations provide referrals to the AAI-funded pre-apprenticeship program? Possibilities include:
Employers
Unions
Workforce system – American Job Centers, Employment Service, and WIOA
Local Education Agencies (LEAs), such as the local public school system
Re-engagement centers (i.e., to link youth that have dropped out of school back to school or other types of training)
Community colleges and other educational institutions including proprietary schools
Courts/correctional system/juvenile justice/probation officers
Faith-based and community-based organizations
Other public and non-profit human service agencies
Which of those organizations have provided the most referrals? Have any organizations fallen short of original expectation for providing referrals? Which of those organizations are also sponsors?
What has been the response of the targeted population(s) to the initiative?
Are you on pace to recruit the intended number of pre-apprenticeship participants? If not, why?
Has the pool of potential applicants for the program been sufficient?
Have there been more applicants than the program can serve (e.g., is there a waiting list to get into the program)? Which program?
How successful has the AAI grant been in recruiting individuals from groups that do not typically enroll in apprenticeship/pre-apprenticeship programs? Which groups have you been successful with and with which groups have you been unsuccessful? Possible reasons for why?
Have there been recruitment challenges? If so, what challenges have been encountered and how have they been addressed? Some possible challenges include:
Difficulty finding eligible participants
Many applicants have insufficient basic skill levels
Apprenticeship period is too lengthy for some applicants
Some of the outreach strategies haven’t resulted in many applicants
Partners that were supposed to provided referrals have not provided them in sufficient numbers
Some applicants have difficulty getting to the program/employer location
Didn’t have enough resources for recruitment
Changing economic conditions in the areas has made recruitment more challenging
Other, please specify.
Overall, what recruitment or referral strategies being used under the AAI pre-apprenticeship program have been most successful? Do you feel any of these strategies are new, unique and/or innovative?
What are the main steps involved in the AAI pre-apprenticeship intake process? Please provide an overview of the flow of new recruits through the intake process, including:
What are the eligibility requirements to enroll in the pre-apprenticeship component funded under the AAI grant? [Note: Request copies of AAI participant eligibility or intake form(s).] Possibilities include:
Complete pre-apprenticeship program (i.e., for apprenticeship program component)
Be from a specific subpopulation
Reside within a specific geographic area
Meet income eligibility requirements (and what are these requirements)
Meet education level requirements (e.g. high school diploma, GED/HSED) or basic skill levels, or complete an assessment?
Who determines eligibility to participate in the AAI grant?
Is there any involvement of employers (and/or sponsors) in the selection process for the pre-apprenticeship program component funded under AAI? Who else is/What other partners or entities are included in the assessment process? Please describe this process including (if applicable):
How does the employer (and/or sponsor) determine whether an applicant is the right fit?
About what percentage of applicants are rejected (e.g., screened out prior to being sent for employer interviews)?
What are the main reasons that applicants are screened out or fail to move through the intake process to be considered by employers?
During the intake process, are any formal assessment tests used by employers, sponsors, or the AAI grantee for pre-apprenticeship participants (including formal tests such as the TABE, interest inventories, substance abuse screening)? [Note: Request copies of assessment forms, if relevant.]
Where and how is training delivered under the pre-apprenticeship component (e.g., classroom, laboratory/shop, employer workplace, via web-based instruction)? Who are the training providers and what is the role of each provider?
Are both classroom instruction and workplace activities incorporated into the pre-apprenticeship program? If so, what is the balance between these two types of activities? If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
Is the pre-apprenticeship program time-based (e.g., completion of specific numbers of hours of training) or competency-based?
What is the usual duration of participation in the pre-apprenticeship component? Is there variation in how long it takes an AAI participant to complete the pre-apprenticeship component? If yes, what is the range (in weeks or months)?
If competency-based, how is it determined that a participant has achieved the necessary levels of competence to complete the pre-apprenticeship component?
If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
How many hours of formal instruction/training (i.e., classroom/web-based) are provided during the pre-apprenticeship component? Over what period of time does formal instruction occur? If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
What is the typical schedule for a pre-apprentice each week – e.g., 16 hours of formal classroom instruction and 20 hours of work experience/internship at a job site)?
How is instruction provided (e.g., classroom, web-based, hybrid)?
Where is instruction provided (e.g., at a community college, at a union, at an employer site)?
Please provide a brief overview of the types and duration of instruction provided (e.g., topics covered in classroom instruction). [Note: If applicable, request a syllabus for the coursework or a topic outline for the training.]
If applicable, as part of the pre-apprenticeship program, are participants also engaged in work experience or an internship at an employer job site? When and how many hours of work experience/internship are involved? Please briefly discuss who sponsors the work experience/internships, whether and how much participants are paid, and the nature of these activities and how they might be connect to formal classroom instruction. If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
Are there innovative instructional methods being used as part of the pre-apprenticeship initiative? Please briefly discuss and name provider. Possibilities might include:
Accelerated learning modules
Contextualized learning
Team teaching
Self-paced learning
Competency-based learning
Prior learning assessments
Technology-enabled learning, such as use of simulators, on-line teaching/learning
Is the pre-apprenticeship training connected to higher education? If so, what organization/institution and how? Do AAI pre-apprenticeship participants receive post-secondary (college) credits, credentials, and/or certificates? Please specify types of credits/credentials/certificates received. If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
Is the pre-apprenticeship training connected to other apprenticeship program(s)? If so, which ones and how? If relevant, are there apprenticeship slots available for all pre-apprentices who successfully complete the pre-apprenticeship program?
To what extent are non-training supports provided, such as case management, career coaches/navigators counseling, or mentoring? If provided, by who and how are these supports structured? If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
To what extent are supportive services provided such as transportation assistance, childcare, work clothing/equipment, or needs-based payments? If provided, who provides them/each and how are these supportive services structured and paid for? If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
What, if any, post-completion services are provided to participants of the AAI-funded pre-apprenticeship program? If applicable, who provides them/each and discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided. Possibilities include:
Referral to apprenticeship programs
Referral to other education or training programs
Help with paying for certification exams or obtaining credentialing
Job search or placement assistance
Job retention services
Provision or referral to supportive services
What percentage of AAI participants complete/have completed pre-apprenticeship training? [Note: If available, collect this data from administrator sources prior to the visit or request data on attrition from the grantee.]
Is attrition greater than expected? If so when and why does attrition typically occur?
Are there certain types of participants that are more/less likely to complete pre-apprenticeship training?
If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
What percentage of AAI pre-apprenticeship completers enter/have entered apprenticeship programs? [Note: If available, collect this data from administrator sources prior to the visit or request data on attrition from the grantee.]
Are there factors that help explain the success or lack of success?
If applicable, discuss differences across industry sectors/occupations for which AAI pre-apprenticeship training is being provided.
What (if any) training participation and performance benchmarks are used for the AAI-funded pre-apprenticeship program?
Are incentives or incentive payments used to influence better performance?
Were any pre-apprenticeship training program components or service elements of the original program design not implemented or abandoned early on? If yes, which ones and why?
Has the grantee faced any challenges in establishing the pre-apprenticeship training component? How have these challenges been addressed?
What aspects, if any, of the pre-apprenticeship program component do you feel are most innovative? Effective? Replicable?
What factors seem to explain success or lack of success for the pre-apprenticeship program?
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | ATTACHMENT 7 |
| Author | admined |
| File Modified | 0000-00-00 |
| File Created | 2021-01-21 |