America’s Promise Employer Partner Protocol MATHEMATICA POLICY RESEARCH
America’s Promise Jobs-Driven Grants Evaluation: Business/Industry Representative Partner Protocol
Good afternoon. My name is ___________________, and this is my colleague _________________ and we are from Mathematica Policy Research/Social Policy Research Associates. We are part of an independent research team that is studying the services being provided through [Program Name] as part of the America’s Promise Job Driven Grants. The study is on behalf of the U.S. Department of Labor. The department wants to learn how businesses and industry representatives are working with other partners in the region to support job training to meet the needs of their industries and whether such efforts help participants learn skills in careers that offer career growth and advancement. Thank you very much for agreeing to participate in this conversation. Your participation is very important to the study. We value the information you will share with us and want to make sure we capture it all. We are recording our discussion today to help us remember what everyone said when we go back to write our report. Although we will be taping the session and taking notes, we will not use names or attribute information to any particular individual, and we will report our findings at an aggregate level. This recording is only listened to by our study team and will be erased at the conclusion of our study.
Before we begin, I just want to clarify that the America’s Promise program at this site is called [PROGRAM NAME]. If I reference “America’s Promise” at any point in the interview, I am referring to [PROGRAM NAME].
Are you okay with our recording the conversation?
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 60 minutes per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is voluntary. Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the U.S. Department of Labor, Office of the Chief Information Officer, Attention: Departmental Clearance Officer, 200 Constitution Avenue, N.W., Room N-1301, Washington, DC 20210 or email [email protected] and reference the OMB Control Number 12XX-XXXX. Note: Please do not return the completed form to this address. |
Respondent name:
Name and type of organization/industry group:
Title and role in organization:
America’s Promise Program Partner/PROGRAM NAME
Before we get started discussing [PROGRAM NAME], can you give me a 1 minute description of your organization (or industry group)? What sector are you in? What services or products do you provide?
[If business] What types and how many customers/clients do you have? How many employees do you have? How many locations?
[If an industry group] What types and how many businesses are in your group? How many employees do they have on average? Where are they located?
I’d also like to hear more about the local context in which your organization operates and learn more about your reasons for becoming involved with [PROGRAM NAME].
Local economic context.
Can you describe the local economic environment in which your organization (or industry group) operates? Is your industry expanding or declining in your region? (RQ4, RQ5, RQ6)
How has COVID-19 affected your industry?
What specific occupations are in high demand for your industry? (RQ4)
What are the biggest challenges and opportunities for businesses right now? What are the primary needs of businesses? (RQ1, RQ5, RQ6)
[IF NOT ADDRESSED] What challenges do businesses face in recruiting and retaining workers? (RQ1, RQ4, RQ5)
Role in the program and reason for participating.
How did you become involved in [PROGRAM NAME]? (RQ7)
Were you involved in the decision to apply for the grant? (RQ7, RQ8)
To what extent were you involved in the grant application process, beyond providing a letter of support? (RQ7, RQ8)
Why did you decide to participate? What did you hope to accomplish through your involvement? (RQ7) Has COVID-19 changed your training needs or your role in the program?
What is your specific role in the program? For example, are you part of an advisory group? Do you help develop industry-specific curricula? Do you provide work-based learning or opportunities for work experience? Do you serve as a job placement site? Or do you fulfill some other role? (RQ7, RQ16)
What specific services do you provide participants? Please describe. (RQ16)
Did you have a preexisting relationship with [LEAD ORGANIZATION] or any of its partners? (RQ7, RQ8, RQ11)
If yes, how long have you partnered with the [LEAD ORGANIZATION]? How long have you partnered with [IDENTIFIED PARTNERS]? Can you describe your prior partnership with the grantee agency and partners, and the purpose? How has the partnership evolved over time? (RQ7, RQ8, RQ11)
Once the lead organization won the grant, were you involved in planning for its implementation? If so, what role did you play in planning efforts (e.g. help deciding on training needs, participate in advisory group, help developing training content)? (RQ16)
How often and in what ways did you interact with regional partners during this planning process? (RQ8, RQ16)
To what extent did you help shape the program components (e.g., identifying target industries, developing career pathways)? (RQ16)
Do you have other decision-making roles in [PROGRAM NAME] (for example, serving as an advisor on employment services, advisory board member, hiring participants, conducting credential certifications)? Explain. (RQ8, RQ16)
How does [PROGRAM NAME] interact with and complement other services available to your organization (or industry group) through the public workforce system, such as at your local American Job Center? (RQ2, RQ5)
Has your organization participated before in on-the-job training, apprenticeships, or other work-based learning activities with the workforce system? (RQ2, RQ5)
Have you screened applicants or hired workers through the workforce system? (RQ2, RQ5)
Specifics of partnership arrangement
Now we’d like to talk about the agreements that you have with [LEAD ORGANIZATION] and how your partnership operates.
Description of partnerships.
Do you have a formal agreement with the lead organization or any of its partners? (RQ7)
If yes, what form is the agreement (e.g. MOU, subgrant, letter of authorization)? What are the terms? Request a copy. (RQ7)
How useful is the MOU/subgrant/LOA in guiding your roles and responsibilities in [PROGRAM NAME]? (RQ7)
What are the terms of the arrangement (whether formal or informal)?
Specified the provision of specific services? (RQ7, RQ15)
Established a number of participants to be served? (RQ7, RQ15)
Allowing case managers to visit? (RQ17)
Provide coaching? (RQ17)
Identify credential that will be earned? (RQ17)
Monitor progress or allow someone else to? (RQ17)
Serve as employer of record? (RQ17)
Included data sharing provisions? (RQ7, RQ15)
Set expected outcomes? (RQ7, RQ15)
Included an agreement for permanent hiring of individuals who complete training? (RQ7, RQ15)
Opportunities for communication, feedback, and collaboration.
How does your partnership operate? With whom do you typically communicate? (RQ8)
In what ways do partners collaborate with each other (e.g., frequent meetings, soliciting ideas for shared vision)? (RQ8)
What are the key issues discussed at partner meetings that affect the program and its participants? (RQ8)
Does the [LEAD ORGANIZATION/OTHER INVOLVED ORGANIZATIONS] solicit your feedback or do anything else to help them improve their ability to meet your needs? (RQ8)
If yes, please describe. (RQ8)
If not, is that type of feedback process needed? If so, what suggestions do you have for improving the feedback process? (RQ8)
Next, we’ll discuss the types of training, education, and services available through [PROGRAM NAME] that are most-relevant to your industry and organization.
Necessary training for employment.
For the occupations in demand at your organization (or in your industry), what type of education, training, and background is necessary? (RQ5, RQ16)
Are certain types of certificates or degrees required for these occupations? If so, which types are needed for each occupation? (RQ5, RQ16)
Work-based learning.
As we discussed already, you indicated that your organization (or your industry) participates in [ADD HERE BASED ON ABOVE]. (RQ15, RQ17)
Work experience opportunities. (RQ15, RQ17)
Internships (paid, unpaid, duration, intensity, virtual). (RQ15, RQ17)
Apprenticeships. (RQ15, RQ17)
On-the-job training. (RQ15, RQ17)
How do you identify potential work-based learning opportunities within your organization (or your industry group)? (RQ15, RQ16)
Do these opportunities need to be approved for the program? If so, how does the approval process work? (RQ15)
Can you describe how the work-based learning is structured? (RQ15)
What positions are included? (RQ15)
What activities or experiences do participants engage in during work-based learning at your organization? (RQ15)
How many weeks is the work-based learning? How many hours per week? (RQ15)
Can participants earn industry-recognized credentials? (RQ20)
Is the training coordinated with classroom instruction? (RQ17)
Are participants paid during work-based learning? (RQ15, RQ17, RQ18)
If so, how much? (RQ15, RQ17, RQ18)
How are these payments funded? Is payment made through the grant or through the employer? Who is the employer of record?(RQ15, RQ17, RQ18)
How are participants monitored, supervised or given feedback? How are supervisors selected for work-based learning participants? What training, if any, do supervisors receive? (RQ15, RQ17, RQ18)
How many participants have participated in work-based learning (WBL) or work readiness training at your organization (or your industry group) through this program? (RQ17, RQ19)
Are there eligibility criteria for participating in work-based learning? (RQ23)
If yes, what are they (e.g. education level, prior work experience, other)? How were they determined? Were they determined by the grant? Does your organization (or industry group) have additional criteria beyond those specified for the grant? (RQ23)
How do you assess eligibility? For example, do you conduct interviews with referred candidates? (RQ24)
What specific supports do participants need to succeed in the work-based learning opportunities? To what extent do you provide these supports? If not you, are they provided by other partners? (RQ15, RQ18, RQ25)
What skills are important for participants to have to be successful in WBL? Do you train them in these skills? (RQ19)
How do you coordinate WBL opportunities with program partners? (RQ8, RQ9, RQ16)
What support does your organization (or industry group) provide to participants engaged in WBL? Such as transportation, child care, support for tools or uniforms. (RQ18, RQ21)
What training or support, if any, have you received from the [PROGRAM NAME] program staff for your work with participants? For example, in establishing expectations, working with participants, reporting back, etc. (RQ9)
Are there any additional supports that you would like to receive but have not? (RQ9)
How do you communicate with the [PROGRAM NAME] program when participants are engaged in WBL? What issues do you work with [PROGRAM NAME] program staff to address? (RQ8, RQ9)
What challenges have you encountered, if any, participating in WBL? (RQ8, RQ9)
What proportion of individuals complete work-based learning? What are common reasons that some do not complete? Are there common characteristics of those who succeed? (RQ19)
Incumbent worker training.
Does your organization (or industry group) facilitate training for incumbent/existing workers through [PROGRAM NAME]? (RQ15)
If yes, how many individuals have had incumbent worker training? (RQ15, RQ17)
What occupations are they being trained for? (RQ6, RQ15)
How are individuals selected for incumbent worker training? (RQ23, RQ24)
What are their education levels and prior work experience? (RQ23, RQ26)
How do you assess eligibility? For example, do you use supervisor referrals or conduct interviews with candidates who express interest? (RQ24)
Do you have agreements with participants that outline expectations, including advancement or training credentials once training is completed?
Can you describe how the incumbent worker training is structured?
What types of training do incumbent workers typically receive? (RQ15, RQ16)
Do participants acquire the skills required to be successful in your field? What skills are they acquiring? (RQ20)
Who provides the training? (RQ16)
What are common activities that occur as part of the incumbent worker training? (RQ15)
How are participants supervised or given feedback? (RQ15)
How many weeks is the training? How many hours per week? (RQ15)
Are participants paid the same wages/salary as before selected for training? If so, what are those typical wages/salaries? (RQ15, RQ18)
What proportion of individuals complete incumbent worker training? What are common reasons that some do not complete? Among those who complete training, are they eligible for higher-skilled and higher-paid jobs? Do they earn credentials in line with industry needs? (RQ17)
Hiring of [PROGRAM NAME] participants. (RQ17)
Among those that participated in training at your organization (industry members), have you permanently hired/promoted some or all of those who completed? (RQ20)
If yes, how many have you hired? How many have you promoted? What occupations do they fill? Are they part-time or full-time? What are typical wages for these positions? (RQ20)
If some were not hired permanent/retained, what were the reasons? (RQ20)
Has your organization (or has your industry) offered permanent job placements to individuals who were trained through other organizations as part of [PROGRAM NAME]? Did you plan to? (RQ20)
If not, what are reasons why you have chosen not to hire these individuals? (RQ20)
If yes, how many have you hired? What occupations do they fill? Are they part-time or full-time? What are typical wages for these positions? (RQ20)
Do the training and skills of these [PROGRAM NAME] participants align with your needs as a business? If yes, in what ways do they align with your needs? If not, can you describe why they don’t align with your needs? (RQ20)
How does the training provided through [PROGRAM NAME] compare to the training typically received by individuals in your workforce or industry? In what ways is it the same? In what ways is it different? How do the skills of [PROGRAM NAME] participants compare to individuals who seek employment who are not part of the program? (RQ2, RQ20)
Financial support and data sharing.
Does your organization receive any grant funds for your involvement in [PROGRAM NAME]? How much and for what activities? (RQ15)
If individuals are paid during work-based learning or incumbent worker training, who pays those wages? Are they paid by the employer, the grant, or a combination? (RQ15)
Does your organization contribute any other resources (either direct or in-kind) for program activities? In-kind resources may include volunteers, equipment, or other contributions. Please describe. (RQ15)
What data do you share with the grantee or other partner organizations on [PROGRAM NAME] participants (such as attendance, performance)? (RQ9)
How often? (RQ9)
In what format? (RQ9)
Have there been any challenges with data sharing? (RQ9)
Now, I’d like to discuss the individuals involved in [PROGRAM NAME] services, including program eligibility requirements, the process for recruitment, and individual characteristics of program participants.
Recruitment/referrals for training/employment.
How are individuals recruited or referred to your organization (or industry group) for employment/internships/apprenticeships? (RQ24)
[IF BUSINESS REFERS EXISTING EMPLOYEES TO PARTICIPATE IN the program]: How do you determine which employees to refer to receive services through [PROGRAM NAME]? (RQ23, RQ24)
How satisfied are you with the individuals referred to your organization through [PROGRAM NAME]? What criteria have you established for eligibility? Has that worked? Why or why not? (RQ22, RQ26)
To wrap up, we’d like to talk about your experience overall to understand any strengths and weaknesses associated with [PROGRAM NAME] and its services.
Changes over time.
Have there been any changes over time in your partnership with [LEAD ORGANIZATION] or your involvement in the [PROGRAM NAME] program during the grant period? If yes, please describe. (RQ11)
Have there been any changes over time in your partnership with other organizations involved in [PROGRAM NAME] during the grant period? If yes, please describe. (RQ11)
Challenges.
Have you encountered any challenges in your partnership with [LEAD ORGANIZATION/OTHER INVOLVED ORGANIZATIONS]? For example, time requirement, conflict between partners, data-sharing issues? If yes, what were they and how did you address them? (RQ12)
Have you experienced any other challenges as a result of your involvement with [PROGRAM NAME]? If yes, what were they and how did you address them? (RQ12)
Effectiveness of [PROGRAM NAME].
To what degree has the America’s Promise grant been helpful to your organization (or industry group) in finding, retaining, or upgrading employees? (RQ22)
What value do you think [PROGRAM NAME] brings to the community and the region? (RQ2)
What effect do you think [PROGRAM NAME] has had on your organization (or industry group)? (RQ6)
What effect do you think [PROGRAM NAME] has had on your industry? (RQ6)
What effect do you think [PROGRAM NAME] has had on regional partnerships? (RQ1, RQ6)
What effect do you think the program has had on participants and the local labor market? (RQ4) (RQ20)
Sustainability.
What interest does your organization/company have in sustaining your partnership with [LEAD ORGANIZATION] and other regional partners beyond the grant period? (RQ14)
[IF THERE IS INTEREST] What elements of your partnership will likely continue beyond the life of the grant? (RQ14)
What obstacles do you foresee to successfully extend the partnership beyond the grant period? (RQ14)
Are there any potential funding sources identified that could be used to support a similar level of education, training, and workforce services currently available through [PROGRAM NAME]? (RQ14)
[IF NO INTEREST] Why are you not interested in continuing the partnerships? What aspects of the partnership would need to change for you to maintain interest? (RQ14)
Strengths and weaknesses of the services.
What are the most successful aspects of service provision through [PROGRAM NAME]? (RQ22)
If you could change three things about the services, what would those be? (RQ21)
Do you have any recommendations that can help improve the services available through [PROGRAM NAME] in the future? (RQ21)
Strengths and weaknesses of the partnerships.
What are the things that work well with the partnerships established through [PROGRAM NAME]? (RQ10, RQ13)
If you could change three things about the partnerships, what would those be? (RQ12)
Do you have any recommendations that can help improve the partnerships established through [PROGRAM NAME] in the future? (RQ12, RQ13)
Final thoughts.
Is there anything else that we haven’t discussed that you’d like to say about [PROGRAM NAME]? (RQ1)
That was all the questions that we had for you. Do you have any questions for us?
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Brittany English |
| File Modified | 0000-00-00 |
| File Created | 2021-01-14 |