Participant focus groups

America’s Promise Participant Focus Group Protocol MATHEMATICA POLICY RESEARCH

^{America’s Promise Jobs-Driven Grants Evaluation: Participant Focus Group Protocol}

Instructions for the Site Visit Team

Prior to going on the site visit, ask your key contact to identify individuals to participate in the focus group. Ideally, this should include individuals currently in services as well as those who have completed and found employment. To ensure a group of 5 to 6 individuals, grantees may need to invite up to 10-12 participants.

Setup Activities (10 min)

• As participants enter the room, ask them their name and write it next to one of the IDs in your list of MPRIDs, as well as on a name tag. Hand them the consent form that corresponds to that ID as well as their name tag and ask them to take a seat.

• Cross out the IDs in your list that were not used.

Introduction (5 minutes)

Good afternoon. My name is ___________________, and this is my colleague _________________ and we are from Mathematica Policy Research/Social Policy Research Associates. We are part of an independent research team that is studying the services being provided through [Program Name] as part of the America’s Promise Job Driven Grants. The study is on behalf of the U.S. Department of Labor. The department wants to learn whether programs like this help people learn skills in careers that offer career growth and advancement. Thank you very much for agreeing to participate in this conversation. Your participation is very important to the study.

To help us better understand how [Program Name] is working, we would like to ask you some questions about your experiences. We are visiting 12 sites similar to yours across the country and holding similar discussions. No one outside of the research team will have access to our notes. We will not share any information you provide with staff in the [PROGRAM NAME], or any employers, and our reports will never identify you by name. Instead, we will combine information from this discussion with information from discussion groups in other states.

To participate in the focus group today, we need to document your consent to participate in our research study. I will read the consent form that we passed out as you arrived, and then ask that each of you sign the form if you agree to participate. [Read the form, verbatim, out loud. Then gather signatures.]

We value the information you will share with us and want to make sure we capture it all. We are recording our discussion today to help us remember what everyone said when we go back to write our report. Although we will be taping the session and taking notes, we will not use names in our reports or attribute anything to a specific individual. We will report our findings at an aggregate level. This recording is only listened to by our study team and will be erased at the conclusion of the study.

Is everyone okay with our recording the conversation? Does anyone object?

I hope you will feel free to be open and frank in our discussion. There are no right or wrong answers to the questions we will ask. I ask that none of you share what you hear with others outside the group so that all of you are more comfortable and candid in your responses. It will also help us if you speak clearly and one at a time. I will try to be mindful of time, and might have to move the conversation along to make sure we can cover all of the topics. We have about one hour for our discussion. Then I will ask you to complete a brief form about yourself before we conclude and provide you with a $25 gift card to show our appreciation for your feedback. Do you have any questions before we begin?

Before we begin, I just want to clarify that the America’s Promise program at this site is called [PROGRAM NAME]. If I reference “America’s Promise” at any point in the interview, I am referring to [PROGRAM NAME].

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 90 minutes per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is voluntary. Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the U.S. Department of Labor, Office of the Chief Information Officer, Attention: Departmental Clearance Officer, 200 Constitution Avenue, N.W., Room N-1301, Washington, DC 20210 or email [email protected] and reference the OMB Control Number 12XX-XXXX. Note: Please do not return the completed form to this address.

Participant Background (5 minutes)

1. Let’s begin by getting to know a bit about your background. Let’s start to my right. Please tell me: (RQ26)

• your first name, (RQ26)

• how long have you been participating in [PROGRAM NAME] (RQ26)

• a sentence or two about your work experience prior to participating. (RQ26)

Motivation for Participating in Americas Promise (5 min)

1. Knowledge of program. How did you hear about [PROGRAM NAME]? (RQ24)

   • Did you seek out this program or were you recruited to participate? If recruited, who recruited you (for example, by your employer)? (RQ24)

2. Reason for participating. Why did you start participating in [PROGRAM NAME] services?

   • What were you doing before deciding to participate? (RQ2, RQ5, RQ6)

   • Was there one thing that attracted you to the program – e.g., work in the particular industry or occupation; the supports provided; something else? (RQ17, RQ24)

3. Participant goals. What did you hope to accomplish by participating? (RQ17)

Recruitment and intake processes (5 min)

4. Application for services. How did you apply for services available through America’s Promise? (RQ23, RQ24)

   • Can you walk me through the process that you went through to enroll in training/services through [PROGRAM NAME]? Did you enroll in [PROGRAM NAME] at [state/local name of AJC], community college, or through some other organization? (RQ24)

   • Did anyone talk to you about the education, training, and other supportive services available through [PROGRAM NAME]? (RQ24)

   • Did you have to complete an application? (RQ23, RQ24)

     1. If yes, what did the application process entail? Did you find it easy or hard to go through? How long did it take before you heard back regarding your application? (RQ23, RQ24)

     2. If not, can you describe the process of becoming enrolled? (RQ23, RQ24)

Types of training, education, and supportive services (15 min)

Now we’d like to learn more about the training, education, and other services you’ve received as part of [PROGRAM NAME].

5. Assessments. Did you have to take any tests or answer any questions-- such as an interest inventory or an assessment test-- to determine what services would be most appropriate for you? Were you required to take (a) any assessment tests, (b) basic skills tests, (c) advanced skills tests or exams, or (d) prerequisite courses? (RQ15, RQ17)

   • If yes, did you have any issues or concerns with the assessments? (RQ17, RQ18)

6. Service planning. How was it determined which training, education, or workforce services you would participate in?

   • Did you have a choice? If yes, how did you decide? What staff helped you decide? (RQ17)

   • Did staff develop an employment or training plan with you? If so, which staff? Is this plan something you refer to? Did you have any concerns with the service plan established for you? Do you think it accurately reflects your skills and interests? (RQ17, RQ18)

7. Type of training, education, and workforce services. In what kind of training, education, or workforce services are you participating or did you participate in as part of [PROGRAM NAME]? If necessary, probe about services provided through [PROGRAM NAME]. Probes: (RQ15)

Case management and career counseling (i.e., staff person who provides 1:1 assistance)

Information about the job market, such as what types of jobs are available and what they require, or what careers you could go into and what they pay

Job readiness or soft skills training

Remediation or GED preparation

Classroom training, on-the-job training, work internships, apprenticeships

Education

Job search and placement assistance, such as resume assistance or interview coaching

Job retention services

• Where did you have to go to participate in each of these? Were they easy or hard to get to? (RQ18, RQ19)

• For those who participated in classroom and work-based learning activities, how well aligned were these? (RQ17, RQ18, RQ19)

• For those in work-based learning, how good of a match was the placement with your skills and interest? (RQ17, RQ18, RQ19)

• How well do you think each services is/was preparing you for employment? (RQ22)

• Will/did you earn a credential? (RQ17)

• Did the availability of a credential factor in your decision to participate? What other reasons did you consider when deciding to participate? (RQ18) Probes:

Career advancement required credential

Personal interest in furthering education

Higher wages

Employer requirement

8. Other supportive services. Have you received any supportive services, referrals for these services, or assistance as part of your involvement with [PROGRAM NAME], either from [MAIN ORGANIZATION] or [OTHER SERVICE PARTNERS]? Where did you have to go to receive the service? (RQ18, RQ19)

If necessary, probe about supportive services provided through [PROGRAM NAME]. Probes:

Assistance with child care

Access to public benefits

Transportation assistance

Specialized services to accommodate disabilities

Payment for work supplies, uniforms, tools, books

9. Staff and partner contacts. What organization(s) or individual(s) do/did you primarily interact with? (RQ16)

• Do you have a case manager or career coach that you interact with regularly? How often did you meet or receive services from your case manager or career coach? (RQ16, RQ17)

• Do you receive services from staff at multiple organizations? If yes, which staff and from what organizations did you interact with? How were you connected to resources or supports from these various organizations? Was it convenient or not to have contact with multiple organizations? (RQ15, RQ16)

10. Financial support. Were you paid any wages or incentives while participating in training or other services? Who paid you? How much were you paid? How was the amount determined? Did that financial support meet your needs? (RQ15, RQ18)

11. Fees. Did you have to pay any fees to participate in any of the services you are receiving through [PROGRAM NAME]? For credential testing or attainment? (RQ18)

12. Unmet needs. Do you need any additional assistance, services, or training beyond what is available or been offered to you? If so, what types of services? Why have you been unable to receive them? (RQ18, RQ19)

Service alternatives (5 min)

Next, we’d like to discuss other services in your community, if any, that are similar to [PROGRAM NAME].

1. Other options. If you did not have the option of participating in this training and/or services, what else might you have done to advance your career? (RQ2)

2. Knowledge of other available services. Do you know of other programs or services similar to [PROGRAM NAME] that are available in your community? (RQ2)

• If yes, who provides them? What is your perception of how the services available through [PROGRAM NAME] compare to other available services? (RQ22)

Anticipated impact of [PROGRAM NAME] services (5 min)

3. [FOR THOSE WHO ARE RECEIVING SERVICES] Do you think that [PROGRAM NAME] will help [or has helped] you in achieving your career goals? How? What do you think [PROGRAM NAME] will offer you that other programs can’t? Do you think you will get a permanent placement at your work-based learning employer? (RQ20, RQ22)

4. [FOR THOSE WHO HAVE COMPLETED SERVICES] Have you been able to make progress towards your career goals as a result of participating in [PROGRAM NAME]? Have you been able to find a good job? (RQ20)

• If yes, what kind of job? What do you do in the job? Is it full- or part-time? In what industry? Is the industry or occupation related to the services or training you received through [PROGRAM NAME]? How important were the services you received through [PROGRAM NAME] in finding that job? (RQ20)

• If no, are you looking for a job? (RQ20)

• [IF CURRENTLY SEEKING EMPLOYMENT] What challenges have you experienced in finding a job? Are you still receiving support to help find a job? (RQ21) Are you working with any staff who are helping your with job placement? (RQ20)

Experiences with America’s Promise (15 min)

To wrap up, we’d like to talk about your experience overall to understand any strengths and weaknesses associated with the [PROGRAM NAME] services you received.

5. Ease of service receipt. How easy was it for you to participate in the training, education, or supportive services offered through [PROGRAM NAME]? (RQ19)

• Was the timing of the training, education, and other services convenient? Was the location of the training, education, and other services convenient? Were your case managers/career counselors/workshop facilitators easily accessible? (RQ18, RQ19, RQ21)

• Did you experience any challenges in accessing services? If so, please describe. (RQ21)

• Has it been easy/hard to stick with the program? Why? (RQ21, RQ22)

6. Strengths of the program. What do you think has been the most helpful or strongest aspect of [PROGRAM NAME]? (that is, case management vs. job readiness vs. vocational training vs. supports) Were your expectations met? (RQ22)

7. Areas for improvement. Do you have any recommendations that can help improve the services available through [PROGRAM NAME] in the future? (RQ21)

• Did you hope to learn certain skills that aren’t being taught? (RQ19, RQ21

• Are there other services you wish had been provided? (RQ19, RQ21)

• Do you feel the training, classroom services were well taught? Were the instructors, supervisors concerned about you and your career path? (RQ18, RQ21, RQ22)

8. Would you recommend these services to others? Why or why not? (RQ18)

Administer Participant Information Forms and Gift Cards (15 minutes)

• Provide each person with a blank participant information form that has their MPRID at the top. Ask them to complete the form, and tell them that information in the form will be kept private and separate from their name, and filling it out is completely voluntary and will not affect their $25 gift card for participating in the focus group (read PIF consent script).

• If anyone declines to fill out the form, write “DECLINE” at the top of the form and keep it separate from the unused forms. Hand them a gift card and a payment receipt record, and ask them to print and sign their name on the record.

• When participants hand back a completed form, hand them a gift card and a payment receipt record, and ask them to print and sign their name on the record.

5