Instrument 17: Participant interview guide

S upporting Statement for OMB Clearance Request

Instrument 17: HPOG 2.0 National Evaluation Participant In-depth Interview Guide

National and Tribal Evaluation of the 2^nd Generation of the Health Profession Opportunity Grants (HPOG)

0970-0462

April 2019

Revised July 2019

Submitted by:

Office of Planning,
Research & Evaluation

Administration for Children & Families

U.S. Department of Health
and Human Services

Federal Project Officers:

Nicole Constance, Hilary Bruck, and Amelia Popham

Program Name: [NAME OF PROGRAM]

Thank you for taking the time to speak with me today. My name is _____ and I work for a research company called Abt Associates. Abt has a contract with the U.S. Department of Health and Human Services to evaluate the Health Profession Opportunity Grants (HPOG) Program. As part of our evaluation, we want to learn more about what the program is like for program participants. As you may know, HPOG funds [NAME OF LOCAL PROGRAM] and other, similar programs across the country. HPOG aims to provide training for jobs in healthcare. We want to hear from people like you about your experiences in HPOG and your ideas for ways to improve the program.

The information that you share during this conversation will be kept private to the extent allowed by law. Our reports will combine your responses with responses from others. Any information that could identify you will be kept private and your name will never be published in any reports. We may use direct quotes from you in our report. If we use a direct quote we will never attribute the quote to you.. Please feel free to share your thoughts openly so that the Administration for Children and Families can learn how it might improve the HPOG program. However, if you do not feel comfortable answering a question, just let me know and we can move on to the next question. This interview will be in person and will take between 60 and 90 minutes. Participation is completely voluntary. If you choose not to participate there will be no effect on:

any services that you or family receive,

your employment, or

your relationship with [NAME OF PROGRAM].

I have here an informed consent document. This document talks about your rights as a research participant and our responsibilities as researchers. Please read the informed consent form. If you wish to continue with the interview after reading it, please sign the document.

According to the Paperwork Reduction Act, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is 0970-0462 and it expires XX/XX/XXXX. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to Gretchen Locke at [email protected]; Attn: OMB-PRA (0970-0462).

[Note to Interviewer: answer any questions the participant has about the informed consent. Ensure that you review with them whether or not they wish to be audio-recorded. Give them a copy of the informed consent].

Here is a copy of the informed consent for your records.

Do you have any questions before we begin?

[Note to interviewer: If respondent asks about what HPOG is or what this study is:

The Health Profession Opportunity Grants (HPOG) Program is designed to deliver training in the health professions to eligible individuals. It was originally created in 2010 as part of the Affordable Care Act. In 2015 a second round of grants were made to programs across 21 states. HPOG funds [NAME OF LOCAL PROGRAM] and other, similar programs across the country. Abt Associates and other researchers are evaluating this second round of grants – what we call HPOG 2.0. We want to better understand how participants like yourself experience the program. To do this, we are interviewing folks like you at 14 HPOG 2.0 programs across the country.]

[Note to interviewer: Throughout the protocol we refer to either HPOG Program or HPOG training. The HPOG Program is the broader HPOG-funded initiative while HPOG training is the specific training course that the participant has selected. This information will be pre-filled from PAGES prior to the interview where possible.]

Introductory questions about interviewee and HPOG participation: (approximately 20 minutes)

Current status with program

We’d like to start by getting to know you and how and why you enrolled in the HPOG program.

1. I’d like to start by hearing about how you came to be in the [LOCAL PROGRAM NAME] HPOG program

2. I want to confirm what specific training you [are/were] enrolled in. Tell me what healthcare occupation [you are pursuing/you pursued] in this program?

1. Are you working toward a certificate or credential in your educational classes?

1. [If yes] What certificate or credential are you working toward? When do you expect to receive this certificate or credential?

2. Have you received any credentials to date from this program?

1. [If yes] What have you received and when did you receive it?

3. Tell me about the jobs you have held during the past three years.

1. For each job, can you tell me about the job, such as what you were doing, and your job title?

2. [For each job, probe for name of job, type of work, whether in healthcare, number of hours per week, stability of hours and schedule, and hourly wage for most recent job]

4. Are you working right now? [If yes] Briefly tell me about your current job.

5. [If no longer enrolled in program] Were you working while participating in [NAME OF PROGRAM]?

1. Tell me about the work you were doing while participating in [NAME OF PROGRAM].

2. What factors influenced your decision to work while you were participating in [NAME OF PROGRAM]?

6. Tell me about where you are currently in the program and how it is going. [Follow up question if they don’t give you this information] Have you completed the program, stopped attending, or are you still enrolled?

1. [If completed program] Were you able to complete the program all at once, or did you have to stop and start again? [Probe] Please tell me more about that.

1. [If completed all at once] Congratulations on finishing! Did you have any challenges in completing the program? If so, what kinds of challenges did you face? What helped you overcome these challenges?

2. [If had interruptions] When and for how long did you stop? [Probe for multiple spells]

1. Can you tell me a little bit about what happened or what was going on that interrupted or stopped your participation?

2. [Probe for dissatisfaction with program, financial issues, employment, family issues]

3. How did the HPOG program staff respond?

4. How did the coursework professors/teachers respond?

5. (If had to drop-out for some period of time) What else could program staff or professors have done to help you stay enrolled/complete the class/or to help you feel more supported?

6. [Probe] Tell me more about that (e.g., experiences with program staff such as case manager, career advisors, etc.).

2. [If still enrolled] How long have you been participating in the HPOG program?

1. Have you had to take breaks from the HPOG program or slow down your participation?

2. [If yes] When and for how long did you stop? [Probe for multiple spells]

1. Can you tell me about what happened or what was going on that interrupted or stopped your participation? [Probe for dissatisfaction with program, financial issues, employment, family issues]

2. Did the HPOG program try to help? If so, how? What else could they have done to help you stay enrolled in those moments? Can you say more about that?

3. [If stopped attending] Did you stop attending altogether, or did you stop attending for a short time?

1. When and for how long did you stop? [Probe for multiple spells]

2. Can you tell me a little bit about what happened or what was going on that interrupted or stopped your participation? [Probe for dissatisfaction with program, financial issues, employment, family issues, different issues at different times]

1. Was there anything that [NAME OF PROGRAM] could have done to help you stay enrolled? Can you say more about that?

Goals and Motivations

I want to come back to talk more about your goals and motivations for enrolling in [NAME OF PROGRAM] in more depth. Earlier in the conversation you said your goals for enrolling in the [NAME OF PROGRAM] were [XX]. Did I understand that correctly?

7. What kind of job do you hope to have in five years?

1. What steps will you need to take to get there?

2. What types of skills do you need to help you meet that goal?

3. What challenges might prevent you from getting there?

4. How might you overcome these challenges?

8. Before doing the program did you have any employment or educational goals? Can you tell me about them?

9. Across the span of your work-life, what have been the key turning points in your education or work experiences that have influenced these goals?

10. Have you noticed any changes in your own self-confidence or motivation as a result of attending this program?

11. What steps [have you taken/are you taking] to help meet your goals?

1. How is [STEPS MENTIONED BY PARTICIPANT] going?

2. Can you tell me what has helped you to take those steps?

3. Can you say more about what makes it challenging?

12. Who have you found to be the most supportive of you in meeting your education and job training goals? Can you give me a specific example of how [INSERT PERSON/PEOPLE] have been helpful.

1. [Probe for family, friends, program staff, employer, mentors] Can you give any specific examples of how [INSERT PERSON/PEOPLE] have been helpful or not?

13. Have you discussed your short- and long-term educational and employment goals with staff at [NAME OF PROGRAM]?

1. In what ways have staff helped you or been supportive of you meeting your long-term and shorter term goals?

1. [Probe: academic advising and supports, career maps, career navigation, and support services]

2. Have you found staff unsupportive of your goals in any ways? Could you give me a specific example?

3. [If not mentioned already] Do you have regular meetings or other opportunities to discuss your short and long-term goals with staff? How useful are these? How have you used them?

4. [If not mentioned already] Do staff follow up routinely with you about your short and long-term goals?

1. [If staff do follow-up] How often do staff check-in with you about these goals?

5. What conversations have you had with program staff about taking more classes or enrolling in further training?

Career Pathways

Now, I’d like to talk a bit about career ladders and pathways.

1. What does “career pathways” mean to you?

2. Can you give an example or two of when you have encountered the concept of career pathways? For example, program staff might have given information about how different healthcare trainings could help you reach your longer-term goals.

1. [If not mentioned before] Are there any times that staff or professors in your program might have talked about career pathways? Or how different health training credentials can help you reach your long-term employment goals?

3. Earlier you mentioned that you are pursuing a career as a [OCCUPATION]. How, if at all, do you see the training from [NAME OF PROGRAM] helping you pursue your career as a [OCCUPATION]?

1. How, if at all, do you see the [NAME OF HPOG TRAINING] fitting together with other trainings that you are doing, plan to do, or have done?

2. What are your plans after you complete [HPOG OCCUPATION OR TRAINING COURSE NAME]?

1. Do you think you’ll go on to take more classes or enroll in further training? Why / Why not?

2. [If yes] Tell me which classes you plan to take and why you plan to take those particular classes or training.

3. What might stop you from continuing with additional training? What might help you overcome those challenges and continue with additional training?

Occupation selection

[If the participant has selected an occupation] You mentioned that you [are/were] pursuing [HEALTHCARE OCCUPATION OR FIELD] as part of the HPOG program. I’d like to talk about how you selected [HEALTHCARE OCCUPATION OR FIELD].

1. What motivated you to pursue an occupation in healthcare?

2. When did you choose your occupation for healthcare training? [Probe for before or during the program]

3. Did you consider any other occupations? [Ask for specific examples]

4. Tell me about any concerns you had in selecting your occupation.

1. How did you overcome those concerns?

5. Did you talk with program staff or your instructors about your choice of [HEALTHCARE OCCUPATION OR FIELD]?

1. [If yes] Can you give me specific examples of a few times when you talked with program staff? [Probe for whether staff provided advice or were hands off]

2. [If staff provided advice] How did program staff advice influence your decisions or thinking about the program and your career?

6. Besides talking with program staff, were any program classes aimed at helping you select an occupation? For example career exploration, interviews with workers in your proposed field, job shadowing.

7. How did the enrollment process or program activities influence your occupational choice in any way? [If yes] Tell me more about that.

1. [Probe for any academic standards required for specific programs or occupations that participant considered or program components aimed at helping select an occupation]

8. During the course of the program, [did you change / have you considered changing] the occupation you are training for?

1. [If yes] What prompted this change?

9. Have you thought about adding any additional trainings while you’re in the current training?

[If the participant has not selected an occupation] You mentioned that you are not yet sure what occupation you intend to pursue as part of the HPOG program.

10. What occupations are you considering?

11. What factors are influencing your decision of these different occupations? [Probe for: enrollment process, support services, instructors or professors, family or friends, length of training, wages of future occupation, other information]

12. Have there been times during the program when you have received advice or support about which occupation to choose?

Experience of program training and support services (20 min)

I’d like to take some time now to talk about your experience at [NAME OF PROGRAM].

Intake and Enrollment in program

Let’s go back to the beginning, when you joined the program. I’d like to hear a bit more about the enrollment process to join the program.

13. Tell me about how you came to participate in [NAME OF PROGRAM].

1. How did you hear about [NAME OF PROGRAM]?

1. [Probe for possible sources: through friends, family members, TV, radio, Internet, newspaper, high school counselor, college staff, employer, etc.]

2. What was your motivation for applying to [NAME OF PROGRAM]?

1. [Probe] Was there anything going on in your life at the time, for example at home or at work, that made you want to enroll?

2. [Probe] When you enrolled, what goals did you hope to achieve?

14. Did you consider applying to any other job training or other education programs?

1. [If yes] What other job training programs did you apply or consider applying to?

15. Other than HPOG, have you participated in any other type of job training?

1. [If yes] What type of training? [Probe for each type of training reported]

1. Who provided it?

2. When did you participate?

3. Did you receive any certificate or credential from the job training?

1. [If yes] What type of certificate (in which occupational field)?

16. Tell me about completing the enrollment process. What was it like?

1. [Probe for: forms completed, meetings with admissions or program staff, information required]

2. Can you describe for me any assessments the program had you do as you were going through the enrollment process?

3. Did you encounter any unexpected challenges? [Probe for issues such as childcare, transportation, finances, documents that were needed, education and qualifications, etc.]

1. [If yes] How did you overcome these challenges?

4. What financial information did you need to provide during the enrollment process?

5. How easy or hard was it to understand the financial costs to you for enrolling in the program?

6. Who did you speak to at [NAME OF PROGRAM] in completing the enrollment process? How well, if at all, did they support you? Can you give me specific examples?

17. What were the standards required for joining the [NAME OF PROGRAM]? [Probe for the following: assessment scores, criminal background history, or any other requirements (if applicable)]

18. When did you begin [NAME OF PROGRAM]?

19. Tell me about the classes you’ve taken as part of the [NAME OF PROGRAM].

1. What did you think of them?

1. [Probe for perceptions of instructor competence/quality, interesting/relevant content, accessibility, interactions with peers etc. Ask for specific examples.]

2. What about the classes has been most useful to you so far? Least helpful?

1. [Probe] Can you give a specific example?

3. What about the classes has been most difficult for you? Have you been able to overcome that challenge? [If no] What has been the impact of that challenge? [If yes] How have you overcome that challenge?

4. What has been most easy in the classes?

1. [Probe] Can you give one or two specific examples?

20. What, if anything, have you had to pay to participate in [NAME OF PROGRAM]?

5. If you had to pay out of pocket, how did you pay for it?

1. [Probe for Pell grant, loans, scholarship, financial assistance from family members, employment or savings]

21. Tell me about the services or supports you received as part of the [NAME OF PROGRAM] that were outside of the classes.

6. [For each service described, probe for the following:]

1. Who in the program provides support or helps you receive supports or services?

2. How often do you meet with this individual? (e.g., on a set schedule, on demand, etc.)

3. What type of support does this person provide?

1. Is the support helpful?

1. [If yes] In what ways?

2. [If no] Why not?

4. How easy was it to meet with advisors, case managers, or other staff?

5. What did you think of the program supports? [Probe how they felt about each of the supports and services that the program provided.]

7. Probe for specific supports provided by [NAME OF PROGRAM] that were not mentioned by interviewee. [PREPOPULATED FOR EACH PROGRAM BY PAGES]

8. [Probe:] What and how much financial support did you receive from [NAME OF PROGRAM]? [Probe for whether respondent will need to pay back any loans]

22. How helpful were the financial supports and resources that you received from [NAME OF PROGRAM]?

23. How easy or difficult [was/is] it to get the financial support you needed from the program? [In particular, probe for challenges navigating the financial aid system, obtaining loans]

1. [Probe] Can you give me specific examples?

9. What about the services and supports has been most useful to you so far? Least helpful?

1. [Probe] Can you explain that further?

24. Did the program meet your expectations of the program? Tell me more about that.

25. If you could wave a magic wand, what would you change about either the classes or the program services and supports?

Role of Finances

Thank you for sharing that. Now I’m going to ask you some questions about your finances.

Let’s start by talking about your finances while you have been enrolled in [NAME OF PROGRAM].

1. Tell me about how you and your family manage/d financially while enrolled in the [NAME OF PROGRAM]?

1. What have been your sources of income during your HPOG training? [Probe for any earned income, gifts, public assistance.]

2. Do you get any gifts that have helped you manage financially while enrolled in your HPOG training? For example, help with groceries, or a parent paying for kids’ clothes.

3. [If working while in training] Earlier you talked about working while you were enrolled in the training. Can you talk me through the decision to keep working while you are in the training?

4. What are the challenges that you face working while in the HPOG training?

5. [If financial challenges are not mentioned] What financial challenges are you currently facing?[For each of the financial challenges mentioned, probe for details.] [Probe for debt, use of public programs to help make ends meet, working while in training]

6. How different is your financial situation compared to when you first started [NAME OF PROGRAM]?

1. Did you experience any changes during your time in HPOG?

2. Could you tell me more about that? [Probe to get information as to what was going on to cause change or to maintain stability]

3. What about now? How would you describe your financial situation?

1. In what ways is it different?

2. What challenges are you currently facing financially? [Probe] Could you tell me more about that?

2. What advice and support have you received from any instructors or staff on these financial issues?

3. Earlier in our discussion you mentioned you were pursuing a career in [INSERT OCCUPATION]. Have any financial issues played a role in your choice of which occupation to go into?

4. Have financial issues influenced your choice of your training in any ways? [Probe for length of training if not mentioned.]

5. How [is/was] your training funded?

6. What, if anything, have you had to pay to participate in [NAME OF PROGRAM]?

1. If you had to pay out of pocket, how did you pay for it?

[Probe for Pell grant, loans, scholarship, financial assistance from family members, employment or savings]

7. Did the program provide other financial supports for costs not related to tuition?

1. [If yes] Describe for me the different supports the program provided for you.

2. [If no] How did you fund non-tuition training-related costs?

8. What other financial support would have been helpful from the program?

9. How helpful were the financial supports and resources that you received from [NAME OF PROGRAM]?

10. How easy or difficult [was/is] it to get the financial support you needed from the program? [In particular, probe for challenges navigating the financial aid system, obtaining loans]

1. [Probe] Can you give me specific examples?

11. Are financial issues affecting your plans to go on to do additional career training?

Balancing program and life

You’ve talked about having to manage the demands of the program with the demands of home and [where relevant the demands of work.]

1. What has it been like to balance the competing demands of [NAME OF PROGRAM] with other aspects of your life and your other responsibilities?

1. Have you encountered any unexpected challenges in managing these competing demands? [Probe for issues such as childcare, transport, difficulty of class, being able to afford tuition or supplies, etc.]

2. [How did you deal / how are you dealing] with that? [Probe for specific information to understand the implications of how this influences their involvement in the program.]

3. Do you have any sources of support from family, friends, or mentors that you can rely on to help you deal with these challenges?

1. [If yes] Who provides support? What support do they provide? Is this support helpful? [If yes] In what ways?

2. Do you feel that program staff and instructors understand the fact that you and your classmates have responsibilities and competing priorities outside of class?

1. [Probe] Can you give me specific examples of when they have been supportive or unsupportive? [Probe particularly for balancing work and training at the same time]

Other Challenges

3. What other challenges have you faced in completing the program that we haven’t talked about yet?

4. How have you overcome these challenges?

Support beyond program (10 min)

Family and Friends support

I’d like to ask you more about your family relationships and their role in your HPOG participation.

12. What do your family and friends say to you about your participation in [NAME OF PROGRAM]?

1. [Probe as to whether they think participation is a good idea, feasible given the respondents’ circumstances, etc.]

13. How supportive are your friends and family regarding your educational goals?

2. Could you tell me more about that? [Probe for concrete examples]

3. How supported do you feel by your family and friends in reaching your goals? Can you give me some specific examples?

1. [Probe] Can you tell me about any times where family and friends have said something positive or negative about your goals?

2. [Probe] Can you tell me about help or support your family and friends have given you?

3. [If taking more classes] Are family and friends supportive of you taking more classes or enrolling in further training?

[If working and still in training]

I’d like to briefly talk about how supportive your employer is of your enrollment in the program.

1. How supported do you feel by your employer in attending [NAME OF PROGRAM]? Can you give me some specific examples of ways your employer supports you? Or does not support you?

[For all working respondents]

I’d like to briefly talk about how supportive your employer is of your longer-term career goals.

2. Do you talk with your employer about your career goals and future education?

3. How supportive is your employer of your long-term career goals and future education?

Closing questions (5 minutes)

Before we wrap up, I’d like to end by asking you a few questions about your overall impressions and insights about [NAME OF PROGRAM] and the HPOG program.

1. What is your overall assessment of your time in [PROGRAM]? Would you recommend this program to others? Why or why not?

2. As I mentioned earlier, [NAME OF PROGRAM] is part of a national evaluation of programs that are trying to help individuals reach their career goals. What would you tell policy makers or other program staff about the types of services, programs, or other changes that people like you need in order to reach those goals?

3. Given everything we’ve talked about today, what else do you think would be useful for the researchers studying this program to know about your experience?

Thank you for your time in talking with us. Your input is invaluable to our research, and will play an important role in helping policy makers improve HPOG programs like [NAME OF PROGRAM]. If you have any further questions, you can contact the task lead for this interview study, Hannah Thomas. Here is her business card.

[GIVE THE PARTICIPANT THEIR GIFT CARD AND HAVE THEM SIGN A GIFT CARD RECEIPT FORM.]