Site Selection Telephone Interviews

Site Selection Telephone Interview Protocol

Introduction

Hello, my name is __________. I am part of the team from Abt Associates evaluating the H-1B Technical Skills Training grantees for the U.S. Department of Labor Employment and Training Administration. ETA has made this grant program and its associated evaluation high priorities. The purpose of the call is to explore the potential for [name of grantee] to participate in the evaluation. Yours is one of a limited number of grantees ETA asked us to contact because of your program’s focus and relative size. This call should take about an hour of your time at the most; we are very appreciative that you can fit this into your busy schedule.

Your participation is voluntary, but we hope that you will respond to this call, as your participation will help the U.S. Department of Labor better understand how training and education programs are working to help individuals improve their skills, and find employment. Your individual responses to the survey will be kept private and no personal information will be shared with the U.S. Department of Labor, the Employment and Training Administration, or any other government agency unless required by law. Please note that an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The OMB approval number for conducting these telephone interview is 1205-0507.

I first want to provide a little background to you about the study. Our study will measure the effectiveness of the job training provided by [name of grant received by this grantee]. The primary measure of effectiveness will be the difference between the long-run earnings for program participants compared to what they would have earned absent the training provided under the grant. To do this, we will use a random assignment research design which requires individual applicants to be assigned to a group who receives the services provided through the grant program and a group that does not. We will discuss how this will work if your program is selected for the study.

To explore whether your program is suitable for the evaluation, I need to ask you a number of questions that range across a variety of topics. We would like to learn about these issues in each of the locations in which your program operates.

• The time frame over which you are operating your [name of grant] program

• Enrollment to date and projected total enrollment

• The extent to which the number of applications exceeds your number of funded “slots”

• Program eligibility rules and target group

• The nature and content of services provided to participants

• Alternative job training services in your community

Do you have any questions before we get started?

Program content and locations

Generally describe your organization (size, operational structure, funding, linkage with other systems like workforce investment boards and community colleges) and how the grant-funded training fits into it.

Describe the key partners and their role in your grant program. How are employers involved in the program (e.g. input to curriculum, hiring, subsidized jobs)?

Describe the training program(s) funded by this grant. For each training program, explain:

• Industry and occupational area

• Length and schedule (hours per week) of classroom training program

• Number of courses funded with occupational area funded by grant

• Credential/degree resulting from program

• Credit/Non-Credit

• Length, schedule, and provider of on-the-job training

• For incumbent worker training, outcome expected (retention, move up to new job)

• How do individuals enroll in the program? Is there a single point of entry or can individuals enroll in multiple locations?

Describe any non-training, support-related services you provide as part of this grant beyond and/or in addition to training. For each such support provided, collect information on content, length, and intensity of the service.

• Needs-related payments

• Pre-enrollment assessment

• Peer support (such as learning communities)

• Job search assistance (direct placement, counseling, etc.)

• Soft or “life” skills components

• Child care and transportation

• Career/vocational counseling

• Academic counseling

• Case management

• Others?

Is this a new training program, or did you add slots to an existing program? If an existing program, how many slots were there before the grant and are there now after the grant?

Does this training program receive funding from any other source? Please explain.

When did you first start serving individuals under this training program (if it existed prior to the grant)?

How many staff work on the program? How many were hired through this grant program? What are their responsibilities?

Are there any other services funded by this grant that you have not described (e.g. capacity building, curriculum development)? Please explain.

For programs with multiple locations

Our information shows you are operating the program at multiple locations. We have a few questions about that:

Describe the different locations for the program (i.e. different cities or different sites within one city).

Do all locations offer the same program? If not, which locations offer which of the programs described earlier? What do the others provide?

Do all locations use the same training providers? If not, which locations use the same training providers?

Do all locations use the same MIS/collect the same information at intake?

Are the locations linked electronically? For example, can an intake counselor tell if an applicant is already enrolled at another location?

For the remainder of the questions, please note where there are differences across locations.

Program size and enrollment period

When did you start serving or plan to start serving individuals under this grant program?

When is your grant program scheduled to end?

How many individuals have you enrolled thus far?

What is the total number of individuals you are expected to enroll in training throughout this grant program? Are there any specific events or activities that you expect to change (either increase or decrease) future enrollment?

When is enrollment in the grant program scheduled to end?

Is enrollment open entry? That is, do you enroll individuals only at the beginning of semesters/school year, on a monthly basis, or other intervals? Please explain.

How many individuals do you enroll per program “cohort” – that is, monthly or per semester (depending on the answer to the above question)?

Is enrollment proceeding on target?

• Do you have a waiting list to get into program services?

• Are you behind schedule on your enrollment targets?

Target population and recruitment

What is the target group for this grant program (e.g. incumbent workers, ex-offenders, high school drop-outs)?

Do a large share of your trainees come from a specific target population– low-wage workers, part-time workers, dislocated workers?

Do you collect information at intake on the characteristics of individuals served through the grant program?

What are the eligibility requirements to enroll in the program (e.g. high school diploma or GED, prerequisites)?

Do you do have any specific recruitment strategies for this program? Have these recruitment strategies been successful in bringing in new applicants?

Are there strategies you plan to undertake to increase potential enrollment?

What do you do in the event of oversubscription (if this is the case)? How do you change recruitment or enrollment processes?

Completion and Placement

How many individuals have completed the training to date? How many of these are incumbent workers (if applicable)?

What is the total number of individuals you expect will complete the training throughout this grant program?

Is training completion proceeding on target? Are certifications or degrees proceeding on target? Are you behind on your completion and certification targets?

Do you have issues with participants dropping out before completion? Do you use any particular retention strategies?

How many individuals have been placed in permanent jobs after completion to date (if there are any completions and this information is tracked)? Are these jobs in the same industry as the training? (Please do not include any incumbent workers in these numbers).

Is it primarily early enrollees who are placed in jobs or most recent enrollees?

Are you on target in terms of your job placements? If not, why not?

Do you have any employers that have committed to hiring graduates?

Do you anticipate any problems with placement in future? If yes, please explain.

Counterfactual conditions

If an individual cannot get into your DOL-funded grant program, do you offer similar training or other training in which they could enroll? Please describe.

If an individual cannot enroll in your DOL-funded grant program, are there other similar programs in the community in which they could enroll (including one-stop career centers, community colleges, or community-based organizations)? Please describe. Can you identify the 3-4 training providers in your community that are most similar to your program?

Random assignment studies

As you know, DOL is planning for a random assignment study of grantees in this program. Do you have any experience with random assignment evaluations?

Are you participating in any other evaluations? Which one(s)? Do any of these involve random assignment?

Conclusion

Do you have any questions for me?

The next step will be to determine whether your program is one of those we want to visit to discuss potential study participation in more detail. We will be in touch shortly to let you know our plans.

For now, let me thank you again for your time today—it has been invaluable to this important research initiative.

8