Interview Protocol - Workforce Investment Systems Partners

Appendix C.7. Site Visit Protocol for Key Stakeholders: Workforce Investment System Partners

National Evaluation of the Trade Adjustment Act Community College Career Training (TAACCCT) Program

Interview Protocol for Key Stakeholders:

Workforce Investment System Partners

INTERVIEW INFORMATION:
GRANTEE:
PROGRAM:
ORGANIZATION/INSTITUTION:
INTERVIEW DATE/TIME:
RESPONDENT:	ADDITIONAL RESPONDENT:
TITLE:	TITLE:
STUDY ID:	STUDY ID:
INTERVIEWER(S):

Interview Topics

1. General Background Information

2. Organizational Characteristics

3. Mission and Vision

4. Organizational Structure of the TAACCCT project

5. Strategic Alignment and Partnerships

6. State/Local Context

7. Targeted Industries

8. Planning

9. Employer Engagement

10. Recruitment and Outreach

11. TAACCCT Project Development and Implementation

12. Implementation Facilitators and Challenges

13. Progress Monitoring

14. Outcomes

15. Sustainment

16. Replicability and Lessons Learned

Introduction/Purpose of the Study

Thank you for agreeing to participate in this interview today.

My name is and I’m a researcher from [Abt Associates, located in Bethesda, MD; The Urban Institute, a non- profit research organization located in Washington, DC; NORC at the University of Chicago, located in Bethesda, MD; Capital Research Corporation, located in Arlington, VA]. With me today is [name and affiliation]. I’m here today because Abt Associates and its partners, on behalf of the United States Department of Labor (DOL), is conducting a national evaluation of the Trade Adjustment Act Community College Career Training (TAACCCT) Program and studying the implementation of selected grantees and their capacity building efforts. This is a five-year study that includes analysis of grantee third party evaluation findings.

Your institution/organization is one of 9 sites we are visiting to conduct in-person interviews with Community College leaders, TAACCCT grant directors, a range of community college staff (i.e., faculty and instructors, advisors, transition coordinators, support staff; institutional research or data staff, technology staff, curriculum developers); employers, industry associations, and unions; workforce investment system partners; technical assistance providers; community and faith-based organizations; and social service agencies.

This interview will cover the organizational characteristics of each institution/organization, as well as its mission and vision, and how it fits within the organizational structure of the TAACCCT project. We will also address the strategic alignment and partnerships of the TAACCCT project. The state/local context of the TAACCCT project will be explored as well as how specific industry sectors were targeted. The interview will address the planning activities for TAACCCT project implementation, how employers were engaged, and the recruitment and outreach strategies in place to reach potential students. With regard to TAACCCT project development and implementation, we will ask about the type of model/intervention that was implemented (and multiple types if applicable). For each model/intervention, we will ask about the replication and adaptation process. Since the TAACCCT grants focus on community college capacity building, we will explore a number of topics, including staff training and new hires, facilities and infrastructure, curricula development, needs assessment, supports and services, transferability and articulation agreements, and funding and leveraged resources. We will also address facilitators and challenges associated with implementation. Progress monitoring strategies and concerns will be addressed. Early outcomes will be identified and verified with program data from the third-party evaluations. The interview will conclude with some questions about sustainment, replicability and lessons learned.

These interviews will provide important qualitative information on the implementation of the TAACCCT grants across a range of institution/organizations, including single sites, single state consortia, and multi-state consortia.

We will use what we learn from these interviews to contribute to a report on TAACCCT implementation and capacity building. This report will be submitted to DOL to inform the federal government of the progress and outcomes of the Round IV TAACCCT grants.

Confidentiality Statement [Interviewer must read this]

Before beginning the interview, I (we) want to thank you for agreeing to participate in this study and remind you that your participation is voluntary. I (we) know that you are busy and will try to be as brief as possible. We have many questions and are going to talk to many different people, so please do not feel as though we expect you to be able to answer every question. The interview today should last about 75 minutes. This interview is not part of an audit or a compliance review. We are interested in learning about your ideas, experiences, and opinions about TAACCCT implementation. There are no right or wrong answers. We want to know what you think.

In addition, before we start, I want to let you know that although we will take notes during these interviews, information is never repeated with the name of the respondent in any reports or in any discussions with supervisors, colleagues, or DOL. When we write our reports and discuss our findings, information from all the people we speak with is compiled and presented so that no one person can be identified.

To help me accurately report on the information you share, it would help if I could tape record this call; of course, I could turn off the recorder at any point if you so desire. I do not have a recorder on now; is it okay with you for me to turn it on?”

Do you have any questions before we begin?

Do I have you permission to begin the interview?

1. GENERAL BACKGROUND INFORMATION

Note: This section to be filled out in advance of the interview based on program documentation.

1. Before we begin, we’d like to get some general information on you and verify some information about your TAACCCT project.

1. TAACCCT project name

2. Institution/organization name

3. Contact information (address, telephone, fax, e-mail)

4. Website address

1. Obtain the following information on each respondent involved in the interview (Note: request a business card from each interviewee):

   1. Name

   2. Institution/organization

   3. Contact information (address, telephone, e-mail)

   4. Title

   5. Position/role under TAACCCT

   6. How long the individual has been involved in TAACCCT

2. ORGANIZATIONAL CHARACTERISTICS

Note: This section to be filled out in advance of the interview based on program documentation and verified with the respondent.

1. Please provide background on your institution/organization. Obtain brochure/recent annual report on the institution/organization.

1. Type of institution/organization (e.g., community college, employers, industry associates, technical assistance contractors, unions, non-profit or for-profit, community- and faith-based organizations, social services agencies, etc.)

2. Area served

3. Target population served (if applicable)

4. Services provided (if applicable)

5. Industry sector (if applicable)

6. Institution/organization’s overall general operating expenses/budget for most current year

$: ______ (for year ending____)

7. Institution/organization’s major sources of funding (e.g., for-profit, funding from federal/state/city agencies, foundations, private contributions, fee for service)

8. Institution/organization’s total # of paid staff: _______

9. Institution/organization’s total # of volunteer staff: _______ (if applicable)

10. Number of students enrolled (community colleges only)

11. Other relevant features about the grantee/subgrantee institution/organization that may have affected TAACCCT project implementation/operations:

3. MISSION and VISION

Note: Prior to the interview review the grantee’s theory of change.

1. What is the mission of your institution/organization? How would you describe its primary goal?

2. How does the mission of your institution/organization relate to the TAACCCT project?

3. What are the goals and intended outcomes of the TAACCCT project?

4. To what extent and in what ways does the TAACCCT project build upon or enhance your institution/organization’s strategic plan?

4. ORGANIZATIONAL STRUCTURE OF THE TAACCCT PROJECT

1. We understand that the organizational structure of TAACCCT project is a:

Note: Obtain an organization chart.

1. Single institution

2. Single state consortium

3. Multi-state consortium

1. Please describe the responsibilities of the Lead Entity (i.e., the institution/organization that was awarded the TAACCCT grant).

Probe: Leadership; Staffing; Where it fits in the structure of the college or institution/organization, etc.

3. What is your institution/organization’s role and responsibilities for the TAACCCT project?

Probe: Program development; Grants management; Education/training/instruction; Curriculum development; Supportive services; Employment; Technical assistance

4. Is this a new project or is it an expansion or enhancement of a previously-funded TAACCCT project?

5. STRATEGIC ALIGNMENT AND PARTNERSHIPS

1. How are the public workforce system, educational institutions, employers and industry, and supportive services involved in the TAACCCT project?

2. What partners are involved in implementing this model/intervention? What are their roles?

Probe: Community Colleges; Public workforce investment system; Human service systems; Employer and employment agencies; Unions; Educational institutions; Faith-based and community-initiatives; Service providers; Economic Development Agencies; State Employment Security Office; Other.

3. Has your institution/organization worked with [name of lead entity; name of subgrantee(s)] before?

1. How was your institution/organization recruited/invited to participate?

2. Who was involved in the decision to participate?

3. Did your institution/organization have any concerns about participating in the TAACCCT project? Was this concern addressed?

4. Is there an MOU in place? What are the terms of the MOU?

4. Which of the partners (noted above) do you work with most frequently?

5. What coordination and communication mechanisms are in place for planning and implementation of the project activities with your partners? Who leads these activities?

Probe: Is this process working well? Could it be improved? How?

6. Overall, how is the partnership functioning?

   1. What are the benefits of this partnership?

   2. Have you experienced any challenges with the partnership? With coordinating across systems? If yes, please describe.

6. STATE/LOCAL CONTEXT

1. The geographic service area for the [Single site (local/region or statewide); Consortia (multi-state)] TAACCCT project is _____________________________. What are the major sources of employment in the area(s) where the TAACCCT-funded program is being offered?

Note: Obtain geographic area and major employer information in advance.

2. From your perspective, describe the local economic environment in which the TAACCCT project operates.

Note: Obtain unemployment rates, job market, and wage rates in advance of the site visit.

1. Unemployment rate for area served (at time of grant application)

2. Availability of job openings in area served (particularly within targeted sector)

3. Wage rates in area served (particularly for occupations in the targeted sectors)

1. Are there any local economic conditions that may have affected the design and implementation of the TAACCCT project?

Probe: In- or out-migration of major employers; Major layoffs; Base closings; Effects of recession; State/local budget cuts; Demographic changes; Other.

7. TARGETED INDUSTRIES

1. What industries and occupations are targeted through the TAACCCT project?

Probe: Aerospace; Agriculture and Food Systems; Energy-related; Engineering; Health Professions; Information Technology; Manufacturing; Natural Resource Management; Public Safety; Skilled Trades; Transit and Transportation; STEM; Other.

2. Why was this sector targeted? Please explain. If multiple sectors are targeted ask about each one.

8. PLANNING

1. Did your institution/organization participate in the planning phase for the TAACCCT project?

1. If so, what was your role?

2. What kinds of issues were addressed?

2. What partners were involved in the planning process? What were their roles?

3. Were any challenges experienced during the planning phase? If so, what were they and how were they resolved?

9. EMPLOYER ENGAGEMENT ¹

1. What are the needs of employers in this area? What workforce challenges have been experienced?

For Supportive Services Providers:

2. Is there an employer engagement strategy in place? If yes, please describe.

Probe: Inform course and program offerings; Gather information about employment and career opportunities or job openings; Projection of demand for different positions; Industry-specific knowledge; Assess and understand trends within local or regional labor market; Determine market’s needs for new workers; Recruit and screen students; Other.

10. RECRUITMENT AND OUTREACH

Target Population

1. Are you involved with identifying TAA-eligible workers for recruitment?

   1. If so, what do you do?

2. What (other) types of individuals were targeted for the TAACCCT project?

Probe: Long-term unemployed; Veterans² and eligible spouses; Specific subpopulations; Dislocated workers; Entry-level workers; Individuals already in the industry; Other.

Outreach and Recruitment

3. What types of students were targeted for training using the model/intervention implemented?

4. From what geographic area(s) were students recruited?

5. Did your institution/organization play a role in recruitment and outreach?

   1. If so, what did you do?

   2. What recruitment and outreach strategies and methods were used? (e.g., social media, TV/radio announcements, newspaper ads, distribution of brochures, community events, word-of-mouth referrals, other)?

   3. Had you used these methods of outreach before TAACCCT?

   4. Did the outreach and recruitment activities build your institution/organization’s capacity in any way?

6. Was the outreach and recruitment strategy coordinated across partners?

7. What was the response of the targeted population to the outreach?

   1. Was it easy to recruit participants into [model/intervention implemented]?

   2. Were there more applicants than the [model/intervention implemented] could serve (e.g., was there a waiting list to get into the program)?

   3. If there were difficulties in recruiting the number of participants originally projected under the TAACCCT project, what factors made recruitment difficult?

8. What incentives (if any) were used to encourage participation?

9. Who determined eligibility to participate in [model/intervention implemented], and what, if any, criteria were used to select among candidates recruited?

Student Population

10. What are the characteristics of the enrolled students in the TAACCCT-funded programs?

Probe: Older students; First-generation students; Students who are not academically prepared; Students who work and have family responsibilities; Students who are self-supporting and have low-incomes; Veterans.

1. Have you served/worked with this population before?

2. What populations do you typically serve?

11. Describe barriers to education/training and employment for the population served.

12. What types of services have been put in place to address these barriers?

11. TAACCCT PROJECT DEVELOPMENT AND IMPLEMENTATION

Capacity Building

We are interested in learning about the capacity-building activities that [name of institution/organization] has engaged in to implement the [evidence-based models/interventions noted above] (e.g., stacked and latticed credentials, technology-enabled learning).

Curricula

1. Were investments made in developing new curricula for the [model/intervention implemented]?

2. What new curricula have been developed?

3. Will the curricula be sustained?

Needs Assessment

4. When a student enrolls in [model/intervention implemented], how is educational readiness or service needs identified?

1. Please describe the assessment process, noting any formal assessment tests that you used under the TAACCCT project (e.g., needs assessment, TABE, interest inventories, substance abuse screening).

5. Was an individual service strategy or employment development plan created for each participant? Request a blank copy of the form used.

6. Is this a new process established as a result of the TAACCCT project? Will it be sustained?

Supports and Services

7. Have academic and non-academic student supports and services been established as a result of the TAACCCT project? If so, please describe:

1. Financial stability supports

Probe: Assistance accessing traditional financial aid; Assistance using nontraditional sources of aid to pay for education and training; Funding for child care; Funding for living expenses (e.g., rent, utilities, food, transportation, health care); Financial emergencies (e.g., car repairs).

2. Academic supports

Probe: Tutoring; Academic advising (e.g., course planning); Academic preparation and course support (e.g., study skills).

3. Personal supports

Probe: Mentoring; Case management; Mental health, substance abuse or other counseling; Targeted, small group interventions; Peer-group support; Life coaches; Group support (e.g., group counseling, learning communities).

4. Sources of public support used to ensure financial stability

Probe: SNAP; WIC; TANF; Medicaid; CHIP; Section 8.

5. Career transition supports

Probe: Career navigation; Job search and placement; Assistance gaining work experience, internship search, and placement.

8. How do you expose students to information about workplace culture, including industry attendance policies and on-the-job behavioral norms?

9. For any of the supports or services described, is this for the training period only? Does the intensity of support decrease with the length of enrollment? Are “bridge programs” (i.e., post-training or intervention services or supports) provided when a student completes the course of study to facilitate his/her transition to further education or employment?

10. For each new type of support provided (financial, academic, personal, public, career transition) what resources have been leveraged?

Funding and Leveraged Resources

11. What funding sources support the implementation of this model/intervention?

Probe: Federal state local public programs; Philanthropic gifts and investments

12. Are federal funds used to provide support services? If so, which ones? See list below. How are these funds used?

Probe: Workforce Investment Act (WIA), Title I (adults and dislocated workers); WIA Title I, Youth; WIA Title II (Adult Education and Family Literacy Act); Pell Grants; Carl D. Perkins Career and Technical Training; TRIO, Student Supportive Services Program; TANF; Community Services Block Grant; Supplemental Nutrition Assistance Program Employment and Training (SNAP E&T); Community Development Block Grant (CDBG).

13. What resources were leveraged to develop and implement the model/intervention at this institution?

14. Are any in-kind resources contributed?

Probe: Local volunteers; AmeriCorps members.

12. IMPLEMENTATION FACILITATORS AND CHALLENEGES

1. How did TAACCCT project start-up and early implementation go (e.g., on-time, slow)?

2. What factors facilitated or helped project start-up?

3. What challenges have you experienced during TAACCCT project implementation?

Probe: Have you experienced challenges associated with the [model/intervention implemented] in any of the following areas? If yes, please describe. How did your institution/organization resolve these issues?

	Challenge	Resolution
	Implementation
	Partnership
	Recruiting and hiring faculty and instructional staff for technical training in the industry sectors targeted
	Leveraging resources to implement the TAACCCT project
	Implementing any of the core components of the TAACCCT project? (Note: Review each component)
	Implementation of the evidence-based models/interventions
	Stacked and latticed credentials
	Use of online and technology-enabled learning
	Transfer and articulation policies/agreements between partner institutions
	Strategic alignment between partners and service systems
	Alignment with Previously-Funded TAACCCT projects
	Obtaining the supplies and equipment/systems necessary for high-tech instructional needs
	Using on-line and technology enabled learning
	Recruitment and retention of students
	Implementing, expanding or providing supportive services to support student retention, completion, and transition to employment
	Financial stability supports
	Academic supports
	Personal supports
	Career transition supports
	Staff turnover associated with the TAACCCT project. Probe: Did this have an effect on program implementation?
	Developing and executing articulation agreements. Probe: Within state? Among consortia partners?
	Data collection across multiple states
	Needed guidance
	Other

13. PROGRESS MONITORING

1. To date, has your institution/organization faced any challenges regarding the data that you are required to collect and report for the TAACCCT grant?

Probe: What have they been (and with which data)?

14. OUTCOMES

1. Has participation in the TAACCCT grant initiative increased or influenced your institution/organization’s aspirations, strategy, organizational skills, systems and/or infrastructure? [See list]

• Aspirations (Mission, Vision, Goals)

• Strategy (Goals/performance targets, Program relevance and integration, Program growth and replication, New program development, Funding model/intervention )

• Organizational Skills (Performance measurement and benchmarking, Strategic planning, Financial planning/budgeting, Operational planning, Human Resource planning, Fund-raising, Revenue generation, Partnership alliance and generation, Local community presence and involvement, Public relations and marketing, Role in policymaking, Organizational development, Staffing capacity, Mission development and strategic planning)

• Systems and Infrastructure (Planning, Decision making framework, Financial operations management, Human resources management – management recruiting, development, and retention, Physical infrastructure, Technological infrastructure)

If so, please provide a concrete example. Request available data, reports, and other materials related to these changes.

2. Has participation in the TAACCCT project initiative increased or influenced the strategic alignment of:

   1. Community college and industry and employers?

   2. Community college and public workforce development system?

   3. Community college and other education institutions?

   4. Community college and local providers?

If so, please provide a concrete example. Request available data, reports, and other materials related to these changes.

SUSTAINMENT

1. We have some questions about the sustainability of the TAACCCT components implemented.

   1. Which components of the TAACCCT project does your institution plan to sustain after the end of the grant? Note: Align with activities referenced in CAPACITY BUILDING section.

      1. Education and training programs overall

      2. New learning strategies/curricula

      3. Stacked/latticed credentials

      4. Technology and online learning

      5. Transfer and articulation agreements

      6. Partnerships within and outside the institution

   2. Will you sustain these specific aspects at the same level or will you scale any of these activities? Within your institution? To other institutions?

   3. Will [name of entity] need to find new funding once the grant ends?

2. Are there some components of the TAACCCT project that will be difficult to sustain due to changes in technology or innovation in the field of study? Employer demand?

3. How does your institution/organization’s plan for sustainability align with your vision of the TAACCCT project?

4. How does the sustainability plan align with the strategic plan of the [name of institution/organization]?

15. PROJECT REPLICABILITY AND LESSONS LEARNED

1. To what extent do you think your program could be replicated in other departments within the college? In other localities?

   1. What features of the TAACCCT project are most amenable to replication?

   2. What features of the project are least amenable to replication? How does location, the target population served, or other distinctive features of your program make it either non-transferable or limit transferability?

2. Based on your experience with the TAACCCT project, are you likely to partner again with [name of lead entity; name of subgrantee(s)]?

3. To date, what do you consider your most important accomplishments under the TAACCCT project?

4. What do you believe to be the main lessons learned from your TAACCCT project?

5. Is there any topic or issue you expected us to cover that we have not brought up yet? (If YES): Please describe the issue(s) and explain why you think it is/they are important.

Thank you for your participation.

1 Employer engagement required for Round IV grantees.

2 Veterans receive first priority for Round IV grant programs, with TAA-eligible workers given second priority.