Interview Protocol - Community College Leadership

Appendix C.4. Site Visit Protocol for Community College Leadership

National Evaluation of the Trade Adjustment Act Community College Career Training (TAACCCT) Program

Interview Protocol for Community College Leadership

INTERVIEW INFORMATION:
GRANTEE:
PROGRAM:
ORGANIZATION/INSTITUTION:
INTERVIEW DATE/TIME:
RESPONDENT:	ADDITIONAL RESPONDENT:
TITLE:	TITLE:
STUDY ID:	STUDY ID:
INTERVIEWER(S):

Interview Topics

1. Mission and Vision

2. Organizational Structure of the TAACCCT project

3. Strategic Alignment and Partnerships

4. State/Local Context

5. Targeted Industries

6. Planning

7. Employer Engagement

8. Recruitment and Outreach

9. TAACCCT Project Development and Implementation

10. Implementation Facilitators and Challenges

11. Progress Monitoring

12. Outcomes

13. Sustainment

14. Replicability and Lessons Learned

Introduction/Purpose of the Study

Thank you for agreeing to participate in this interview today.

My name is , and I’m a researcher from [Abt Associates, located in Bethesda, MD; The Urban Institute, a non- profit research organization located in Washington, DC; NORC at the University of Chicago, located in Bethesda, MD; Capital Research Corporation, located in Arlington, VA]. With me today is [name and affiliation]. I’m here today because Abt Associates and its partners, on behalf of the United States Department of Labor (DOL), is conducting a national evaluation of the Trade Adjustment Act Community College Career Training (TAACCCT) Program and studying the implementation of selected grantees and their capacity building efforts. This is a five-year study that includes analysis of grantee third party evaluation findings.

Your institution/organization is one of 9 sites we are visiting to conduct in-person interviews with Community College leaders, TAACCCT grant directors, a range of community college staff (i.e., faculty and instructors, advisors, transition coordinators, support staff, institutional research or data staff, technology staff, curriculum developers); employers, industry associations, and unions; technical assistance providers; community and faith-based organizations; and social service agencies.

This interview will cover the organizational characteristics of each institution/organization, as well as its mission and vision, and how it fits within the organizational structure of the TAACCCT project. We will also address the strategic alignment and partnerships of the TAACCCT project. The state/local context of the TAACCCT project will be explored as well as how specific industry sectors were targeted. The interview will address the planning activities for TAACCCT project implementation, how employers were engaged, and the recruitment and outreach strategies in place to reach potential students. With regard to TAACCCT project development and implementation, we will ask about the type of model/intervention that was implemented (and multiple types if applicable). For each model/intervention, we will ask about the replication and adaptation process. Since the TAACCCT grants focus on community college capacity building, we will explore a number of topics, including staff training and new hires, facilities and infrastructure, curricula development, needs assessment, supports and services, transferability and articulation agreements, and funding and leveraged resources. We will also address facilitators and challenges associated with implementation. Progress monitoring strategies and concerns will be addressed. Early outcomes will be identified and verified with program data from the third-party evaluations. The interview will conclude with some questions about sustainment, replicability and lessons learned.

These interviews will provide important qualitative information on the implementation of the TAACCCT grants across a range of institutions/organizations, including single sites, single state consortia, and multi-state consortia.

We will use what we learn from these interviews to contribute to a report on TAACCCT implementation and capacity building. This report will be submitted to DOL to inform the federal government of the progress and outcomes of the Round IV TAACCCT grants.

Confidentiality Statement [Interviewer must read this]

Before beginning the interview, I (we) want to thank you for agreeing to participate in this study and remind you that your participation is voluntary. I (we) know that you are busy and will try to be as brief as possible. We have many questions and are going to talk to many different people, so please do not feel as though we expect you to be able to answer every question. The interview today should last about 60 minutes. This interview is not part of an audit or a compliance review. We are interested in learning about your ideas, experiences, and opinions about TAACCCT implementation. There are no right or wrong answers. We want to know what you think.

In addition, before we start, I want to let you know that although we will take notes during these interviews, information is never repeated with the name of the respondent in any reports or in any discussions with supervisors, colleagues, or DOL. When we write our reports and discuss our findings, information from all the people we speak with is compiled and presented so that no one person can be identified.

To help me accurately report on the information you share, it would help if I could tape record this call; of course, I could turn off the recorder at any point if you so desire. I do not have a recorder on now; is it okay with you for me to turn it on?”

Do you have any questions before we begin?

Do I have you permission to begin the interview?

1. MISSION and VISION

Note: Prior to the interview review the grantee’s theory of change, if available.

1. What is the mission of your institution/organization? How would you describe its primary goal?

2. How does the mission of your institution/organization relate to the TAACCCT project?

3. What are the goals and intended outcomes of the TAACCCT project?

4. To what extent and in what ways does the TAACCCT project build upon or enhance your institution/organization’s strategic plan?

2. ORGANIZATIONAL STRUCTURE OF THE TAACCCT PROJECT

1. What is your institution/organization’s role and responsibilities for the TAACCCT project?

Probe: Program development; Grants management; Education/training/instruction; Curriculum development; Supportive services; Employment; Technical assistance

3. STRATEGIC ALIGNMENT AND PARTNERSHIPS

1. How are the public workforce system, educational institutions, employers and industry, and supportive services involved in the TAACCCT project?

2. What partners are involved in implementing this model/intervention? What are their roles?

Probe: Community Colleges; Public workforce investment system; Human service systems; Employer and employment agencies; Unions; Educational institutions; Faith-based and community-initiatives; Service providers; Economic Development Agencies; State Employment Security Office; Other.

3. Overall, how is the partnership functioning?

   1. What are the benefits of this partnership?

   2. Have you experienced any challenges with the partnership? With coordinating across systems? If yes, please describe.

4. STATE/LOCAL CONTEXT

1. Are there any local economic conditions that may have affected the design and implementation of the TAACCCT project?

Probe: In- or out-migration of major employers; Major layoffs; Base closings; Effects of recession; State/local budget cuts; Demographic changes; Other.

5. TARGETED INDUSTRIES

1. What industries and occupations are targeted through the TAACCCT project?

Probe: Aerospace; Agriculture and Food Systems; Energy-related; Engineering; Health Professions; Information Technology; Manufacturing; Natural Resource Management; Public Safety; Skilled Trades; Transit and Transportation; STEM; Other.

2. Why was this sector targeted? Please explain. If multiple sectors are targeted ask about each one.

6. PLANNING

1. During the planning phase, did you seek input from TAACCCT project partners about this model/intervention (e.g., other community colleges, employers, industry associates, technical assistance contractors, unions, non-profit or for-profit, community- and faith-based organizations, social services agencies, other)?

   1. If so, how was this done?

   2. What partners were involved in the planning process? What were their roles? What kinds of issues were addressed?

7. EMPLOYER ENGAGEMENT ¹

1. What are the needs of employers in this area? What workforce challenges have been experienced?

2. Is there an employer engagement strategy in place? If yes, please describe.

Probe: Gather information about labor market trends (local wisdom); Align courses to local industry cluster and high-growth, high demand occupations; Establish and update curricula; Assess the adequacy of training curriculum; Solicit advice on program effectiveness; Encourage businesses to serve on advisory committees.

8. RECRUITMENT AND OUTREACH

Outreach and Recruitment

1. What types of students were targeted for training using the model/intervention implemented?

Student Population

2. What are the characteristics of the enrolled students in the TAACCCT-funded programs?

Probe: Older students; First-generation students; Students who are not academically prepared; Students who work and have family responsibilities; Students who are self-supporting and have low-incomes; Veterans.

1. Have you served/worked with this population before?

2. What populations do you typically serve?

3. Describe barriers to education/training and employment for the population served.

4. What types of services have been put in place to address these barriers?

9. TAACCCT PROJECT DEVELOPMENT AND IMPLEMENTATION

Type(s) of Model/Intervention(s) Implemented

1. According to the grant application, the TAACCCT-funded project(s) is/are implementing the following evidence-based model/intervention for the industry sector(s).

Note: Refer to list below and identify the model/intervention. If multiple models/interventions are being implemented, cover each one separately with question 2

	Accelerated learning models/interventions
	Modularized learning
	Stacked and latticed credentials
	Integrated Basic Education and Skills training (I-BEST)
	Contextualized learning
	Team teaching
	Self-paced learning
	Prior learning assessments
	Other:
	Technology-enabled learning
	Simulations
	Assessment technology
	Online teaching/learning
	Real-time online interactions
	Other:
	Career Pathways
	Other

2. Let’s talk about [model/intervention].

   1. Why was this model/intervention chosen? Were other models/interventions considered? If so, which ones?

   2. Did the evidence for this model/intervention show relative advantage over other models/interventions? What are the gaps?

   3. Is this model/intervention being used for a particular industry sector? Or for multiple industry sectors?

Capacity Building

We are interested in learning about the capacity-building activities that [name of entity] has engaged in to implement the [evidence-based models/interventions noted above] (e.g., stacked and latticed credentials, technology-enable learning).

Staffing, New Hires, and Training

3. Were there staffing changes made to implement the TAACCCT project?

Facilities and Infrastructure

4. Have there been improvements to the community colleges’ facilities or infrastructure resulting from the implementation of the [model/intervention implemented]? Please describe.

Probe: Access to instructional supports including computer labs, tutors, and student support; Investment in technology; Technology integration; Dedicated training or classroom space; Offsite facilities; Other.

5. Have any TAACCCT project partners been involved in facility or infrastructure development or improvement? Please describe.

6. What resources have been leveraged to develop or improve facilities or infrastructure?

Curricula

7. Were investments made in developing new curricula for the [model/intervention implemented]?

8. What new curricula have been developed?

9. Will the curricula be sustained?

Supports and Services

10. Have academic and non-academic student supports and services been established as a result of the TAACCCT project? If so, please describe:

1. Financial stability supports

2. Academic supports

3. Personal supports

4. Sources of public support used to ensure financial stability

11. What internal partnerships or relationships have been formed to implement the support or service?

1. Will this relationship be sustained?

2. Will the support or service be sustained?

12. What external partnerships have been formed to provide this support or service?

Transferability and Articulation (for credit and credentials)

13. What transfer policies were in place prior to the TAACCCT project?

14. What criteria, policies, and procedures are now in place to enable students to transfer academic and technical courses from a two-year institution to a four-year institution for TAACCCT-funded courses and credentials?

15. What articulation policies were in place prior to the TAACCCT project?

16. What agreements are now in place to facilitate bridging non-credit courses and credentials from one institution to another among the TAACCCT project partners?

1. What institutions are involved?

Probe: Board of Regents; The office of career-technical & adult education of the state Department of Education; Public secondary/adult career-technical education institutions; State-supported institutions of higher education.

Funding and Leveraged Resources

17. What funding sources support the implementation of this model/intervention?

Probe: Federal state local public programs; Philanthropic gifts and investments.

10. IMPLEMENTATION FACILITATORS AND CHALLENEGES

1. How did TAACCCT project start-up and early implementation go (e.g., on-time, slow)?

2. What factors facilitated or helped with project start-up?

3. What challenges have you experienced during TAACCCT project implementation?

Probe: Have you experienced challenges associated with the [model/intervention implemented] in any of the following areas? If yes, please describe. How did your institution/organization resolve these issues?

	Challenge	Resolution
	Implementation
	Partnership
	Recruiting and hiring faculty and instructional staff for technical training in the industry sectors targeted
	Leveraging resources to implement the TAACCCT project
	Implementing any of the core components of the TAACCCT project? (Note: Review each component)
	Implementation of the evidence-based models/interventions
	Stacked and latticed credentials
	Use of online and technology-enabled learning
	Transfer and articulation policies/agreements between partner institutions
	Strategic alignment between partners and service systems
	Alignment with Previously-Funded TAACCCT projects
	Obtaining the supplies and equipment/systems necessary for high-tech instructional needs
	Using on-line and technology enabled learning
	Recruitment and retention of students
	Implementing, expanding or providing supportive services to support student retention, completion, and transition to employment
	Financial stability supports
	Academic supports
	Personal supports
	Career transition supports
	Staff turnover associated with the TAACCCT project. Probe: Did this have an effect on program implementation?
	Developing and executing articulation agreements. Probe: Within state? Among consortia partners?
	Data collection across multiple sites
	Needed guidance
	Other

11. PROGRESS MONITORING

1. How are results of the TAACCCT-funded project shared with potential students and the general public?

12. OUTCOMES

Note: Before the interview, ask the grantee to provide the third-party outcome data necessary to complete this section. If this is not possible, request available data/reports on TAACCCT performance/outcomes during the site visit.

1. What student outcomes can be attributed to the TAACCCT project?

1. Increased student educational attainment (certificates, degrees)?

2. Increased student attainment of Industry Recognized Credentials?

3. Increased student attainment of degrees in high-demand occupations?

4. Increased graduation rates?

5. New or strengthened relationships with employers?

2. Has the provision of new academic and non-academic supports enabled the community college to:

   1. Serve more students?

   2. Serve students from diverse communities?

   3. Facilitate student recruitment?

   4. Increase student retention?

3. Has participation in the TAACCCT grant initiative increased or influenced your institution/organization’s aspirations, strategy, organizational skills, systems and/or infrastructure? [See list]

• Aspirations (Mission, Vision, Goals)

• Strategy (Goals/performance targets, Program relevance and integration, Program growth and replication, New program development, Funding model/intervention)

• Organizational Skills (Performance measurement and benchmarking, Strategic planning, Financial planning/budgeting, Operational planning, Human Resource planning, Fund-raising, Revenue generation, Partnership alliance and generation, Local community presence and involvement, Public relations and marketing, Role in policymaking, Organizational development, Staffing capacity, Mission development and strategic planning)

• Systems and Infrastructure (Planning, Decision-making framework, Financial operations management, Human resources management – management recruiting, development, and retention, Physical infrastructure, Technological infrastructure)

If so, please provide a concrete example. Request available data, reports, and other materials related to these changes.

4. Has participation in the TAACCCT project initiative increased or influenced the strategic alignment of:

   1. Community college and industry and employers?

   2. Community college and public workforce development system?

   3. Community college and other education institutions?

   4. Community college and local providers?

If so, please provide a concrete example. Request available data, reports, and other materials related to these changes.

13. SUSTAINMENT

1. We have some questions about the sustainability of the TAACCCT components implemented.

   1. Which components of the TAACCCT project does your institution plan to sustain after the end of the grant? Note: Align with activities referenced in CAPACITY BUILDING section.

      1. Education and training programs overall

      2. New learning strategies/curricula

      3. Stacked/latticed credentials

      4. Technology and online learning

      5. Transfer and articulation agreements

      6. Partnerships within and outside the institution

   2. Will you sustain these specific aspects at the same level or will you scale any of these activities? Within your institution? To other institutions? Note: These probes may overlap with replicability.

   3. Will [name of entity] need to find new funding once the grant ends?

2. Are there some components of the TAACCCT project that will be difficult to sustain due to changes in technology or innovation in the field of study? Employer demand?

3. How does your institution/organization’s plan for sustainability align with your vision of the TAACCCT project?

4. How does the sustainability plan align with the strategic plan of the [name of institution/organization]?

14. PROJECT REPLICABILITY AND LESSONS LEARNED

1. To what extent do you think your program could be replicated in other departments within the college? In other localities?

   1. What features of the TAACCCT project are most amenable to replication?

   2. What features of the project are least amenable to replication? How does location, the target population served, or other distinctive features of your program make it either non-transferable or limit transferability?

2. Based on your experience with the TAACCCT project, are you likely to partner again with [name of lead entity; name of subgrantee(s)]?

3. To date, what do you consider your most important accomplishments under the TAACCCT project?

4. What do you believe to be the main lessons learned from your TAACCCT project?

5. Is there any topic or issue you expected us to cover that we have not brought up yet? (If YES): Please describe the issue(s) and explain why you think it is/they are important.

Thank you for your participation.

1 Employer engagement required for Round IV grantees