New Instrument #3 Program Leadership/Managers/Supervisors Interview Guide

Supporting Statement for OMB Clearance Request

Appendix I: Program Management Interview Guides

Innovative Strategies for Increasing Self-Sufficiency (ISIS) – Follow-up Data Collection

OMB No. 0970-0397

March 2013

Submitted by:

Brendan Kelly

Office of Planning, Research
and Evaluation

Administration for Children
and Families

U.S. Department of Health and Human Services

Appendix I: Program Management Interview Guides

Script for interviews:

We are members of the Innovative Strategies for Increasing Self-Sufficiency—ISIS—Implementation Research Team. We are visiting all of the ISIS sites for the implementation study. We are talking with individuals who help to implement and/or operate the programs included in ISIS as well as individuals in other organizations who are part of your program, either as service providers, referral partners, or other functions.

Today we’d like to ask about your program’s design and the local context in which it operates, including any services control group members might access during their participation in the study. We would like to discuss if and how the program as changed over the course of the program’s inclusion in ISIS. We would also like to hear how outreach, recruitment, intake, and random assignment and other activities are progressing. Neither your name nor the names of your colleagues we speak to will appear in any public document.

The interview will take about an hour. Your participation is voluntary. There are no penalties for choosing not to take part in the interviews. Although your answers are important to the study, you can refuse to answer any questions or stop the interview at any time without penalty. Information you provide will not be shared with others and information provided will not be attributed to you in any publications.

The Paperwork Reduction Act Burden Statement: This collection of information is voluntary and will be used to document changes in program operations or the provision of services, as well as implementation challenges. Public reporting burden for this collection of information is estimated to average 60 minutes per response. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to [Contact Name]; [Contact Address]; Attn: OMB-PRA (0970-XXXX).

Do you have any questions about the ISIS study or this implementation research visit before we begin?

Topic List for Second Round Site Visits

Changes to Program Design, Components, or Operation (and reason(s)/impetus for changes)

• Goals of the program

• Occupations or pathways addressed (added, changed)

• Target population (eligibility criteria, characteristics and outreach strategies)

• Partnerships

• Nature of relationship (formal/informal, funding arrangement (if any), type of partner (referral, service provider, advisory), length of relationship, level of collaboration/integration)

• Program Context

(Potential Interviewees: program managers, program staff, key partners)

• Institutional/organizational environment

• Local labor market

• Area or participant demographics

• Programs and services available to the control group

• Assessment

• Assessment tools used (academic and non-academic)

• When administered and how used

• Career plan development

• Monitoring academic progress (process, staff roles and coordination in monitoring and addressing progress)

• Core Curriculum

• Articulated Pathways

• Courses required

• Connections between courses on the career pathway

• Completion requirements

• Use of credits, ability to transfer credits

• Exams or licenses needed to move into employment, assistance provided

• Contextualization

• Integration of applied content into basic skills instruction

• Infusion of basic skills into occupational content

• Connections between basic and applied courses

• Acceleration

• Reorganization of delivery of instruction and curriculum to allow completion in a shorter time period

• Integration of basic skills students in college-level courses

• Flexible Delivery

• Schedule, location, hours and duration of courses, including open entry/open exit

• Class size

• Technology-supported distance learning

• Self-paced instruction

• Active learning

• Instructional methods used

• Variation by course content, instructor, class size

• Participant Supports

• Participant supports added/modified/eliminated

• Personal guidance and supports

• Instructional supports

• Social supports

• Support services

• Financial assistance

• Retention strategies added/modified/eliminated

• Follow-up components

• Employment Connections

• Role of employers in program (development of curriculum, hiring after program completion, key employers involved)

• Job search and placement services

• Work-based instruction

• Staffing

• Staff roles and responsibilities

• Staff qualifications and characteristics

• Staff evaluation

• Staff development activities

• Program management

• Staff morale and autonomy

• Participation Patterns

• Trends

• Barriers to participation

• Facilitators to participation

Program Challenges/Successes

• How program addressed challenges (if any)

• Hypotheses about factors contributing to challenges/successes

Scaling Up

• Program components added for the ISIS study

• Level of scaling up that occurred

• Planning process

• Organizational and staffing changes needed

• Financial resources utilized/needed

• Challenges (if any) that arose and how addressed

• Lessons learned from the program’s experience with scaling up

• Implications for replication, adaptation in other areas

Ability to Sustain Programs

• Alignment of program with the organization’s mission and practices

• Support for the program both within and outside the organization

• Types and amounts of resources (e.g., staff, physical, financial) required to operate the program as implemented for ISIS

• Potential future funding needs and targeted sources

• Ability to maintain partnerships developed for the study

• Process for addressing regarding sustaining the program beyond current funding

Evaluation

• Sample build-up (progress towards achieving desired sample)

• Changes (if any) to evaluation procedures

• Reflections on ISIS participation, evaluation in general

• Plans (if any) to evaluate program post-ISIS