Appendix B_Site Visit and Key Informant Interview Topic Guide

CB Evaluation: Regional Partnership Grants (RPG) National Cross-Site Evaluation and Evaluation Technical Assistance [Implementation/descriptive study and Outcomes/Impact analyses]

Appendix B_Site Visit and Key Informant Interview Topic Guide

OMB: 0970-0527

Document [docx]
Download: docx | pdf

APPENDIX B

TOPIC GUIDE FOR SITE VISIT and phone INTERVIEWS







This page has been left blank for double-sided copying.

Regional Partnership grants (RPG) cross-site evaluation

TOPIC GUIDE FOR site visit and phone interviews

The RPG cross-site evaluation will include site visits to 11 projects and phone interviews with the remaining 6 in 2020. Researchers will interview RPG project directors, partners, managers, supervisors, and frontline staff who work directly with families during the site visits. Interviews will be conducted either individually or in small groups, depending on staffing structure, roles, and the number of individuals in a role. Researchers will interview RPG project directors and partners by phone for those projects that do not receive a site visit. This topic guide includes the full breadth of topics that will be covered in the multiple interviews, although each individual or small group interview may not include all topics.

Topic

Subtopic

Informant characteristics

Informant characteristics

Job title

Education background/licensing qualifications

Years in current position and with agency

Role on RPG and prior experience with RPG project

Partnerships

Goal setting

Organizations/stakeholders that participated in planning (during proposal stage and planning phase), if not the same as current partners

  • Child welfare and substance use treatment agencies’ involvement in RPG planning

How partners were involved in developing a shared vision and goal setting

How partners and other community organizations/stakeholders were involved in the planning and decision-making processes, and how concerns were addressed

Key design decisions made during the planning phases and rationale for those decisions

Challenges encountered during the planning process and how/if they were resolved

Grantees’ and partners’ prior experience with similar programs and how prior experience informed the RPG project design

Partnership composition and roles

How and why particular partners were selected

How partnerships came to be/developed, such as partnerships with organizations prior to RPG project; type and length of prior relationship

Grantee and partner organizations’ roles in RPG

  • Child welfare and substance use treatment agencies’ roles and responsibilities in RPG

  • Development and maintenance of formal or informal agreements

Changes in partnerships and the rationale for those changes (such as turnover of partner organizations and key staff within partners)

Changes in grantee, partner, or RPG project leadership staff that occurred during the grant period and may have affected the direction of the RPG project

Interagency collaboration and service coordination

Whether and how partners collaborate on joint activities (such as training)

Competing priorities for partner organizations

  • Process for decision making and resolving conflicts within the partnership

Policy or process changes within partner agencies (such as mental health service providers, courts) resulting from collaboration on RPG

Process to share data and information about families across partners

Process for coordination of screening, assessment, referrals, treatment, or other services

Partnership successes, challenges, and lessons learned about interagency collaboration/partnerships

Child welfare and SUD treatment agencies’ collaboration

Child welfare and substance use treatment agencies’ history of working together; successes, challenges, and lessons learned

Child welfare and substance use treatment agencies’ views on the goals of RPG

Process for defining and delineating the roles and responsibilities of each agency to meet the goals of the RPG project

  • Clarity of roles for each agency while families were served during RPG

  • Guidelines and delineation of roles for each agency especially for follow-up of services referrals

Child welfare agency’s capacity to offer SUD assessment and treatment improved or changed resulting from collaboration with SUD treatment agency

The extent of collaboration between the two agencies on four collaboration activities with drug and alcohol service providers (as defined in He, 2015)1: (a) a memorandum of understanding (MOU) or other formal interagency agreement, (b) cross-training of staff, (c) colocation of staff, and (d) joint budgeting or resource allocation

  • Intensity of collaboration, that is the number of collaboration activities

  • Types of collaboration, such as policy (such as having an MOU) versus practice collaboration (such as colocation of staff).

Alignment of RPG goals with child welfare and substance use treatment agencies’ goals and priorities

  • Process for reconciling competing priorities (if applicable)

Process for reconciling differing treatment and permanency timelines; how child welfare and SUD treatment RPG staff at every level interact with each other (such as frontline staff, managers/supervisors, and administrators/directors across the two agencies)

How lessons learned from prior collaborations between child welfare and SUD treatment agencies have been integrated into the RPG project

Process for child welfare and SUD treatment agencies to identify and address challenges related to RPG collaboration

Policy or practice changes within the child welfare and substance use treatment agencies resulting from collaboration on RPG

Perceptions of RPG project partners

Perceptions of partnership quality; frequency of partner interaction

Partners’ views about the grantee organization as a convener (an organization with enough credibility to bring together stakeholders across sectors)

Implementation teams2

Organizational structure of the implementation team for the RPG project

Development of implementation team for the project partnership; timing of development, relative to project implementation

How grantee determined members of project partnership implementation team; qualifications established for team membership; member characteristics

Roles and responsibilities of project partnership implementation team and its members

Duration of operation of implementation team, frequency of meetings, forms of communication by team members

Barriers and facilitators to effectiveness of implementation team in RPG project

Accomplishments of implementation team

Staff perception of usefulness of implementation team

Services

Implementation plans

Development of plans and procedures used to monitor project activities for compliance with plans and consistency; what details were included in plan (such as types of tasks, timeline for activities, partners responsible for tasks)

Modifications to the RPG project implementation plan that have occurred since implementation began; reasons for modifications; whether they were planned or unplanned; how decisions about modifications were made

How RPG projects developed strategies to address barriers to the delivery of high-quality services

Staff perceptions of whether implementation plan was communicated sufficiently, executed successfully, and useful in proactively identifying roadblocks to implementation

Barriers and facilitators to success of implementation plan

Service selection process

Process by which RPG project selected planned RPG services, including

  • Identified community need to be addressed

  • Conducted needs and resource assessment (including need for and availability of space, technology, financial, and other resources, including in-kind contributions by grantee and/or partners)

  • Assessed alignment of services with planned target population

  • Assessed organization capacity/readiness

  • Considered other programs or services

  • Considered need for adaptation of planned service

  • Assessed alignment of services with grantee and partners’ goals and mission

Referral processes to RPG services

How and when RPG project determined referral pathways

Sources of referrals, length of relationship with these referral sources, how relationships were established, relative size of enrollment from each referral source

Referral sources that consistently refer individuals that meet eligibility criteria and engage in the RPG project

Process used by partners to refer potential participants to RPG

Any changes to outreach and referral strategies and why

How staff accept referrals for RPG services

Barriers and facilitators to establishing pathways and translating referrals into participation

Referral processes from RPG services

Types of community services to which RPG project staff refer participants

Extent to which needed services are available and accessible in the community

Plan for conducting initial and ongoing assessments of participants’ needs and linking them to appropriate services

Extent to which participants follow up on referrals and take up the services

Process for tracking referrals, how often progress is monitored, and who is responsible for monitoring

Staffing roles and perceptions

Staffing structure for the RPG project, including frontline staff and those who support implementation (project directors, managers, and supervisors)

Responsibilities and expectations for each staff role

Staff perception on extent to which RPG project goals and purpose of partnership were explained

Involvement of frontline staff in the planning and decision-making processes, and how concerns were addressed

Who and how RPG staff interact with other RPG staff in partner organizations

Staff perceptions of the utility of an integrated approach to the provision RPG services

RPG project’s ability to provide sufficient training to all necessary staff

Internal evaluation and continuous quality improvement

RPG project expectations about the quality of services delivered through RPG; how RPG project defines high quality delivery for RPG services, and why project defines service quality in this manner

Efforts to monitor RPG service delivery, quality, and project performance; adherence to curricula or other programming, engagement, participation, and participant outcomes; who completes monitoring; what is monitored and how often; how information is used by staff

Strategies for identifying successes and challenges to implementation for purposes of continuous project improvement

Use of improvement cycles or other continuous quality improvement strategies

Use of data systems to monitor progress toward goals and RPG project performance

How staff use the data to make project decisions

Staff perception of the relevance and usefulness of project data and management information system

Barriers and facilitators to using systems and conclusions derived from data

Facilitators and barriers to ongoing evaluation and project improvement

Sustainability

Plans and activities to sustain services and partnership after grant period ends

Decision-making process for sustaining services/partnership

Strategies to engage external systems (such as health, education, housing) that may not have been involved in partnership in provision of financial, organizational, or other resources after the grant period ends

How partners identified funding sources/financing to sustain services

  • Mix of financial resources (such as, state, local, federal, private)

Sustainability of implementation team for scale-up

Sustainability of implementation plan as grant period ends

Sustainability of referral pathways

RPG project’s ability to provide sufficient training to all necessary staff for sustainability

Sustainability of data systems and processes needed to monitor and improve project implementation

Federal, state, local, tribal, and community context

Federal, state, local, tribal, and community context

Federal, state, or local policies and policy climate, and how they impeded or supported project development

Role of the courts and willingness of family court judges to support and participate in RPG

For projects who serve many American Indian/Alaska Native participants,

  • How partnerships formed, operate, and serve clients

Characteristics of communities in which RPG is offered

Unexpected events that altered RPG project activities; how they affected the project and how they were addressed

1 He, A. S. (2015). Examining intensity and types of interagency collaboration between child welfare and drug and alcohol service providers. Child Abuse & Neglect, 46, 190–197. http://dx.doi.org/10.1016/j.chiabu.2015.07.004

2 An implementation team is a team of individuals focused on supporting the implementation of services. The team may help increase the buy-in and readiness of staff, coordinate the supports staff may need to implement the services (particularly evidence-based programs or practices [EBPs]) with fidelity, assess the fidelity of the implementation of the services, and problem-solve implementation challenges. Collectively, the team possesses an in-depth knowledge of the services, knowledge of implementation best practices, and experience using data to improve program quality (Metz & Bartley, 2012).


This page has been left blank for double-sided copying.


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorSclark
File Modified0000-00-00
File Created2021-05-24

© 2024 OMB.report | Privacy Policy