1 PIPPilot

CFSR PIP Pilot: Focus Group Guide

THE PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The control number for this project is 0970-0401. The control number expires on 5/31/2021.

Facilitator Guidelines

Provide an introduction. Introduce leaders of the focus group and roles of each in supporting the meeting (i.e., facilitator, note taker).

Briefly discuss the purpose of the interview:  The purpose of today’s focus group is to gather feedback about your experiences with the Round 3 CFSR PIP Pilot process. Your contributions to the evaluation effort is extremely valuable and will be used to improve the PIP process.

Convey to participants our confidentiality policy: (1) the focus group is voluntary; (2) they can decline to answer any questions, or can stop participating in the focus group at any time (3) Data from this focus group will be held in confidence by the evaluation team. The Information from today’s focus group will be reported and may be shared with the Children’s Bureau. When reported this information will be aggregated across all States involved in the pilot and individual’s names will not be associated with it. The names of individuals who participated in the focus group will not be shared. In some instances, however, the information provided about the strengths and challenges of the pilot process might be tied to the specific site in our reporting (4) please respect others’ privacy by not sharing any information from the focus group outside the focus group.

Confirm permission to tape record the focus group: We would like to record today’s focus group to ensure we capture the discussion accurately and completely. Only evaluation team members will have access to this recording.  If any person in the group chooses not to have the focus group recorded, we will be taking notes instead. In our confirmation e-mails to each participant before today’s call, there were no objections to recording today’s focus group. I would like to confirm that we still have the group’s permission to record the session. Would anyone prefer we not record this focus group? [Start recording if there are no objections.]

Focus Group Questions

Note to facilitator: Italicized questions are to be used as optional probes to encourage respondents to expand upon their responses. Questions in bold font are key questions that should be prioritized if time becomes an issue.

1. How well did the PIP Pilot (pre-onsite and onsite work) achieve its stated goals and objectives?

Optional probes:

• Engage key stakeholders in a meaningful way

• Model a process for developing a sound program improvement approach

• Support reaching consensus

• Facilitate open and constructive dialogue

• Promote partnership

• Identify underlying causes of state performance

• Develop a credible theory of change

• Identify strategies that are likely to improve outcomes

• Develop a clear and useful logic model

2. Considering the goals for the activities planned, do you think the right people were involved for the [pre-onsite work, onsite meeting]? Why or why not?

Optional probe:

• What important roles, responsibilities, or organizations were missing during the [pre-onsite work, onsite meeting]? (e.g. leadership, youth, birth parents, foster parents)

3. What facilitated the success of the pilot? What kept it from working better?

Optional probes:

Which of these activities was most helpful to you? Least helpful?

• Pre-onsite work

  • Gathering data

  • Reviewing and analyzing the data

  • Reviewing example root causes

  • Conducting root cause analysis

• Onsite activities

  • Developing theory of change

  • Developing logic model

  • Developing strategies to achieve outcomes

4. In what ways (if any) did the pre-onsite work prepare you for the onsite meeting?

5. How well planned, organized, and supported (e.g., technology, meeting space) was the…

• Pre-onsite work

• Onsite meeting

Optional probes:

• What worked well?

• What didn’t work well?

6. How would you describe the interaction among participants that you experienced during the…

• Pre-onsite work

• Onsite meeting

Optional Probes:

• Did you feel like members of the [CW Agency, Federal Team, Legal/Judicial Community, TA Team] worked with you as a partner?

• What made it feel like a partnership? Or not?

7. What types of data did you use during the…

• Pre-onsite work

• Onsite meeting

Optional probes:

• Was there any data missing that should have been explored?

• Was there too much data?

• How was this data used?

8. How well did the PIP Pilot facilitate the identification or development of…

• Underlying/root causes

• Theory of change

• Logic model

Optional probes:

• How would you assess the quality of [root causes, theory of change, logic model] identified/developed?

• How confident were you that the critical root causes were identified?

• How credible was the theory of change?

• How useful was the logic model?

• What could have been done differently to strengthen these products of the groups’ work?

9. How did the group reach agreement about which strategies might be the most important to pursue?

Optional Probes:

• Was consensus reached about the strategies? Do any important differences in opinion remain?

• Is it realistic to expect that these strategies can be implemented by the state?

• How well do the strategies align with the state’s vision and future goals?

10. Why do you think the strategies developed will create improvements?

Optional Probes:

• Are the strategies strongly connected to issues/root causes?

• Are they realistic?

• Are they aligned with the agency’s vision?

• Do they take into account future goals?

• Are they easy to implement?

11. (State, Stakeholder, Federal Groups Only) How would you describe the quality of the TA services during…

• Pre-onsite work

• Onsite meeting

Optional Probes:

• Which of services or activities seemed most helpful?

• Which services or activities seemed least helpful? 

12. (TA Provider Group Only) What types of TA activities and services did you use with the State team?

Optional probes:

• What made you choose these activities/strategies?

• Which of services or activities seemed most or least helpful?

13. Was four days the right amount of time for the onsite work? Why or why not?

14. Which aspect(s) of the PIP Pilot did you find most promising or successful?

15. If you could change one thing about the onsite meeting what would it be?

1