Natinal Public Health Improvement Initiative: Program Assessment

Form Approved

OMB No.: 0920-0879

Exp. Date: 04/30/2017

Attachment F –Interview Guide

National Public Health Improvement Initiative (NPHII): Awardee Agency Leaders Interview Guide

Name and Title of Interviewee:	Interviewee Agency:
Respondent Number:	Verbal Consent Provided? Circle one. YES NO Audio Recording Consent Provided? Circle one. YES NO
Name of Interviewer:	Date of Interview:
Start Time of Interview:	End Time of Interview:

 READ TO PARTICIPANT: Thank you for agreeing to speak with me today. My name is [insert name] from ICF International, and we are working with the Centers for Disease Control and Prevention (or “CDC,” for short) and the National Network of Public Health Institutes to gain a better understanding of the impact of CDC’s National Public Health Improvement Initiative (also known as NPHII [“Nifee”]).

As you may be aware, the purpose of NPHII is to provide support in three main areas:

• First, accelerating public health accreditation readiness activities;

• Second, implementing quality improvement practices; and,

• Third, advancing performance management and the use of data to drive decisions.

The program’s measurable outcomes are aligned with three performance goals:

1. Increased efficiency of programs’ services and/or operations (for example, saving time or money),

2. Increased effectiveness (such as the use of evidence-based policies and practices, improvements in health outcomes, improvements in the quality of services, or enhanced customer satisfaction) and,

3. Increased readiness for applying for and/or achieving accreditation by the Public Health Accreditation Board (or “FAB” for short).

ICF staff members are conducting in-depth interviews with state health officials, other public health leaders from NPHII-funded agencies, as well as partners from select national agencies and thought leaders in the fields of quality improvement (QI), accreditation, and performance management. The goals of these interviews are to determine first, the extent to which NPHII has supported improvements in public health practice and performance, or influenced public health research, and, second, to understand the value of NPHII and lessons learned from multiple perspectives. These findings will be used, in conjunction with other data on NPHII, to report on the impact of the initiative and to inform current and future performance management, quality improvement, and accreditation readiness initiatives.

Do you have any questions before we begin?

[Pause to allow for questions].

If there are no questions: Great, I’m going to begin by reviewing the informed consent.

After answering any questions: If there are no further questions, I’m going to begin by reviewing the informed consent.

You were selected to participate in this interview as a leader within your agency to share your perceptions related to NPHII. We greatly value your experience, thoughts, and opinions related to the impact of NPHII on you and your agency, and we appreciate your willingness to participate in this interview. Your comments and answers to questions will be shared with CDC and NNPHI in transcripts and a final report. Please note that your name, role, and/or title will be linked to your specific responses in reports to CDC and NNPHI.

We want to learn from you, so we encourage you to speak openly and honestly. Both positive and negative feedback can be very helpful in understanding the overall value of NPHII. This discussion will take approximately 60 minutes of your time.

Your participation is voluntary, and you may choose not to answer some of the questions. You may also discontinue the discussion at any time without penalty. With your permission, we will record the interview to ensure we do not miss any comments, and this recording will be transcribed. If you choose not to participate, we will share this information with CDC so that we can recruit another individual in your place.

If you have any questions about this interview, please contact the ICF Project Manager, Tamara Lamia at 404-321-3211 or at [email protected].

Before we begin with the discussion, I would like to get verbal consent to proceed. Do you consent to participate in the interview?

IF “NO”: Thank the participant for his or her time and end the conversation.

IF “YES”: Thank you. I confirm that you are willing to participate in the interview and will note your verbal consent. As previously stated, we would also like to record the conversation to make sure we don’t miss anything. We will transcribe the recording, and the transcripts will be provided to CDC and the National Network of Public Health Institutes. We will destroy the audio files at the end of the project. Do I have your permission to turn on the audio recorder?

IF “YES”: Thank you. [Turn on recorder].

IF “NO”: Thank you. I will refrain from recording the session and will take detailed notes instead.

Please feel free to ask questions at any time during the discussion.

The interview will cover your level of interaction with NPHII, where NPHII and the Performance Improvement Manager (or “PIM”) are situated in your agency, your views on the most important accomplishments achieved with NPHII support, and then move to how NPHII has impacted you as a leader, your agency (or program depending on the respondent), and the relationship of your agency (or program depending on the respondent) to the broader public health community. We will end the interview by discussing the overall value of NPHII and sustainability of activities initiated through NPHII support.

I would like to start by learning more about your level of engagement with NPHII.

Instructions to interviewer:

1. The listed probes are suggestions to help you flesh out participants’ responses. Not all probes will be relevant to all participants. However, please touch on each probe topic to ensure consistent data are collected across all respondents.

2. If program-level leader participants address the interview questions only from an agency-level perspective, be sure to probe the participants on their perspectives from the program-level as well. It is fine for program-level leaders to provide responses from both perspectives; however, we are most interested in the program-level perspective from this respondent group and need to ensure this is captured in the discussions.

Q1. Describe your level of interaction with or engagement in NPHII.

[Note to interviewer: If the respondent indicates direct or hands-on involvement in NPHII activities, the following probe should NOT be asked.]

• How frequently (for example, daily, weekly, monthly, quarterly) are you updated on NPHII-related activities within the agency?

Q2. Where is NPHII housed within your agency (that is, in what office, department, or center is the initiative managed)? Where is the Performance Improvement Manager (or “PIM”) located, and what is the role of the PIM within your agency?

• What is your relationship with the PIM? How do you interact with the PIM?

• Describe the PIM’s key responsibilities.

 As we reviewed earlier, NPHII funded a variety of activities, including 1) implementing activities to accelerate your agency’s accreditation readiness; 2) conducting an organizational assessment to identify gaps in meeting PHAB standards; 3) identifying and implementing performance improvement or quality improvement initiatives to increase the efficiency and effectiveness of a program; and 4) implementing performance management activities.

We will now turn to discussing your perspectives on the most important accomplishments that have been achieved within your agency with NPHII-support.

.

Q3. Within the scope of the activities funded by the cooperative agreement, describe the most important accomplishments that have been achieved within your agency with NPHII funding or support.

• How has NPHII helped your agency [or program, if interviewee is a program-level leader] achieve these accomplishments?

[Note to interviewer: If the respondent asks for examples or clarification, you may give funding, technical assistance, and staffing as examples.]

• What other factors beyond NPHII enabled the achievement of these accomplishments?

• Were there any activities within the scope of the cooperative agreement that your agency was unable to achieve? If so, please describe?

  • What were some of the barriers your agency [or program, if interviewee is a program-level leader] faced in implementing these activities?

 I would now like to turn to how NPHII may have impacted you, personally, as a leader within your agency, how it impacted your organization, and how it impacted your agency’s relationship to the public health community overall.

Q4. Has NPHII affected how you manage and/or lead within your agency? If so, please describe.

• Has NPHII changed how you view your role and/or your leadership approach? If so, please describe.

• Has NPHII helped you use data to inform decision making? If so, please explain.

Q5. Has NPHII changed the overall capacity within your agency to provide high quality programs and services to your customers? If so, please describe.

[Note to interviewer: If a respondent asks for the definition of “capacity” and/or examples, you may provide the definition from Turnock, 2009: “Capacity consists of the resources and relationships necessary to carry out the core functions and essential services of public health; these include human resources, information resources, fiscal and physical resources, and appropriate relationships among the system components.” Examples of capacity may include staff time committed to performance management, quality improvement, or accreditation readiness (e.g., PIM) and workforce development.]

• Has NPHII changed the awareness of or focus on quality improvement efforts within your organization? If so, please describe.

• To what extent has NPHII impacted workforce development within your agency (for example, were you able to offer new or expanded training as a result of NPHII funding, were there changes in job descriptions of functions, or were there leadership development opportunities)?

Q6. What operational changes, if any, have occurred within your agency due to NPHII?

[Note to interviewer: If a respondent asks for a definition of “operational changes” and/or examples of operational changes, you may provide the following definition: Operational changes involve the assessment and application of specific processes necessary to ensure that public health operations are efficient in terms of using as few resources as needed, and effective in terms of meeting customer requirements. Examples of operational changes may include efficiencies in the amount of time to process grants/procurement requests, more effective fiscal management policies, and working across silos to ensure effective staff collaboration.]

• Has NPHII changed the way you use resources within your agency? If so, how? (for example, availability of data for decision-making, greater efficiency in contracts/fiscal management)

• Has NPHII helped with the identification or implementation of performance improvement or quality improvement initiatives within your agency? Has it helped you improve efficiency overall or increase your effectiveness overall? Please explain.

• Has NPHII changed the extent to which you are able to integrate efforts across departments or units within your agency? If so, please explain.

• Has your agency created or changed any operational or public health policies based on NPHII activities? If so, please explain.

Q7. In what ways has NPHII impacted your agency’s ability to work with the public health community and provide service to your community overall?

• Have NPHII-related activities helped change the visibility or reputation of your agency? If so, how?

• From your perspective, has NPHII changed external partners’ and/or the public’s perceptions of your agency? If so, please explain.

• From your perspective, has NPHII changed the value of your agency for external partners and/or the public? If so, please explain.

• Has your NPHII related work resulted in any new opportunities for partnerships or collaborations? If so, please describe.

Q8. Were there any additional outcomes or changes, intended or not, that occurred as a result of NPHII funding? If so, please describe.

• Were there any unintended costs to planning and implementing NPHII-funded initiatives? If so, please explain.

[If the interviewee needs help understanding the question, a few examples include: burden as a result of reporting requirements, staffing and role changes as a result of NPHII, resistance among staff (such as anxiety over job security, difficulty with change).]

• Were there any unanticipated benefits or outcomes to planning and implementing NPHII-funded efforts? If so, please explain.

[Examples if the interviewee needs help understanding the question: unanticipated benefits to meeting reporting requirements, unanticipated staffing changes.]

Q9. What value does having a Performance Improvement Manager (PIM) bring to your agency?

• In what unique ways did the PIM facilitate achieving NPHII outcomes?

Q10. What do you perceive as the overall value of NPHII to your agency?

• What were the most important outcomes resulting from the implementation of NPHII?

• 

   Great. Thank you. I would like to end the interview by asking you to think about sustainability.

  Were there any opportunities that NPHII afforded your agency that would not have been possible without this funding?

• What legacy, if any, will NPHII leave behind in your agency?

Q11. Does your agency plan to sustain NPHII-related efforts once CDC NPHII cooperative agreement funding is no longer available? If so, please describe.

• What alternative forms of funding has your agency identified or secured to sustain these efforts?

• 

   I would like to take a minute or two to review my notes and make sure we covered all the questions. [Review your notes and revisit any questions that were not touched upon].

  Thank you for taking the time to speak with me today. This has been very informative.

  In the future, CDC and/or NNPHI may wish to use one of your quotes, with your name, role, and/or title linked to these quotes, in reports released to the public. Prior to publication, CDC and/or NNPHI will contact you to confirm the quote, and you may choose to confirm it and consent to its use with your identifying information attached, or to decline, and the quote will either not be used, or will have the identifying information removed.

  Do you have any final questions? [Pause for participant questions]

  If any questions arise after today, please feel free to contact the ICF Project Manager, Tamara Lamia, at 404-321-3211 or by email at [email protected].

  What additional resources would help your agency sustain these efforts?

Public reporting burden of this collection of information is estimated to average 60 minutes per response, including the time for reviewing the instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspects of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Information Collection Review Office, 1600 Clifton Rd, NE, MS D-74, Atlanta, GA 30333; ATTN: PRA (0920-0879).