Interview Protocol - Assessment of NPHI Capacity

OMB Control No. 0920-1050

Exp. Date 05/31/2022

The public reporting burden of this collection of information is estimated to average 45 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to - CDC/ATSDR Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333 ATTN: PRA (0920-1050)

Assessment of National Public Health Institutes’ Capacity to Strengthen Essential Public Health Functions in Countries Supported by U.S. Centers for Disease Control and Prevention

[U.S. CDC/CGH/DGHP/WIDB]¹

INTERVIEW PROTOCOL

1. Begin Interview: Background

I would like to start the interview with a few questions about you and your role.

1. What is your current position or role?

2. How do you engage with NPHIs (or the NPHI in [country]) in your role as [role]? (Probes: managing different public health functions, logistics, establish or revise policy, policy implementation, report to higher authorities including international authorities, coordination, etc.)

3. How long have you been in this role with [NPHI name]?

4. What position did you hold just prior to starting in your current role?

5. In your own words, what is the major role of NPHI here in [country]? I.e. what is it (are they) trying to accomplish? (Probes: to improve essential public health functions, increase coordination among various public health functions, respond to emerging outbreaks, etc.)

2. NPHIs role in strengthening the key public health functions² and linkages among these functions

Next, I would like to focus in on the roles that [NPHIs] {the NPHI in [Country]} play(s) in the strengthening of key public health functions. I’d especially like to focus our discussion on the four key public health functions: surveillance, public health laboratory, emergency preparedness & response, and workforce development.

1. What public health functions does [country] NPHI focus on? (Probe: all types of functions can be mentioned- not just the 4 key public health functions)

2. Which of the four public health functions (surveillance, public health laboratory, emergency preparedness & response, workforce development) are housed directly under the country’s NPHI?

   1. Of those housed under the NPHI directly, can you tell us more about how these functions were implemented and coordinated before they were housed under the NPHI (if known)?

   2. What do you think has been the main difference, in each of the public health functions, since the establishment or strengthening of the NPHI in [country]? [Note: help the interviewee go through each of the four public health functions when responding to this question]

   3. In your opinion, in what way (if any) does the NPHI in [country] contribute to the process and functioning of the key public health functions?

   4. Can you provide a few examples of a way the support from U.S. CDC’s NPHI program affected the four public health functions in the [country] NPHI? (Probe: coordination of these functions, staff capacity, public health activities, political will and government-stakeholder coordination, resource utilization, etc.)

3. Are there public health functions that are not housed directly under the NPHI but are significantly linked to or are coordinated by the NPHI?

   1. If yes, which public health functions are significantly linked to the country’s NPHI?

   2. How are these public health functions coordinated?

   3. How does the NPHI in [country] work with these public health functions?

4. Are there other key public health functions your NPHI facilitates or coordinates that has not yet been mentioned?

NOTE TO INTERVIEWER: The following section is about identifying CDC’s impact, go through these questions even if interviewees discussed the impact of U.S. CDC support under (A)

Focused on U.S. CDC in general

1. In what way has U.S. CDC’s support helped [country] NPHI to coordinate the four public health functions, or other public health functions that are a priority to your country’s NPHI?

2. In your opinion, what type of support provided by U.S. CDC has been the most valuable or impactful for [country] NPHI(s)? (Probe: planning around strategy and operations, prioritization and activity planning, technical assistance for NPHI operations, technical assistance for NPHI functions, funding NPHI activities, etc.)

3. Are there any gaps and/or challenges in NPHI functions or operations that U.S. CDC has helped [country] NPHI(s) identify or address? If yes, can you please describe them (give specific examples).

4. In your opinion, what can U.S. CDC do better to support [country] NPHI(s)?

Focused on CDC NPHI Team in particular

5. In what way has the U.S. CDC’s NPHI team (provide names of NPHI team if needed) assisted your country’s NPHI? (Probe: helped with planning, focusing, and implementing systems strengthening, etc.)

6. Can you think of any specific examples of support that was provided by U.S. CDC’s NPHI team that impacted or improved the functioning or operation of your country’s NPHI?

In the next section, we would like you to take some time to fill out Table 1. This table is a maturity framework that assesses an organization’s ability to improve across key process areas and domains. As you fill this table, please let us know if you have any questions or want clarifications.

Table 1: Maturity Ratings of Key Organizational Capacity and Key Public Health Functions

Country		[Country Name]
Domain		Area	Maturity Level Description and Score Basic (1-2), Developing (3), Advanced/Mature (4-5); (does not know = 0); (if domain is outside the influence of the NPHI = N/A)
			Description	Score
ORGANIZATION	Systems and Readiness	Planning	The annual operational plan is designed to support [country] NPHI’s strategies articulated in the NPHI strategic plan.
		Internal communication	Effective communication mechanisms are used consistently to share information across the NPHI’s organizational units, and among staff at different levels.
		External communication	The NPHI is widely known, due in part to the quality and reach of its communication efforts and has evidence that its communication efforts have increased its ability to impact public health.
		Monitoring and evaluation	The organization regularly monitors its progress, evaluations results, and uses the findings to improve services and plan its next phase of work
		Human resource management	The NPHI has human resources policies and procedures in place, and managers use them consistently to hire and retain talented and committed staff.
		Financial Management	The NPHI follows a long-term financial management plan strategy, balancing diverse sources or revenue to meet current and future needs.
		Quality assurance	The NPHI has an established ongoing system for assessing and improving the quality of services, and trained staff regularly use this system.
	Leadership and Management	Strategic Direction	Leaders/Managers provide sustained focus on achieving the vision, mission and long-term goals, which inspires staff and stakeholders commitment to the NPHI. Even during unexpected events, leaders/managers remain focused on the long-term.
		Systems	The NPHI’s system make it easy for leaders/managers at all levels to use data to track and improve performance. Leaders/managers are expected to provide regular performance assessments to employees, including plans to help employees improve or learn new skills.
		Quality	The NPHI’s leaders/managers uses 360 and other review processes to improve performance. The NPHI seeks to use best leadership and management practices, regularly engaging with other sectors to identify the best new approaches.
		Impact	Leaders/managers at every level can document how their team’s work contributes to the NPHI’s goals. The NPHI is widely admired and trusted both inside and outside the NPHI. The NPHI’s leadership and management are models for other NPHIs.
PUBLIC HEALTH FUNCTIONS	Surveillance Systems	Strategic Direction	The NPHI’s surveillance is designed to provide data to guide policies and programs. In designing systems, all aspects of surveillance are considered, from data collection through analysis and use of the data.
		Systems	The NPHI’s surveillance SOPs are models for other organizations. The NPHI routinely reviews systems for quality and relevance, including whether the data are being used. Evaluations usually lead to improvements in the system.
		Resources	The NPHI has resources to provide substantial assistance to reporting entities to improve data collection. NPHI staff have the skills and resources to collect and analyze data, including sophisticated analyses, and to use data to make recommendations.
		Quality	The NPHI’s surveillance data collection and analysis is of generally high quality. Analyses often involve advanced methods; and reports are completed in a timely manner.
		Engagement	The NPHI actively seeks input from a range of stakeholders to inform its data collection and analysis efforts, and also proactively shares results. The NPHI maintains engagement as projects proceed, increasing the likelihood that results will be used. Impactful findings are disseminated using a variety of approaches.
		Impact	Decision-makers often rely on the NPHI’s surveillance data for informing programs and policies. The NPHI can provide several examples where problems were identified earlier because of surveillance.
	Laboratory Systems	Strategic Direction	The NPHI’s laboratories] figure substantially in its strategic plan. They are critical to the NPHI achieving its public health mission.
		Systems	The NPHI’s laboratories have systems to ensure safe handling and processing of specimens. The NPHI uses a laboratory information system (LIMS) to track specimens, and support data management, but specimens are sometimes lost or information is not entered in the system.
		Resources	The laboratory has adequate infrastructure and resources to handle laboratory work associated with large emergencies without compromising routine work. It can conduct almost any needed test, and has the ability to develop new tests if needed. The NPHI routinely upgrades equipment and helps staff enhance their skills.
		Quality	Most of the NPHI’s laboratories have received Strengthening Laboratory Management Toward Accreditation (SLMTA) or International Organization for Standardization (ISO) certification, if appropriate, for consistently implementing strong QA procedures. The NPHI provides training, proficiency testing, technical assistance, or other efforts, to improve quality in key laboratories in the country.
		Engagement	The NPHI] has formalized relationships with key laboratories and clinical facilities throughout the country.
		Impact	The NPHI and other organizations throughout the country often use NPHI’s laboratory results to inform public health program and policies decisions. The NPHI’s efforts have had demonstrable impact on quality in some laboratories in the county.
	Emergency Management and Response	Strategic Direction	The NPHI is a critical part of national preparedness and response efforts. The NPHI’s plans for national preparedness and response are up-to-date and address many types of emergencies.
		Systems	The NPHI has a well-functioning Incident Command System (IMS); policies, plans, and procedures; informational materials; and response teams to support emergency preparedness and response.
		Resources	The NPHI has material resources and trained staff to respond to most emergencies without major disruption of routine functions, although very large emergencies stress the NPHI. The NPHI stockpiles supplies to respond to most emergencies and can procure most others in a timely manner. It sometimes offers preparedness training to subnational levels and other organizations.
		Quality	The NPHI’s responses to emergencies are rapid and effective, except during extreme events. The NPHI uses stimulations to prepare for emergencies.
		Engagement	The NPHI has strategic relationships with important partners, including from other sectors, and integrates them into emergency planning and exercises. During emergencies, the NPHI places a high priority on communications with those responding to the event and with the public and other stakeholders.
		Impact	The NPHI‘s planning and response efforts often reduce the public health impact of emergencies.
	Workforce Development	Strategic Direction	The NPHI conducts workforce assessments in strategically selected areas and works with partners to address the most critical gaps.
		Systems	The NPHI works closely with partners to systematically assess workforce needs in a range of areas and addresses gaps. The NPHI has established agreements with national or subnational level organizations (e.g. to ensure that many new public health graduates have field experiences during training.
		Resources	Resources are adequate for the NPHI to support a range of workforce assessment and development activities. It has workforce and training specialists on staff, but these are not sufficient for the NPHI’s needs. The NPHI uses technology to increase the reach and effectiveness of its recommendations and training.
		Quality	The NPHI conducts comprehensive assessment in high priority areas to guide its efforts to fill workforce gaps. The NPHI provides detailed recommendations for addressing identified gaps and has well-designed training materials and well trained trainers.
		Engagement	A range of stakeholders are engaged in workforce assessments, developing strategies to address workforce gaps, and conducting joint trainings and other activities.
		Impact	The NPHI’s assessments of workforce gaps informs its programs and those of others. Training and other approaches are addressing public health workforce needs.

3. Success, sustainability, and institutionalization

Now we are going to switch gears and talk about NPHIs sustainability and institutionalization. First, I would like to focus in on some definitions.

Success

1. How would you define the success of [country] NPHI(s)? Would you say [country] NPHI is currently successful? If yes, why do you think it has been successful? If no, what do you think it would take or how can it become more successful?

Sustainability

2. In your own words, how would you define sustainability of a government agency? (Probe: What would it take for (or how can) [country] NPHI to (be) sustainable? E.g. mention in national plan, critical government functions like outbreak response)

3. In your opinion, what do you think might help [country] NPHI(s) achieve sustainability?

4. What do you think might hinder or prevent [country] NPHI(s) from achieving sustainability?

5. What factors do you think will help facilitate the continuation of NPHI(s) in [country]?

6. What factors do you think might prevent the continuation of [country] NPHI(s)?

Institutionalization

7. How would you define institutionalization of [country] NPHI(s)? (Probe: What would it take for (or how can) [country] NPHI(s) be institutionalized? What does institutionalization look like to you? What would some of the characteristics be for an institutionalized NPHI?

8. What do you think might help [country] NPHI(s) achieve institutionalization? (Probe: political will, legislative mandate, stakeholder commitment, inter-sectoral cooperation, using experience of other countries, involvement of research communities and other stakeholders)

9. What do you think might hinder or prevent [country] NPHI(s) from achieving institutionalization?

3. Looking to the future

Next, I would like to ask you a few more questions around [country] NPHI(s) future.

1. What do you think is needed for [country] NPHI’s future? What does this planning look like?

2. Who are the key stakeholders or partners you think should be involved in this process?

3. To what degree do you feel U.S. CDC’s involvement has increased communications between [country] NPHI and other partners?

In the next section, we would like you to take some time to fill out Table 2. This table is an assessment framework that measures [country] NPHI’s sustainability and institutionalization using key sustainability and institutionalization enabling factors. As you fill this table, please let us know if you have any questions or want clarifications.

Table 2: Sustainability [and Institutionalization] Assessment Framework

COUNTRY	[COUNTRY NAME]
Category	Sustainability [or Institutionalization] Enabling Factor	Measure/Indicator	Response options	Score
POLITICAL WILL AND PUBLIC SUPPORT	Leadership buy-in	How supportive do you feel that this country’s leaders are with regards to strengthening or developing the country’s NPHI?	(1=very unsupportive to 5=very supportive, don’t know )
	Country ownership of NPHI	To what degree do you feel this country’s government perceives it owns the NPHI program?	(1=very little ownership to 5=very strong ownership, don’t know )
		To what degree do you feel this country perceives it owns the NPHI’s data and information systems?	(1=very little ownership to 5=very strong ownership, don’t know )
		How aware do you feel the public is of the NPHI and its role of protecting the population’s health?	(1=very little awareness to 5=very strong awareness, don’t know )
	Structure/Strategy	Do you feel the NPHI supports government health and public health priorities?	(1=little support to 5= high support, don’t know )
		To what extent do feel the NPHI staff understand the NPHI organizational structure and their role in it?	(1=little understanding to 5= high understanding, don’t know )
		In your view, did the NPHI use change management methods to implement modifications to its structure or new strategies?	yes/no/ don’t know
RESOURCES	(Co)-Financing	What percentage of funding for [country] NPHI comes directly from [country] government?	0 to 100%, don’t know
		What percentage of funding for NPHI comes from other sources? (E.g. donors, bilateral/multi-lateral funders, etc.)	0 to 100%, don’t know
		Does the NPHI have a line in the government budget in [country]?	yes/no/ don’t know
		In your view, what is the level of financial management capacity within the [country] NPHI? (E.g. grant writing, financial reports, budget projections, etc.)	(1=very low to 5=very high, don’t know )
		To what degree do you feel most NPHI staff have knowledge of financial processes and understand the importance of these processes to the NPHIs financial health?	(1=not at all to 5=A lot, don’t know)
	People, expertise, equipment, processes	In your view, what is the level of capabilities of the NPHI’s in-country technical support and analytical personnel (e.g. subject matter experts, public health experts, etc.)?	(1=very low to very high, don’t know)
		To what degree do you feel the public health laboratory functions in [country] NPHI have the adequate resources and equipment for ongoing effective operation?	(1=very ineffective to 5=very effective, don’t know)
		To what degree do you feel [country] emergency response during public health emergencies has improved since [country] NPHI development?	(1= not improved to 5= improved a lot, don’t know)
		In your view, what is the level of capacity of in-country technical data support and analytical personnel to manage NPHI surveillance functions?	(1=very low to 5=very high, don’t know )
		In your view, do you feel the skills and experience of technical data support and analytical personnel have improved since NPHI’s development?	(1= not improved to 5= improved a lot, don’t know)
		In your view, do you feel the skills and experience of public health management personnel have improved since NPHI’s development?	(1= not improved to 5= improved a lot, don’t know)
		In your view, do you feel the skills and experience of field epidemiologists have improved since NPHI’s development?	(1= not improved to 5= improved a lot, don’t know )
		In your view do you think the development of NPHIs has affected FETP’s institutionalization, i.e. organization integration within MoH or NPHI?	(1=not at all to 5=a lot, don’t know)
		In your view, is the NPHI staffing plan informed by the NPHI’s strategic plan and address current and future staffing and expertise needed?	(1=not at all to 5=a lot, don’t know )
		To what degree does the NPHI staff have clear development plans and understand ways they can progress in their careers?	(1=not at all to 5=a lot, don’t know)
PROGRAMMATIC OR OPERATIONAL	Strategic Planning	Did NPHI strategic planning engage feedback from a group of diverse stakeholders?	yes/no/ don’t know
		Is there a group that manages implementation of the NPHI’s strategic plan? If yes, how effective is the group?	(1=very ineffective to 5=very effective, don’t know)
	Operational Planning	How satisfied are you with the government’s commitment to implementing the NPHI’s operational plan?	(1=very unsatisfied to 5=very satisfied, don’t know )
		To what degree do you feel NPHI/MOH leaders understand NPHI's goals and functions?	(1=not at all to 5=a lot, don’t know )
		To what degree do you feel NPHI/MOH staff understand NPHI's goals and functions?	(1=not at all to 5=a lot, don’t know )
	Accountability	To what degree do you feel that the government uses data from [country] NPHI to inform, improve, or respond to the country’s various public health needs?	(1=doesn’t use data at all to 5 = uses data a lot for this, don’t know)
		To what degree do you feel that actions and decisions taken by NPHI officials are subject to oversight so as to guarantee that government initiatives meet their stated objectives and respond to the needs of the community?	(1=not at all to 5=a lot, don’t know )
		To what degree do you feel the NPHI officials provide information about their decisions and actions to the public and those institutions of accountability tasked with providing oversight?	(1=not at all to 5=a lot, don’t know)
	Stakeholder Involvement	To what degree are international stakeholders (U.S. CDC, IANPHI, and other NPHIs) engaged with the various functions and process of the country’s NPHI?	(1=not at all involved to 5 = very involved, don’t know)
		To what degree are local organizations or universities engaged with the various functions and processes of the country’s NPHI?	(1=not at all involved to 5 = very involved, don’t know)
		To what degree are other ministries and other domestic stakeholders (e.g. private sector, NGOs, etc.) engaged in various functions and processes of the country’s NPHI?	(1=not at all involved to 5 = very involved, don’t know)
	Network Platforms	In your view, what is the level of communication and collaboration between [country]’s NPHI and other NPHIs?	(1=very low to 5=very high, don’t know )
		Does the country NPHI have a communications platform with other NPHIs during a cross-border public health emergency?	yes/no/don’t know
	Public Health Report	Does the country have a regular (e.g. weekly, monthly) report on public health data?	yes/no/ don’t know
		Do you feel the country’s national data report provides information to the public on current health threats, what the government is doing to reduce the threats, and how the public can protect themselves?	(1= disagree strongly; to 5=agree strongly; don’t know )
		Do you feel data sharing by government is encouraged (i.e. has sustained funding, strong stakeholder support, adequate staff, etc.?	(1= disagree strongly to 5=agree strongly; don’t know )

3. Closing Remarks

1. Thank you very much for your time and important insights. Is there anything else we did not talk about today that you would like to discuss? For instance, is there anything else you think the U.S. CDC’s NPHI program should be aware of with respect to [insert country]’s NPHI(s)?

2. Is there anyone else you think we should speak to in order to understand the big picture (specifically, colleagues in your country?)

3. Please do not hesitate to get in touch with us by phone or email if you think of anything else we should know. Information on how to contact Dr. Mahlet Woldetsadik is provided in the informed consent form that was shared with you. Thank you again!

Additional notes/comments:

1 U.S. Centers for Disease Control and Prevention/ Center for Global Health/ Division of Global Health Protection/ Workforce and Institute Development Branch

2 Surveillance, public health laboratory, emergency preparedness & response, and workforce development