Evaluation questions

Attachment G. NPHII Evaluation Questions

NPHII Evaluation Questions; Data Sources; Timing of Measurement and Notes

In collaboration with Centers for Disease Control and Prevention (CDC) as well as local, state, and national partners, the National Network of Public Health Institutes (NNPHI) has and will continue to implement a formative evaluation of NPHII to examine the achievements and impact of the program and to inform improvements to the program in future years. The focus of the evaluation is to examine the NPHII investment and its impact on increased efficiencies and effectiveness of programs, processes, and services, and increased readiness for applying for accreditation by the Public Health Accreditation Board (PHAB).

General Note: Much of the ability to currently answer evaluation questions depends on data provided by CDC. The timely reporting of this data and high quality of data provided are critical to the success of the evaluation. Additional data collection approaches proposed will assure the success of the evaluation.

From 8/22/12 meeting, CDC priorities for evaluation include the following:

1. Determining the extent to which grantees are achieving QI outcomes for initiatives to improve efficiency and effectiveness identified in their workplans

2. Examining the relationship between grantee capacity (including the role of the PIM), infrastructure and foundation for performance management and grantee achievement of key outcomes.

3. Examining the successes and challenges related to accreditation readiness work.

4. Examining the contribution of NPHII to improvements in accreditation readiness, efficiency/effectiveness, and organizational capacity and support for performance management / quality improvement.

5. Determining extent to which grantees are successfully implementing their workplans and achieving objectives and performance targets.

NPHII Objective/Content Area 1: Overall Implementation and Progress

Overarching Evaluation Question	Specific Evaluation Question	Data Source(s)	Notes
How much progress have grantees made in their overall implementation of stated activities and in meeting goals and objectives as outlined in their work plans and performance measures?	P - What percent of grantees have met the following requirements of the NPHII Cooperative Agreement? Continue to have a full time equivalent (FTE) Performance Improvement Manager (PIM). Participation in the National Performance Management Network, and What percent of grantees have completed all activities in their work plans? What percent of grantees have demonstrated gains in efficiency and/or effectiveness as evidenced through their performance measures? In which areas have grantees demonstrated, in aggregate, progress? In which areas have grantees encountered challenges to completing activities in their workplans?	Grantee Workplans, performance measures, Assessment, IPR/APR	Semi-annual/Annual
In what ways does NPHII support the establishment of QI/PI as a core organizational capacity in the organization?	In what ways did NPHII support the establishment of QI/PI as a core organizational capacity in the organization?	Assessment, Qualitative approaches
What is the contribution of the NPHII intervention components on improvements in accreditation readiness, efficiency/ effectiveness, and organizational capacity?	0 – In what ways have the following intervention components supported improvements accreditation readiness, efficiency/effectiveness, and organizational capacity for PM/QI (really 3 separate questions): Establishment of a performance management system PIM PIM Network Annual Conference TA/CBA Self-assessment Pre-requisites QI initiatives	Synthesis of available data

Page 2 Revised 09-10-2012 Revised 09-10-2012