NPHII Assessment - Word

Attachment I. NPHII Annual Assessment (Word Version)

Form Approved

OMB No. 0920-0879

Expiration Date: 03/31/2014

NPHII Annual Assessment of Performance Management and Improvement Practices

Thank you for participating in the National Public Health Improvement Initiative Annual Assessment. Your feedback will help the Centers for Disease Control and Prevention (CDC) and the National Network of Public Health Institutes (NNPHI) gather information about the impact of the performance improvement activities your agency engaged in during the past NPHII funding year. This questionnaire is estimated to take approximately 20-25 minutes. Your open and honest feedback in this assessment is appreciated and will provide critical information in assessing the impact of the NPHII project. As a recipient of NPHII funding, it is expected that you will participate in evaluation activities.

This assessment is being conducted on behalf of CDC and NNPHI by NORC at the University of Chicago. NORC is a not-for-profit social science research organization affiliated with the University of Chicago. Please feel free to contact us if you have any questions:

• For assessment technical support, please contact Arika Garg ([email protected], 301-634-9479).

• For information about this assessment, please contact Anita McLees ([email protected] , 404-498-0316) or Nikki Lawhorn Rider ([email protected] , 251-928-8534).

Instructions

You do not have to complete the assessment in one sitting; you will be able to save it and return at a later point. Pages of the assessment are saved automatically when you progress to the next section. Therefore, in order to save your answers, scroll to the bottom of the page and click on ‘Next’. You will see the next section of the assessment on your screen. The previous page has been saved and you may exit the assessment and return to it at any time. (To make sure your responses are saved, please use the “Next” and “Previous” buttons in the assessment. If you use your Internet browser’s forward and back buttons, the pages may not save.)

Throughout the assessment, some of the terms appear in underlined, italicized font. These terms are defined in the next screen of the assessment. Those definitions are also available in the PDF version of the assessment that was sent to you earlier.

Please note that each organization is being sent a unique link. If you lose this link or have difficulty accessing the assessment, please contact Arika Garg ([email protected], 301-634-9479). While you are encouraged to ask your colleagues for information, each organization can only complete one assessment.

How will the findings from this assessment be used?

NORC will be providing CDC and NNPHI with the data collected in this assessment, including identifiers. However, the information collected in this assessment focuses on your official capacity and the activities of your organization. There are no foreseeable risks to you in participating in this assessment and your answers are needed to enable comparisons with previously collected data and to establish an accurate, specific reference point for future evaluations of NPHII. We anticipate that CDC and NNPHI will use these data for two purposes—first, data will inform the broader NPHII evaluation in order to help CDC and NNPHI better understand the impact of the NPHII. Second, CDC and the NPHII national partners may use the information in this assessment to identify technical assistance needs and provide support to grantees. Aggregate findings from the assessment will be shared with grantees. Findings from this assessment may also be included in reports for NNPHI, CDC, and the national partners (American Public Health Association, Association of State and Territorial Health Officials, National Association of County and City Health Officials, and Public Health Foundation). Findings may also be synthesized in articles and reports that may be publicly available. When the findings are reported publicly, data and quotations will not be linked to the identity of a particular respondent or organization; however please be aware that due to the small sample size, organizations may be indirectly identifiable.

Definitions: Components of a Performance Management System

These definitions were adapted from a variety of sources.

• A performance management system is the continuous use of performance standards, performance measures, routine performance reports and quality improvement so that they are integrated into a public health department’s core operations.

• Performance measures are quantitative measures or indicators of capacities, processes, or outcomes relevant to the assessment of an established performance goal or objective (e.g., time to award contracts, number of staff hours required to complete a specific process or deliver a specific service, percentage of target population that has been offered, received, or completed a specific public health service or program).

• Performance standards are objective standards or guidelines that are used to assess an organization’s performance (e.g., 100% of contracts awarded within 30 days, 100% of children receive all required vaccines upon entry into kindergarten). Standards may be set by benchmarking against similar organizations, or based on national, state/territory, or scientific guidelines.

• Quality improvement refers to a formal, systematic approach—such as plan-do-check-act—applied to the processes underlying public health programs and services in order to achieve measurable improvements.

• Routine performance reporting means regular documentation and reporting of progress in meeting standards and targets, and sharing of such information through feedback.

Organization: Throughout this document, the term “organization” is used to refer to the entities that were awarded National Public Health Improvement Initiative (NPHII) funding. Awardees include 49 health departments representing 48 states and the District of Columbia, nine local health departments, eight organizations representing the US Territories and Pacific Islands, and eight organizations representing tribal health departments or consortiums.

SECTION ONE: PERFORMANCE IMPROVEMENT MANAGER

1. Please identify the name of your organization: ______________________________

2. Are you the organization’s Performance Improvement Manager?

1. Yes, and I was working at this organization prior to October 2010.

2. Yes, and I was first hired to work at this organization through NPHII funding.

3. No, our organization has not hired a Performance Improvement Manager.

[IF YES (1 or 2), GO TO QUESTION 3]

If you are not the Performance Improvement Manager, what role do you play in support of NPHII? _____________________________

[AFTER ANSWERING THIS QUESTION, SKIP TO QUESTION 15]

3. Did you personally complete the baseline assessment (March/April 2011) or year one assessment (November/December 2011) for the NPHII program? (These assessments were conducted using a similar online format.)

1. Yes, only baseline

2. Yes, only year one

3. Yes, both baseline and year one

4. No

5. I don’t remember

[IF YES (1, 2, or 3), SKIP TO QUESTION 7, IF NO/DON’T REMEMBER (4 or 5), GO TO QUESTION 4]

Tenure and Formal Training

4. How long have you been working in public health?

1. One year or less

2. More than one year but less than six years

3. Six or more years but less than 10 years

4. 10 or more years but less than 16 years

5. 16 or more years but less than 20 years

6. 20 years or more

5. How long have you been working at your current organization?

1. One year or less

2. More than one year but less than six years

3. Six or more years but less than 10 years

4. 10 or more years but less than 16 years

5. 16 or more years but less than 20 years

6. 20 years or more

6. Please indicate the length of time that you have been working on quality improvement activities, in public health and/or other fields.

1. One year or less

2. More than one year but less than three years

3. Three or more years but less than five years

4. Five or more years but less than eight years

5. Eight or more years but less than 10 years

6. 10 or more years

7. I have not worked on quality improvement activities

Competencies

For each of the following competencies, think about your own knowledge, skill, or ability level. Then rate your level of proficiency on each competency using the scale below:

0. N/A: This competency is not applicable to my job as a Performance Improvement Manager; I do not do this.

1. None: I am unaware of, or have very little knowledge of, this competency; I do not understand the relationship between this competency and my role as a Performance Improvement Manager.

2. Aware: I have heard of this competency; my knowledge of or ability to perform this competency is limited. In order to better fulfill my role as a Performance Improvement Manager, I need more information about this topic.

3. Knowledgeable: I am comfortable with my knowledge of this topic or my ability to apply the skill; I use this knowledge or skill on a regular basis in my job.

4. Proficient: I am very comfortable with, or expert on, this knowledge or skill; I could teach it to others.

Please rate your level of proficiency on the following competencies.

	N/A	None	Aware	Knowledgeable	Proficient
	0	1	2	3	4
Implement organizational and system-wide strategies for continuous quality improvement
Implement mechanisms to monitor and evaluate programs for their effectiveness and quality
Use evaluation results to improve performance
Integrate data and information to improve organizational processes and performance
Use cost-effectiveness, cost-benefit, and cost-utility analyses in programmatic prioritization and decision making
Establish a performance management system
Integrate “systems thinking” into public health practice (e.g., cross-programmatic, cross-organizational approaches)
Ensure the measuring, reporting, and continuous improvement of organizational performance

SECTION TWO: PHAB accreditation

15. What communications and activities has your organization undertaken in the last year that will increase your readiness for accreditation? (Please select all that apply.)

1. Conducted communications or meetings with leadership

2. Conducted communications or meetings with staff

3. Implemented activities to complete the Public Health Accreditation Board (PHAB) Readiness Checklist

4. Designated individual(s) to coordinate accreditation readiness activities

5. Developed a timeline for the agency’s application to PHAB’s accreditation program

6. Developed a “roadmap” for the agency’s application to PHAB’s accreditation program

7. Organized agency documentation for accreditation

8. Promoted accreditation readiness activities among other health departments in our jurisdiction

9. Promoted accreditation readiness activities among other health departments outside our jurisdiction

10. Submitted a statement of intent to pursue PHAB accreditation

11. We have not undertaken any of these activities

12. Other

13. I don’t know

If you chose “Other,” please specify.

SECTION THREE: Performance Management Systems and Quality Improvement

We are interested in the extent to which your organization has established any or all components of an ORGANIZATION-WIDE performance management system to systematically assess and improve performance across processes or programs. For each component, please respond to the following questions.

16. Has your organization established organization-wide performance standards?

1. Yes

2. No

3. I don’t know

[IF YES (1), GO TO 17, ELSE (2 or 3) SKIP TO 18]

17. When did your organization establish organization-wide performance standards?

1. Established in past funding year only

2. Established prior to past funding year and not updated in past year

3. Established prior to past funding year and updated in past year

4. Not yet established

18. Has your organization established organization-wide performance measures?

1. Yes

2. No

3. I don’t know

[IF YES (1), GO TO 19, ELSE (2 or 3) SKIP TO 20]

19. When did your organization establish organization-wide performance measures?

1. Established in past funding year only

2. Established prior to past funding year and not updated in past year

3. Established prior to past funding year and updated in past year

4. Not yet established

20. Has your organization established organization-wide routine performance reporting?

1. Yes

2. No

3. I don’t know

[IF YES (1), GO TO 21, ELSE (2 or 3) SKIP TO 22]

21. When did your organization establish organization-wide routine performance reporting?

1. Established in past funding year only

2. Established prior to past funding year and not updated in past year

3. Established prior to past funding year and updated in past year

4. Not yet established

22. Has your organization established an organization-wide process for quality improvement?

1. Yes

2. No

3. I don’t know

[IF YES (1), GO TO 23, ELSE (2 or 3) SKIP TO 24]

23. When did your organization establish an organization-wide process for quality improvement?

1. Established in past funding year only

2. Established prior to past funding year and not updated in past year

3. Established prior to past funding year and updated in past year

4. Not yet established

24. What are the top three challenges that your organization experienced in the past funding year regarding implementing performance management on an organization-wide basis? (Please select no more than three options.)

1. Limited staff available for this work

2. Limited number of staff trained in performance management or quality improvement

3. Staff attitudes that do not support this work

4. Turnover in staff or leadership

5. Lack of leadership buy-in

6. Economic barriers (e.g., staffing cuts, reorganization, elimination of services)

7. Political environment that does not support this initiative

8. Competing priorities

9. Public health crises (e.g., outbreak, natural disaster)

10. Not knowing how or where to begin

11. Other

12. Our public health organization has not experienced any of these barriers

If you chose “Other,” please specify.

25. In the past funding year, has your organization used performance reports from your performance management system for any of the following purposes? (Please select all that apply.)

1. Developing administrative regulations

2. Developing agency or tribal policy (e.g., informing policies for organizational or public health improvement)

3. Establishing health priorities and plans

4. Monitoring program or project performance

5. Identifying level of investment needed or return on investments

6. Allocating funds

7. Administering programs

8. Other

9. None of the above

10. I don’t know

If you chose “Other,” please specify.

26. The following organizations’ resources support performance management and quality improvement efforts. Please indicate below those organizations / resources that have been useful to your organization’s performance management or quality improvement efforts in the past funding year. (Please select all that apply.)

1. Public Health Foundation resources, such as Public Health Memory Jogger or Public Health Quality Improvement Handbook

2. National Network of Public Health Institutes resources from the Multi-State Learning Collaborative and Public Health Performance Improvement Toolkit

3. Association of State and Territorial Health Officials (ASTHO) resources, such as the ASTHO Accreditation and Performance Improvement Guide

4. National Association of County and City Health Officials (NACCHO) resources, such as NACCHO Accreditation and Quality Improvement Toolkit, quality improvement webcasts, or Mobilizing for Action through Planning and Partnerships

5. Institute for Healthcare Improvement resources, including Seven Leadership Leverage Points and other public health improvement tools

6. American Society for Quality resources, including The Quality Toolbox and other quality improvement tools

7. Michigan Local Public Health Accreditation Program resources, such as Embracing Quality in Public Health: A Practitioners Quality Improvement Guidebook

8. CDC’s Public Health Law Program resources

9. Network for Public Health Law resources

10. Other organization, program, or resource

11. My organization has not used any of these resources

12. I don’t know

If you chose “Other organization, program, or resource ,” please specify.

27. There are many different tools and techniques that can be used for performance management and quality improvement. Have the following tools or techniques for quality improvement been used in your organization in the past funding year? (Please select all that apply)

Tool	Yes	No	I don’t know
Affinity diagram
Brainstorming
Cause and effect diagrams
Check sheet
Control chart
Control and influence plots
Flow chart
Fishbone diagram
Five whys
Force field analysis
Histogram
Interrelationship digraph
Know and don’t know matrix
Multi-voting technique
Pareto chart
Process Decision Program chart
Prioritization matrix
Process maps
Radar chart
Root cause analysis
Run chart
Scatter diagram
SMART chart
SWOT (i.e., strengths, weaknesses, opportunities, threats) analysis
Surveys
Tree diagrams
Other
None of the above

If you chose “Other,” please specify.

28. In addition to specific tools, there are many different methods that can be used for performance management and quality improvement. Have the following methods for quality improvement been used in your organization in the past funding year? (Please select all that apply.)

Method	Yes	No	I don’t know
Adaptive Promising Practice
Baldridge Performance Excellence Criteria or state version Balance Scorecard
Business Process Analysis
Kaizen Event
Lean/Six Sigma
Model for Improvement
Plan, Do, Check/Study, Act Cycle
Rapid Cycle Improvement
Standarize-Do-Check-Act Cycle
Total Quality Management
Turning Point Performance Management Framework
Other
None of the above

If you chose “Other,” please specify.

28. Approximately what percentage of your organization’s staff has received training in quality improvement methods or tools in the past funding year?

1. None

2. 1–25%

3. 26–50%

4. 51–75%

5. 76–100%

6. I don’t know

28. Approximately what percentage of your organization’s staff has received training in quality improvement methods or tools ever?

    1. None

    2. 1–25%

    3. 26–50%

    4. 51–75%

    5. 76–100%

    6. I don’t know

28. Approximately what percentage of other public health organizations within your jurisdiction has your organization trained in quality improvement methods in the past funding year?

1. None

2. 1–25%

3. 26–50%

4. 51–75%

5. 76–100%

6. I don’t know

28. In the past funding year, have your organization’s performance or quality improvement efforts focused on increasing efficiencies, such as saving time, saving money, etc.?

1. Yes

2. No

3. I don’t know

[IF YES (1), GO TO 33, ELSE (2 or 3) SKIP TO 35]

28. The rows in the following table list potential outcomes associated with increased efficiencies. For each outcome, please select the response option that best represents your NPHII-funded work towards that outcome during the past funding year. If you have not implemented performance or quality improvement efforts targeting the select outcome in the past funding year, select “not applicable.”

	In progress	Completed with measurable outcomes	Not applicable
Saving time (i.e., reducing time to complete a process or deliver a service)
Saving money (i.e., decreasing cost of process implementation or service delivery)
Increasing revenue
Reducing the number of steps required to complete a process or deliver a service
Reducing staff hours required to complete a process or deliver services
Other

If you chose “Other,” please specify.

28. For the outcomes above that you identified as having completed with measurable outcomes (or those that are still in progress if none are completed), please 1) identify your best example, 2) describe the quality improvement method(s) used, and 3) describe the results obtained.

28. In the past funding year, have your organization’s performance or quality improvement efforts focused on improving effectiveness of programs, services, or processes, such as increased customer satisfaction, increased reach of service delivery, etc.?

1. Yes

2. No

3. I don’t know

[IF YES (1), GO TO QUESTION 36, ELSE (2 or 3) SKIP TO QUESTION 38]

28. The rows in the following table list potential outcomes associated with improved effectiveness of programs, services, or processes. For each outcome, please select the response option that best represents your NPHII-funded work towards that outcome during the past funding year. If you have not implemented performance or quality improvement efforts targeting the select outcome in the past funding year, select “not applicable.”

	In progress	Completed with measurable outcomes	Not applicable
Increased staff satisfaction
Increased customer satisfaction
Increased reach of service delivery
Quality enhancement of information systems or service delivery
Increase in funds leveraged
Increase in preventive behaviors
Decrease in incidence or prevalence of disease
Other

If you chose “Other,” please specify

28. For the outcomes above that you identified as having completed with measurable outcomes (or those that are still in progress if none are completed), please 1) identify your best example, 2) describe the quality improvement method(s) used, and 3) describe the results obtained.

SECTION FOUR: Environment for Quality Improvement and Performance Management

This section focuses on how the environment within your organization currently supports quality improvement, and efforts currently underway to build quality improvement capacity and implement improvement initiatives. Please rate your agreement with the following statements.

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
	1	2	3	4	5
38. Leaders (e.g., board, senior management team) of my public health department are receptive to new ideas for improving organization programs, services, and outcomes.					
39. The impetus for improving quality in my organization is largely driven by an internal desire to make our services and outcomes better.					
40.The board or management team of my organization work together for common goals.					
41.Staff consult with and help one another to solve problems.					
42. Staff members are routinely asked to contribute to decisions at my organization.					
43. The leaders of my organization are trained in basic methods for evaluating and improving quality, such as Plan-Do-Study-Act.					
44. Staff at my organization who provide public health services are trained in basic methods for evaluating and improving quality, such as Plan-Do-Study-Act.					
45. Many individuals responsible for programs and services in my organization have the skills needed to assess the quality of their program and services.					
46. My organization has objective measures for determining the quality of many programs and services.					

47. Many individuals responsible for programs and services at my organization routinely use systematic methods (e.g., root cause analysis) to understand the root causes of problems.						
48. Many individuals responsible for programs and services at my organization routinely use best or promising practices when selecting interventions for improving quality.						
49. Programs and services are continuously evaluated to see if they are working as intended and are effective.						
50. The quality of many programs and services in my organization is routinely monitored.						
51. My organization has a quality improvement council, committee, or team.						
52. My organization has a quality improvement plan.						
53. Job descriptions for many individuals responsible for programs and services at my organization include specific responsibilities related to measuring and improving quality.						
54. Good ideas for measuring and improving quality in one program or service USUALLY are adopted by other programs or services in my organization.						
55. Staff members at all levels participate in quality improvement efforts.						
56. Customer satisfaction information is routinely used by many individuals responsible for programs and services in my organization.						
57. Accurate and timely data are available for program managers to evaluate the quality of their services on an ongoing basis.						
58. Improving quality is well integrated into the way many individuals responsible for programs and services work in my organization.						
59. Agency staff is aware of external quality improvement expertise that can help measure and improve quality.						
60. Spending time and resources on quality improvement is worth the effort.						
61. The key decision makers in my organization believe quality improvement is very important.						
62. Using quality improvement approaches will impact the health of my community.						
63. Organization staff and stakeholders will notice changes in programs and services as a result of our quality improvement efforts.						
64. Many individuals responsible for programs and services in my organization have the authority to change practices or inform and/or educate about policies to improve services within their areas of responsibility.						
65. When trying to facilitate change, staff has the authority to work within and across program boundaries.						

66. Please briefly provide any additional information about your organization’s NPHII cooperative agreement activities that you would like to share that have not been addressed by this survey.

Thank you!

Thank you for taking the time to participate in the annual assessment for NPHII. This annual assessment is being conducted on behalf of CDC and NNPHI by NORC at the University of Chicago, a not-for-profit research organization. Please feel free to contact us if you have questions:

• For technical support, please contact Arika Garg ([email protected], 301-634-9479)

• For information about this assessment, please contact Anita Lees ([email protected], 404-498-0316) or Nikki Lawhorn Rider ([email protected], 251-928-8534).

Public reporting burden of this collection of information is estimated to average 25 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Information Collection Review Office, 1600 Clifton Road NE, MS D- 74, Atlanta, Georgia 30333; ATTN: PRA (0920-0879).

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	DOMAIN ONE: PERFORMANCE IMPROVEMENT MANAGER
Author	zdu5
File Modified	0000-00-00
File Created	2021-01-31