National Public Health Performance Standards Program
Local Public Health System Performance Assessment Instrument
(OMB Control Number: 0920-0555)
May 31, 2007
Office of Chief of Public Health Practice
Office of the Director
Centers for Disease Control and Prevention
A. JUSTIFICATION
The Office of Chief of Public Health Practice is requesting a revision and three-year clearance for OMB No. 0920-0555, National Public Health Performance Standards Program, Local Public Health System Performance Assessment instrument.
1. Circumstances Making the Collection of Information Necessary
The mission of the Centers for Disease Control and Prevention (CDC) is to promote health and quality of life by preventing and controlling disease, injury, and disability. The National Public Health Performance Standard Program (NPHPSP) information collection is intended to contribute to this mission by providing optimal standards for public health practice and by measuring the achievement of those standards at the state and local levels. The Local Public Health System Performance Assessment Instrument queries respondents and generates data for use in health policy development, resource allocation, and quality improvement efforts.
State and local public health practice form the backbone of the nation’s health system, but little is known about capacity and performance. The NPHPSP was established to address this problem and is based on the following three principles:
Public health must be accountable to its constituencies.
Public health professionals need a system for assessing the provision of Essential Public Health Services.
The public health decision-making process must be based on strong scientific evidence and assessment of current needs.
The NPHPSP is a volunteer data collection effort. The assessment instruments are designed to collect the evidence necessary to refine the domestic public health infrastructure. During the past decade, CDC has worked with other Department of Health and Human Services (DHHS) agencies, key national public health associations, state and local health officials, boards of health, and academic institutions to explore and better articulate the state and local public health infrastructure. Through the identification of infrastructure objectives for Healthy People 2010, the development of a national public health systems research agenda, and other related efforts, these organizations and constituencies have identified the need for better data on the status of the public health infrastructure. The NPHPSP was designed, in part, to address this urgent need. These assessments facilitate development of a strong national infrastructure that will result in improved national, state, and local capacity to detect and effectively respond to public health threats.
The NPHPSP is intended to help users answer questions such as, “What are the components, activities, competencies, and capacities of our public health system?” and “How well are the Essential Services being provided?” The dialogue that occurs in answering these questions will identify strengths and weaknesses; this information can be used to improve and better coordinate public health activities at the state and local levels. Lastly, the results gathered will provide an understanding of how state and local public health systems and governing entities are performing. This information will help local, state, and national policymakers make better and more effective policy and resource decisions that will improve the nation’s public health as a whole.
The NPHPSP is intended to improve the quality of public health practice and the performance of public health systems by:
Providing performance standards for public health systems and encouraging their widespread use;
Engaging and leveraging national, state, and local partnerships to build a stronger foundation for public health preparedness;
Promoting continuous quality improvement of public health systems; and
Strengthening the science base for public health practice improvement.
The NPHPSP is a collaborative effort of seven national partner organizations:
Centers for Disease Control and Prevention, Office of Chief of Public Health Practice (CDC / OCPHP)1,
American Public Health Association (APHA),
Association of State and Territorial Health Officials (ASTHO),
National Association of County and City Health Officials (NACCHO),
National Association of Local Boards of Health (NALBOH),
National Network of Public Health Institutes (NNPHI), and
Public Health Foundation (PHF).
The NPHPSP includes three instruments:
The State Public Health System Performance Assessment Instrument (State Instrument) focuses on the “state public health system.” This system includes state public health agencies and other partners that contribute to public health services at the state level. The instrument was developed under the leadership of ASTHO and CDC. (OMB Control Number 0920-0557)
The Local Public Health System Performance Assessment Instrument (Local Instrument) focuses on the “local public health system” or all entities that contribute to the delivery of public health services within a community. This system includes all public, private, and voluntary entities, as well as individuals and informal associations. The local instrument was developed under the leadership of CDC and NACCHO. (OMB Control Number 00920-0555)
The Local Public Health Governance Performance Assessment Instrument (Governance Instrument) focuses on the governing body ultimately accountable for public health at the local level. Such governing bodies may include boards of health or county commissioners. The governance instrument was developed under the leadership of CDC and NALBOH. (OMB Control Number 0920-0580)
Although each instrument was developed under the collaborative leadership of a specific partner organization and CDC, all partners were involved throughout the entire process. Additionally, the instruments were collectively reviewed to ensure that each is complementary and supportive of the others and includes consistent terminology and concepts.
The national partners represent many of the organizations and individuals that use the assessment instruments. Through working groups and field test activities, representatives from these organizations have been continuously involved in developing, reviewing, testing, and refining the instruments. A peer-guided development process occurred during 1998-2002 during the development of the original instruments; this process was replicated recently during the recent revision activities of each instrument.
During 2005-2006, the three NPHPSP instruments were updated based on experience from the field and new developments in public health practice. Updates were undertaken for each of the three NPHPSP instruments: the state public health system assessment, local public health system assessment and local governance assessment. Three work groups of practitioners (representing ASTHO, NACCHO, and NALBOH constituencies) were convened to oversee each set of updates. The general purpose of the process was to assure the standards remain current and also to improve the language and user-friendliness of the instruments. During the revision process, CDC also worked with subject matter experts and key organizations to determine content areas that needed to be updated or modernized. Expert input was solicited in areas such as preparedness, informatics, health marketing, partnerships, workforce, public health law, and laboratory issues. As a result of this entire process, new versions of the instrument (subsequently referred to as “Version 2”) were developed.
A limited field testing process, using eight repeat local sites, was undertaken to identify areas for improvement within the instruments, assess the extent to which improvements in utility have been achieved, demonstrate a longitudinal linkage to the currently available instruments, assess the impact of changes made, and gather an understanding of the implementation process related to the updated instruments (including information to inform a revised time burden for OMB clearance).
The use of the NPHPSP instruments is intended to result in numerous benefits, including:
Improving organizational and community communication and collaboration, by bringing partners to the same table.
Strengthening the diverse network of partners within state and local public health systems, which can lead to more cohesion among partners, better coordination of activities and resources, and less duplication of services.
Providing a mechanism for measuring public health practice and performance.
Identifying strengths and weaknesses that can be addressed in quality improvement efforts.
Providing a benchmark for public health practice improvements, by setting a “gold standard” to which public health systems can aspire.
There are four concepts that have helped frame the National Public Health Performance Standards into their current format:
The standards are designed around the ten Essential Public Health Services. The use of the Essential Services framework assures that the standards cover the gamut of public health action needed at state and community levels. 2,
The standards focus on the overall public health system, rather than a single organization. A public health system includes all public, private, and voluntary entities that contribute to public health activities within a given area. This ensures that the contributions of all entities are recognized in assessing the provision of essential public health services.
The standards describe an optimal level of performance rather than provide minimum expectations. This ensures that the standards can be used for continuous quality improvement.
The standards are intended to support a process of quality improvement. In responding to the questions, system partners determine which elements of the model standards they do/ do not meet. They then should develop action plans for improving their performance in the low-scoring areas.
2. Purpose and Use of Information Collection
This data collection is authorized under Section 301 of the Public Service Act (42, USC 241) (Attachment A). The CDC, state, and local public health systems use this instrument to assess the capacity of public health systems to deliver the ten Essential Public Health Services, and assist in targeting resource investments. This instrument assists both agencies and public health systems representatives. The public health agency submits on behalf of the public health system. The public health system is defined as “The collection of public, private and voluntary entities, as well as individuals and informal associations that contribute to the public’s health within a jurisdiction.”
The NPHPSP is applied as part of a public health system self-assessment process. Public health systems and local boards of health voluntarily conduct data collections for infrastructure self-assessment and quality improvement. CDC and NPHPSP partners support the process by providing technical assistance and training tools, computer-generated data analysis and reports of results.
States and localities self-select and participate voluntarily. Local jurisdictions can choose to undertake this individually, but the process is generally encouraged through a statewide coordinated approach. The concept of using the local instrument through a statewide process is critical since it assists states and localities in maximizing the assessment results for planning and improvement.
States and localities can also undertake the NPHPSP Local Assessment as part of a broader effort to implement the NACCHO Mobilizing for Action Through Planning and Partnerships (MAPP) process. MAPP is a strategic planning tool principally developed through a cooperative agreement between NACCHO and the CDC. The Local Public Health System Performance Assessment Instrument (local instrument) is one of four assessments in the MAPP strategic planning process. Within MAPP, the local instrument is used to define the activities, competencies, and capacities of local public health systems and the results are used as part of a broad community health improvement plan.
Regardless of whether the instrument is supported through a statewide approach or by a local jurisdiction volunteering to undertake the assessment individually, the assessment and data collection are accomplished in the same manner. CDC recommends that the governmental public health agency serve as the lead organization in submitting the instrument responses, although in some jurisdictions other entities have been empowered with this authority. Technical assistance resources, such as the training workshops, the User Guide and NPHPSP staff, instruct responding jurisdictions in how to complete the assessment. Jurisdictions are also informed that they will need a User ID and survey password to enter data into the limited-access website. If they are completing the assessment through a statewide approach, CDC provides the User IDs to the state coordinator. Other jurisdictions can contact CDC and PHF (a partner that assists in providing technical support for the limited-access website and reporting system) directly at 1-800-747-7649 or by email at [email protected] or [email protected]. The User IDs are disseminated with an instruction sheet (see Attachment G).
Since the national release in July 2002, 521 local jurisdictions have used the local assessment instrument and submitted data to CDC. The majority of these local jurisdictions conducted the assessment as part of a statewide process, with encouragement or support from their state health department or another state entity such as a state association. The concept of using the local instrument through a statewide process is critical since it assists states and localities in maximizing the assessment results for planning and improvement.
Furthermore, the Public Health Infrastructure chapter in Healthy People 2010 includes an objective measuring the use of the performance standards. Objective 23-11 cites the NPHPSP as its sole data source and seeks to monitor and provide targets for state and local public health systems that use the national standards. Without continued data collection, there will be no ongoing data source for this important national objective. The NPHPSP also has been discussed as an important tool in significant national policy documents such as the 2003 Institute of Medicine’s report, The Future of the Public’s Health in the 21st Century.
Other strategic linkages have been made to best assure the utmost value of the NPHPSP assessment instruments. In states such as Colorado, Oklahoma, and New Mexico, the NPHPSP was used as part of their bioterrorism planning activities to identify statewide strengths and weaknesses and priorities for public health infrastructure improvement. Further, the Local Instrument is used within a community health improvement tool entitled Mobilizing for Action through Planning and Partnerships (MAPP). MAPP includes four complementary assessments – one of which uses the NPHPSP local instrument to measure the performance of the local public health system. The linkage between MAPP and the NPHPSP encourages the use of performance standards within the context of a broader health improvement effort. The Turning Point Performance Management Collaborative (PMC), funded by the Robert Wood Johnson Foundation, also has emphasized the role the NPHPSP standards can play in improvement efforts.
Additionally, the NPHPSP standards have been incorporated into state regulations or legislation for public health infrastructure. This includes states such as Ohio, Illinois, and New Jersey. Both Ohio and New Jersey have incorporated state and local use of the NPHPSP as a required component of larger performance improvement and infrastructure reform efforts. Illinois has included the performance standards as a required assessment in their newly enacted State Health Improvement Plan Act. In these and many other states, the NPHPSP is having a positive impact as a tool for assessing the performance of public health systems.
Finally and most notably, the NPHPSP is also playing a valuable and critical role within new efforts to strengthen public health practice. CDC and its national partners (led by NACCHO, ASTHO, NALBOH, and APHA) have been engaged in an effort to design and build a national accreditation system. The model, which has been approved by a national steering committee, mentions the NPHPSP standards as an important building block for developing the accreditation standards as well as a critical tool for preparing the field for accreditation. Additionally, there is great interest in assuring that the NPHPSP remains focused on strengthening the public health system while accreditation focuses, in a complementary manner, on strengthening state and local public health agencies.
Due to the many factors mentioned above, there has been increasing momentum in the voluntary use of the performance standards. The NPHPSP has held annual training workshops; attendance has grown steadily during the four years that this workshop has been offered. The Fourth Annual NPHPSP Training Workshop in April 2006 hosted 80 participants from 28 different states; registration and interest has grown so much that NPHPSP partners have created concurrent topic tracks for workshop attendees so as to retain the small-setting value of such a practical in-person training workshop. The availability of a stable NPHPSP instrument will enable this momentum to continue.
In the years since the initial request for approval of data collection and the subsequent release of the NPHPSP state and local instruments in July 2002, the landscape of public health has changed considerably. The attention of public health leaders – at the national, state, and local levels – has been significantly diverted to bioterrorism preparedness and planning, in addition to other emerging public health issues such as West Nile Virus, the flu vaccine shortage, and SARS. While bioterrorism investments have increased dramatically since 2002, state and local public health revenues have suffered from major state fiscal shortfalls. This series of unanticipated challenges for public health agencies slowed the early opportunities to use the NPHPSP after their 2002 launch. However, despite this diverted attention, it is vitally important to recognize and acknowledge the substantial and invaluable contributions the NPHPSP has made in providing public health leaders, policy makers, and program staff in a multitude of jurisdictions with an effective and efficient assessment and quality improvement process. In fact, many of the sites that voluntarily chose to use the NPHPSP did so in an effort to better respond to the changing landscape of public health.
Although it is described above that a statewide coordinated approach is the ideal context for using the NPHPSP, this approach also requires a time investment in order to appropriately plan and coordinate the timing of such a statewide process. Many statewide coordinators will seek to use the assessment instruments during a key juncture in a multi-year health improvement and planning cycle. For example, Virginia has incorporated their use of the NPHPSP into a larger state and local performance measurement and improvement program, which establishes a timeline stating their intent to use the NPHPSP in 2008. Other states have sought to incorporate the NPHPSP into legislation prior to initiating its use. As an example, the Illinois State Health Improvement Plan Act, enacted in August 2004, supports the use of the performance standards as a tool in improving the public health system and health status of Illinois residents. However, the time required to develop and enact this legislation delayed Illinois’ initiation of the assessment instruments. From this experience CDC and the national partners have learned that the time required to prepare for the optimal timing of the NPHPSP assessment has delayed some states’ use and they are now planning on initiating the process. Continued availability of the data collection instrument will assist them in assuring this is feasible.
Since its release in 2002, it is clear that the NPHPSP has proven itself an important step toward achieving more consistently effective, high-performing public health systems in the United States. By providing national performance standards, a means for jurisdictions to assess their performance, and a catalyst for improvement strategies, the NPHPSP supports performance improvement and accountability of public health practice at both the state and local levels. The updated instruments, which have been revised to reflect recent changes in public health practice and have been streamlined to improve user-friendliness, are heavily anticipated by state and local jurisdictions. The availability of these tools and the data collection instrument is critical to sustaining our ability to support these efforts and to build our understanding of public health practice. With the release of the revised Version 2 NPHPSP instruments, we anticipate that the instruments and additional responses will yield a significantly stronger database for research and improvement opportunities.
3. Use of Improved Information Technology and Burden Reduction
To minimize respondent burden, the surveys are web-based. Data collection, analysis, and reporting are automated. The web-based survey is the preferred method of choice; however, CDC recognizes that compliance depends on the availability of appropriate technology at the state and local levels. In areas with limited access to technology, CDC in conjunction with a state liaison can provide technical assistance using hard copy surveys. The coordinated efforts of CDC and state liaisons along with automated data processing minimize the burden to respondent.
4. Efforts to Identify Duplication and Use of Similar Information
Extensive literature searches were conducted using online databases such as Medline, Psychino, and Sociofile. No duplicative data collection has been conducted to date. Certain national organizations like ASTHO and NACCHO have published profiles of state and local health departments but such data does not contain in-depth information on performance or infrastructure. According to literature searches and evidence provided by public health systems research, to date, no duplicative assessment has been conducted. Although several states have developed performance assessment tools, none of the efforts duplicate the ability to collect and compare national data using a common assessment framework.
The NPHPSP instruments are designed to collect and analyze performance data, and to improve system-wide performance. These tools provide a common framework for measuring performance, define the desired optimal level of public health practice, provide practitioners with specific information on areas for improvement, provide objective data for development of health policy, and provide information to decision-makers to better target the use of their resources.
We are submitting similar requests for the State Public Health System Performance Assessment and the Local Governance Performance Assessment. The primary difference between the instruments is the scope. The local instrument focuses on the local public health system or all the entities that contribute to the delivery of public health services within a community. The local governance instrument focuses on the board of health or other similar governing entity. The state instrument focuses on the state public health system. The state public health system includes state public health agencies and other partners that contribute to public health services at the state level.
5. Impact on Small Businesses or Other Small Entities
The respondents for this survey will be local health departments. There are no small entities involved.
6. Consequences of Collecting the Information Less Frequently
It is critical to assure the data collection instruments remain viable instruments for the field of public health practice and for the jurisdictions that are seeking to use these tools to understand their state public health system. Numerous states are planning for use of the State Instrument and, if not available, will not have a viable tool for health improvement planning. Some have incorporated the use of the NPHPSP instruments into their legislation or regulation and therefore require its use during prescribed times. Other states have written the use of NPHPSP instruments into federal cooperative agreements and require their use during 2007-08; there will be consequences for these states’ ability to comply with their federal grant deliverables if the instruments are not available for use. Additionally, if the information collection is undertaken on a less frequent basis, there will be consequences to the availability of current knowledge about state and local public health systems and boards of health. There are no legal obstacles to reduce burden.
7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
This assessment is expected to take 16 hours to be completed by a workgroup. The table below explains the burden hours.
Explanation of Burden Hours |
TASK |
Time Needed (in Hrs) |
Explanation |
Time for reviewing instructions and preparing for the assessment |
3 |
Based on field test experience it requires 3 hours to review the instructions and prepare for the assessment. This assessment is a group effort, therefore, this step includes initial contact and orientation of workgroup members regarding the effort. |
Review data sources, discuss, and respond to questions |
10 |
This assessment will require input from a number of different data sources and partners. The various entities that contribute to the delivery of public health services within a community will make up a workgroup that is responsible for this collective response. During this step, the workgroup compiles information based on their own knowledge and data sources and then discusses the input in order to make a well-informed decision. |
Completing and submitting the information
|
3 |
Based on field test results, it takes about 3 hours to review the information and enter this information into the on-line database for analysis.
|
Total Hours |
16 |
|
The table above was produced based on field test results from eight sites, which indicate that there is a slightly decreased burden (16 hours instead of 24 hours) for the revised Version 2 instrument. The currently approved Version 1 instrument has a burden estimate of 24 hours. CDC and its NPHPSP partner organizations have found that local jurisdictions’ use since the national release in 2002 continue to support these burden estimates.
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
8a: Federal Register Notice
The 60-day Federal Register Notice was published on December 22, 2006, Vol. 71, No. 246, pages 77024-77025. No comments were received. A copy of this notice is provided for reference in Attachment B.
8b: Consultations
Representatives from the following organizations reviewed and guided the original development and recent revision of the data collection instrument and have worked with CDC since 2002 to support its use in local jurisdictions. Refer to Attachment C for a list of these individuals.
American Public Health Association (APHA)
Association of State and Territorial Health Officials (ASTHO)
National Association of County and City Health Officials (NACCHO)
National Association of Local Boards of Health (NALBOH)
National Network of Public Health Institutes (NNPHI)
Public Health Foundation (PHF)
9. Explanation of Any Payment or Gift to Respondents
None
10. Assurance of Confidentiality Provided to Respondents
The OMB justification has been reviewed and it has been determined that the Privacy Act is not applicable. While names and titles of contact persons are being collected, individuals will be not be providing personal individually identifiable data, but instead speaking from their professional roles as being capable of collecting data to measure the capacity of the local public health system to deliver the 10 Essential Public Health Services. Demographic information requested deals not with the point of contact but with information on the local public health agency. Therefore, the data for the project do not meet the definition of a Privacy Act system of records. A password protected electronic database has been created to store survey results at CDC. Access is limited to individuals with a bona fide need to know for official duties. Respondents will be identified by unique identifiers developed under a National Public Health Registry. Data management procedures have not changed since previous approval.
11. Justification for Sensitive Questions
The local instrument does not contain questions which are sensitive in nature.
12. Estimates of Annualized Burden Hours and Costs
12a: Hours
Local public health agencies will complete the NPHPSP local instrument with the consensus responses of local public health system representatives. Local instrument completion will consume approximately 16 hours of local health department staff time. This includes time necessary to conduct an orientation, convene representatives of the local public health system, collate responses, and submit data for analysis.
Although some states have provided funding to their local jurisdictions for use of this instrument, there is no dedicated grant funding for this activity. For that reason, it is critical to support states and local jurisdictions in this effort at a time that best meets their needs and resources. Therefore, CDC and its partners considers it important that this data collection instrument be available to be used on a voluntary and rolling basis.
The universe for this data collection is 2,146 local jurisdictions. The table below reflects estimates for respondents during the next three years. These estimates are based on past use of the Version 1 instruments as well as the expert opinion of the NPHPSP staff and partner organizations who are working with states preparing to undertake the assessment. Burden hours also are shown in table A.12.a.
Table A.12.a: Estimates of Organizational Hourly Burden
Respondents |
No. of Respondents |
No. of Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
Local Public Health Systems |
350 |
1 |
16 |
5600
|
12b: Costs
A local health official or senior designee will coordinate completion of the instrument by local public health system representatives. Using estimated wages for local health department senior staff and the calculated burden hours, this study represents an estimated annual cost of $107,688.
Table A.12.b: Estimates of Cost Burden
National Public Health Performance Standards Program Local Public Health System Assessment Annual Cost Burden Per Respondent
|
Local Public Health Systems |
Respondents
|
Responses per respondent |
Total Burden per response |
Wage rate** (per hour) |
Total cost |
Annualized Estimate |
350 |
1 |
16 |
$19.23 |
$ 107,688 |
**This estimate is based on an average annual salary of $40,000 for all respondents. NACCHO provided an estimated annual salary for health department staff. The total cost to respondents were arrived at by multiplying the number of respondents by total burden hour per response times wage rate (350x16x$19.23 =$107,688).
Basis for Burden
The universe of respondents (n=2146) are represented in Tables A.12.a and A.12.b. The response estimates are based on past use as well as the expert opinion of the NPHPSP staff and partner organizations who are working with states preparing to undertake the assessment.
13. Estimates of Other Annual Cost Burden to Respondents or Record Keepers
There are no annualized capital and maintenance costs to the respondents.
14. Annualized Cost to the Government
Four FTEs are dedicated to implement the NPHPSP within CDC, Office of Chief of Public Health Practice in the CDC Office of the Director. Based on time allocations for the program, an average annual salary of $75,000 per FTE (including benefits), is dedicated to the NPHPSP and its three data collection instruments (OMB control numbers: 0920-0555, 0920-0557, and 0920-0580). This is a cost of $300,000 per year. In addition to the salary, the cost of attending and presenting at state and regional meetings is estimated to be approximately $10,000 in travel expenses each year.
National Public Health Performance Standards Program Centers for Disease Control and Prevention Office of Chief of Public Health Practice
|
Number of FTEs For Program Implementation |
Annual Salary charged to NPHPSP- Including Benefits (per FTE) |
Total Salary Cost (per year) |
|
Number of FTEs attending Regional Meeting |
Total Cost of Travel for each meeting |
Number of Trips |
Total cost of Travel |
4 |
$75,000 |
$300,000 |
|
1 |
1000 |
10 |
$10,000 |
|
|
|
|
|
|
|
|
The total cost to the government of the integrated, three-prong program is estimated to be $310,000. The portion of that cost allocated to each of the three surveys is based on an estimate of 40% for the State instrument, 40% for the Local instrument and 20% for the Governance instrument. For the Local Public Health System Performance assessment instrument, the estimated cost to the government is $124,000.
15. Explanation for Program Changes or Adjustments
This is a revision request of a currently-approved data collection instrument. The instrument has been revised to assure the standards reflect the most current public health practice and to incorporate some improvements for user-friendliness and ease of use. Below is a description of the key changes made during this revision process:
Modernized and updated the content – CDC and its partners worked with subject matter experts and key organizations on the content covered in the NPHPSP to determine what needs to be updated or modernized. Expert input was solicited in areas such as preparedness, informatics, health marketing, partnerships, workforce, public health law, and laboratory issues.
Included new “discussion boxes” and deleted long lists of subquestions – the instruments now include non-scored "discussion boxes" at key points, such as in locations where there previously had been a series of listed points at the lower subquestion level. This helps the instrument retain enough specificity to adequately measure performance, but at the same time provide a format change that may prompt more fruitful discussion by users. The discussion boxes can be used as checkboxes or prompting points to inform the response on the question above and identify areas for improvement. This change was recommended based on the results of psychometric analysis of the Version 1 instruments.
Added a fifth response option to capture “Absolute No” responses – the original instruments included 4 response options - yes, high partial, low partial and no. The “no” response reflected that between 0 – 25% of the activity was being done. Based on user input and to better address the need to separate those without any activity at all and those performing at a minimal level, we have added a “no activity” (or 0%) response option as well as a new option to represent "minimal activity" (greater than zero but no more than 25%). The new and slightly reworded response options are: No activity (0%); Minimal activity (1-25%); Moderate activity (26-50%); Significant activity (51-75%); and Optimal activity (76-100%).
Moved the summary question about the contribution of the agency into a separate optional questionnaire) – as recommended by work group members and field test sites, the agency contribution question is best answered separately from the questions about overall system performance. This is now in a separate and optional questionnaire. The data will not be used in a national database by CDC or its partners, since these questions are only intended to assist the responding site itself.
Added an optional priority questionnaire – sites are provided with a supplemental and optional questionnaire that asks about the "priority" of addressing each model standard. Those that complete these questions get an additional component to their reports which ranks their scores in relation to how they have prioritized standards. This new component is intended to strengthen and better catalyze the performance improvement activities that should occur as a result of the assessment process. The data will not be used in a national database by CDC or its partners, since these questions are only intended to assist the responding site itself.
Because of the changes described above and because we have a newly revised data collection instrument, this is a revision request. There are four components to this information collection effort – the performance standards assessment, a Respondent Information Form which elicits basic information about the responding site, an optional priority questionnaire (described above) and an optional agency contribution questionnaire (described above). These four components are part of one information collection that is implemented at the same time by the same respondents. These are found in Attachment E.
Additionally, the universe of respondents has been revised, as described in Section 12. Due to these changes, the Estimated Burden Hours and the Estimates of Cost Burden are now lower than the estimates reflected in the previous requests to OMB (the original request in 2002 and the extension request submitted in 2004).
16. Plans for Tabulation and Publication and Project Time Schedule
CDC and PHF (a grantee partner organization that assists by providing technical support for the limited-access website) accept data from respondents participating in a statewide process as well as from those who elect to complete the assessment outside of a statewide approach. Ideally, state and local public health agencies will choose to conduct the performance assessment through a coordinated statewide approach. CDC and the NPHPSP partner organizations provide training to orient personnel that play key roles in coordinating a statewide process. Generally these personnel include representatives from state health department, state or regional public health institutes, or state associations of local health officials. CDC and NPHPSP partner organizations also provide statewide “kickoff” training at state conferences, if requested.
Task |
Estimated Time Frame |
Make instruments and technical assistance resources available for all local sites, so that any site interested in completing the process is able to do so (even outside of a statewide coordinated approach). |
Within first month of approval and ongoing throughout the three years of approval |
Identify first set of states for implementation; obtain an indication of commitment from the state |
Within two months after approval |
Provide training and/or work with the state liaison to plan the assessment |
2-4 months after approval |
Provide access to web-based assessment; provide support to the state liaison and local jurisdictions in using the assessment; conduct orientation and kick-off activities, if CDC or partner presence is requested at a state conference |
4-7 months after approval |
Receive and analyze data; provide automated reports to responding jurisdictions; provide aggregate report to the state |
7-9 months after approval |
Encourage states and local jurisdictions to use the results for performance improvement |
9-11 months after approval |
Select second set of states for implementation; obtain an indication of commitment from the state |
12 months after approval |
Provide training and/or work with the state liaison to plan the assessment |
14-16 month after approval |
Provide access to web-based assessment; provide support to the state liaison and local jurisdictions in using the assessment; conduct orientation and kick-off activities, if CDC or partner presence is requested at a state conference |
16-19 months after approval |
Receive and analyze data; provide automated reports to responding jurisdictions; provide aggregate report to the state |
19-21 months after approval |
Encourage states and local jurisdictions to use the results for performance improvement |
21-23 months after approval |
Select third set of states of implementation. Obtain an indication of commitment from the state |
24 months after approval |
Provide training and/or work with the state liaison to plan the assessment |
24-26 months after approval |
Provide access to web-based assessment; provide support to the state liaison and local jurisdictions in using the assessment; conduct orientation and kick-off activities, if CDC or partner presence is requested at a state conference |
26-29 months after approval |
Receive and analyze data; provide automated reports to responding jurisdictions; provide aggregate report to the state |
29-31 months after approval |
Encourage states and local jurisdictions to use the results for performance improvement |
31-33 months after approval |
Publication |
36-48 months after approval |
Publication
Results generated by the NPHPSP assessment instruments will be primarily used for national public health infrastructure improvement. Results will also be presented to the public health community at professional and CDC-sponsored conferences. Further, results from this collection will be prepared for publication in professional reports and journals. To date, manuscripts utilizing results from the NPHPSP assessment instruments have been published in journals including the Journal of Public Health Management and Practice, Milbank Quarterly, Public Health Reports, Health Affairs, American Journal of Preventive Medicine, and the American Journal of Public Health. Similar publication opportunities will be sought for disseminating future data.
Analysis Plan
The local instrument is a qualitative self-assessment designed to provide local public health systems with a point-in-time analysis of their capacity to deliver the Essential Public Health Services. Data collected using the local instrument are analyzed according to standardized algorithms that generate electronic reports. These reports illustrate strengths, weaknesses, opportunities for improvement, and barriers to infrastructure development for local public health systems. Data analysis and reporting are fully automated.
After local public health systems complete the assessment, results are submitted to the limited access data collection website. Technical assistance resources, such as the training workshops, the User Guide and NPHPSP staff, instruct responding jurisdictions that they will need a User ID and survey password to enter data into the limited-access website. If they are completing the assessment through a statewide approach, CDC provides the User IDs to the state coordinator. Other jurisdictions can contact CDC and PHF directly at 1-800-747-7649 or by email at [email protected] or [email protected]. The User IDs are disseminated with an instruction sheet. (See Attachment F for the Facilitator’s Guide; Attachment G for the data entry instruction sheet; and Attachment H for example screen shots of the web-based system for submitting data and accessing reports).
Based on field testing and sound statistical methods, a detailed scoring methodology was developed for the NPHPSP. It is applicable to all three instruments. It was used for the previously-approved instruments and has been tested with the new Version 2 instruments, with the necessary slight alterations (such as inclusion of the new 5th response option). It is described, in brief, below:
Scores are developed for four different levels:
First-tier or “stem” question scores – This score is developed by establishing the weight value for each question, and then multiplying the weight value by the response value. The weight value of each question grouping totals 1 point – lead-in questions are given 0.5 weight while subquestions are assigned 0.5 weight collectively. The weight of each question is multiplied by its response value (“no” responses are given a zero value; “minimal activity” is 0.25; “moderate activity” is 0.50; “significant activity” is 0.75; and “optimal activity” is 1.0). The scores for each question in the question grouping are totaled up to produce the “stem” question score.
Model Standard scores – the average of all stem question scores found within this indicator section.
Essential Service scores – the average of all model standards found within this Essential Service section.
Overall Score – the average of all ten Essential Service scores.
Local public health systems should strive for scores of 75% or above to “fully meet” the model standard. The 75% level was determined by consensus agreement between CDC and national partner organizations. In addition to the numerical scores provided in the reports, respondents are heavily encouraged to record qualitative discussion points that will help to describe areas of weakness in the delivery of the Essential Services. Local public health systems are encouraged to review the scores and qualitative data to identify opportunities for performance and infrastructure improvement planning. CDC and NPHPSP partner organizations provide technical assistance resources and training to assist states and local jurisdictions with using the results for performance improvement. Such technical assistance includes linking the state and local agencies with peers to improve sharing of best practices and providing web-based resources that provide practice models for making improvements in weaknesses identified in their assessment.
An automated sample report is generated for each respondent. Sample reports are available on the CDC website so that potential users can view the reports prior to submitting data.
17. Reason(s) Display of OMB Expiration Date is Inappropriate
CDC is not seeking an exemption for displaying an expiration date.
18. Exemptions to Certification for Paperwork Reduction Act Submissions
There are no exemptions.
List of Attachments
Attachment A: Public Health Service Act, Section 301
Attachment B: Federal Register Notice
Attachment C: List of Project Consultants at Partner Organizations and List of State, Local, and Governance Work Group Members
Attachment D: Spreadsheet of Possible Respondents by State
Attachment E: Local Public Health System Performance Assessment Instrument, Respondent Information Form, Optional Priority Questionnaire, and Optional Agency Contribution Questionnaire
Attachment F: National Public Health Performance Standards Program (NPHPSP) Facilitator Guide (for Local Public Health System Instrument)
Attachment G: Web-based Data Entry Instruction Sheet
Attachment H: Screen Shots of Example Pages from Web-based System – Submitting Data and Accessing Reports
Attachment I: Letters of Support
1 The original clearance package was submitted from CDC’s Public Health Practice Program Office, but due to CDC reorganization, on October 1, 2004, the National Public Health Performance Standards Program was transitioned to the CDC Office of the Director, Office of the Chief of Public Health Practice.
2The Essential Public Health Services are: Monitoring Health Status; Diagnosing and Investigating Health Problems; Informing, Educating, and Empowering People; Mobilizing Community Partnerships; Developing Policies and Plans; Enforcing Laws and Regulations; Linking People to Needed Services; Assuring a Competent Workforce; Conducting Evaluations; and Conducting Research.
| File Type | application/msword |
| File Title | SUPPORTING JUSTIFICATION FOR |
| Author | lmc5 |
| Last Modified By | T. Taylor |
| File Modified | 2007-05-31 |
| File Created | 2007-05-31 |