Supporting Statement A

This information collection is being conducted using OMB No. 0920-0879 “Information Collections to Advance State, Tribal, Local and Territorial Governmental Agency System Performance, Capacity, and Program Delivery” nicknamed the “CSTLTS Generic.” The respondent universe for this information collection aligns with that of the CSTLTS Generic. Data will be collected from a total of 278 respondents representing 26 state, 8 territorial, and 244 local government employees. Respondents acting in their official capacities include emergency preparedness program directors and program coordinators.

This information collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. 241). This information collection falls under the essential public health service(s) of

1. Monitoring health status to identify community health problems

2. Diagnosing and investigating health problems and health hazards in the community

3. Informing, educating, and empowering people about health issues

4. Mobilizing community partnerships to identify and solve health problems

5. Development of policies and plans that support individual and community health efforts

6. Enforcement of laws and regulations that protect health and ensure safety

7. Linking people to needed personal health services and assure the provision of health care

when otherwise unavailable

8. Assuring a competent public health and personal health care workforce

9. Evaluating effectiveness, accessibility, and quality of personal and population-based

health services

10. Research for new insights and innovative solutions to health problems¹

Many different sectors have roles to play in preventing, preparing for, responding to, and recovering from the health effects of disasters and emergencies. This complexity poses challenges for understanding the collective strength of the health security system as a whole, and for managing and improving this system over time. The National Health Security Preparedness Index provides a tool for assessing how health security capabilities vary across the U.S. and how these capabilities change over time. Using a multi-sector perspective, the Index combines measures from more than 50 data sources and multiple vantage points to offer a broad assessment of preparedness.² State and local public health preparedness programs function as critical components of the nation’s health security system, but relatively little is known about how accurately and completely the Index measures the contributions of public health preparedness programs. Moreover, little is known about how public health preparedness programs can best use the Index to strengthen health security in their communities. Of the 140 individual measures currently used in the Index, as many as half of these measures reflect capabilities that are not under the direct control and responsibility of governmental public health agencies and their preparedness programs. Many Index measures reflect capacities in healthcare, emergency management, education, social services, employers, and other sectors with which public health agencies must communicate, coordinate, and advise. State and local public health preparedness programs face uncertainties about how best to interpret and use the Index results, particularly when many of the Index measures lie outside their direct control.

Working with a broad set of subject matter experts, the U.S. Centers for Disease Control and Prevention defined a set of 15 public health emergency preparedness and response (PHEP) capabilities that specify the actions that state and local public health preparedness programs should be able to perform in preparing for and responding to public health emergencies.³ Initially developed in 2011, the PHEP capabilities have been tested and evaluated using multiple iterations of review, exercises and real events, culminating with an updated set of capabilities released in 2018. This updated set of PHEP capabilities provides a framework that will guide the collection of data about the validity and utility of the Index for public health preparedness programs.

The purpose of the data collection is threefold: (1) to assess how accurately and completely the Index measures the contributions of state and local PHEP programs to community preparedness and overall health security; (2) to identify a core set of PHEP metrics that these programs can use to assess and improve their activities; and (3) to identify strategies for improving the usability and usefulness of the Index for state and local PHEP program stakeholders.

CDC has contracted with a research team led by the University of Kentucky to collect the data, supported by subcontracts with the University of Colorado (to lead the overall analysis strategy), University of California at Los Angeles (to lead survey data collection), New York University (to lead qualitative data collection), the Association of State and Territorial Health Officials (ASTHO, to facilitate state preparedness program engagement), and the National Association of County and City Health Officials (NACCHO, to facilitate local preparedness program engagement).

Overview of the Information Collection System

Data will be collected via two data collection instruments: an online assessment (n=278) and a semi-structured interview guide (n=42). (see Attachment A—Online Assessment Instrument Word Version, Attachment B—Online Assessment Instrument Web Version, and Attachment C—Semi-Structured Interview Guide). The instruments will be used to gather information from employees of state, territorial and local public health preparedness programs who are knowledgeable about PHEP program operations, as identified by each program’s chief administrator, regarding how these preparedness programs can best use the Index to inform and improve their operations.

The information collection instruments were pilot tested by four (4) public health preparedness professionals. Feedback from this group was used to refine questions as needed, ensure accurate programming and skip patterns and establish the estimated time required to complete the information collection instrument.

Items of Information to be Collected

Online Assessment

The online assessment consists of a self-administered questionnaire containing a total of 129 main questions of various types, including dichotomous (yes/no), categorical multiple response, and ordinal (rating scale) items, along with four open-response items. The instrument will collect data from a total of 278 respondents on the following constructs:

Experience in using the Index, including usefulness and usability
Relevance and comprehensiveness of each Index measure

Semi-Structured Interview Guide

The Semi-Structured Interview Guide Instrument consists of a three-part protocol containing a total of 41 open-ended questions and prompts, designed for use during individual key informant interviews that will include a total of 42 respondents. In contrast to the online assessment, the interview guide instrument contains open-ended questions that collect detailed qualitative information about difficulties encountered in using the Index, and about enhancements that could improve the Index. Each part of the instrument is designed for use with one of three specific subgroups of respondents, such that an individual respondent will complete no more than one part of the instrument. The instrument collects data on the following constructs:

Part 1: Index familiarity, use, and utility (asked of respondents selected for the Use and Utility Subgroup)
Part 2: Assessment of core PHEP metrics (asked of respondents selected for the Core Metrics Subgroup)
Part 3: Assessment of Index guidance documents and tools (asked of respondents selected for the Tools Subgroup).

Purpose and Use of the Information Collection

The data collected through the two instruments will be used for three purposes: (1) to assess how accurately and completely the Index measures the contributions of state and local PHEP programs to community preparedness and overall health security; (2) to identify a core set of PHEP metrics that these programs can use to assess and improve their activities; and (3) to identify strategies for improving the usability and usefulness of the Index for state and local PHEP program stakeholders.

Data collected using the two instruments will be linked with additional information about PHEP programs, community characteristics, hazards and risks from existing public databases and public datasets. Quantitative and qualitative data analysis will be used to characterize the relevance, completeness, and utility of the Index for PHEP programs. These results, in turn, will be used to expand and improve the Index measurement set and to develop a collection of guides and tools that help PHEP programs use the Index to strengthen their activities.

Use of Improved Information Technology and Burden Reduction

Data will be collected via computer, tablet, or smart phone from any location chosen by the respondent for the Online Assessment Instrument. This method was chosen to reduce the overall burden on respondents by means of making the data collection processes easily accessible from respondents’ preferred locations while maintaining strict data security. The data collection instruments are designed to collect the minimum information necessary for the purposes of this project (i.e., limited to 129 questions).

The Semi-Structured Interview Guide Instrument is divided into 3 parts, with each part designated for a different subgroup of respondents. This design minimizes respondent burden by distributing questions among different respondents, ensuring that no duplication occurs and no single respondent is burdened by an excessive volume of questions. Respondents will be purposively assigned to one of the three subgroups based on their knowledge and use of the Index as self-reported on the Online Assessment and based on the recommendation of the state PHEP director.

Efforts to Identify Duplication and Use of Similar Information

A thorough literature review was conducted of the peer-reviewed and grey literature relevant to PHEP programs and capabilities and to the Index. A small but growing body of literature examines strategies for measuring PHEP capabilities, but none of the existing studies evaluates measures that are included in the Index.⁴ A relatively small body of literature examines the general validity and reliability of measures included in the Index, but none of these existing studies assesses the relevance of measures specifically for PHEP programs and capabilities. Three past data collection projects were conducted using the CSTLTS Generic ICR (OMB Co. No: 0920-0879) to assess needs and evaluate capabilities relevant to the PHEP Cooperative Agreement program in 2015, 2018 and 2019; however, these projects did not collect information about Index use and utility among PHEP awardees. The Robert Wood Johnson Foundation commissioned a small and limited qualitative study (unpublished) to examine preliminary stakeholder experiences with Index use and utility in 2017, which was conducted by members of the project team that is responsible for this data collection project. The Foundation study included fewer than five stakeholders from state and local PHEP programs, and therefore the results were insufficient to support conclusions about Index use and utility among the larger population of PHEP programs across the U.S. Moreover, this Foundation study was very limited in scope and did not examine the relevance of specific Index measures, nor did it seek to identify possible new measures of relevance to PHEP programs. Nevertheless, results from the small Foundation study were used to inform the development of the data collection instruments to be used in this project. The project team that is responsible for this data collection is also responsible for producing and disseminating results from the Index, and as such this team continuously monitors the research literature, websites, social media, and mainstream media for publications and communications that mention the Index (using Pubmed, Google Scholar, Google Alerts, Lexus-Nexus, Twitter, and other automated search engines). These monitoring activities ensure that new studies and new data collections relevant to the Index are rapidly identified. Additionally, the project team regularly communicates with the collaborating organizations at ASTHO and NACCHO regarding their data collection and data use activities relevant to PHEP, including by serving as advisory committee members for the periodic ASTHO and NACCHO surveys of health agencies. Collectively, these activities ensure that the proposed data collection activities do not duplicate or overlap with existing or planned data collection activities conducted by other entities.

Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this information collection.

Consequences of Collecting the Information Less Frequently

This request is for a one-time data collection. There are no legal obstacles to reduce the burden. If no data are collected, CDC will be unable to:

Advise and assist PHEP programs in using the Index to strengthen program operations and performance
Identify a core set of metrics from the Index and other sources that can be used to assess the capabilities of state and local PHEP programs
Develop guidelines and tools that help PHEP programs use Index results for data-driven planning, decision-making, and community engagement.

Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances with this data collection package. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.

Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

This data collection is being conducted using the Generic Information Collection mechanism of the CSTLTS Generic Information Collection Service (CSTLTS Generic) – OMB No. 0920-0879. A 60-day Federal Register Notice was published in the Federal Register on April 27, 2017, Vol. 82, No. 80, pp 19371-19373. One non-substantive comment was received. CDC sent forward the standard CDC response.

CDC partners with professional STLT organizations, such as the Association of State and Territorial Health Officials (ASTHO), the National Association of County and City Health Officials (NACCHO), and the National Association of Local Boards of Health (NALBOH) along with the National Center for Health Statistics (NCHS) to ensure that the collection requests under individual ICs are not in conflict with collections they have or will have in the field within the same timeframe.

Explanation of Any Payment or Gift to Respondents

CDC will not provide payments or gifts to respondents.

Protection of the Privacy and Confidentiality of Information Provided by Respondents

The Privacy Act does not apply to this data collection. STLT governmental staff will be speaking from their official roles.

Institutional Review Board (IRB) and Justification for Sensitive Questions

No information will be collected that are of personal or sensitive nature. This data collection is not research involving human subjects.

Estimates of Annualized Burden Hours and Costs

The estimate for burden hours is based on a pilot test of the Online Assessment by four (4) public health preparedness professionals. In the pilot test, the average time to complete the Online Assessment Instrument including time for reviewing instructions, gathering needed information and completing the instrument, was approximately 42 minutes (range: 32 –52). For the purposes of estimating burden hours, the upper limit of this range (i.e., 52 minutes) is used.

The estimate for burden hours is based on a pilot test of the Semi-Structured Interview Guide data collection instrument by four (4) public health preparedness professionals. In the pilot test, the average time to complete the instrument including time for reviewing instructions, gathering needed information and completing the instrument, was approximately 57 minutes (range: 43 –70). For the purposes of estimating burden hours, the upper limit of this range (i.e., 70 minutes) is used.

Estimates for the average hourly wage for respondents are based on the Department of Labor (DOL) Bureau of Labor Statistics for occupational employment for Emergency Management Directors http://www.bls.gov/oes/current/oes_nat.htm. Based on DOL data, an average hourly wage of $39.01 is estimated for all 278 respondents. To account for potential increases in fringe and overhead costs due to the COVID-19 response, the hourly wage rate has been doubled to $78.02 (https://aspe.hhs.gov/pdf-report/guidelines-regulatory-impact-analysis).Table A-12 shows estimated burden and cost information.

A total of 278 respondents are expected to provide a total of 320 responses. NOTE: from the 278 respondents invited to participate in the online assessment, a subset of 42 respondents will also be invited to participate in the semi-structured interviews, resulting in a total of 320 responses (278+42=320). See table below.

Table A-12: Estimated Annualized Burden Hours and Costs to Respondents

Data collection Instrument: Form Name	Type of Respondent	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Online Assessment Instrument	Public Health Emergency Preparedness Staff*	278	1	52/60	241	$78	$18,798
Key Informant Interview Guide Instrument	Public Health Emergency Preparedness Staff*	42	1	70/60	49	$78	$3,822
	TOTALS	320**	1**		290		$22,620

*Salaries taken from May 2017 National Occupational Employment and Wage Estimates for Emergency Management Directors at https://www.bls.gov/oes/current/oes_nat.htm

**Based upon selection criteria, all respondents who complete the Key Informant Interview Guide will also complete the Online Assessment Instrument. A total of 278 respondents are expected to provide a total of 320 responses.

Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There will be no direct costs to the respondents other than their time to participate in each data collection.

Annualized Cost to the Government

There are no equipment or overhead costs. The only cost to the federal government would be the salary of CDC staff and project staff from the University of Kentucky, University of Colorado, University of Colorado School of Public Health, New York University, the University of California Los Angeles, the National Association of County and City Health Officials (NACCHO), and the Association of State and Territorial Health Officials (ASTHO) to develop the data collection instrument, collect data, and perform data analysis. The total estimated cost to the federal government is $4,769 per collection (i.e. per response), based on a total of 320 responses received from a total of 278 respondents. Table A-14 describes how this cost estimate was calculated.

Table A-14: Estimated Annualized Cost to the Federal Government

Staff (FTE)	Average Hours per Collection	Average Hourly Rate	Total Average Cost
Senior Health Scientist – GS14-10, To manage the project on behalf of the Government; provide scientific input into assessment development, interpretation of data, and communication of findings.	12 hrs (remaining project period)	$68.67	$824
University of Kentucky Principal Contract Organization To manage and coordinate the overall study and ensure that all tasks and deliverables are completed on schedule by subcontracting organizations.			$483
University of Colorado Subcontract To oversee the design of data collection instruments, data collection, data analysis, and dissemination for all study aims, and to lead the analyses of Index validity and relevance (Aim 1).			$1,069
New York University Subcontract Organization To lead the analysis of core PHEP metrics (Aim 2).			$1,100
University of California Los Angeles Subcontract Organization To lead the analysis of Index use and usability (Aim 3).			$1,661
NACCHO Subcontract Organization To help design, recruit and field the Online Assessment Instrument.			$188
ASTHO Subcontract Organization To help design, recruit and field the Online Assessment Instrument and help disseminate Index guidance tools for state health officials.			$188
Estimated Total Cost of Information Collection			$4,769

Note: cost estimates in Table A-14 indicate the average cost per collection (i.e. per response), based on a total of 320 responses received from a total of 278 respondents.

Explanation for Program Changes or Adjustments

This is a new data collection.

Plans for Tabulation and Publication and Project Time Schedule

As resources and respondents may be impacted by the COVID-19 pandemic, we propose that data collection begin in July 2020 so that activities are completed prior to possible seasonal surges in disease transmission. Quantitative data collected using the Online Assessment Instrument will be stored on an encrypted and password-protected file server at the University of Colorado for data cleaning and analysis. Once data cleaning procedures are complete, the resulting quantitative data file will be linked with the other publicly available secondary data sources to be used in the analysis, including data from the National Health Security Preparedness Index, the Spatial Hazard Events and Losses Database (SHELDUS), the NACCHO and ASTHO Profile surveys, and the Area Health Resources File. Quantitative analyses will be used to learn about the relevance, completeness and utility of the Index for PHEP programs.

For the qualitative data collected using the Semi-Structured Interview Guide Instrument, electronic files for all recordings, transcripts, and interviewer notes from interviews and focus groups will be maintained on encrypted and password protected computer servers and computers in compliance with requirements for data security. Qualitative data files will be coded and analyzed using ATLAS.ti and NVIVO text analysis software. Coded qualitative data files will be linked with the quantitative analytical file to support mixed-method analyses by using the unique study identifiers assigned to each qualitative and quantitative observation.

Findings from the analyses will be used to expand and improve the Index measurement set and to develop a collection of guides and tools that help PHEP programs use the Index to strengthen their activities all results will be reported in aggregate form only and results will be used to develop guidance documents and tools, and findings may be published in scientific journals.

Project Time Schedule

Design instrument (COMPLETE)

Develop protocol, instructions, and analysis plan (COMPLETE)

Pilot test instrument (COMPLETE)
Prepare OMB package (COMPLETE)
Submit OMB package (COMPLETE)

OMB approval (TBD)
Conduct Online Assessment Instrument data collection (Open 5 weeks)
Code data, conduct quality control, link and analyze survey data (8 weeks)
Prepare summary report on data (4 weeks)
Disseminate results/reports on data (12 weeks)
Conduct Key Informant Interview data collection (Open 8 weeks)
Code data, conduct quality control, link and analyze interview data (4 weeks)
Prepare summary report on interview data (4 weeks)
Disseminate results/reports on interview data (8 weeks)

Reason(s) Display of OMB Expiration Date is Inappropriate

We are requesting no exemption.

Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.

LIST OF ATTACHMENTS – Section A

Attachment A – Online Assessment Instrument, Word Version
Attachment B – Online Assessment Instrument, Web Version
Attachment C – Semi-Structured Interview Guide

REFERENCE LIST

Centers for Disease Control and Prevention (CDC). "National Public Health Performance Standards Program (NPHPSP): 10 Essential Public Health Services." Available at http://www.cdc.gov/nphpsp/essentialservices.html. Accessed on 8/14/14.
Uzun Jacobson E, Inglesby T, Khan AS, Rajotte JC, Burhans RL, Slemp CC, Links JM. Design of the National Health Security Preparedness Index. Biosecurity and Bioterrorism. 2014;12(3):122-31.
U.S. Centers for Disease Control and Prevention (CDC). “Public Health Emergency Preparedness and Response Capabilities: National Standards for State, Local, Tribal and Territorial Public Health.” Available at: https://www.cdc.gov/cpr/readiness/00_docs/CDC_PreparednesResponseCapabilities_October2018_Final_508.pdf
Qari SH, Yusuf HR, Groseclose SL, Leinhos MR, Carbone EG. Public Health Emergency Preparedness System Evaluation Criteria and Performance Metrics: A Review of Contributions of the CDC-Funded Preparedness and Emergency Response Research Centers. Disaster Medicine and Public Health Preparedness. 2019;13: 626-638.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	gel2
File Modified	0000-00-00
File Created	2021-01-13