Ssb

Data will be collected from 730 respondents. Respondents are self-identified as having the authority and knowledge to provide responses that reflect their agency’s perspective on topics and matters pertaining to emergency response leadership training and exercises.

These respondents are comprised of preparedness leads of state, territorial, and local health departments, which includes Program Directors of the 62 Public Health Emergency Preparedness (PHEP) cooperative agreement jurisdictions (50 states, eight territories and freely associated states, and four large metropolitan areas); and a randomly generated and stratified sample of 668 preparedness coordinators representing distinct local health departments (i.e., one preparedness coordinator per local health department) drawn from the Emergency Preparedness Coordinators database of the National Association of County and City Health Officials (NACCHO), excluding four PHEP metropolitan areas (see Attachments A – Respondent Universe and Attachment B – Sample of Respondents).

The local health departments will be stratified by two key variables:

Population size served:

< 50,000
Between 50,000 – 499,999
≥ 500,000

Geography: distribution across different regions of the United States, grouped by Northeast, Midwest, South, and West.

Focus Groups

Data will be collected from 32 additional state, territorial and local health department officials who have served or will serve in public health emergency response roles, not exclusively in leadership roles, and voluntarily wish to participate. These respondents will be recruited and selected by convenience sampling from registered attendees at the 2019 NACHHO Preparedness Summit who meet criteria and indicate willingness to participate. Recruitment will continue until enrollment goals are achieved. Informed by published peer-reviewed literature suggesting that three to six focus groups are sufficient to discover 90% of key themes and reach thematic saturation, a total of four in-person focus groups will be conducted at the 2019 NACCHO Preparedness Summit. This will include two in-person focus groups with state/territorial health department officials and two in-person focus groups with local health department officials; each focus group will include up to eight participants. In contrast to the web-based assessment, focus groups will provide a platform for organic, dynamic, and interactive discussions with context in discovering key themes through qualitative analysis.

Procedures for the Collection of Information

Web-Based Assessment

Respondents will be given a total of six weeks to complete the electronic assessment. At the beginning of week one, potential respondents for the web-based assessment at the state/territorial health department level will be invited to participate via an invitation email sent jointly by CNA and JHU (see Attachment F – Web-Based Assessment PHEP Director Invitation Email, Word Version). Potential respondents for the web-based assessment at the local health department level will be invited to participate via an invitation email sent by NACCHO (see Attachment G – Web-Based Assessment Local Invitation Email, Word Version). The invitation email, including a factsheet (Attachment H – Web-Based Assessment One-Page Factsheet), will explain to respondents:

The purpose of the data collection, and why their participation is important;
Instructions for participating;
Method to safeguard their responses;
That participation is voluntary; and
Contact information for the CNA/JHU project team.

The respondents will have until the end of week six to complete the assessment. Respondents may complete the assessment in multiple sessions, if necessary.

Email notification and reminder emails will be sent to all initial invitation email recipients to maximize response rates. Two reminder emails will be sent to state and local health department points of contact, a method that has proven useful in improving response rates in prior projects. These reminder emails, which provide the link to the electronic data collection instrument, will be sent at the three-week mark after the assessment has opened (see Attachment I – Web-Based Assessment PHEP Director Reminder Email, Word Version and Attachment J – Web-Based Assessment Local Reminder Email, Word Version) and again the week prior to the closing date of the assessment. Those who do not respond by the close of the survey period will be considered non-responders. All information collected will be kept on secure, password-protected servers.

Web-Based Assessment Analysis

Electronic assessment data will be collected via a web-based information collection instrument, programmed using Qualtrics (Provo, UT, 2009), allowing respondents to complete and submit their responses electronically. CDC, in collaboration with CNA and JHU, will analyze the data through a mixture of quantitative methods, including descriptive and statistical analysis, as appropriate, for the web-based assessment data. Descriptive statistics informed by software analytics will encompass state or local health department characteristics (i.e., governance type, size of population served, and geographic location) with a focus at the organizational/programmatic level. In addition, certain paired characteristics of state/territorial and local health departments potentially linked to particular findings may also be explored. In addition, statistical analysis, as appropriate, (e.g., statistical testing for significant differences in opportunities for, or barriers to, leadership training/exercise among health departments serving populations of differing size) will be used to explore key selected state/territorial or local health department characteristics with focus on the organizational/programmatic level, as feasible.

Focus Groups

Recruitment

Four in-person focus groups will be conducted in-person at the 2019 NACCHO Preparedness Summit, scheduled from March 26, 2019 to March 29, 2019, in St. Louis, Missouri; specifically, two focus groups will be conducted on March 27, 2019, and two focus groups will be conducted on March 28, 2019. In order to recruit focus group participants, CNA and JHU will first engage with NACCHO. Among registrants for the 2019 NACCHO Preparedness Summit who may self-identify as employees of state, territorial, or local health departments and have roles relating to public health emergency response, NACCHO will recruit potential in-person participants through an electronic invitation (see Attachment K – Generic Focus Group Invitation Email, Word Version). A factsheet of the focus group purpose and methodology (see Attachment L – Focus Group One-Page Factsheet) will be attached to the invitation email from NACCHO. The following criteria will be used to determine participation in the focus groups:

No more than two officials from the same agency will be permitted to enroll; and
If interested focus group participants—through self-reporting—meet one or more of the following criteria:
1. have had a role, currently have a role, or expect to have a role in public health preparedness;
2. have had a role, currently have a role, or expect to have a role, in a public health emergency or incident response;
3. have held a leadership position relating to public health emergency or incident response;
4. currently hold a leadership position relating to public health emergency or incident response; and/or
5. will—or anticipate having to—fill a leadership position relating to public health emergency or incident response

Note: Eligibility based on criterion number 1 (above) will be monitored by CNA/JHU through enrollment and waiver form submissions (Attachment M – Focus Group Enrollment and Waiver Form, Word Version). Eligibility based on criterion number 2 (above) will be based on self-reporting and will not be validated by other sources. CNA/JHU will rely on potential participants to be honest and truthful in their disclosure of prior experiences and roles related to public health preparedness. While determining eligibility through self-reporting is not without limitations, due to resource constraints, it is the most realistic and feasible approach for this study.

Those willing to participate in the focus groups will be asked to complete an enrollment form (see Attachment M – Focus Group Enrollment and Waiver Form, Word Version) that collects the participant’s name, work email address, work phone number, health department/agency name, preferred focus group session, and his/her consent to be recorded during the focus group discussions. CNA/JHU will monitor the number of focus group enrollees; once the desired number of participants has enrolled for all focus group sessions, CNA/JHU will inform additional interested participants that focus groups have reached capacity, but will notify interested participants if space becomes available.

Business contact information will be gathered by CNA/JHU exclusively to facilitate scheduling of, reminders about, and consent to be recorded for the focus groups. The Enrollment and Waiver Form will also clearly state inclusion and exclusion criteria for participation, which is limited to officials who have served or will serve in a public health emergency response role, not exclusively restricted to leadership roles, and who have not previously participated in the web-based assessment. All identified state and local health department public health emergency response officials will be given a total of four weeks to enroll in the focus groups.

All participants who complete the Enrollment and Waiver Form will receive a confirmation email (see Attachment N – Focus Group Participation Confirmation Email, Word Version) to confirm their willingness to participate in a designated focus group discussion and the designated date of an appropriate focus group for each respective participant. The confirmation email will detail the following:

The purpose and significance of the data collection;
The voluntary nature of participation and conditions for withdrawal;
Focus group estimated duration;
Methods to safeguard responses; and
Contact information for the CNA/JHU project team.

The CNA/JHU project team, in collaboration with NACCHO, will conduct reminder phone calls to participants (see Attachment O – Focus Group Reminder Call Script, Word Version) a week prior to Summit commencement. Two business days before each in-person focus group is scheduled to take place, CNA/JHU will send a reminder email to its participants (see Attachment P – Focus Group Conduct Reminder Email, Word Version).

Focus Group Data Collection

Between March 26, 2019 and March 29, 2019, four in-person focus group discussions will be conducted at the 2019 NACCHO Preparedness Summit. This will include two in-person focus groups with state/territorial health department officials and two in-person focus groups with local health department officials. Each session will include up to eight participants. All focus group discussions will be recorded. The CNA/JHU project team will also take detailed notes during focus group discussions. Audio recordings, transcripts, and notes will be stored securely on a password-protected system for analysis.

Focus Group Data Analysis

Transcripts from focus groups will be prepared from audio recordings. Responses to the primary discussion and secondary probing questions will be collated across focus groups. Following all data collection and preliminary data analysis, the CNA/JHU project team will create a content summary document of all findings from the primary discussion questions and notable secondary probing questions. The summary document will be used to develop a codebook conveying key themes and findings. The CNA/JHU project team will then use the codebook to analyze all focus group transcripts through axial coding to delineate the relationship among key themes and findings. Each focus group transcript will be reviewed by each coder to confirm that key themes and findings detailed with the summary document accurately reflect the content within focus groups. The CNA/JHU project team coders will convene to discuss and resolve any inconsistencies for refinement of the codebook. To organize the data and test inter-coder reliability, the CNA/JHU project team will code all focus group transcripts using NVivo 12 (QSR International). Coded data will be reviewed prior to a second round of coding using the finalized codebook to identify common themes and subthemes and for consistency.

Methods to Maximize Response Rates, Deal with Nonresponse

Web-Based Assessment

Participation in the web-based assessment is voluntary with every effort made to maximize the rate of response. The web-based assessment was designed with particular focus on clarity of questions and response options as well as to streamline the need to obtain information from other sources, thereby minimizing response burden.

Following the distribution of the invitation to participate in the web-based assessment, (see Attachment F – Web-Based Assessment PHEP Director Invitation Email, Word Version and Attachment G – Web-Based Assessment Local Invitation Email, Word Version), respondents will have six weeks to complete the instrument. Those who do not respond within three weeks will receive a reminder via email encouraging them to complete the instrument; a subsequent email reminder will be sent one week prior to the close of the assessment (see Attachment I – Web-Based Assessment PHEP Director Reminder Email, Word Version and Attachment J – Web-Based Assessment Local Reminder Email, Word Version). Those who do not respond by the close of the survey period will be considered non-responders.

Focus Groups

Participation in the focus groups is voluntary with every effort made to maximize the rate of participation and response. The Focus Group Facilitation Guide includes overarching key questions and additional probing questions. Focus group facilitators will be instructed to skip questions, as appropriate, based on responses to previous questions, thereby minimizing response burden and improving discussion flow.

Initially, an invitation to participate in a focus group will be sent to potential participants. Subsequently, a reminder phone call will be made to enrolled participants (see Attachment O – Focus Group Reminder Call Script, Word Version) one week prior to each focus group session. In addition, a reminder email (see Attachment P – Focus Group Conduct Reminder Email, Word Version) will be sent two days before each focus group to maximize participation. In order to account for schedule conflicts and no shows, the CNA/JHU project team will offer multiple dates and sessions for focus group participation during the enrollment period.

Test of Procedures or Methods to be Undertaken

A total of nine public health professionals pilot tested the web-based assessment instrument and participated in a pilot focus group discussion. Feedback was used to refine the instruments as needed, and in the case of the web-based instrument, to ensure accurate programming and skip patterns. Feedback was also obtained to establish the estimated time required to complete the information collection.

For the web-based assessment, in the pilot test, the time to complete the instrument including time for reviewing instructions, gathering needed information and completing the instrument, ranged between 15 – 20 minutes. For the purposes of estimating burden hours, the upper limit of this range (i.e., 20 minutes) is used.

For focus groups, in the pilot test, the time to complete a review of focus group instructions and discussions of questions within the guide ranged from 60 – 75 minutes. For the purposes of estimating burden hours, the upper limit (i.e., 75 minutes) was used.

Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Dale A. Rose, PhD, MSc

Associate Director for Science

Division of Emergency Operations (DEO)

Center for Preparedness and Response

1600 Clifton Road

Atlanta, GA 30329

Email: [email protected]

Phone: 404.639.5115

Xiaohong Mao Davis, PhD

Behavioral Scientist

Division of Emergency Operations (DEO)

Center for Preparedness and Response

1600 Clifton Road

Atlanta, GA 30329

Email: [email protected]

Phone: 404.639.2015

Silvia Trigoso, MPH

Emergency Management Specialist

Division of Emergency Operations (DEO)

Center for Preparedness and Response

1600 Clifton Road

Atlanta, GA 30329

Email: [email protected]

Phone: 404.639.7057

CAPT Gail Stennies, MD, MPH, FACPM

Medical Officer

Division of State and Local Readiness (DSLR)

Center for Preparedness and Response

1600 Clifton Road

Atlanta, GA 30329

Email: [email protected]

Phone: 404.718.5995

Yang Li, MPH, MS

Senior Research Scientist

CNA

3003 Washington Blvd

Arlington, VA 22201

Email: [email protected]

Phone: 703. 824.2370

NhuNgoc Pham, MPH

Senior Research Specialist

CNA

3003 Washington Blvd

Arlington, VA 22201

Email: [email protected]

Phone: 703. 824.2939

Juliana Pearson, MA

Associate Research Analyst

CNA

3003 Washington Blvd

Arlington, VA 22201

Email: [email protected]

Phone: 703.824.2447

Edbert Hsu, MD, MPH

Associate Professor

Department of Emergency Medicine

Johns Hopkins University

5801 Smith Ave., Davis Building

Baltimore, MD 21287

Email: [email protected]

Phone: 443.271.1686

Daniel Barnett, MD, MPH

Associate Professor

Department of Environmental Health and Engineering

615 N. Wolfe Street

Room E7036

Baltimore, Maryland 21205

Email: [email protected]

Phone: 410.502.0591

Carol Thompson, MS

Associate Scientist

Department of Biostatistics

615 N. Wolfe Street

Room E3142

Baltimore, Maryland 21205

Email: [email protected]

Phone: 410.502.9142

Laura Biesiadecki, MSPH

Senior Director of Preparedness

National Association of County and City Health Officials (NACCHO)

1201 Eye Street, NW/Fourth Floor

Washington, DC 20005

Email: [email protected]

Phone: 202.507.4205

LIST OF ATTACHMENTS – Section B

Attachment A – Respondent Universe

Attachment B – Sample of Respondents

Attachment F – Web-Based Assessment PHEP Director Invitation Email, Word Version

Attachment G – Web-Based Assessment Local Invitation Email, Word Version

Attachment H – Web-Based Assessment One-Page Factsheet

Attachment I – Web-Based Assessment PHEP Director Reminder Email, Word Version

Attachment J – Web-Based Assessment Local Reminder Email, Word Version

Attachment K – Generic Focus Group Invitation Email, Word Version

Attachment L – Focus Group One-Page Factsheet

Attachment M – Focus Group Enrollment and Waiver Form, Word Version

Attachment N – Focus Group Participation Confirmation Email, Word Version

Attachment O – Focus Group Reminder Call Script, Word Version

Attachment P – Focus Group Conduct Reminder Email, Word Version

Page 1 of 12

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Graaf, Christine (CDC/OSTLTS/DPHPI)
File Modified	0000-00-00
File Created	2021-01-15