Evaluation on Impact and Effectiveness of Pilot Project: This is a Tabletop Exercise Simulation Tool (TEST)
OMB Control No. 0920-1154
Supporting Statement Part B- Collections of Information Employing Statistical Methods
May 22, 2024
Kristin Delea, MPH
Epidemiologist
Capacity Building and Technical Assistance Development Branch
Office of Readiness and Response/Division of State and Local Readiness
Centers for Disease Control and Prevention (CDC)
770-488-7065
B. Collections of Information Employing Statistical Methods 3
B.1. Respondent Universe and Sampling Methods 3
B.2. Procedures for the Collection of Information 3
B.3. Methods to Maximize Response Rates and Deal with No Response 4
B.4. Test of Procedures or Methods to Be Undertaken 4
The respondent universe for the proposed data collection includes people participating in use of the training and exercise tool titled, “This is a TEST – A Tabletop Exercise Simulation Tool” (TEST). Participants will be staff who work at state, territorial, and local health departments, or other partner agencies such as emergency management. Ideally, a combination of senior decision makers and emergency planning/response staff will participate in the exercise. Due to competing priorities, it may be unrealistic to integrate senior decision makers and leaders.
The sampling method relies on jurisdictions to volunteer to put on a TEST activity. TEST is a new preparedness tool that US jurisdictions can use for training or exercising emergency preparedness and response partners. The sampling method relies on jurisdictions to volunteer to put on a TEST activity. The TEST tool has been introduced to the Public Health Emergency Preparedness (PHEP) cooperative agreement in CDC Division of State and Local Readiness (DSLR) Weekly Friday Updates distributed to all CDC Public Health Emergency Preparedness (PHEP) funded state, local, and territorial health authorities (n=62 jurisdictions) and during several conferences targeting state, local, territorial and tribal public health jurisdiction.
A minimum of 5 exercise sites will be selected with the potential for up to 15 sites. Sites will be selected based on how well they fit the requirements listed below:
have previously conducted tabletop exercises in the past 2 years.
have/confirm at least 3 agencies available to participate in the exercise.
The call for participation will be open to all state and local governmental agencies across the United States. We anticipate a mix of local and state partners will participate in each exercise. Each site will need to ensure at least 20 participants.
We will travel to administer and evaluate TEST at up to 15 sites. We will not travel to administer TEST to more than 15 sites, but we will provide additional sites access to the exercise materials and an OMB- approved generic survey used to evaluate online resources and tools (OMB Control # 0920-1050).
Additionally, TEST will be used at up to 3 conferences with each conference anticipated to have up to 50 participants. Participation will be open to conference attendees. Conferences will promote workshops through their channels. Determination of which conference will host TEST will be based on conference abstract acceptance and available travel funds.
Immediately following completion of the exercise, we will provide participants with a QR code that they will be directed to scan. This will take them to the online survey (Attachment A), and they will be provided time to fill out the survey prior to exercise debriefing.
If participants voluntarily provide their contact information in the initial survey, we will select a limited number for focus group (Attachment C) which will be held virtually via Teams (maximum of 12 focus groups). Invites will be emailed to the email provided. Focus groups will use the provided questions to guide discussion; one CDC staff member will be a facilitator and another will be a note taker for data collection.
Two month and four months following the exercise, participants will be provided a survey link by the exercise host site. We will provide the link to the survey to the Exercise Coordinator for each host site to share via email. This link will take them to the online survey (Attachment B).
Emphasis will be made to participants at the start and end of the exercise stating that this is a pilot project and data is needed for future development and implementation. In pilot exercises, survey response rates have been approximately 90%. To further maximize response rates, we will designate time at the end of the exercise for participants to fill out the initial survey.
To maximize response rates for follow-up surveys, we will include reminders in the DSLR Friday updates for those who participated. PHEP Project Officers will remind their recipients (jurisdictions which receive PHEP funds) to fill out the surveys if they previously participated in an exercise. Project officers talk to their PHEP recipients on a monthly basis.
One potential way to deal with no response is to use the exercise documentation after action reports (AARs) or implementation plans (IP), which are commonly submitted to CDC following exercises to gain credit for PHEP funding requirements. Within these documents, it’s possible that participants would provide some information on the impact of the exercise and gaps identified although these documents are very high level and brief.
Exercises routinely survey participants. This evaluation will survey participants but include more information on the impact and effectiveness of the use of TEST. A pilot of the initial survey was implemented at an exercise with 9 local public health staff. The survey has been revised to be more concise and reduce time spent.
Follow-up surveys are common to determine if action has been taken following the exercise. This will provide useful information to determine long-term implications of use of TEST.
Given that the purpose of conducting the focus groups is to discover information that will be helpful in evaluating TEST, the focus groups are intended to be semi-structured. We expect to learn and gather useful information by allowing the conversation to proceed organically. The focus group instrument was not pilot tested but extensively reviewed by researchers experienced in conducting qualitative data collection.
B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
The following people are primarily responsible for the design of the surveys and focus groups and will be primarily responsible for data analyses. Lauren Finklea is the primary contact for data collection and analyses.
Kristin Delea, MPH
Epidemiologist
Centers for Disease Control and Prevention, Office of Readiness and Response
Keira Wickliffe Berger, MPH, MSN, RN
Epidemiologist
Centers for Disease Control and Prevention, Office of Readiness and Response
Lauren Finklea, MS
Health Scientist
Centers for Disease Control and Prevention, National Center for Environmental Health
Centers for Disease Control and Prevention (CDC). (2019). Public Health Emergency Preparedness (PHEP) Notice of Funding Opportunity Exercise Requirements.
Federal Emergency Management Agency (FEMA) (2020). Homeland Security Exercise and Evaluation Program. Retrieved September 25, 2022 from https://www.fema.gov/sites/default/files/2020-04/Homeland-Security-Exercise-and-Evaluation-Program-Doctrine-2020-Revision-2-2-25.pdf
Attachment A – Immediate Post Exercise Survey Questions
Attachment B – Follow-Up Survey Questions
Attachment C – Focus Group Script
Attachment D – ORR Research Determination Form
Attachment 1 – TEST CRC edition._Gameboard
Attachment 1a – TEST CRC_Game instructions
Attachment 2 – TEST POD edition_Gameboard
Attachment 2a – TEST POD edition_Game instructions
Attachment 3 – TEST ICS edition_draft Gameboard
Attachment 3a – TEST ICS edition_draft instructions
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2025-05-19 |