Evaluation on Impact and Effectiveness of Pilot Project: This is a TEST
OMB Control No. 0920-1154
Supporting Statement Part B- Collections of Information Employing Statistical Methods
Lauren Finklea, MS
Heath Scientist
Technical Assistance and Guidance Team
CPR/DSLR/PIB
Centers for Disease Control and Prevention (CDC)
770-488-0703
B. Collections of Information Employing Statistical Methods 3
B.1. Respondent Universe and Sampling Methods 3
B.2. Procedures for the Collection of Information 3
B.3. Methods to Maximize Response Rates and Deal with No Response 4
B.4. Test of Procedures or Methods to Be Undertaken 4
The respondent universe for the proposed data collection includes people participating in use of a new training and exercise tool titled, “This is a TEST – A Tabletop Exercise Simulation Tool” (TEST). Participants will be staff who work at state, territorial, and local health departments, or other partner agencies such as emergency management. Specifically, radiation emergency planners will make up a large portion of respondents. Ideally, a combination of senior decision makers and emergency planning/response staff will participate in the exercise. Due to competing priorities, it may be unrealistic to integrate senior decision makers and leaders.
The sampling method relies on jurisdictions to volunteer to be an exercise site for TEST. Opportunities will be advertised (Attachment A) in up to four CDC Division of State and Local Readiness (DSLR) Weekly Friday Updates distributed to all CDC Public Health Emergency Preparedness (PHEP) funded state, local, and territorial health authorities (n=62 jurisdictions).
Due to the timelines of the project, a minimum of 5 exercise sites will be selected with the potential for up to 15 sites. Sites will be selected based on how well they fit the requirements listed below:
have previously conducted tabletop exercises in the past 2 years.
have/confirm at least 3 agencies (including radiation control) available to participate in the exercise.
The call for participation will be open to all state and local governmental agencies across the United States. We anticipate a mix of local and state partners will participate in each exercise. Each site will need to ensure at least 20 participants. Pending which jurisdictions volunteer, we will attempt to select one site from each HHS region.
We will travel to administer and evaluate TEST at up to 15 sites. We will not travel to administer TEST to more than 15 sites, but we will provide additional sites access to the exercise materials and an OMB- approved generic survey used to evaluate online resources and tools (OMB Control # 0920-1154).
Additionally, TEST will be used at up to 5 conferences with each conference estimated to have 30 participants. Participation will be open to conference attendees. Conferences will promote workshops through their channels. Determination of which conference will host TEST will be based on conference abstract acceptance and available travel funds.
Immediately following completion of the exercise, we will provide participants with a QR code that they will be directed to scan. This will take them to the online survey (Attachment B), and they will be provided time to fill out the survey prior to exercise debriefing.
If participants voluntarily provide their contact information in the initial survey, we will schedule them for focus group testing (Attachment D) which will be held virtually via Teams. Invites will be emailed to the email provided. Focus groups will use the provided questions to guide discussion; one CDC staff member will be a facilitator and another will be a note taker for data collection.
One month and three months following the exercise, participants will be provided a survey link by the exercise host site. We will provide the link to the survey to the Exercise Coordinator for each host site to share via email. This link will take them to the online survey (Attachment C).
Emphasis will be made to participants at the start and end of the exercise stating that this is a pilot project and data is needed for future development and implementation. In pilot exercises, survey response rates have been approximately 90%. To further maximize response rates, we will designate time at the end of the exercise for participants to fill out the initial survey.
To maximize response rates for follow-up surveys, we will include reminders in the DSLR Friday updates for those who participated. PHEP Project Officers will remind their recipients (jurisdictions which receive PHEP funds) to fill out the surveys if they previously participated in an exercise. Project officers talk to their PHEP recipients on a monthly basis.
One potential way to deal with no response is to use the exercise documentation after action reports (AARs) or implementation plans (IP), which are commonly submitted to CDC following exercises to gain credit for PHEP funding requirements. Within these documents, it’s possible that participants would provide some information on the impact of the exercise and gaps identified although these documents are very high level and brief.
Exercises routinely survey participants. This evaluation will survey participants but include more information on the impact and effectiveness of the use of TEST. A pilot of the initial survey was implemented at an exercise with 9 local public health staff. The survey has been revised to be more concise and reduce time spent.
Follow-up surveys are common to determine if action has been taken following the exercise. This will provide useful information to determine long-term implications of use of TEST.
Given that the purpose of conducting the focus groups is to discover information that will be helpful in evaluating TEST, the focus groups are intended to be semi-structured. We expect to learn and gather useful information by allowing the conversation to proceed organically. The focus group instrument was not pilot tested but extensively reviewed by researchers experienced in conducting qualitative data collection.
B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
The following people are primarily responsible for the design of the surveys and focus groups and will be primarily responsible for data analyses. Lauren Finklea is the primary contact for data collection and analyses.
Lauren Finklea, MS
Health Scientist
Centers for Disease Control and Prevention, Center for Preparedness and Response
Brandy Maddox, MPH
Health Scientist
Centers for Disease Control and Prevention, National Center for Environmental Health
Fuyuen Yip, Ph.D.
Epidemiologist
Centers for Disease Control and Prevention, National Center for Environmental Health
Amy Schnall, DrPH
Epidemiologist
Centers for Disease Control and Prevention, National Center for Environmental Health
Centers for Disease Control and Prevention (CDC). (2019). Public Health Emergency Preparedness (PHEP) Notice of Funding Opportunity Exercise Requirements.
Federal Emergency Management Agency (FEMA) (2020). Homeland Security Exercise and Evaluation Program. Retrieved September 25, 2022 from https://www.fema.gov/sites/default/files/2020-04/Homeland-Security-Exercise-and-Evaluation-Program-Doctrine-2020-Revision-2-2-25.pdf
Attachment A – Participant Information
Attachment B – Immediate Post Exercise Survey Questions
Attachment C – Follow-Up Survey Questions
Attachment D – Focus Group Script
Attachment
Attachment E- CPR Research Determination Form
A
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2025-05-19 |