November 26, 2024
GenIC Clearance for CDC/ATSDR
Formative Research and Tool Development
Communication
Evaluation:
Assessing Foodborne, Waterborne and Mycotic Disease
Prevention Messages
Contact: Sara Bresee, MPH
Office of the Director
Division of Foodborne, Waterborne, and Environmental Diseases
Centers for Disease Control and Prevention
1600 Clifton Road, NE
Atlanta, Georgia 30333
Phone: (404) 639.3371
Email: [email protected]
TABLE OF CONTENTS
1 Circumstances Making the Collection of Information Necessary 3
2 Purpose and Use of the Information Collection 4
4 Efforts to Identify Duplication and Use of Similar Information 5
5 Impact on Small Businesses or Other Small Entities 5
6 Consequences of Collecting the Information Less Frequently 5
7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 5
8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency 5
9 Explanation of Any Payment or Gift to Respondents 6
10 Assurance of Privacy Provided to Respondents 7
11 Justification for Sensitive Questions 8
12 Estimates of Annualized Burden Hours and Costs 9
13 Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers 10
14 Annualized Cost to the Federal Government 10
15 Explanation for Program Changes or Adjustments 11
16 Plans for Tabulation and Publication and Project Time Schedule 11
17 Reason(s) Display of OMB Expiration Date Is Inappropriate 11
18 Exceptions to Certification for Paperwork Reduction Act Submissions 11
Exhibits
Exhibit A.8.1. Individuals Consulted During A-4
Exhibit A.12.1. Estimated Annualized Burden Hours A-7
Exhibit A.12.2 Estimated Annualized Burden Costs A-8
LIST OF ATTACHMENTS
Eligibility Screener
Recruitment Materials
Eligible Participant Screener
Privacy Agreement
Respondent Consent Form for Focus Groups
Standard Invitation for FGs
Participant Confirmation Email
Focus Group Moderator Guide
Eligibility Survey for Rapid Survey
Screenshot of Eligibility Screener for Survey
Rapid Survey
Respondent Consent Form for Survey
Recruitment Materials for Survey
Screenshot of Rapid Survey
Messages to be Tested
Foodborne, waterborne, and fungal diseases cause serious illness and death each year in the United States. Approximately 48 million people (1 in 6) get sick from foodborne disease each year in the United States, including 128,000 hospitalizations and 3,000 deaths. In 2021, the foodborne pathogens with the highest incidence per 100,000 population included Campylobacter (17.8), Salmonella (14.18), Shinga toxin-producing Escherichia coli (5.04), and Shigella (3.37), according to CDC FoodNet surveillance data. Dietary culture and health eating beliefs continue to effect behaviors like eating and preparing raw or underprepared food or meat products, which brings risk to human health (Zhou et al., 2020). In 2014, seventeen waterborne pathogens caused 7.15 million illnesses and 6,630 deaths in the United States. Additionally, fungal disease accounts for more than 75,000 hospitalizations every year. In 2021, approximately 7,199 deaths occurred from fungal diseases. A fungal meningitis outbreak associated with procedures under epidural anesthesia in Matamoros, Mexico, occurred in 2023 with 10 confirmed cases and 12 deaths, along with additional probable cases.
Health communication materials and resources produced by CDC’s Division of Foodborne, Waterborne, and Environmental Diseases (DFWED) can help prevent diseases caused by contaminated food, water, and fungi. Message testing and behavioral questions are important tools that can be used to evaluate the effectiveness of health communication messages among priority audiences. Engaging directly with audience members in focus group settings allows for CDC to collect direct feedback like audience perceptions, beliefs, and attitudes towards CDC messages and apply that feedback to update and improve those messages.
CDC’s contractor, Banyan Communications, will implement qualitative focus groups and quantitative online surveys. The focus group respondents for this project will be a maximum of 64 individuals recruited by Banyan Communications. The online survey respondents for this project will be a maximum of 1500 individuals recruited by Banyan Communications. The project will work with volunteer respondents. Participants must meet a set of criteria to ensure all focus groups and surveys will include a maximally diverse group of participants considering age, educational level, and socioeconomic status, gender, and ethnicity and include a mix of geographical areas and urban/rural residents. The focus groups will be conducted between adults (18+) and at least one research staff member. The survey will be conducted with adults (18+). The goal is to obtain feedback to support food safety communication initiatives.
Data to be collected include the following: socio-demographics; reactions and receptivity to foodborne, waterborne, and fungal prevention messages and content. Questions shall assess whether/how the participants intend to change their behavior based on the message. Participants shall also elaborate on ways in which the presented messages, through text or presentation changes, could be improved so that they are more effective. In addition, some behavioral questions will help provide an understanding of what motivates specific behaviors.
The data collection will use
a 5-minute Eligibility Screener before the virtual focus group (Attachment 1)
a 5-minute Eligible participant screener (Attachment 3)
a virtual 60-minute focus group (Attachment 8)
a 5-minute Eligibility Screener before the online survey (Attachment 9)
a 10-15-minute online survey (Attachment 11)
This information collection does not involve websites or website content directed at children less than 13 years of age.
The purpose of this project is to conduct focus group discussions (FGDs) and online surveys with U.S. adults (parents of children ages 0–4, older adults ages 65+, pregnant adults aged 18+, and immunocompromised adults 18–64) to improve foodborne, waterborne, and fungal prevention messages and web content. Banyan Communications will conduct the focus groups and administer the online survey.
The objectives of this project are to:
Identify appropriate and effective messages for the public to increase awareness on preventing foodborne, waterborne, and environmental illness and following proper safety practices.
Gather data on the preferred tone, format, and placement of those messages on CDC’s communication channels.
Understand motivation for behaviors which might lead to increased exposure of foodborne, waterborne or fungal illness
The data collected will be used to:
Continue refining foodborne, waterborne, and fungal disease prevention messages and dissemination via the CDC website, resulting in content that is targeted to reach more consumers, with a focus on people who are at higher risk for illness;
Tailor content to address current perceptions and concerns, make content easier to access, understand, and implement, and ensure content is presented attractively and engagingly;
Assist in developing materials and messages that can be shared with other consumer foodborne, waterborne, and environmental safety education programs within the U.S. government and public- and private-sector partners to help improve acceptability and understanding of safety messages.
3 Use of Improved Information Technology and Burden Reduction
We will record each focus group to use for preparing reports. Our data collection requires that we employ qualitative research methods using one-time virtual focus group discussions. We will receive recorded verbal confirmation from participants to record the discussion. Questions (within the focus group discussions and online survey) will be kept to a minimum required for the intended use of the data.
There are no other federal generic collections that duplicate the project types included in this request. Health messages developed by CDC are unique in their mix of intended audience, health behavior, concept, and execution. Therefore, in most cases, there is no similar data available. We have reviewed existing published data and consulted with outside experts to identify information that could facilitate message development prior to conducting any data collection.
DFWED leads an interagency working group with other U.S. government agencies. In this working group we discuss research and communication projects to ensure there is a lack of redundancy.
This project does not have an impact on small businesses or other small entities.
The activities involve a one-time collection of data over a 12-month period.
This request fully complies with regulation 5 CFR 1320.5. Race and ethnicity data will be collected using minimum categories. The benefit of collecting detailed race and ethnicity data does not justify the additional burden to the agency or the public, and the risk to respondents’ privacy or confidentiality.
For subcollection requests under an approved generic ICR, Federal Register notices are not required, and none were published.
Exhibit A.8.1. Outside Consultation
Name |
Affiliation |
Phone |
|
Sara Bresee |
CDC |
Office: 404.639.3371 |
|
Laura Whitlock |
CDC |
Office: 404.718.4574 |
|
Deanna Amarosa |
CDC |
Office: |
|
Mmesoma Saffold |
CDC |
- |
- |
Nora Kuiper |
Banyan Communication |
- |
- |
Tola Aina |
Banyan Communications |
- |
- |
Sharanya Thummalapally |
Banyan Communications |
- |
- |
Abigail Mariani |
Banyan Communications |
- |
- |
Bria Berry |
Banyan Communications |
- |
- |
To ensure there is no duplication or redundancy of effort across projects and programs, program staff will consult with a variety of sources on the availability of data, frequency of collection, clarity of instructions, and record keeping, disclosure, and reporting format (if any), and on the data elements to be recorded, disclosed, or reported. CDC staff has consulted with relevant Federal agencies and national associations that conduct food safety messaging (e.g., USDA, FDA).
We will provide a token of appreciation of $75 for each individual who participates in the focus group. Tokens of appreciation were determined based on previous projects and experience with conducting focus groups with individuals. The range of monetary reward is consistent with current rates for participation in formative projects. Tokens of appreciation will take the form of gift cards.
Reviewed literature revealed the payment of incentives can provide significant advantages to the government in terms of direct cost savings and improved data quality (see references). As participants often have competing demands for their time, a token of appreciation for participation in a study is warranted. The use of a token of appreciation treats participants justly and with respect by recognizing and acknowledging the effort participants expend to participate. Numerous empirical studies have also shown that a token of appreciation can significantly increase response rates in cross-sectional studies and reduce attrition in longitudinal studies (e.g., Abreu & Winters, 1999; Castiglioni et al., 2008). It also should be noted that message testing is a marketing technique, and it is standard practice among commercial market researchers to offer incentives as part of respondent recruitment.
We are over-sampling those disproportionately affected from foodborne, waterborne and fungal diseases, including racial and ethnic minorities, and pregnant people. DFWED has had difficulties recruiting sufficient samples of these subpopulations in previous messaging projects. Having insufficient representation from these subgroups means their perspectives are not adequately included in message development and results in less effective messaging to support DFWED's goals to "improve public health nationally and internationally through the prevention and control of disease, disability, and death caused by foodborne, waterborne, and environmentally transmitted infections." An appropriate incentive improves the chances for these subgroups to participate, therefore increasing the government's efficiency in data collection and reducing redundancies for future efforts.
These subgroups have been difficult for DFWED to reach for several reasons.
The racial and ethnic subgroups who are being asked to participate are historically less likely to participate in research activities due to mistrust in the medical system fostered by research institutions. Specifically, Hispanic/Latino populations participate in research activities at a lower rate than their White counterparts. Offering a higher token of appreciation addresses issues brought on by historically unjust research practices, by encouraging participation from a more diverse pool of participants.
Racial and ethnic subgroups such as Hispanic/Latino populations are
more likely to be low-income and economically disadvantaged. Their
social economic situation makes it harder for these groups to
take off from or miss work to participate in research. Though the
groups are virtual, low-income populations are less likely to
have jobs where they work from home and may have to miss work or
leave work to participate. Offering a higher token of
appreciation may address this issue.
A similar communication evaluation project conducted in the summer of 2023 was approved for $75 per person for a 60-minute focus group discussion (OMB: 0920-1154, Agency IC Tracking Number: 0920-23DK). During this project, the team was very successful and was able to recruit 115 individuals (the goal was to recruit 144).
Contractors and anyone listening to the project will be required to sign a privacy agreement prior to the start of the project (Attachment 4). CDC’s contractor, Banyan Communications, will retain notes, audio/video files, and any other project-related documents on secure servers or in locked file cabinets; only project staff members will be able to access the servers via password-protected computers. Focus group findings and survey findings will be reported in summary form, and participants’ names and identifying information will not be included in the findings. Identifiable information will be kept separate from focus group data and survey data, so that participants’ responses cannot be linked with their names. All audio and video files will be destroyed three years after completion of the project. No identifiable information describing individual respondents will be included in the analyzed data and aggregate reports provided to CDC.
In review of this application, it has been determined that the Privacy Act is not applicable.
Banyan Communications will identify, screen, and recruit potential participants through a recruitment firm, using a proprietary recruitment list/database. Banyan Communications will use additional recruitment methods, such as including social media notices and snowball sampling as needed.
Individuals will first be screened to assess if they are eligible to be a part of the focus groups (Attachment 1). Those who meet the screening criteria for the focus groups will then receive a second demographic screener to assess which focus groups they will be put into (Attachment 3). Finally, they will be invited to attend a virtual 60-minute focus group. Participants will be asked to give verbal consent on a recording prior to the start of the focus group and will also fill out Attachment 5 before starting. They will receive a copy for their records.
Individuals will be screened for the online survey using Attachment 9 and those who agree to participate will be directed to the 10-15-minute online survey (Attachment 11). Participants will be asked to consent electronically (Attachment 12) prior to the start of the online survey.
The screeners will be stored in an encrypted online file hosted by Banyan Communications throughout the project’s duration. Once the project ends, the screeners will be destroyed. Banyan Communications will retain notes, video files, and any other project-related documents on secure servers; only project staff members will have access to the servers via password-protected computers. Findings will be reported in summary form and participants’ names and identifying information will not be included in the findings. Identifiable information is kept separate from focus group data and survey data so that participants’ responses cannot be linked with their names. All video files will be destroyed at the completion of the project.
During the focus group, the moderator will go over key parts of the informed consent during the introduction to the focus group. The moderator will inform participants that the focus group is voluntary, and that they may choose not to answer any question and end participation at any time. The moderator also will inform participants that Banyan Communications will report findings in summary form so that participants cannot be identified and that their identifiable information will be kept secure and separate from the focus group notes and video recordings. The moderator will inform the participant that there is a note taker listening/watching. The informed consent includes both the number for Banyan Communications in case participants have questions about their rights as a participant, as well as the principal investigator in case participants have questions about the project itself.
This data collection was reviewed by NCEZID’s Human Subjects Advisor, which deemed it was not human subjects’ research and gave it a non-research determination (Attachment 16).
There is a minimal risk that some questions may make respondents feel uncomfortable. There will be potentially sensitive information collected such as race and income. These questions are critical to the project because messages need to be tested with a diverse population. Therefore, the team needs to gather data surrounding race, ethnicity, income etc.
Race and ethnicity data will be collected using minimum categories. The benefit of collecting detailed race and ethnicity data does not justify the additional burden to the agency or the public, and the risk to respondents’ privacy or confidentiality.
The respondent consent form includes a statement about risks and informs participants that they may choose not to answer a particular question if they wish and/or end the session at any time without penalty.
We estimate the total annualized response burden at 650 hours (Exhibit A.12.1). For the focus group discussions, every individual will be pre-screened using a 5-minute Eligibility and Demographic Screener. This process will be used to get the final focus group participants not to exceed 64 participants. Those who screen in and agree to participate in the project will participate in a 60-minute focus group; consent activities will be included in the 60 minutes. For the online survey, every individual will be pre-screened using a 2-minute Eligibility and Demographic Screener. This process will be used to get the final sample not to exceed 1500 participants. Those who screen in and agree to participate in the project will participate in an online survey that will last about 10-15 minutes. Timing is based on our previous experience conducting research with this population using these methods to determine the overall burden per respondent.
Exhibit A.12.1. Estimated Annualized Burden Hours
Type of Respondent |
Form Name |
No. of Respondents |
Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
|
Individual |
Eligibility Screener for Focus Groups Attachment 1 |
480 |
1 |
5/60 |
40 |
|
Eligible Participant Screener for Focus Group Attachment 3 |
250 |
1 |
5/60 |
21 |
||
Eligibility Screener for Survey Attachment 9 |
4500 |
1 |
2/60 |
150 |
||
Focus Group Moderator Guide Attachment 8 |
64 |
1 |
60/60 |
64 |
||
Rapid Online Survey Attachment 11 |
1500 |
1 |
15/60 |
375 |
||
Total |
|
|
|
650 |
The estimates of the annualized cost to respondents for the burden hours for the collection of information is derived from the October 2024 mean hourly wage of $35.46 across all occupations (average hourly earnings for all employees on private nonfarm payrolls), per the U.S. Department of Labor (DOL) October 2024 (the most up-to-date non-provisional data) National Occupational Employment and Wage Estimates. The total annualized burden cost is estimated at $15,673.32 per year.
Exhibit A.12.2 Estimated Annualized Burden Costs
Activity |
No. of Respondents |
No. of Responses per Respondent |
Average Burden per Response (in Hours) |
Total Burden Hours |
Hourly Wage Rate |
Total |
Eligibility Screener for Focus Groups
Attachment 1 |
480 |
1 |
5/60 |
40 |
$35.46 |
$1,418.40
|
Eligible Participant Screener for Focus Group Attachment 3 |
250 |
1 |
5/60 |
21 |
$35.46 |
$744.66 |
Eligibility Screener for Survey Attachment 9 |
4500 |
1 |
2/60 |
150 |
$35.46 |
$5319.00 |
Focus Group Moderator Guide Attachment 8 |
64 |
1 |
60/60 |
64 |
$35.46 |
$2,269.44
|
Rapid Online Survey Attachment 11 |
1500 |
1 |
15/60 |
375 |
$35.46 |
$13,297.50
|
Total |
|
|
|
442 |
|
$23,043.09
|
There are no costs to respondents other than their time for participation.
The contractor’s costs are based on estimates provided by the contractor, who will carry out the data collection activities. With the expected period of performance, the annual cost to the federal government is estimated to be $195,592.38 (Exhibit A.14.1). This is the cost estimated by the contractor, Banyan Communications, and includes the estimated cost of coordination with CDC, data collection, analysis, and reporting.
Exhibit A.14.1. Estimated Cost to the Government
Expense Type |
Expense Explanation |
Annual Costs (dollars) |
Direct cost to the federal government |
||
CDC oversight of contractor and project |
CDC Project Officer |
$27,319.50 |
|
CDC Co-Principal Investigator |
$22,974.25 |
Subtotal, Direct Costs to the Government |
|
|
Contractor and Other Expenses |
||
Recruitment, data collection, analysis and reporting (contractor) |
Labor hours and other direct costs |
$145,298.63 |
Subtotal, contracted services |
|
|
Total cost to the government |
$195,592.38 |
No change in burden is requested, as this is a new information collection.
During qualitative data collection, the Banyan Communications note taker will enter data from the focus group discussion into ATLAS.ti, which will be stored on a password-protected computer. Analysis of the focus group data will start immediately after completion of data collection and will be conducted under the supervision of a senior staff member with extensive experience in qualitative research. Banyan Communications will conduct thematic or ground theory analysis of the data to understand participants’ reactions to the messages in as rigorous and detailed manner as possible. Banyan Communications will summarize results in a final report. The final report will include key data from the online eligibility and demographic screener and report it in descriptive data tables with accompanying narrative in the summary and final reports. Exhibit 16.1 lists the key events and reports.
During quantitative data collection, the Banyan Communications team will conduct descriptive statistics. Banyan Communications will summarize results in a final report.
Exhibit A.16.1. Project Time Schedule
Activity |
Time Schedule |
Begin recruitment |
March 25, 2025 |
Conduct focus groups |
Weeks of 4/1, 4/8, 4/15, 4/22, 4/29 of 2025 |
Administer online survey |
Weeks of 3/25, 4/22, 5/27, 6/24 of 2025 |
Report due |
July 1, 2025 |
OMB Expiration Date will be displayed on necessary materials and documents.
There are no exceptions to the certification.
Abreu, D.A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. Proceedings of the Survey Research Methods Section of the American Statistical Association.
Bonevski, B., Randell, M., Paul, C., Chapman, K., Twyman, L., Bryant, J., ... & Hughes, C. (2014).
Reaching the hard-to-reach: a systematic review of strategies for improving health and medical
research with socially disadvantaged groups. BMC medical research methodology, 14(1), 1-29.
Castiglioni, L.,
Pforr, K., & Krieger, U. (2008). The effect of incentives on
response rates and panel
attrition: Results of a controlled
experiment. Survey Research Methods, 2(3), 151-158.
Krueger, R. and Casey, M. (2009) Focus Groups: A Practical Guide for Applied Research. Sage
Publications: Thousand Oaks, CA.
Robinson, K.A., Dennison, C.R., Wayman, D.M.,
Pronovost, P.J., and Needham, D.M. (2007).
Systematic
review identifies number of strategies important for retaining study
participants. J
Clin Epidemiol; 60(8): 757-765.
Shettle, C., & Mooney, G. (1999). Monetary incentives in U.S.
government surveys. Journal of Official
Statistics,
15, 231–250.
Singer, E., N. Gelber, J. Van Hoewyk, and J.
Brown (1997). Does $10 Equal $10? The
Effect of
Framing on the
Impact of Incentives. Paper
presented at the American Association for Public Opinion;
Norfolk, VA.
Singer, E., Van Hoewyk, J., and Maher, M.P.
(2000). Experiments with Incentives in Telephone
Surveys.
Public Opinion Quarterly
64(3):171-188.
U.S. Bureau of Labor Statistics. Economy at a Glance. Retrieved from https://www.bls.gov/eag/eag.us.htm, on November 4, 2024.
Zhou, M., Zhang, N., Zhang, M. et al. Culture, eating behavior, and infectious disease control and prevention. J. Ethn. Food 7, 40 (2020).
Restricted
Use/Any User (No encryption)
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
Author | Peyton Williams |
File Modified | 0000-00-00 |
File Created | 2025-05-18 |