Supporting Statement A

Goal of the project: The purpose of this project is to conduct a formative evaluation to understand the attitudes, behaviors, knowledge, and communication channel preferences among U.S. adults regarding foodborne illness outbreaks and to evaluate the efficacy and utility of public communications from the Outbreak Response and Prevention Branch to inform the public about foodborne illness outbreaks.
Intended use of the resulting data: To improve the design and delivery of public communication efforts regarding foodborne illness outbreaks in the United States.
Methods to be used to collect: Focus groups and hybrid groups involving cognitive interviewing and focus groups.
The subpopulation to be studied: General population
How data will be analyzed: Thematic or grounded theory analysis of qualitative data

Circumstances Making the Collection of Information Necessary

The Centers for Disease Control and Prevention (CDC) requests approval for a new Gen-IC titled “Focus Group Study of U.S. Adults’ Knowledge, Attitudes, and Practices of Foodborne Disease Outbreaks and Illness Prevention” under OMB Control No. 0920-1154. Information collection activities are limited to formative work that will result in the development of improved communication materials and efforts to inform the public about emerging foodborne illness outbreaks and recalls.

Each year, fourteen foodborne pathogens cost the American public 61,000 quality-adjusted life years and $14 billion dollars in economic loss (Hoffmann, Batz, and Morris, 2012). Once a foodborne outbreak of enteric bacteria has occurred, consumers need the ability to find and understand reliable information to avoid joining one of the 48 million Americans that become sick from a foodborne illness each year (CDC, 2018).

The Outbreak Response and Prevention Branch (ORPB) reduces additional incidences of foodborne illnesses by educating the public via web announcements with current and accurate information on the outbreak so that consumers, retailers, media, and medical professionals can act accordingly. Though the webpage metrics indicate high viewership of these messages since their inception in 2006, the format and content of these webpages has not undergone rigorous message testing to determine the level of understanding and behavioral responses from the general public.

A contractor, EurekaFacts, will employ a mixed-mode approach that combines one-on-one cognitive interviewing with focus groups to derive qualitative data at the individual and group level that will help direct ORPB’s communication efforts.

Purpose and Use of Information Collection

The purpose of this project is to conduct a formative evaluation to understand the knowledge, attitudes, behaviors, perceptions, and communication channel preferences among U.S. adults regarding foodborne illness outbreaks and to evaluate the efficacy and utility of public communications from ORPB to inform the public about foodborne illness outbreaks.

The contractor will conduct a series of focus groups and hybrid groups, combining cognitive interviewing and focus group methods, to explore these topics with both the general public and older adults, defined as age 65 years and older.

Data collection will include a screening process to determine participant eligibility for recruitment. The contractor will recruit from the EurekaFacts panel of previous research participants in the DC-Arlington-Alexandria, Maryland, Virginia, and West Virginia metropolitan region. The EurekaFacts participant database includes tens of thousands of individuals in the DC Metro area. In addition, EurekaFacts will employ a recruitment strategy that integrates multiple outreach/contact methods and resources such as Internet ads, individual emails, telephone recruiting, and on-site location-based recruiting (attachments B and C). Furthermore, EurekaFacts will implement snowball referrals from personal and community networks during general recruitment efforts and after completing focus groups.

The evaluation results will be used to develop and redesign ORPB communication materials and to inform the U.S. public about foodborne illness outbreaks.

Use of Improved Information Technology and Burden Reduction

Respondents will be screened using the pre-approved screener script to be programmed into a computer-assisted telephone interview (CATI) system to ensure that the screening procedure is uniformly conducted, instantly quantifiable, and may be checked and monitored throughout the recruitment effort (attachments B and C).

If the respondents meet the recruitment criteria, potential participants will be provided with the project description. The screening interviewers will inform respondents about the project objectives, purpose, and participation requirements of the data collection effort, the activities that it entails, and, if applicable, any potential risks associated with participation. If respondents agree to participate in the project, EurekaFacts will collect all contact information, including telephone numbers, email, and postal contact information. Following the determination of eligibility, participants will be contacted by email to schedule participation in the focus group. The initial screening is short, estimated at 3 to 5 minutes. Eligible respondents will participate with informed consent and all responses will be kept private to the extent permitted by law. The briefness of the screening will reduce the burden to non-participants. Where possible and upon consent from the participant, the contractor will audio-record the groups to capture all information and help prepare reports.

Efforts to Identify Duplication and Use of Similar Information

CDC is not aware of the availability of any similar information. The contractor searched for “gray” literature online to identify duplication and use of similar information. Duplication of this effort or the existence of similar information was not identified during this process.

Impact on Small Businesses or Other Small Entities

This data collection will not involve small businesses

Consequences of Collecting the Information Less Frequently

This is a one-time information collection.

Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A. A Federal Register notice was published for this generic package on July 18, 2016, Vol. 81, No. 137, pp. 46680. No public comments were received.

B. The contracting company, EurekaFacts, will be used to conduct all data collection related to the proposed evaluation. Data collection will include recruiting and screening participants into the formative evaluation and conducting four 100-minute long in-person focus groups and four 100-minute long in-person hybrid groups (combing cognitive interviews and focus groups) with adults in the United States.

Explanation of Any Payment or Gift to Respondents

Both the focus groups and hybrid groups are estimated to take 100 minutes. Participants will receive $75 at the end of the session as a token of appreciation for their participation. It is assumed that many of these participants will be taking time either during work hours or personal time to complete the groups, and may have children. The respondents must also get to the site where the groups are taking place. Therefore the monetary gift may serve to offset out-of-pocket expenses related to participating in the evaluation.

Numerous empirical studies have shown that incentives can significantly increase response rates (e.g., Abreu & Winters, 1999; Shettle & Mooney, 1999). The token of appreciation amounts were determined through discussions with contract staff with expertise in conducting interviews with the study population. Removing the token of appreciation could negatively affect recruitment, which could threaten the development of a messaging strategy resulting from this testing.

Protection of the Privacy and Confidentiality of Information Provided by Respondents

The National Center for Emerging and Zoonotic Infectious Diseases (NCEZID) reviewed this submission and determined that the Privacy Act does not apply. Focus group participants will be recruited and moderated by a contracting company. Personally identifiable information (PII) will not be transmitted to CDC.

The screening instrument for this evaluation is provided in Attachment B. Before data collection, participants will be given time to read the consent form (Attachment D) and ask questions. They will be given two copies of the informed consent: one to keep and one to sign to indicate consent and return to the research team. The consent form includes the phone number and email for a point of contact at EurekaFacts in case participants have any questions about the focus group after participating.

The contractor will conduct the formative work in the following ways:

Screening: EurekaFacts will recruit participants by emailing their database of individuals and households, using advertisements in newspapers (print and online), and referrals from personal networks from those who have responded (Attachment B). Individuals interested in participating will use a link provided which will take them to an online screening form. Those who do not meet inclusion criteria will be notified during the screening of their ineligibility. Those that meet inclusion criteria will be notified during the screening of their potential eligibility. EurekaFacts will send participants a confirmation email and letter with the date, time, and location of their focus group, and provide a telephone reminder at least 24 hours prior to their appointment.

PII (e.g., name, address, e-mail address, and telephone number) will be used by the contractor to make contact with, and send reminders, to participants. All PII will be maintained by the contractor in a locked file cabinet or on secure online servers. No PII will be sent to CDC. The de-identified screening data for recruited participants will be password protected and encrypted when sharing with CDC. CDC will keep these screening data on CDC’s secure network in a drive accessible only to CDC staff involved in this project.

Focus Groups: Four in-person focus groups and four in-person hybrid groups will be conducted. Each focus group will last no more than 100 minutes. Prior to starting the focus groups, all participants will complete an informed consent form and will be provided with an additional consent form to take home (Attachment D). During the focus groups, participants will not use their full names but first names only to protect their identity. The contracting company will keep all consent forms in locked file cabinets or on secure online servers at their facility, and they will not share these documents with CDC. Audio recordings of the focus groups, and field notes taken during the focus groups will be kept in locked file cabinets or on secure online servers at the contracting company’s facility. The contracting company will password protect and encrypt the audio files, field notes, and focus group transcripts for sharing with CDC. CDC will keep all focus group materials on CDC’s secure network in a drive accessible only to CDC staff involved in this project.

All findings will be reported in aggregate only. Reports will not include PII and will be stored on a secure share drive and password-protected computers. The contracting company will be instructed to destroy their project-related records (i.e., screening data, contact logs, informed consent forms, audio files, and field notes) within one year of completion of the evaluation.

Institutional Review Board (IRB) and Justification for Sensitive Questions

Institutional Review Board (IRB)

This project was reviewed by NCEZID’s human subjects advisor and determined to not meet the definition of research under 45 CFR 46. IRB review is not required (Attachment H).

Justification for Sensitive Questions

There are no planned sensitive questions regarding topics such as criminal behavior, sexual behavior and attitudes, drug or alcohol use, religious beliefs, or other matters that are commonly considered private. The facilitators will ask questions to capture and explore participant understanding and knowledge of concepts, such as disease messaging or warning communication. All participants will be informed that they need not answer any question that makes them feel uncomfortable or that they simply do not wish to answer.

Estimates of Annualized Burden Hours and Costs

Estimated annualized burden associated with information collection

The estimated burden for recruitment assumes attrition throughout the process. The contractor anticipates screening a maximum of 500 individuals from the general population, concluding with 54 participants completing the focus groups and hybrid groups, resulting in a total of 157 burden hours.

Four focus groups and four hybrid groups (combination of cognitive interviews and focus group discussion) will be conducted. Two of the focus groups will explore knowledge and practices (identify participant perceptions, knowledge, attitudes about foodborne illnesses, outbreaks, risks, and prevention, as well as their knowledge, attitudes, and practices about food items that pose a high risk of contamination). The other two focus groups will address warning communication messages with participants, especially in terms of their preferences with tone, format, and placement of risk communication messages (Attachment G).

Methodology	FG Objective – Knowledge & Practices	FG Objective – Preferred Communication
Number of Standard Focus Group Sessions	2 Focus Groups (9 participants each)	2 Focus Groups (9 participants each)
Number of Hybrid: Cognitive Interview and Focus Groups	2 Hybrid groups (4 - 5 participants each)	2 Hybrid Groups (4 – 5 participants each)

The screener (Attachment C) will be completed 500 times. It is expected to take 3 minutes per response.
Follow-up contact with respondents who meet inclusion criteria (confirmation email (Attachment B) and consent form (Attachment D)) is expected to take 10 minutes per response.
36 general population respondents will be split into four focus groups of 9 participants each. Two of the focus groups will cover Knowledge and Practices (Attachment E), and two will cover Communication Messages (Attachment F). Each focus group is estimated to take 100 minutes.
18 general population respondents will participate in hybrid focus groups which will also include cognitive interviews (Attachment F):
- 9 general population respondents will be split into two hybrid groups (4-5 participants in each) involving cognitive interviews and focus groups (Attachments E and F). The total time for the hybrid group is estimated to take 100 minutes.
- 9 older adult respondents will be split into two hybrid groups (4-5 participants in each) involving cognitive interviews and focus groups (Attachments E and F). The total time for the hybrid group is estimated to take 100 minutes.

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Avg. Burden per response (in hrs.)	Total Burden (in hrs.)
General population	Screener	500	1	3/60	25
General population	Follow-up contact	250*	1	10/60	42
General population focus group	Moderator guide for focus group	36	1	100/60	60
General population -Hybrid group	Moderator guide for focus group and cognitive interview	18	1	100/60	30
Total					157

* Subset of initial contact group, not double counted in the total number of respondents.

B. Estimated Annualized Burden Costs

The estimated burden for recruitment assumes attrition throughout the process. The contractor anticipates screening a maximum of 500 individuals concluding with 54 participants completing the focus groups and hybrid groups, resulting in a total of 157 burden hours. Total estimated cost burden is $2,844.84.

Type of Respondent	Form Name	Total Burden Hours	Hourly Wage Rate*	Total Respondent Costs
General population	Screener	25	$18.12	$453.00
General population	Follow-up contact	42	$18.12	$761.04
General population focus group	Moderator guide for focus group	60	$18.12	$1087.20
General population - hybrid group	Moderator guide for focus group and cognitive interview	30	$18.12	$543.60
Total				$2,844.84

*Hourly Wage data obtained from https://www.bls.gov/oes/current/oes_nat.htm using Median Wage for All Occupations. Accessed on November 15, 2018.

Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no costs to respondents other than their time to participate.

Annualized Cost to the Government

The average annualized cost to the Federal Government to collect this information is $40,197.48. The federal government personnel estimate is based on cost of five CDC staff. Federal staff responsibilities include overall management and oversight of the project, provision of content matter expertise in the development of the research strategy and data collection instruments, oversight of focus group composition, and overseeing all data analyses and dissemination activities.

Costs for the contractor, EurekaFacts, include direct labor for recruiting and screening participants, conducting focus groups, transcribing focus group audio recordings, analyzing qualitative data, and preparing and presenting the summary report.

		Percent Time	Total ($)
Federal Government Personnel Costs	CDC Epidemiologist (GS-13)	5%	$6,726.12
	CDC Branch Manager (GS-13)	5%	$7,893.92
	CDC Health Communication Specialist (GS-11)	5%	$6,182.40
	CDC Supervisory Medical Officer (GS-15)	5%	$11,068.37
	ORISE Fellow (GS-9)	10%	$8,326.67
Total Annualized Cost to Government			$40,197.48

Explanation for Program Changes or Adjustments

No changes are requested as this is a new information collection.

Plans for Tabulation and Publication and Project Time Schedule

Project Time Schedule
Activity	Time Schedule
Begin recruitment of participants	1 week after OMB approval
Conduct focus groups	3 months after OMB approval
Final report due	4 months after OMB approval
In-person presentation	5 months after OMB approval

The main reporting mechanism will be in the form of a final formative research report submitted to CDC. The final report will include the following information: an executive summary, overview of background literature to provide contextual information about the purpose of the research, a detailed summary of the formative research methods and results, a discussion of findings, strengths and limitations of the research, and future directions regarding message strategy.

In addition, EurekaFacts will schedule and conduct in-person briefings for the CDC team and staff with a visual display materials and handouts in PowerPoint. Comments on draft presentations will be incorporated into final presentations. Each briefing and presentation will include an overview of the project, key findings from the focus groups, and recommendations.

Reason(s) Display of OMB Expiration Date is Inappropriate

The display of the OMB Expiration date is not inappropriate.

Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

References

Abreu, D. A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. Proceedings of the Survey Research Methods Section of the American Statistical Association.

Centers for Disease Control and Prevention (2018, February 16). Foodborne Illnesses and Germs. Retrieved from https://www.cdc.gov/foodsafety/foodborne-germs.html

Hoffmann, Sandra, Michael B. Batz, and J. G. Morris. (2012). Annual cost of illness and quality-adjusted life year losses in the united states due to 14 foodborne pathogens. Journal of food protection 75(7) (07), 1292-302. Retrieved from http://proxygw.wrlc.org/login?url=https://search-proquest-com.proxygw.wrlc.org/docview/1024158105?accountid=11243 (accessed August 14, 2018).

Shettle, C., & Mooney, G. (1999). Monetary incentives in U.S. government surveys. Journal of Official Statistics, 15, 231–250.

Attachments

Sampling Plan
Recruitment Materials
Screener
Consent forms
CDC Foodborne Outbreak Knowledge and Practices FG Moderator Guide
CDC Foodborne Message Test FG Guide & Cognitive Interview
Messages to Test
CDC non-research determination

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	0000-00-00

Supporting Statement A