Study Protocol

ICR ATTACHMENT A

U.S. Food and Drug Administration
Study on the Occurrence of
Foodborne Illness Risk Factors
in Selected Retail and Foodservice Facility
Types (2013-2024)

Protocol for the Data Collection

Revised April 28, 2015
OMB Control No. 0910-0744
Expiration Date: 08/30/2024

Study on the Occurrence of Foodborne Illness Risk Factors
in Selected Retail and Foodservice Facility Types (2013-2024)
Protocol for the Data Collection
Table of Contents
I. Background ................................................................................................................. 2
II. 2013-2024 Study Objectives and Purpose .................................................................. 2
III.
Industry Segments and Facility Types Included in the Study.................................. 3
IV. Data Collection Cycles ............................................................................................ 5
V. Selection of Data Collectors ....................................................................................... 6
VI. Random Selection of Establishments....................................................................... 6
VII. Sample Size.............................................................................................................. 7
VIII.
Preparing for the 2015-2016 Data Collection in Selected Healthcare; School;
and Retail Food Store Establishments ................................................................................ 7
IX. Conducting the Data Collection ............................................................................... 9
Conduct a Quick Walk-Through ................................................................................. 9
Focus on the Primary Data Items .............................................................................. 10
Other Areas of Interest – Data Items ........................................................................ 10
Information Statements ............................................................................................. 11
Documenting Observations of Food Safety Practices .............................................. 12
Handwashing Frequency Assessment ....................................................................... 14
Assessment of Food Safety Management Systems ................................................... 14
Establishment Information ........................................................................................ 16
Corrective Actions – Observations that Pose a Significant Public Health Risk ....... 16
Exit Briefing with Person in charge .......................................................................... 17
X. Entering the Data – Transition from ACCESS Database to Web-Based Platform... 17
XI. Piloting the Use of Hand Held Technology ........................................................... 18
XII. Reports from Previous FDA Retail Food Risk Factors Studies............................. 18
Attachment A – Example Introductory Letter for Establishments Selected for the Study 19

1

I. Background
In 1998, the U.S. Food and Drug Administration’s (FDA) National Retail Food Team
initiated a ten-year voluntary study to measure trends in the occurrence of foodborne
illness risk factors - preparation practices and employee behaviors most commonly
reported to the Centers for Disease Control and Prevention (CDC) as contributing factors
to foodborne illness outbreaks at the retail level. Specifically, the study included data
collection inspections of various types of retail and foodservice establishments at fiveyear intervals (1998, 2003, and 2008) in order to observe and document trends in the
occurrence of the following foodborne illness risk factors:
•
•
•
•
•

Food from Unsafe Sources
Poor Personal Hygiene
Inadequate Cooking
Improper Holding/Time and Temperature
Contaminated Equipment/Protection from Contamination

FDA developed reports summarizing the findings for each of the three data collection
periods (1998, 2003, and 2008). Data from all three data collection periods were
analyzed to detect trends in improvement or regression over time and to determine
whether progress had been made toward the goal of reducing the occurrence of foodborne
illness risk factors in selected retail and foodservice facility types.
The research obtained from these Studies provides FDA a solid foundation for
developing a national retail food program model that can be used by federal, state, local,
and tribal agencies to:
•
•
•
•

Identify essential food safety program performance measurements;
Assess strengths and gaps in the design, structure, and delivery of program
services;
Establish program priorities and intervention strategies focused on reducing the
occurrence of foodborne illness risk factors; and
Create a mechanism that justifies program resources and allocates them to
program areas that will provide the most significant public health benefits.

II. 2013-2024 Study Objectives and Purpose
Using this ten-year study as a foundation, FDA has developed a new study design. The
design of the new study will determine the following for each facility type included in the
study:
•
•

The foodborne illness risk factors that are in most need of priority attention during
each data collection period;
Trends of improvement or regression in foodborne illness risk factor occurrence
over time;

2

•
•

•

Potential correlations between operational aspects of the industry, such as average
number of meals per day, number of employees, complexity of food preparation,
and the control of foodborne illness risk factors;
Potential correlations between elements within regulatory retail food protection
programs, such as enrollment in the FDA Voluntary National Retail Food
Regulatory Program Standards, timing of regulatory inspections, grading
systems, posting of inspections results, manager certification requirements and
required food handler training, and the control of foodborne illness risk factors,
and
The impact of industry food safety management systems in controlling the
occurrence of foodborne illness risk factors.

The results of the study will be used to:
• Provide FDA research information that will assist the agency develop retail food
safety initiatives and policies focused on the control of foodborne illness risk
factors;
• Identify retail food work plan priorities and allocates resources to enhance retail
food safety nationwide;
• Generate nationally representative estimates of the prevalence of foodborne
illness risk factors and trends of improvement and regression over time; and
• Recommend best practices and targeted interventions strategies to assist the retail
and foodservice industry and state, local, and tribal regulatory professionals with
reducing the occurrence of foodborne illness risk factors.

III.

Industry Segments and Facility Types Included in the Study

The scope of the Study includes four major segments of the retail and foodservice
industries that account for over a million varied and diverse types of operations in the
United States:
• Restaurants
• Healthcare Facilities
• Schools (K-12)
• Retail Food Stores
Each of these four major industry segments is comprised of specific facility types that
have been defined in Tables 1-4.

3

Table 1: Description of Facility Types that Comprise the Restaurant Industry Segment

Facility Type
Full Service
Restaurants
Fast Food
Restaurants

Description
Establishments where customers place their order at their table; are
served their meal at the table, receive the service of the wait staff,
and pay at the end of the meal.
Also referred to as quick service restaurants and defined as any
restaurant that is not a full service restaurant. Customers generally
order and pay for their meals at a counter.

Table 2: Description of Facility Types that Comprise the Health Care Industry Segment

Facility Type
Hospitals
Long-Term
Care

Description
Foodservice operations that provide for the nutritional needs of
inpatients, by preparing meals and transporting them to the patient’s
room and/or serving meals in a cafeteria setting (meals in the
cafeteria may also be served to hospital staff and visitors).
Foodservice operations that prepare meals for residents in a group
care living setting such as nursing homes and assisted living centers.

Table 3: Description of Facility Types that Comprise the School (K-12) Segment

Facility Type
Base Kitchen

On-site Kitchen
Combination
Kitchen

Description
School foodservice facility where meals are fully prepared in the onsite kitchen. Some meals are served to students on-site; other meals
are shipped to other locations (including multiple locations within the
same school.
School foodservice facility where all meals are prepared and serviced
on-site.
School foodservice facility in which some meals are prepared and
served on-site; but some meals are fully prepared or partially
prepared in a central or base kitchen.

4

Table 4: Description of Facility Types that Comprise the Retail Food Store Segment

Facility Type

Description
Areas is a retail food store where foods, such as luncheon meats and
cheeses, are sliced for the customers and where sandwiches and
salads are prepared on-site or received from a commissary in bulk
containers, portioned, and displayed. Parts of the deli
Deli Department department/operation may include:
• Salad bars, pizza stations, and other food bars managed by the deli
/ Operation
department manager,
• Areas where meat and poultry products are cooked and offered for
sale as ready-to-eat and are managed by the deli department
manager.
Seafood
Areas in a retail food store where seafood is cut, prepared, stored, or
Department /
displayed for sale to the customer.
Operation
Areas in a retail food store where produce is cut, prepared, stored, or
Produce
displayed for sale to the consumer. A produce operation may include
Department /
salad bars or juice stations that are operated under the same manager
Operation
who has responsibility for the produce department.

IV.

Data Collection Cycles

In 2013, FDA obtained Office of Management Budget (OMB) approval to initiate the
first phase of the Study focused on the data collection within the restaurant segment of
the industry. The restaurant data collection was initiated November, 2013 and completed
in early October of 2014.
In order to assess trends over time, a minimum of three data points are required for
statistical purposes. For the restaurant segment of the industry, two additional data
collections are scheduled. The 2nd data collection period for the restaurant industry is
scheduled to start October, 2017, and the 3rd data collection period is slated to being
October, 2021 as outlined in Table 5
TABLE 5. SUMMARY OF DATA COLLECTION TIME FRAMES
INDUSTRY
SEGMENT

FACILITY
TYPE

Restaurants

Full Service Restaurants
Fast Food Restaurants

2nd Data
Collection
Industry
Segments

Health Care Facilities, Retail
Food Stores, and Schools (K12)

YEAR FOR
INITIAL DATA
COLLECTION
(Baseline
Measurement)
Nov. 15, 2013
To
Sept. 30, 2014

2ND
DATA
COLLECTION
PERIOD

3RD
DATA
COLLECTION
PERIOD

Oct 1, 2017
To
Sept 30, 2018

Oct. 1, 2021
To
Sept 30, 2022

Oct. 1, 2015
To
Dec. 31, 2016

Oct 1, 2019
To
Dec. 31, 2020

Oct. 1, 2023
To
Dec. 31, 2024

5

The second phase of the data collection is planned to be launched in October, 2015. This
second data collection period will focus on three industry segments that include
healthcare facilities, schools (K-12), and retail food stores. This data collection period
will run 15 months and be completed in December, 2016. As with the restaurant segment
of the industry, two additional data collection periods are scheduled to provide a basis for
assessing trends in the control of foodborne illness risk factors over time. The 2nd data
collection period for healthcare facilities, schools, and retail food stores is scheduled to
start October, 2019, and the 3rd data collection period is slated to being October, 2023 as
outlined in Table 5 above.

V. Selection of Data Collectors
FDA has approximately 25 Regional Retail Food Specialists (Specialists) who serve as
the data collectors for the ten-year study. The Specialists are geographically-dispersed
throughout the U.S. and possess technical expertise in retail food safety and a solid
understanding of the operations within each of the facility types selected for the study.
The Specialists are also standardized by FDA’s CFSAN personnel in the application and
interpretation of the FDA Food Code.

VI.

Random Selection of Establishments

A geographical information system (GIS) database containing a listing of businesses
throughout the U.S. is used as the establishment inventory for the data collection. The
geographical distribution of Specialists throughout the U.S. allows for a broad sampling
of facility types in all regions of the U.S.; therefore, establishments are randomly selected
to participate in the study from among all eligible establishments located within a 150mile radius of each of the Specialists’ home locations. This model provides a reasonably
convenient, cost-effective design for generating nationally representative estimates of the
prevalence of foodborne illness risk factors and trends of improvement and regression
over time.
The random selection of establishments from the GIS database is performed by the
FDA’s Center for Food Safety and Applied Nutrition (CFSAN) Biostatistics Branch.
Prior to distributing the selected establishments to the Specialists, the Biostatistics
Branch, working with members of FDA’s National Retail Food Team, performs an initial
review to ensure establishments are correctly classified and considered eligible to
participate in the study based on the facility type descriptions in Tables 1-4.
To further determine the pool of establishments eligible for selection, an assessment is
made to exclude operations that handle only pre-packaged food items or conduct low-risk
food preparation activities. Annex 5, Table 1 – Risk Categorization of Food
Establishments of the 2009 FDA Food Code contains a grouping of establishments by
risk, based on the type of food preparation that is normally conducted within the
operation. The vast majority of selected establishments are to be chosen from risk
categories 2 through 4.

6

VII. Sample Size
In order to obtain a sufficient number of observations to conduct statistically significant
analysis, the FDA CFSAN Biostatistics Branch has determined, based on the previous
ten-year foodborne illness risk factor study, that approximately 400 data collection
inspections of each industry segment facility types listed below are needed during initial
and subsequent data collection periods:






Full Service Restaurants (400 data collections/period)
Fast Food Restaurants (400 data collections/period)
Healthcare (400 data collections/period)
Schools (400 data collections/period)
Retail Food Stores (400 data collections/period)

This sample size provides sufficient observations to be 95% confident that compliance
percentages derived from the data collections are within 5% of their actual occurrence.
The sample for each data collection period is evenly distributed among the Specialists.
Given the participation in the study by the industry is voluntary and the status of any
given randomly selected establishment is subject to change, substitute establishments will
be provided to each Specialist for cases where the facility is misclassified, closed, or
otherwise unable or unwilling to participate. The inventory of substitute establishments
will be randomly selected by the FDACFSAN Biostatistics Branch and used to replace an
ineligible establishment that was included on their original list.

VIII. Preparing for the 2015-2016 Data Collection in Selected
Healthcare; School; and Retail Food Store Establishments
With only a few minor exceptions, the data collection protocol and methodology planned
for healthcare facilities; schools (K-12); and retail food stores will mirror the restaurant
study protocol. The few exceptions are related to unique characteristics related to the
different facility types. The data collection elements related to food safety procedures
and practices is the same for all the facility types.
Data Collection Procedures and Training
Each Specialist will attend a training webinar prior to initiating the data collection. The
training will be provided by members of the FDA National Retail Food Team that have
been responsible for the design and assessment of all the Retail Food Risk Factor Study
elements. The training will cover all the study components with particular emphasis on
the data collection protocol and marking instructions for the data collection form 1. The
workshop training will include written instructions for completing the data collection
form that will be used to record the observations made during the visits to the
establishments that have been selected for the study.

1

The data collection forms used by the FDA are available at www.fda.gov/RetailFoodProtection.

7

Verification of Eligibility of Randomly Selected Restaurants
Each Specialist will receive from FDA’s CFSAN Biostatistics Branch, a set of restaurant
facilities within their primary area of responsibility that have been randomly selected for
the study. Prior to conducting the data collection, the Specialist will contact the state or
local jurisdiction that has regulatory responsibility for conducting retail food inspections
for the selected establishment. The Specialist verifies that the restaurant facility has been
properly classified (in the correct facility type category) for the purposes of the study and
is still in operation. The Specialist ascertains whether the selected restaurant is under
legal notice from the state or local regulatory authority. If the selected restaurant is under
legal notice, the Specialist will not conduct a data collection in that establishment. The
Specialist will remove the establishment from their sample inventory and select another
establishment from their list of substitute establishments provided by the FDA CFSAN
Biostatistics Branch.
Working with State and Local Regulatory Authorities
As part of the initial contact with the state or local regulatory authority, the Specialist
obtains information from the jurisdiction pertaining to the items listed under the heading,
“Information on the Regulatory Authority,” found on pages 2 and 3 of the Data
Collection Form. The data collection fields in this section are the same for all three
industry segments (healthcare; schools; retail food stores). At that time, the Specialist
will collect the information for the following data collection fields:
●
●
●
●
●
●
●
●
●
●

Name of Jurisdiction with Regulatory Oversight;
Enrolled in FDA Retail Food Program Standards;
Jurisdiction Meets Standard 1;
Dates of the Two Most Recent Regulatory Routine Inspections;
Jurisdiction Uses a Grading System;
Type of Grading System;
Jurisdiction’s Program Includes Public Reporting of Inspection Results;
Jurisdiction Has a Mandatory Food Protection Manager Certification Requirement;
Scope of Food Protection Manager Certification Requirement; and
Jurisdiction Requires Food Handler Cards

Guidance for completing these data collection information fields is included on pages 17
– 24 in the FDA Retail Food Program, Foodborne illness Risk Factor Study - Marking
Instructions for the Data Collection Form 2.
The Specialist will extend an invitation to the state or local regulatory authority to
accompany him or her on the data collection visit. Should the regulatory authority accept
and accompany the Specialist, the Specialist should strongly recommend that the state or
local regulatory authority refrain from conducting a regulatory compliance inspection
during the data collection visit.
Calibration of Temperature Measuring Devices
Specialists must ensure that thermometers used for each data collection are accurate. The
2

A copy of this document is available at www.fda.gov/RetailFoodProtection.

8

Specialists must calibrate their thermometers prior to each establishment data collection
visit.

IX.

Conducting the Data Collection

Unannounced Data Collection Visits
Each data collection visit is to be unannounced. The intent is to observe the operation in
its normal mode, without special preparation to accommodate the data collection visit.
Discuss Purpose of the Data Collection
Upon arrival to the establishment, the Specialist will explain to the owner the purpose of
the visit. An introductory letter that explains the purpose of the data collection visit and
the study must be used in addition to a verbal explanation. A sample letter is provided in
Attachment A. If entry into the selected establishment is denied by the owner or person
in charge, the Specialist will not conduct a data collection. The Specialist will select a
new establishment from the substitute establishment list provided by the FDA CFSAN
Biostatistics Branch.
Conduct a Quick Walk-Through
The primary purpose of the data collection is to observe food safety practices and
employee behaviors that are associated with the control of foodborne illness risk factors.
After discussing the purpose of the data collection and developing a rapport with the
person in charge, the Specialist is to conduct a quick (two to three minute) walk-though
of the establishment’s kitchen. The goal of the quick walk-though is to identify the
critical food preparation processes being conducted at the time of the inspection so that
inspection priorities and flow can be determined. For each critical activity observed
during the walk-through, the Specialist should determine whether the activity is static
(one that will likely be the same over the course in the inspection) or dynamic (one that
will likely be completed soon or will change quickly over the course of the inspection).
In addition, the Specialist will need to consider the data that will be needed over the
course of the inspection to adequately assess the activities being performed. For instance,
if cooling or reheated for hot holding are observed during the quick walk-through, the
Specialist will likely need multiple temperature measurements over time to ascertain
whether the procedures being used are effective.
During the quick walk-through, the Specialist should ask the operator whether cooking,
preparation, cooling, reheating, or receiving are currently being conducted.
Specialists should set priorities for the inspection based on the quick walk-through and
responses to the operator’s questions about the specific activities being conducted at the
time of the inspection.
A review of the establishment’s menu can provide important information on the type of
processes conducted in the operation, but it should be integrated as part of the data
collection and not done as a separate interview activity with the person in charge. The
Specialist is to use the menu as an information resource as the data collection is being
conducted.

9

Focus on the Primary Data Items
The data collection is intended to be targeted on the control of foodborne illness risk
factors. It is not intended to be a comprehensive assessment of compliance with Food
Code requirements. The focus of the data collection is to be on observations of the
primary data items listed on the data collection form.
Data items 1 through 10 are considered primary data items. Each of the primary data
items has been placed under the appropriate FDA foodborne illness risk factor category
which will be used as the key indicators for FDA’s statistical analysis for the study:
 Risk Factor –Poor Personal Hygiene
1. Employees practice proper handwashing
2. Food Employees do not contact ready-to-eat foods with bare hands
 Contaminated Equipment / Protection from Contamination
3. Food is protected from cross-contamination during storage, preparation,
and display
4. Food contact surfaces are properly cleaned and sanitized
 Improper Holding / Time and Temperature
5. Foods requiring refrigeration are held at the proper temperature
6. Foods displayed or stored hot are held at the proper temperature
7. Foods are cooled properly
8. Refrigerated, ready-to-eat foods are properly date marked and discarded
within 7 days of preparation or opening
 Inadequate Cooking
9. Raw animal foods are cooked to required temperature
10. Cooked foods are reheated to required temperatures
For each data collection, the Specialists should make every effort to observe procedures
and practices related to the primary data items. Comprehensive guidance for marking
observations of primary data items is provided on pages 62 – 82 of the FDA Retail Food
Program, Foodborne Illness Risk Factor Study – Marking Instruction for the Data
Collection Form.
Other Areas of Interest – Data Items
Data items 11 through 19 are listed under the heading “Other Areas of Interest.”
These food safety practices and procedures directly support active managerial control of
the foodborne illness risk factor areas addressed under the primary data items:
 Other Areas of Interest
11. Handwashing facilities are accessible and properly maintained
12. Employees practice good hygiene
13. Consumers are properly advised of risks of consuming raw or
undercooked animal foods
14. Time alone is properly used as a public health control
15. Facilities have adequate equipment and tools for ensuring food
temperature control and sanitization of food contact surfaces

10

16. Special processes are conducted in compliance with issued variance /
HACCP Plan, when required
17. Food is received from safe sources
18. Toxic materials are identified, used and stored properly
19. Management and food employees are trained in food allergy awareness as
it relates to their assigned duties
Specialists should be cognizant of opportunities to observe these data items during the
data collection. The same type of risk assessment and dynamic-static evaluation used for
the primary data items can also be applied to those listed under the “Other Areas of
Interest” in establishing priorities for the data collection. For example, assessing whether
an establishment has an accurate thermometer for checking internal food temperatures or
whether there is a chemical test kit for checking sanitization concentration, which are part
of data item 15, can be done at anytime during the data collection because these items are
static in nature. In contrast, the opportunity to assess a reduced oxygen packaging
process during the data inspection is dynamic because quantitative measurements must be
made at critical production points. A reduced oxygen packaging process also has an
inherently high food safety risk if done improperly.
Comprehensive guidance for marking observations of data items listed under the “Other
Areas of Interest” is provided on pages 83 – 96 of the FDA Retail Food Program,
Foodborne Illness Risk Factor Study – Marking Instruction for the Data Collection Form.
Information Statements
Under most of the data items, a list of information statements is provided. These
information statements are preceded by a letter for organization purposes and describe a
specific observation (food safety practice) associated with the overarching data item
under which it is listed. For example, the information statements for the Data Item #1 –
Employees practice proper handwashing are:
A. Hands are cleaned and properly washed using hand cleanser / water supply /
appropriate drying methods / length of time as specified in Section 2-301.12
of the Food Code.
B. Hands are cleaned and properly washed when required as specified in 2301.14 of the Food Code.
The information statements provide a method for:
● Conducting comparisons with the previous ten-year risk factor study (19982008). Some of the information statements were included as data items on the
data collection form used for the first study;
● Recording observations made. Data collectors have an option to check a box
rather than write a narrative statement; and
● Enhancing quality assurance pertaining to the interpretation of the data
collected. Standard statements provide a means for maintaining uniformity
and consistency among multiple data collectors.

11

Documenting Observations of Food Safety Practices
Using the current version of the FDA Food Code, the data collector will determine
whether the observations made of the employee food safety practices or behaviors
contained in the information statements were IN Compliance, OUT of Compliance, Not
Observed (NO), or Not Applicable (NA). The recorded markings of the information
statements are then used to determine the compliance status of the corresponding data
item.
An observation is based on an evaluation of one or more occurrences of a data item or
information statement at an establishment. Specific instructions for marking each data
item and information statement are provided in the FDA Retail Food Program,
Foodborne Illness Risk Factor Study – Marking Instruction for the Data Collection Form.
The four marking options are defined as follows:
•
•

•

•

IN – means that all observed occurrences were IN Compliance with the
appropriate FDA Food Code provision for the data item or information statement.
OUT – means that one or more of the observations made were OUT of
Compliance with the appropriate FDA Food Code provision for the data item or
information statement. An explanation of the specific criteria used for
determining OUT of Compliance for each data item is to be recorded by the data
collector on the data collection form.
NO – means the data item or information statement was Not Observed during the
inspection. The NO marking is used when an information statement is a usual
practice in the food establishment, but the practice is NOT observed during the
time of the inspection.
NA – means the data item or information statement is Not Applicable. The NA
marking is used when a data item or information statement is NOT a function of
the food establishment.

Quantitative measurements are to be made with calibrated thermocouples, heat sensitive
tape or maximum registering thermometers, and chemical test strips. Quantitative
temperature measurements are to be recorded in the food temperature charts provided on
the data collection form. Sanitization measurements should be recorded in the comment
section for the specific data item observed.
Recording Food Product Temperatures
The Specialist will record ALL food product temperatures measured during the data
collection in the charts provided under data items that contain specific product
temperature critical limits. A partial illustration for the temperature chart for data item #5
– is provided below:

12

Table 5: Cold Holding Temperatures Recorded During the Data Collection (List all Temperatures
Taken)
FOOD
PRODUCT

FOOD
TEMP

FOOD CODE
CRITICAL
LIMIT

TYPE OF
COLD HOLDING
EQUIPMENT

FOOD
PRODUCT

FOOD
TEMP.

FOOD CODE
CRITICAL
LIMIT

TYPE OF
COLD HOLDING
EQUIPMENT

Cooked
Chicken

40ºF

41ºF

Walk-in Cooler

Diced Ham

44ºF

41ºF

Refrigerated
Sandwich
Preperation
Table

Raw
Hamburger
Patty

52ºF

41ºF

Refrigeration Drawer
Preparation Line

Cooked Pasta

39ºF

41ºF

Walk-in Cooler

Sliced
Tomatoes

48ºF

41ºF

Refrigerated
Sandwich
Preparation Table

Cooked Ribs

41ºF

41ºF

Walk-in Cooler

41ºF

Refrigerated
Sandwich
Preparation Table

41ºF

Refrigerated
Sandwich
Preparation
Table

Egg
Salad

42ºF

Tuna Salad

42ºF

The database that will be used to record the data has been designed to provide a drop
down menu for the Food Code Critical Limits for each temperature-based data item.
Using the food product temperature entered by the Specialist, the database has been
programmed to automatically calculate the difference between the food product
temperature recorded by the Specialist and the Food Code critical limit. The database
system will then use this information to automatically enter the correct totals in the
summary of product temperatures table depicted below. The Specialist will not have to
manually complete the product temperature summary tables.
NUMBER OF
FOOD PRODUCT
TEMPERATURES

SUMMARY
COLD HOLDING PRODUCT TEMPERATURE
CATEGORIES

3

I. Number of product temperature measurements IN Compliance with Food Code critical limits
II. Number of OUT of Compliance product temperature measurements 1ºF - 2ºF above Food Code critical
limits
III. Number of OUT of Compliance product temperature measurements 3ºF - 4ºF above Food Code critical
limits
IV. Number of OUT of Compliance product temperature measurements 5ºF - 9ºF above Food Code critical
limits
V. Number of OUT of Compliance product temperature measurements 10ºF or more above Food Code critical
limits

2
1
1
1

13

Handwashing Frequency Assessment
The Specialist will record all of his or her handwashing observations during the regular
data collection using the “Handwashing Frequency Assessment” located under data item
#1 – Employees practice proper handwashing on the Data Collection Form. Over the
course of the data collection visit, the Specialist will record a tally of each time an
employee is observed doing the following:
•
•
•

Washing hands properly and when required,
Washing hands improperly, or
Failing to wash hand when required.

Specialists should recognize their limitations with this aspect of the Study. The
assessment of handwashing frequency in the context of this study is intended to provide a
broad-based indicator of handwashing practices and will not be used to draw statistical
correlations. It will be impossible to assess every activity during which handwashing
should occur so the precision needed for statistical analysis will not be achievable.
Specialists should not forgo an opportunity to observe a food safety practice or procedure
related to a primary data item in order to observe food employees who may need to wash
their hands at some point in an ongoing food preparation activity.
Handwashing frequency data will be collected throughout the normal course of the data
collection for other food safety procedures and practices. Additional inspection time
should not be allocated for collection of this data.
Assessment of Food Safety Management Systems
In addition to collecting information on compliance with the FDA Food Code, Specialists
will obtain information on the extent to which food establishments have developed and
implemented food safety management systems. FDA will use this information to
examine the correlations, if any, between the degree to which management systems are in
place and the control of foodborne illness risk factors.
The Food Safety Management System Assessment will be conducted during the same
establishment visit but independent from the determination of Food Code compliance for
individual data items. The Food Safety Management System Assessment is to be
conducted at an appropriate time so it does not compromise a Specialist’s opportunity to
observe food safety practices or procedures related to the primary data items.
The 2013 data collection will focus on the food safety management system in place to
control four key foodborne illness risk factors and selected items for each as presented
below:
 Poor Personal Hygiene
1. Employees practice proper handwashing
2. Employees do not contact ready-to-eat foods with bare hands

14

 Contaminated Equipment / Protection from Contamination
3. Food is protected from cross-contamination during preparation, and
display
4. Food contact surfaces are properly cleaned and sanitized
 Improper Holding / Time-Temperature Control
5. Foods requiring refrigeration are held at the proper temperature
6. Foods displayed or stored hot are held at the proper
temperature
7. Foods are cooled properly
8. Refrigerated, ready-to-eats foods are properly date marked and
discarded within 7 days of preparation or opening
 Inadequate Cooking
9. Raw animal foods are cooked to required temperatures
10. Cooked foods are reheated to required temperatures
Each randomly selected establishment will have a management system assessment
conducted for ONE of the four foodborne illness risk factor areas described above. The
FDA CFSAN Biostatistics Branch will randomly select the risk factor area for which a
food safety management system assessment is to be conducted for each establishment.
The data collector will evaluate the presence of three key food safety management system
elements (procedures, training, and monitoring) for all the primary data items listed
under the assigned risk factor.
 Procedures – A defined set of actions adopted by food service management for
accomplishing a task in a way that minimizes food safety risks.
 Training – Management informs employees what the procedures are and teaches
the employees how to carry them out. This is not to be used for determining
manager knowledge or certification
 Monitoring – Routine observations and measurements made by management to
determine if procedures are being followed and maintained.
For each of these three food safety management system elements, the data collector will
determine if the information provided by the establishment management adequately
addresses the essential critical limits for the assigned risk factor area. A food safety
management system assessment questionnaire has been developed for each of the
foodborne illness risk factor areas. The questionnaire for each of the risk factor areas
requires the Specialist to enter a YES or NO response for the following four statements:
•
•

Management is able to describe the critical limits for (the specific risk factor
procedure or practice) as they apply to their establishment.
Management is able to describe the steps / tasks (how and when) that are
performed to ensure the identified critical limits for (the specific risk factor
procedure or practice) are achieved.

15

•
•

Management is able to identify specific employees that have been assigned the
responsibility to correctly perform the (the specific risk factor procedure or
practice).
Management is able to produce written materials (SOPs; posters; wall charts;
wallet cards; etc.) that support the implementation of their (the specific risk factor
procedure or practice) within their establishment

Using the food safety management system assessment tool, the data collector will add up
the total number of “YES” responses for each of the management system elements
(Procedures, Training, and Monitoring). The number of “YES” responses on the
assessment tool will determine how to mark the Procedures, Training, And Monitoring
sections for the data item on the data collection form. Comprehensive guidance for
marking the food safety management system assessment (Procedures; Training; and
Monitoring) for selected risk factor areas is provided on pages 55 – 58 of the FDA Retail
Food Program, Foodborne Illness Risk Factor Study – Marking Instruction for the Data
Collection Form.
Establishment Information
During the course of the data collection, the Specialist will obtain information from the
owner/person in charge related to items listed in the following Sections on pages 1, 4, and
5 of the Data Collection Form:






Establishment Information
Establishments that are Part of Multi-Unit Operations
Manager Certification
Employee Health Policy
Highly Susceptible Populations (Only Healthcare facilities)

Guidance for completing the information fields associated with these sections of the data
collection form is provided on pages 1 – 16 and 25 – 40 of the FDA Retail Food
Program, Foodborne Illness Risk Factor Study – Marking Instructions for the Data
Collection Form.
This information can be obtained at any time during the data collection and should not
take precedence over, or inhibit, the Specialist’s observations of actual food safety
practices and procedures related to the primary data items. Specialists should consider the
static nature of this information and prioritize the collection of this information
accordingly.
Corrective Actions – Observations that Pose a Significant Public Health Risk
Though industry participation in the Study is voluntary, correction action is to be
obtained for observations that pose a significant public health risk. If conditions
observed during the data collection visit pose a significant public health risk, the
Specialist is to discuss the situation with the person in charge and seek to obtain
voluntary corrective action. FDA’s experience from data collections performed as part of
its previous study indicate that in all but a few instances, industry responded in a
cooperative and responsible manner to alleviate potential public health risks.

16

Should an instance occur where an observation during the data collection poses a
significant public health risk and corrective action cannot be voluntarily obtained, the
Specialist should contact the appropriate regulatory authority to ensure appropriate
corrective actions are taken. This is an example of a situation where it is advantageous to
have the responsible regulatory authority accompany the Specialist during the data
collection.
Exit Briefing with Person in charge
The data collection visit is conducted as part of a research project and is not intended to
be a regulatory compliance inspection. No written report is left with the establishment.
Upon completion of the data collection, the Specialist conducts an exit briefing with the
owner or person in charge to discuss significant findings and answer any questions.

X. Entering the Data – Transition from ACCESS Database to WebBased Platform
During the 2013-2014 restaurant data collection, each Specialist was provided with a
copy of an ACCESS database software program that had been specifically formatted to
store and analyze data collected during the study. The Specialists manually entered their
observations for each of the data items and information statements for the selected
establishment into the ACCESS database.
Over the past three years, FDA has been working with the National Center for Food
Protection and Defense (NCFPD) to develop and maintain a web-based database platform
for the retail food risk factor study. The web-based database platform will be located as
part of the FoodSHIELD site. The FoodSHIELD database has been designed not only for
use by FDA but for state/local/tribal jurisdictions seeking a system to store and maintain
data from their own studies. Firewalls have been built into the design of the database
system to maintain the security and integrity of data entered by each jurisdiction.
The database system also contains the option of “permission-based” sharing of data
between jurisdictions. This feature has the potential of creating more robust data sets for
analyzing trends in the occurrence of foodborne illness risk factors over time within the
retail food segment of the industry.
For the 2015-2016 Healthcare, School, and Retail food store data collection, FDA will
use the web-based database platform located in FoodSHIELD to store and maintain the
data from the Study. During this period of time, FDA is planning on uploading the data
from its recently completed restaurant data collection from the ACCESS database to the
FoodSHIELD system.

17

XI.

Piloting the Use of Hand Held Technology

The FoodSHIELD retail food risk factor study database has been designed to
accommodate both manual entry of data and uploading electronic reports from hand held
technology. During the 2015-2016 data collection, FDA will begin the transition process
from manual data entry to the use of hand held technology. As part of an agency-wide
initiative to assess the use of hand held technology, FDA will have 5 Specialists conduct
their data collections using hand held tablets. The information from the hand held tablets
will be transferred “real-time” into the FoodSHIELD database via a web browser. Based
on the lessons-learned from the hand-held technology pilot, FDA will assess the type of
equipment and support system needed to fully integrate hand held technology for future
data collections. Specialist who are not provided hand held technology will continue to
manually enter data directly into the FoodSHIELD system.
Webinar trainings and pilot data entries will be conducted so Specialists have a solid
understanding of the data entry features built into the FoodSHIELD system. Specialist
designated for the use of hand-held technology will conduct field pilot testing of the
equipment and upload data to the FoodSHIELD database prior to the start of the 20152016 data collection.

XII. Reports from Previous FDA Retail Food Risk Factors Studies
The following reports from FDA’s previous risk factors studies are available from the
following web links:
Report of the FDA Retail Food Program Database of Foodborne Illness Risk Factors (2000)

http://www.fda.gov/downloads/Food/FoodSafety/RetailFoodProtection/FoodborneIllness
andRiskFactorReduction/RetailFoodRiskFactorStudies/ucm123546.pdf
Report on the Occurrence of Foodborne Illness Risk Factors in Selected Institutional
Foodservice, Restaurant, and Retail Food Store Facility Types (2004)

http://www.fda.gov/downloads/Food/GuidanceRegulation/RetailFoodProtection/Foodbor
neIllnessRiskFactorReduction//ucm423850.pdf
Report on the Occurrence of Foodborne Illness Risk Factors in Selected Institutional
Foodservice, Restaurant, and Retail Food Store Facility Types (2009)

http://www.fda.gov/downloads/Food/FoodSafety/RetailFoodProtection/FoodborneIllness
andRiskFactorReduction/RetailFoodRiskFactorStudies/UCM224682.pdf
Trend Analysis Report on the Occurrence of Foodborne Illness Risk Factors in Selected
Institutional Foodservice, Restaurant, and Retail Food Store Facility Types (1998 – 2008)

http://www.fda.gov/downloads/Food/FoodSafety/RetailFoodProtection/FoodborneIllness
andRiskFactorReduction/RetailFoodRiskFactorStudies/UCM224152.pdf

18

Attachment A – Example Introductory Letter for Establishments
Selected for the Study
[DATE]
Dear Owner/Manager:
Your facility has been randomly selected as part of a nationwide research project
designed to assess food preparation procedures and practices specific to the various
segments of the retail food industry. The U.S. Food and Drug Administration (FDA) will
use this research for identifying best practices within the industry and directing limited
resources to areas that will provide the most significant public health benefits.
This is not a regulatory visit. Your participation is voluntary. No inspection report will
be left with your facility. This is a research project designed to focus on the
implementation of food safety procedures and practices within the retail food industry
that are designed to protect the public health. The expected length of the data collection
will be 90-120 minutes. Approximate 30 minutes of the data collection will focus on
obtaining information on the nature of your operation.
Should an observation be made of a food safety procedure or practice that poses a
significant public health risk, every effort will be made to work with you to ensure that
the appropriate corrective action is taken to alleviate the hazard. Should a situation arise
where a significant public health risk cannot be resolved during the data collection, the
regulatory authority that has issued your permit will be contacted to work with you to
ensure corrective action is taken.
An exit briefing will be provided at the end of the visit to discuss significant findings that
may assist you in enhancing the effectiveness of your food safety system. If significant
food safety issues are identified, they will be brought to the attention of the person-incharge or responsible employee to determine the appropriate corrective action based on
the current FDA Food Code. Your questions regarding the data collection process or
food safety issues in general are encouraged as part of the visit to your facility.
Your facility’s name will not appear on any reports or public documents. The research
project is designed to protect the privacy of participating establishments to the extent the
law permits. The data collected is tabulated using broad industry segments and is not
associated with any specific establishment.
FDA is responsible for providing technical assistance to approximately 75 state and
territorial agencies and more than 2,300 local departments that assume primary
responsibility for working with the industry on preventing foodborne illnesses.
Beginning in 1998, FDA began collecting data related to direct observations made of
food safety practices within institutional foodservice, restaurant, and retail food segments
of the industry. From the data collected, FDA provides guidance to regulatory and

19

industry food safety professionals to assist them in addressing food safety issues that
have the most significant impact on protecting the public health.
FDA’s previous research studies can be accessed and downloaded from the following
web link:
http://www.fda.gov/Food/FoodSafety/RetailFoodProtection/FoodborneIllnessandRiskFactorReduction/RetailFoodRiskFactorStudies/default.htm

Paperwork Reduction Act Statement: According to the Paperwork Reduction Act of
1995, an agency may not conduct or sponsor, and a person is not required to respond to a
collection of information unless it displays a valid OMB control number. The valid OMB
control number for this information collection is 0910-0744 (Expiration Date:
08/30/2024). The time required to complete this information collection is estimated to
average 73 minutes per response for the person in charge of a fast food restaurant, 106
minutes for the person in charge of a full service restaurant, 180 minutes for the person in
charge of a retail food store, and 30 minutes for the program director (or designated
individual) of the regulatory authority. This includes the time for reviewing instructions,
searching existing data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. Send comments regarding this
burden estimate or any other aspects of this collection of information, including
suggestions for reducing burden to [email protected].
Thank you for your willingness to cooperate in this important endeavor. It is through
this type of cooperative effort that government and the food service industry seek to
provide safe and wholesome food to the consuming public.
In the future, should you have any questions regarding this study or other food safety
issues, please do not hesitate to contact me at [Specialist’s phone number].
Sincerely

[Specialist’s contact information]

20