SUPPORTING STATEMENT Part B FOODBORNE IRFSR 7-18-13

SUPPORTING STATEMENT Part B FOODBORNE IRFSR 7-18-13.doc

Survey on the Occurrence of Foodborne Illness Risk Factors in Selected Retail and Foodservice Facility Types (2013)

OMB: 0910-0744

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0910-0744 can be found here:

Document [doc]

Download: doc | pdf

Revised 5-8-13; add’l revisions 7-17-13

B. Statistical Methods (used for collection of information employing statistical methods)

Respondent Universe and Sampling Methods

The respondent universe will be 1,776 respondents: 176 respondents for the pilot study and 1,600 respondents for the baseline restaurant study.

A geographical information system (GIS) database containing a listing of businesses throughout the U.S. will be used as the establishment inventory for the data collections. The data were purchased from the Environmental Systems Research Institute (ESRI) Inc. The restaurant data is the partial of ESRI’s USA Business Locations and Business Summary. It is updated annually. The latest version FDA has is July 2012. The data is stored as GeoDataBase.

The restaurants list contains data for restaurant establishment name, location, franchise code, industry classification code, number of employees, and estimated sales volume (expressed in thousands of dollars).

ESRI and its partner Infogroup references several sources including directory listings such as the Yellow Pages and business white pages; annual reports; 10Ks and Securities and Exchange Commission (SEC) information; federal, state, and municipal government data; business magazines; newsletters and newspapers; and information from the U.S. Postal Service. To ensure accurate and complete information, Infogroup conducts annual telephone verifications with each business listed in the database.

The addresses of restaurants are geocoded to assign latitude and longitude coordinates to the restaurant site. The quality of the local address system varies; address matching is better in urban areas that use street-level address system than in rural areas; restaurants that cannot be assigned to a census block group are assigned to a census tract or county. We use the geographic codes to do spatial sampling for the risk factor study.

In preparation for the retail food risk factor study, FDA reviewed ESRI’s nationwide restaurants data beginning with 2010. We found a turnover of 31% of restaurants in 2011 compared with 2010; while 36% restaurants of 2012 changed compared with 2011.

A full analysis of coverage will be submitted to OMB within 2 months of clearance.

FDA will perform a three-tiered filtering process to ensure establishments are correctly classified into the appropriate facility type described in Table 4 and considered eligible to participate in the survey. The filter types include: the subclass the establishment belongs to, the name of the establishment, and keywords are the filter types. The term “eligible” in this context means that the establishment is contained in the geographic areas that are being sampled from. Any establishment in the geographic areas can be selected.

TABLE 4 – Description of the Facility Types Included in the Survey

Industry Segment

Facility Type

Description

Restaurants

Full Service Restaurants

Establishments where customers place their order at their table, are served their meal at the table, receive the service of the wait staff, and pay at the end of the meal.

Fast Food Restaurants

Also referred to as quick service restaurants and defined as any restaurant that is not a full service restaurant.

To further determine the pool of establishments eligible for selection, an effort will be made to exclude operations that handle only pre-packaged food items or conduct low-risk food preparation activities. Annex 5, Table 1 – Risk Categorization of Food Establishment of the 2009 FDA Food Code contains a grouping of establishments by risk, based on the type of food preparation that is normally conducted within the operation.^⁶ The vast majority of selected establishments are to be chosen from risk categories 2 through 4.

Currently, FDA has 22 Regional Retail Food Specialists (Specialists) who will serve as the data collectors for the pilot. These individuals possess technical expertise in retail food safety and a solid understanding of the operations within each of the facility types to be surveyed. The Specialists are also standardized by FDA Center for Food Safety and Applied Nutrition (CFSAN) personnel in the application and interpretation of the FDA Food Code. For the FY14 restaurant data collection, we may utilize up to three additional Specialists if we can backfill vacant positions and these individuals can be standardized by CFSAN and trained to collect data. This will not affect the burden since the 800 data collections will be evenly distributed among all available standardized Specialists.

For the purposes of the study, a sampling zone is equal to the 150 mile radius around a Specialist’s home zip code. Restated, establishments will be randomly selected to participate in the study from among all eligible establishments located within a 150 mile radius of each of the Specialists’ home locations (i.e. sampling zone).

The Specialists are located near major metropolitan areas (i.e. population centers) across the contiguous United States. Population centers usually contain a large concentration of state and local regulatory jurisdictions. Given that the main role of an FDA Regional Retail Food Specialist is to support state and local regulatory jurisdictions, it is prudent from an economical and customer service standpoint for FDA to locate Specialists in population centers. Using the 150 mile radius sampling zones around the Specialists’ home locations provides three advantages to the study:

It provides a relatively good cross section of urban and rural areas from which to sample the eligible establishments.
It represents a good mix of small, medium, and large regulatory entities having jurisdiction over the eligible establishments.
It reduces overnight travel and therefore reduces travel costs incurred by the Agency to collect data.

In the interest of cost efficiency, we are applying one caveat to the sampling zones as follows. The actual driving distance to a few of the selected establishments may exceed 150 miles to geographic barriers of one form or another. Since travel time is not included in the Specialists’ work plan hours and excessive overnight travel would be financial burden to the Agency, if an establishment on the inventory list exceeds a 150 mile driving distance from the Specialist’s home, as confirmed via Google Maps, the Specialist has the option of requesting a substitute establishment. Specialists are encouraged to still conduct data collections at establishments that may exceed the 150 mile radius by only a few miles (or where travel time is not significantly impacted by the extra distance). When requesting a substitute establishment based on driving distance exceeding 150 miles, the Specialist is to include the Google Map showing the mileage distance from their home to the establishment.

The total number of restaurants in the database is 472,243 and the total number within the 22 sampling zones is 295,003. This means that the 22 sampling zones contain approximately 62% of all restaurant establishments in the contiguous United States. If additional FTEs are utilized in the FY14 restaurant data collection, then an even greater percentage of restaurant establishments would be contained within approximately 25 sampling zones. All analysis reports will clearly indicate that the sample drawn was purposeful and that estimates generated from the study cannot be generalized to the U.S. as a whole.

The following map and table illustrate the location of FDA’s 22 currently standardized Specialists and the corresponding 150 mile radius.

FIGURE 1. Location of FDA’s 22 Currently Standardized Regional Retail Food Specialists’ home zip code and the surrounding 150 mile radius (restricted by FDA region boundaries) TABLE 5 – List of FDA’ Currently Standardized Regional Retail Food Specialists by region, home city, state, and zip code.

#	FDA Region	Name	Home City/State	Home Zip Code
1	NER	Al Pistorio	Manchester, NH	03103
2	NER	Mary Leong	Brooklyn, NY	11211
3	NER	Ray Duffill	Danvers, MA	01923
4	NER	Steve Nattrass	Avon, CT	06001
5	CER	Kris Moore	Georgetown, IN	47122
6	CER	Larry Edwards	Clifton, VA	20124
7	CER	Akeila Randle	Chicago, IL	60661
8	CER	John Powell	Kenosha, WI	53142
9	CER	Barbara Kitay	Bryn Mawr, PA	19010
10	CER	Greg Abel	Minneapolis, MN	55405
11	SER	Diane Kelsch	Orlando, FL	32803
12	SER	Chris Smith	Milton, GA	30004
13	SER	Joseph Dan Redditt	Snellville, GA	30078
14	SER	Donna Wanucha	Mocksville, NC	27028
15	SWR	Mario Seminara	Lakewood, CO	80228
16	SWR	Cindy Kunkel	Kansas City, MO	64151
17	SWR	Scott Krause	Sachse, TX	75048
18	PAR	John Marcello	Gilbert, AZ	85233
19	PAR	Katey Kennedy	Brush Prairie, WA	98606
20	PAR	Brad Tufto	Spokane Valley, WA	99212
21	PAR	Richard Ramirez	San Clemente, CA	92673
22	PAR	Lisa Whitlock	Loma Rica, CA	95901

We estimate the response rate to be 98% response rate. We base this estimate on the number of entry refusals and closures we had during the previous 10-year study.

Substitute establishments will be selected for each Specialist for the cases when an establishment is misclassified, closed, or otherwise unavailable, unable, or unwilling to participate. The inventory of substitute establishments will remain with the FDA CFSAN Biostatistics Branch until needed by a Specialist to replace an ineligible establishment that was included on their original list.

Procedures for the Collection of Information

In order to obtain a sufficient number of observations to conduct statistically significant analysis, FDA has determined, based on the previous ten-year foodborne illness risk factor study (1998-2008), that approximately 400 data collection inspections of each facility type are needed during restaurant data collection. This sample size was calculated to provide for sufficient observations to be 95% confident that the compliance percentage is within 5% of the true compliance percentage.

The rationale for this calculation follows.

The previous study that was designed prior to the initial 1998 data collection did not take into account any effect of intracluster correlation (ICC). A random selection mechanism including all establishments in a geographic area was not used since, at the time, we did not have GIS technology. Instead a comparison set list approach was utilized.

During the 1998 data collection period, each Specialist developed five Comparison Set Establishment Lists for each of the facility types. In most cases, the comparison set lists were comprised of between 10 and 20 establishments located in the same geographical area. Establishments were listed in alphabetical order. In order to maintain data reliability and to ensure confidentiality of the selected establishments, the comparison set lists, as well as the inspectional observations, were retained in a central database by number rather than by establishment name or location.

In order to maintain consistency between data collection periods, the Specialists used the 1998 Comparison Set Establishment Lists in 2003 and in 2008. Selection bias was prevented by using a random number table to choose the establishments that were to be inspected.

Only one establishment was inspected from each comparison set list during the data collection. In addition, an establishment on a comparison set list could only be selected once for inspection. For instance, if in 2003, a Specialist randomly picked an establishment that had already been inspected in 1998, the Specialist would have had to draw another random number until an establishment on the comparison set list that had not been inspected was chosen.

The data from the previous study was used to estimate the intra cluster correlation and to estimate the variance for the sample size computation. This would seem to be a conservative approach since the comparison set list approach would be much more likely to produce geographic correlation than the new design that employs GIS technology and establishment lists that contain all establishments contained in a particular geographic region.

In the previous study there were 42 data items comprising six risk factor areas. If a data item was applicable in the establishment being surveyed and it was observed by the specialist it was marked either IN compliance or OUT of compliance. For each facility type an IN compliance percentage was calculated by summing all of the IN compliance observations and dividing this number by the number of observations IN compliance plus the number of observations OUT of compliance. The baseline IN compliance percentage was calculated in 1998. Data collections in 2004 and 2008 utilized the same 42 data items and the IN compliance percentages for the three data collection periods were then used to track trends over time. Within each facility type the risk factors and individual data items were also analyzed and compared over time. As we dug deeper into the data and looked at risk factors and data items the sample size became smaller and fewer inferences were made.

Although many different population parameter estimates will be made using this survey data, the sample size was calculated to ensure that the primary goal of the study was achieved. The required sample size was calculated based on the ten primary data items. Each of the ten primary data items should have a response (IN compliance or OUT of compliance) based on the information statements which are contained within each data item. We expect that all or almost all of the data items will have a response (see B.3). We will have a compliance percentage for the ten primary data items which will simply be the total number of IN compliance observations divided by the total number of IN compliance observations plus the total number of OUT of compliance observations. Therefore, each of the 400 establishments will have 10 observations that will be used to compute the IN compliance percentage for the facility type.

Using data from the previous study the “effective sample size” was calculated as follows:

Where m = 160 responses (10 per establishment) in a geographical area, k= number of geographic areas, and DE is the design effect. .

JMP 10 was utilized to calculate the ICC in EMP-results obtained by the Measurement Systems Analysis platform.

In order to calculate the sample size we needed an estimate of the variance of the proportion, a confidence level. Utilizing the ESS calculated above and estimates for the IN compliance percentages from the previous study, the precision was estimated as follows:

Where Z_.025 = 1.96, P=.71, Q = .29 and n =630 and e is the margin of error. Solving for e gives 3.50%.

Once the data is collected, the observed sample ICC and variance will be used when reporting the results. FDA feels that the sample size will be sufficient to have a margin of error of less than 5% of the estimated proportion of IN compliance observations.

Each Specialist will receive from FDA’s CFSAN Biostatistics Branch, a set of establishments within their primary area of responsibility that have been randomly selected for the study. The sample for each data collection period will be evenly distributed among the Specialists.

In addition, each Specialist will attend a training workshop prior to initiating the data collection. The training will be provided by members of the FDA National Retail Food Team that have been responsible for the design and assessment of all the Retail Food Risk Factor Study elements. The training will cover all the study components with particular emphasis on the following:

Data collection protocol (Attachment A)
Marking instructions for the data collection form (Attachment B)
Data collection form (Attachment C)
Introductory Letter (Attachment D)

The form is divided into three sections: Section 1 - Establishment Information; Section 2 - Jurisdiction with Regulatory Authority Information; and Section 3 for tabulating the Specialists’ observations of (a) the food employees’ behaviors and practices related to personal hygiene and food storage, preparation, and service, (b) the industry food safety management being employed, and (c) the frequency of food employee hand washing.

Prior to conducting any data collection inspection, the Specialist will interview the personnel of the State/local health or agriculture agency that has regulatory oversight over the selected establishment. During the interview, the Specialist will verify that the establishment has been properly classified for the purposes of the study and is still in operation. The Specialist will also ascertain whether the selected facility is under legal notice from the state or local regulatory authority. If the selected facility is under legal notice, the Specialist will not conduct a data collection and a substitute establishment will be used. The Specialist will extend an invitation to the state or local regulatory authority to accompany him or her on the data collection visit. Should the regulatory authority accept, the Specialist should strongly recommend that the state or local regulatory authority refrain from conducting a regulatory compliance inspection during the data collection visit.

In addition, the Specialist will complete Section 2 - Jurisdiction with Regulatory Authority Information of the data collection form. Guidance for completing the information fields in this section of the form is included on pages 4 – 11 of the marking instructions document (Attachment B). Information collected in this section of the form includes:

Name of the jurisdiction;
Whether the jurisdiction is enrolled in the FDA Voluntary Retail Food Regulatory Program Standards and whether the jurisdiction meets Program Standard 1^⁷;
Dates of the two most recent inspections;
Whether the jurisdiction uses a grading system, and if so, what type;
Whether the results of the inspections are posted for consumers to review, and if so, how
Whether the jurisdiction requires food protection manager certification and food handler training, and if so, whether training, test, or both training and test are required

Each data collection inspection will be unannounced. The intent is to observe the operation in its normal mode, without special preparation to accommodate the data collection visit.

Upon arrival to the establishment, the Specialist will explain to the owner the purpose of the visit. An introductory letter that explains the purpose of the data collection visit and the study should be used in addition to a verbal explanation. The letter is provided as Attachment D to the Supporting Statement. If entry into the selected establishment is denied by the owner or person-in-charge, the Specialist will not conduct a data collection. The Specialist will contact the FDA CFSAN Biostatistics Branch and request a substitute restaurant establishment as a replacement.

After discussing the purpose of the data collection and developing a rapport with the person-in-charge, the Specialist will conduct a quick (two to three minute) walk-though of the establishment’s kitchen to identify the critical food preparation processes being conducted at the time of the inspection so that inspection priorities and flow can be determined.

After priorities and inspection flow are established, the Specialist will collect the data necessary for completing the Section 3 of the form. Section 3 of the form is comprised of three parts (A – C).

Part A includes 19 data items relating to food employees’ behaviors and practices in limiting contamination, proliferation, and survival of food safety hazards:

Data Item #1 – Employees practice proper hand washing
Data Item #2 – Employees do not contact ready-to-eat foods with bare hands
Data Item #3 – Food is protected from cross-contamination during storage,
preparation, and display
Data Item #4 – Food contact surfaces are properly cleaned and sanitized
Data Item #5 – Foods requiring refrigeration are held at the proper temperature
Data Item #6 – Foods displayed or stored hot are held at the proper temperature
Data Item #7 – Foods are cooled properly
Data Item #8 – Refrigerated, ready-to-eats foods are properly date marked and discarded within 7 days of preparation or opening
Data Item #9 – Raw animal foods are cooked to required temperatures
Data Item #10 – Cooked foods are reheated to required temperatures
Data Item #11 – Hand washing facilities are accessible and properly maintained
Data Item #12 – Employees practice good hygiene
Data Item #13 – Consumers are properly advised of risks of consuming raw or undercooked animal foods
Data Item #14 – Time alone is properly used as a public health control
Data Item #15 – Facilities have adequate equipment and tools for ensuring food temperature control and sanitization of food contact surfaces
Data Item #16 – Special processes are conducted in compliance with issued variance / HACCP Plan, when required
Data Item #17 – Food is received from safe sources
Data Item #18 – Toxic materials are identified, used and stored properly
Data Item #19 – Management and food employees are trained in food allergy awareness as it relates to their assigned duties

The Specialist must follow the guidance provided on pages 22-34 and 42-86 of the marking instructions document (Attachment B) when assessing the data items in Section C, Part A.

The Specialist will use the current version of the FDA Food Code as a basis for assessing control of each of the data items. Quantitative measurements will be made with calibrated thermocouples, heat sensitive tape or maximum registering thermometers, and chemical sanitizer test strips. Infrequent, non-standard questions may be asked by the Specialist if clarification is needed on the food safety procedure or practice being observed. The Specialist will record all food product temperatures measured during the data collection in the charts provided under data items that contain specific product temperature critical limits.

Under most of the data items, a list of information statements is provided. These information statements are preceded by a letter for organization purposes and describe a specific observation (food safety practice) associated with the overarching data item under which it is listed.

An observation is based on an evaluation of one or more occurrences of a data item or information statement at the establishment. The Specialist will determine whether the observations made of the employee food safety practices or behaviors contained in the data item and information statements are IN Compliance, OUT of Compliance, Not Observed (NO), or Not Applicable (NA). The Specialist will then mark the appropriate compliance status of the data item and information statement as follows:

IN – means that all observed occurrences were IN Compliance with the appropriate FDA Food Code provision for the data item or information statement.

OUT – means that one or more of the observations made were OUT of Compliance with the appropriate FDA Food Code provision for the data item or information statement. An explanation of the specific criteria used for determining OUT of Compliance for each data item is to be recorded by the data collector on the data collection form.

NO – means the data item or information statement was Not Observed during the inspection. The NO marking is used when an information statement is a usual practice in the food establishment, but the practice is NOT observed during the time of the inspection.

NA – means the data item or information statement is Not Applicable. The NA marking is used when a data item or information statement is NOT a function of the food establishment.

Section 3, Part B of the form includes an assessment of the industry food safety management system in place at the time of the visit. The Food Safety Management System Assessment will be conducted during the same establishment visit but independent from the determination of Food Code compliance for individual data items.

The data collection will focus on the food safety management system in place to control four key foodborne illness risk factors including Poor Personal Hygiene, Contaminated Equipment/Protection from Contamination, Improper Holding/Time and Temperature Control, and Inadequate Cooking/Reheating Temperatures. Each randomly selected establishment will have a management system assessment conducted for one of the four risk factor areas described above. FDA will randomly select which risk factor area will have a food safety management system assessment for each of the establishments.

The Specialist will evaluate the presence and adequacy of three management system components (procedures, training, and monitoring) for all the data items listed under the selected risk factor. For each data item that falls under the assigned risk factor, a separate assessment will be made of the three food safety management system components using a rating scale of 1 to 4. The rating number reflects the relative degree to which each component of the management system is developed and implemented by the food establishment. Higher ratings are intended to reflect an increasing likelihood that the management system components are in place. Each rating number is broadly defined below:

Non-Existent – No system in place.

Underdeveloped – System is in early development. Efforts are being made, but there are crucial gaps in completeness and/or consistency.

Well Developed – System is complete, consistent and oral or a combination of oral and written. The preponderance of the management system is oral.

Well Developed and Documented – System is complete, consistent and primarily written. The preponderance of the management system is written. This is the goal for all establishments.

The Specialist must follow the guidance provided on pages 35-39 of the marking instructions document (Attachment B) when completing Section 3, Part B of the form. The Specialist will ask industry management the questions on page 36 of the marking instructions to obtain information on the extent to which the food establishment has developed and implemented procedures, training, and monitoring as part of a comprehensive food safety management system.

Section 3, Part C of the form, which is associated with Data Item #1 – Employees practice proper hand washing, is a hand washing frequency assessment and will include only direct observations by the Specialist. Specialists will use the guidance provided on page 40 of the marking instructions document (Attachment B) when completing this section of the form. Over the course of the data collection visit, the Specialist will record a tally of each time an employee is observed doing the following:

Washing hands properly and when required,

Washing hands improperly, or

Failing to wash hand when required.

During the course of the data collection, not necessarily at any particular time, the Specialist will use the guidance provided on pages 1-4 and 12-21 of the marking instructions document (Attachment B) to interview the owner/person-in-charge to complete the fields in Section 1 - Establishment Information of the data collection form relating to:

The establishment’s name, address, city, state, and zip code; average number of meals sold per day; seating capacity; total number of employees; maximum number of employees per shift; number of employees present at the time of visit; activity level at the time of visit; and if the establishment is a multi-unit operation, the type and size of chain, whether the unit is company- or privately-owned or a franchise. If a franchise, the number of units owned;
The status of food protection manager certification including whether one is employed by the establishment, whether the person-in-charge is a certified manager, whether a certified food protection manager is on-site during the visit, whether the establishment has a policy to have a certified manager present during all hours of operation, and the organization providing the food protection manager certification; and
The implementation of a written or unwritten employee health policy that is consistent with the requirements in the FDA Food Code.

Although industry participation in the study is voluntary, correction action will be obtained for observations that pose a significant public health risk. If conditions observed during the data collection visit pose a significant public health risk, the Specialist will discuss the situation with the person-in-charge and seek to obtain voluntary corrective action. Should an instance occur where an observation during the data collection poses a significant public health risk and corrective action cannot be voluntarily obtained, the Specialist will contact the appropriate regulatory authority to ensure appropriate corrective actions are taken.

Upon completion of the data collection, the Specialist will conduct an exit briefing with the owner or person-in-charge to discuss significant findings and answer any questions. No written report is left with the establishment.

Each Specialist will be provided with a copy of an ACCESS database software program that has been specifically formatted to store and analyze data collected during the study. The Specialist will enter his or her observations for each of the data items and information statements for the selected establishment into the ACCESS database.

Before saving a record, the Specialist will conduct a quality assurance check that has been integrated as part of the ACCESS database, to ensure that all required data entry fields have been completed and are accurate. A menu icon has been integrated into the database. Clicking on the icon will trigger a database search of data collection fields that may have been inadvertently left blank or data collection field where the Specialist has entered information that is inconsistent with the marking instructions for the study. The Specialists will be prompted to correct the data collection error. This quality assurance function will continue automatically until all data entry errors have been rectified.

When the Specialist has completed collecting data for all their randomly selected facilities, the ACCESS database that has been installed on his or her computer will be exported to a central database. A QA review will be conducted to ensure that no duplicate records or overriding of existing records has occurred.

Methods to Maximize Response Rates and Deal with Non-response

The expected response rate is 98%. The study design includes assignment of substitute establishments to a Specialist when the originally selected establishment is misclassified, closed, or otherwise unavailable, unable, or unwilling to participate. The inventory of substitute establishments will remain with the FDA CFSAN Biostatistics Branch until needed by a Specialist to replace an ineligible establishment that was included on his or her original list.

4. Test of Procedures on Methods to be Undertaken

Within three days of receiving OMB approval, FDA will initiate a pilot data collection to practice the use of the data collection form and methods and test exportation of the pilot data into a central repository. For the pilot, the 22 Specialists will conduct four data collection inspections each. Following completion of the pilot, there will be a debriefing session during which the Specialists will report any challenges or confusion they had, if any, with marking the data collection form utilizing the study protocol and marking instructions provided. This exercise may result in minor clarifications to the marking instructions to ensure the quality, consistency, and uniformity of the restaurant baseline data collection effort.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Sampling and Statistical Methods/Data Analysis:

Marc Boyer

Mathematical Statistician

FDA/CFSAN/OFDCER/DPHB

5100 Paint Branch Parkway

College Park, MD 20740

(240) 402-1686

File Type	application/msword
Author	ila.mizrachi
Last Modified By	Mizrachi, Ila
File Modified	2013-07-18
File Created	2013-07-18