REVISED Supporting Statement A

Supporting Statement Part A

EHS-Net Listeria Retail Deli Study

EHS-NET Generic Information Collection Request

OMB No. 0920-0792

Supporting Statement - A

Submitted: November 5, 2012

Program Official:

Laura Green Brown, Ph.D.

Behavioral Scientist

Centers for Disease Control and Prevention

National Center for Environmental Health

Emergency and Environmental Health Services

Environmental Health Services Branch

4770 Buford Highway, NE F – 60

Atlanta, GA 30341-3724

770-488-4332 (Phone)

770-488-7310 (Fax)

E-mail: [email protected]

Table of Contents

A. Justification	3
1. Circumstances Making the Collection of Information Necessary	6
2. Purpose and Use of Information Collection	8
3. Use of Improved Information Technology and Burden Reduction	9
4. Efforts to Identify Duplication and Use of Similar Information	9
5. Impact on Small Businesses or Other Small Entities	9
6. Consequences of Collecting the Information Less Frequently	10
7. Special Circumstances Related to the Guidelines of 5 CFR 1320.5	10
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency	10
9. Explanation of Any Payment or Gift to Respondents	10
10. Assurance of Confidentiality Provided to Respondents	11
11. Justification for Sensitive Questions	11
12. Estimates of Annualized Burden Hours and Costs	11
13. Estimates of Other Total Annual Cost Burden to Respondents or Recordkeepers	13
14. Annualized Cost to the Federal Government	13
15. Explanation for Program Changes or Adjustments	13
16. Plans for Tabulation and Publication and Project Time Schedule	13
17. Reason(s) Display of OMB Expiration Date is Inappropriate	32
18. Exceptions of Certification for Paperwork Reduction Act Submissions	32
References

EHS-NET Listeria Retail Deli Study

A. Justification

1. Circumstances Making the Collection of Information Necessary

This data collection is being conducted using the Generic Information Collection mechanism of the EHS-Net - OMB No. 0920-0792. The respondent universe for this data collection aligns with that of the EHS-Net GIC. Data will be collected from six state and local health departments (California, Minnesota, New York, New York City, Rhode Island, and Tennessee).

This data collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. 241) (Attachment 1).

Background

The Environmental Health Specialists Network (EHS-Net) program, developed by the Centers for Disease Control and Prevention (CDC), conducts research designed to 1) identify and understand environmental factors associated with food- and water-borne illness and outbreaks, and 2) identify and understand the strengths and weaknesses of environmental public health regulatory programs responsible for food and water safety. EHS-Net data collections are typically conducted in response to food- and water-borne illness outbreaks, and provide timely data on the causes of outbreaks, including environmental factors associated with outbreaks. These data are essential to environmental public health regulators’ efforts to respond more effectively to outbreaks and prevent future, similar outbreaks.

The Environmental Health Specialists Network (EHS-Net) is a collaborative project of the CDC, the U.S. Food and Drug Administration (FDA), the U.S. Department of Agriculture (USDA), the U.S. Environmental Protection Agency (EPA), and six state and local public health departments (California, Minnesota, New York, New York City, Rhode Island, and Tennessee). In total, EHS-Net provides funding for 6 full-time and 3 part-time personnel in these state and local health departments, and they are responsible for collaborating with CDC on study design, collecting the data, and co-analyzing the data with CDC. The federal partners provide funding and input into study design and data analysis. EHS-Net also has industry partners that support its goals and research by collaborating on study design and data analysis; Attachment 2 contains a list of industry partners. To date, EHS-Net has summarized its research efforts in 19 publications; Attachment 3 contains a bibliography of EHS-Net publications.

Given the need to conduct data collections rapidly, EHS-Net requested a generic OMB clearance for all EHS-Net data collections conducted through 2011. On October 21, 2008, OMB gave generic clearance (no. 0920-0792) to CDC for the EHS-Net Program. Recently (February 6, 2012), OMB renewed the generic package, and it now supports EHS-Net data collections until February 28, 2015. CDC is now requesting OMB approval of a new retail food establishment study under this program. Under the EHS-Net Program generic clearance, OMB has agreed to expedite review of EHS-Net Program’s data collections. Thus, no additional Federal Register notices are necessary, and the expected turn-around time for requested packages submitted under this clearance is six weeks or less. The data collection proposed in this package is the first to be submitted under the current EHS-Net Program generic clearance.

In spite of continued public health and food industry prevention efforts, the number of cases of listeriosis, the clinical manifestation of the foodborne pathogen, Listeria monocytogenes (L. monocytogenes), is not continuing to decline (CDC, 2008), suggesting that L. monocytogenes remains a persistent public health concern for the United States. Among the major foodborne pathogens, L. monocytogenes is second and third only to Toxoplasma gondii and Salmonella, respectively, in the number of deaths that result annually from exposure (Scallan et al., 2011).

Although illnesses caused by L. monocytogenes occur at low rates relative to those caused by other major foodborne pathogens, the severity of the health effects of certain subpopulations are of great concern.

Since 1981, epidemiologic investigations have repeatedly indicated that ingestion of contaminated food is the primary vehicle of transmission of L. monocytogenes. The ubiquitous nature of L. monocytogenes makes it difficult to eradicate the pathogen from the food supply; therefore, many individuals are likely to be exposed to it. Yet, certain subpopulations are more susceptible to acquiring listeriosis and suffering from severe health conditions or even death. Of the listeriosis cases in high-risk subpopulations, two-thirds occur in immunocompromised individuals, including the elderly (≥65 years of age) and individuals with cancer or HIV/AIDS, and the infection primarily results in meningitis. The remaining one-third of the cases of listeriosis occur in pregnant women and carry the risk of feto-placental infection that could result in severe fetal health conditions such as septicemia, pneumonia or meningitis and in about 25% of cases, the occurrence of pre-term delivery or stillbirth (Mylonakis et al., 2002).

According to the FDA, ready-to-eat foods such as deli meats, deli salads, and soft cheeses are among those foods that have a greater potential for L. monocytogenes contamination. Of 23 ready-to-eat food categories linked to L. monocytogenes, deli meats have been found to pose the greatest risk of listeriosis per year and per serving (FDA/USDA/CDC, 2003). In addition, a more recent risk assessment indicated that approximately 80% of the listeriosis cases associated with deli meats resulted from those sliced at retail, despite the fact that retail-sliced deli meats accounted for only 53% of consumption (Endrikat et al., 2010; Pradhan et al., 2010).

In 2011, the University of Florida’s Emerging Pathogens Institute published their top-10 ranking of pathogen-food combinations that caused the greatest burden to U.S. public health; L. monocytogenes in deli meats was ranked third. Listeria in deli meats costs the U.S. $1.1 billion in medical fees, productivity losses due to morbidity along with pain and suffering losses due to premature mortality; it also causes a loss of 4,000 Quality-Adjusted Life Years (QALYs) annually. QALYs is a standardized measure of disease burden that includes pain and suffering as well as the impact on normal activities caused by an illness.

Studies (Friedman et al., 2004; Jones et al., 2004; Kassenborg et al., 2004; Olsen et al., 2000) indicate that retail food establishments are an important source of foodborne illnesses, highlighting a critical need to study these establishments to further understand the potential causes and mechanisms of foodborne illnesses. In fact, a recent study by Cornell University found that approximately 30% of retail food establishments examined had persistent non-food contact surface bacterial contamination, and the levels were sometimes quite high: 10² – 10⁶ cfu/sponge (Wiedmann and Olivier, 2011). Based on these data, cross-contamination (a known foodborne illness risk factor) is likely a contributor to L. monocytogenes-contaminated ready-to-eat foods in retail food establishments. However, little is known about how this cross-contamination occurs. Specifically, we need to identify gaps in retail deli policies and practices and manager and worker food safety knowledge and training that may contribute to cross-contamination of equipment and ready-to-eat foods and explore how retail deli policies and manager and worker knowledge and training are related to practices that may contribute to cross-contamination of equipment and ready-to-eat foods. Data collection will involve interviewing and/or observing retail delis’ environments, managers and workers to better understand food safety practices and policies. The data collected will be used to develop effective strategies for the intervention and prevention of future L. monocytogenes contamination in deli ready-to-eat foods.

This data collection supports the U.S. Department of Health and Human Services’ Healthy People 2020 Goal to, “Improve food safety and reduce foodborne illnesses.” Specifically, these data can be used to develop educational materials and food safety trainings that are targeted towards decreasing the number of cases of listeriosis as L. monocytogenes is identified as one of the key pathogens transmitted in food in the Health People 2020 objectives. This data collection also supports one of CDC’s Winnable Battles, “Reducing foodborne diseases.”

Privacy Impact Assessment

Overview of the Data Collection System. Data will be collected by environmental health specialists in the participating EHS-Net sites. Retail deli managers and workers are the respondents in this study. After obtaining consent, data will be collected using: 1) a manager interview, 2) a manager survey (completed by manager with pen and paper), 3) a worker interview, 4) a structured observation conducted by the environmental health specialists of the retail deli environment and workers’ food safety practices (observations based on a questionnaire), and 5) a notational observation conducted by the environmental health specialists, in which uninterrupted sequences of work-related actions of deli workers are captured.

These multiple data collection methods are necessary to adequately collect the multi-faceted data needed to achieve the study’s objectives. The deli manager interview is necessary for collecting data on retail deli demographics (i.e., the number of customers served), and manager demographics (i.e., training and certification). Interviewing the manager is also necessary for collecting information about deli food safety policies and practices related to cross-contamination (i.e., cleaning equipment and hand washing). The manager survey is necessary for collecting data on manager food safety knowledge. The deli worker interview is necessary for collecting data on worker food safety knowledge and the deli’s usage and cleaning policies regarding equipment, namely food slicers. The structured observation is necessary for collecting data on food safety practices. The notational observation is necessary for collecting data on the sequential actions of deli workers. Attachments 4, 5, 7, and 8 contain the manager interview, manager survey, worker interview, and structured observation and notational observation instruments, respectively.

Data will be reported to CDC by EHS-Net data collectors through a 10-year old web-based information system, the Environmental Health Specialists Network Information System (EHSNIS). User accounts will be issued to the EHS-Net specialist in each state. Account privileges identify the data each specific user is authorized to access and the functions they are authorized to perform. Each EHS-Net specialist is responsible for the administration of the system for his or her own site, and includes user administration and correction and deletion of records capabilities. All data records are owned by the site entering the data. Each site has authority over its records and must grant permission to other sites or agencies who would like to use the data. Each site’s data will be stored for seven years.

Items of Information to be Collected. Below is a description of the types of information to be collected with each method used.

• Deli manager interview

• retail deli demographics (e.g., ownership [independent or chain])

• manager food safety training and certification

• retail deli policies (e.g., hand washing policies)

• Deli manager survey

• worker demographics

• manager food safety knowledge

• Deli worker interview

  • worker demographics

  • worker knowledge and training

  • retail deli practices, particularly those concerning slicers (e.g., cleaning and sanitization)

• Structured observation

• retail deli practices (e.g., employees’ glove use)

• Notational observation

  • sequences of actions of work-related duties performed by deli workers

No individually identifiable information is being collected.

Identification of Website(s) and Website Content Directed at Children Under 13 Years of Age. Study information will be reported through a web-based system. This system is password protected- only people given permission by the CDC can access it. There is no public web site, nor is there any content directed at children less than 13 years of age.

2. Purpose and Use of the Information Collection

The first objective of this study is to collect descriptive data on individual deli workers’ actions involved in performing work-related tasks, such as slicing deli meat for a customer. Through an interagency agreement (Attachment 9), EHS-Net will be collecting these data for the USDA- Food Safety and Inspection Service (FSIS), whom will in turn use these data in their ongoing Listeria risk assessment efforts. The overall goal of the Listeria risk assessment is to better understand how L. monocytogenes is transmitted in the retail environment. The risk assessment can be described as a retail-to-table process model that includes a module focused on cross- contamination of ready-to-eat foods by L. monoctyogenes in the retail deli environment. The data from this study describing deli workers’ actions will serve as the backbone of this cross-contamination module. Specifically, USDA-FSIS will use these data to (1) identify individual events that occur during sequences of work-related tasks that could lead to cross-contamination of ready-to-eat foods (e.g., deli meats) with L. monocytogenes and (2) through computer simulations, and in combination with data obtained from other sources, estimate the risk of cross-contamination of ready-to-eat foods associated with particular sequences of actions. Both identifying potential cross-contamination events and predicting the associated risks require detailed data on deli workers’ uninterrupted sequences of actions regarding sanitation and handling of ready-to-eat foods. These data will be collected through notational observation, a type of observation in which a worker’s action sequences (e.g., worker put on gloves, worker touched door handle and worker picked up meat) related to a particular task/activity (e.g., slicing meat) are captured. USDA-FSIS will also use some of the data from the manager and worker interviews and the structured observation in their risk assessment. Attachment 12 describes how the data from these data collection activities will be used in the risk assessment.

EHS-Net will use the data collected from this study to develop a better understanding of retail deli food safety policies and practices related to cross contamination. Specifically, EHS-Net will use the data to fulfill the second objective of this study: collect descriptive data on retail delis’ food safety policies and practices (focused on cross contamination) and manager and worker knowledge and training. Examples of the type of data that will be collected include: retail deli policies on equipment (cutting boards, knives, etc.) usage and cleaning; retail deli practices on the storage of raw animal products and preparation of deli salads; and manager and worker food safety knowledge and training. Of particular interest will be policies and practices related to food slicers, as recent data have suggested that deli food slicers may be linked to L. monocytogenes outbreaks. The data collected on retail delis’ policies and practices and manager and worker knowledge and training will be used to identify gaps in policies, practices, knowledge and training that may contribute to cross contamination of equipment and ready-to-eat foods. Data collection will involve conducting interviews with deli managers and workers and making structured observations of retail deli environments.

The third objective of the study will be to assess the relationships among manager and worker knowledge and training, and retail deli food safety policies and practices that may contribute to cross contamination of equipment and ready-to-eat foods. We will also examine the relationships among these variables and retail deli establishment, manager and worker characteristics (e.g., number of customers served on the deli’s busiest day, manager experience). These data will be collected primarily through interviews/surveys and structured observations.

Collectively, the data collected for this study will contribute a great deal to the understanding of how L. monocytogenes is transmitted in the retail detail environment and about retail delis’ gaps in food safety policies, practices and manager and worker knowledge and training. This information can be used by USDA and CDC, and state and local health departments to influence changes regarding training and certification that focus on cross contamination and L. monocytogenes prevention and develop intervention recommendations for food safety programs. These data can also be used in the development of educational materials and outreach efforts for food service employees and even consumers.

Again, EHS-Net is comprised of retail food establishments in selected geographical areas in California, Minnesota, New York City, New York State, Rhode Island, and Tennessee. While the number of areas included in EHS-Net is small, they are demographically diverse and provide good geographical coverage of the U.S. (northeast, mid-west, south, and west). When the statistical methods outlined here for ensuring a representative sample in the current study are used, the results of the collection covered by this OMB package can be used to generalize to the population of retail food establishments in the given EHS-Net site(s). Furthermore, the geographic and demographic variability across these sites suggests that CDC may be able to use data collected from these studies to draw conclusions about relationships that are likely relevant to establishments in other parts of the U.S.

Privacy Impact Assessment

Why is the information being collected. The information in this study is being collected because currently there is a lack of data on how cross-contamination occurs among ready-to-eat food, deli equipment, etc. and L. monocytogenes in retail delis. This information is essential in reducing the risk of human exposure to L. monocytogenes. These data can be used by regulators to develop more targeted training, interventions, and preventive strategies regarding L. monocytogenes in foodservice establishments, specifically retail delis.

Intended use of the information being collected. A portion of the data will be used to fill in data gaps in the cross-contamination module of USDA-FSIS’s national risk assessment for L. monocytogenes in retail delis. More specifically, the data will be used to identify practices that could lead to cross-contamination of ready-to-eat foods and through computer simulations estimate the risk of cross-contamination of ready-to-eat foods associated with particular sequences of actions. The data describing deli establishment, manager and worker characteristics, food safety practices and policies, and manager and worker knowledge and training will be used to identify areas that need to be improved in retail delis’ food safety policies and practices in order to reduce and/or prevent cross contamination of equipment and ready-to-eat foods. Collectively, the data can be used by federal regulators and industry to develop guidelines regarding cross contamination and L. monocytogenes in retail delis and other food service establishments.

No IIF is being collected.

3. Use of Improved Information Technology and Burden Reduction

The primary burden to respondents of participation in this study involves their participation in interviews. Some of the interview questions require detailed responses. It is less burdensome for respondents to provide these types of responses verbally than to have to type their responses into an electronic reporting system. Thus, we have chosen not to collect interview data electronically, but rather, collect the data through face-to-face verbal interviews with respondents. This data collection method has been used in previous, successful EHS-Net studies (Green et al., 2006; Kirkland et al., 2009; Lee et al., 2004; Sumner et al., 2011).

After manager respondents complete their interview, they will be asked to complete an 8 item written survey. To complete this survey, respondents will need to choose their response to each question on the survey. As simply choosing 8 responses by checking a box is less burdensome to respondents than teaching them how to enter their data in an electronic reporting system, we will collect the survey data with pen and paper, rather than with an electronic reporting system.

Participation in this data collection is voluntary, and every effort was made to keep the data collection as short as possible and still meet the needs of the data collection.

4. Efforts to Identify Duplication and Use of Similar Information

We have searched relevant scientific bibliographical databases (e.g., PubMed, Ovid, Agricola), attended national meetings (e.g., National Environmental Health Association, International Association of Food Protection), and consulted with other organizations (e.g., FDA, USDA-FSIS) concerning research on this topic. Previously, USDA-FSIS conducted a study in which they described retail deli worker behavior using notational observation (Lubran et al., 2010). However, the study was conducted in only nine establishments in one region of the United States; thus, the data obtained are not generalizable to retail delis located in other regions of the United States. Consequently, data are needed from a more geographically and demographically diverse population of retail delis in order to increase the precision of estimating relationships among retail deli food safety policies and practices, deli manager and worker food safety knowledge and practices, and the risk of cross-contaminating ready-to-eat foods with

L. monocytogenes. Thus, this EHS-Net data collection will not be a duplication of effort.

5. Reducing Impact on Small Businesses or Other Small Entities

Some proportion (an estimated 30%) of the retail establishments contacted for participation in this study will be small businesses. Given that small businesses are likely to have different experiences and practices than larger businesses, it is important that small businesses be included in this data collection. Short forms for small businesses will not be developed. We will, however, strive to hold the number of questions to the minimum needed for the intended use of the data.

6. Consequences of Collecting the Information Less Frequently or Not at All

Participants will be asked to respond to this data collection only one time. Without this data collection, USDA-FSIS will not be able to complete its risk assessment, which will be used to predict how deli workers’ practices and behaviors influence the risk of cross-contamination of ready-to-eat foods with L. monocytogenes. In turn, it will be more difficult for FDA, state and local environmental public health regulators, and the food service industry to provide the much needed preventive or control strategies for L. monocytogenes in retail delis. Thus, it would also be difficult for CDC to fully address CDC’s Winnable Battle of “Decreasing foodborne diseases.” There are no legal obstacles to reduce the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for this data collection. It will fully comply with 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

1. The notice for the renewal Generic Information Collection did not receive any comments.

The 60-Day Federal Register notice was published June 23, 2011 in Volume 76, Pages 36924-36925. The 30 day Federal Register notice was published September 21, 2011 in Volume 76, Page 58517.

2. Partners from USDA-FSIS began working with CDC to develop this data collection in March 2011. Additionally, personnel from our EHS-Net sites began consulting on the data collection in April 2011. Names and contact information of USDA-FSIS and EHS-Net research partners are provided in Attachment 10. Electronic correspondence among CDC, USDA-FSIS, and the EHS-Net sites regarding these data collection consultations are on file and will be made available upon request.

A.9. Explanation of Any Payment or Gift to Respondents

There will be no payments or gifts to respondents.

A.10. Assurance of Confidentiality Provided to Respondents

The proposed project has been reviewed and it has been determined that the Privacy Act does not apply. No assurances of confidentiality will be provided to respondents. While face to face interviews will be conducted, CDC will not directly be engaged in data collection nor will we receive identifying information on any of the retail delis, managers, or workers. Therefore, CDC’s IRB ruled that the current study does not qualify for a review. However, EHS-Net sites will obtain approval from their respective IRBs as appropriate. The manager’s informed consent script can be found at the beginning of the manager interview in Attachment 4, whereas the worker’s informed consent script is combined with the worker recruiting script and can be found in Attachment 6.

Privacy Impact Assessment Information

1. This submission has been reviewed by CDC’s Privacy Officer, who determined that the Privacy Act does not apply.

2. No paper files will be collected. All electronic data will be stored on secure CDC networks. Access to the data will be limited to those with a bona fide need-to-know in order to perform job duties related to the project. User accounts will be issued to the EHS-Net specialist whom will serve as the administrator of the system for his or her own site. Through these password protected accounts, EHS-Net specialists will be granted privileges including entering and accessing data, and correction and deletion of records capabilities. As previously stated, all data records are owned by the site entering the data. Each site possesses ownership of its records and must grant permission to other sites or agencies who would like to use the data.

3. Verbal consent will be obtained from respondents. The consent scripts can be found in Attachments 4 and 6. As a part of the informed consent, respondents will be made aware of their ability to retrieve a summary of the study’s findings by contacting their health department 12 months following data collection.

4. Participation in this data collection is voluntary, and respondents are informed of the voluntary nature of the data collection during the recruiting call (Attachment 11) and when they are read the informed consent (Attachments 4 and 6).

No IIF is being collected.

11. Justification for Sensitive Questions

There are no sensitive questions in this data collection.

12. Estimates of Annualized Burden hours and costs

Six EHS-Net sites will collect data for this study; each site will collect data in 50 retail delis. Thus, there will be 300 retail deli manager respondents. Each manager respondent will be interviewed only once; the interview will last approximately 22 minutes. Each manager respondent will also complete a short survey; the survey will take approximately 10 minutes. The average manager burden per response will be approximately 32 minutes (160 burden hours). We expect a manager response rate of approximately 70 percent; thus, we will need to contact 429 managers via telephone in order to meet our goal of 300 respondents (Attachment 11 contains the manager recruiting script). Each respondent to the script will respond only once, and the average burden per response will be approximately 3 minutes (22 burden hours). In total, the average burden per response for each manager respondent will be 35 minutes (182 burden hours).

We will also ask the deli manager to help recruit a worker respondent who speaks English and has experience cleaning, sanitizing and operating the retail deli’s slicers. Each worker respondent will respond only once. Each worker respondent will be interviewed; the interview will last approximately 10 minutes (50 burden hours). The data collectors will use the script found in Attachment 6 to recruit a deli worker to participate in this study. For the present study, the worker recruiting script and informed consent script were combined. Each respondent to this script will respond only once, and the average burden per response will be approximately 3 minutes (17 burden hours). We expect 90 percent of these worker respondents to agree to participate; thus we will need to ask approximately 334 workers to participate in this study. In total, the average burden per response for each worker respondent will be 13 minutes (67 burden hours). The estimated burdens to the deli managers and workers in this study are comparable to those seen in previous EHS-Net studies.

The data collectors themselves will then conduct an observation of the retail deli environment which will take approximately 30 minutes. They will also observe 1 to 3 deli workers per establishment which will take from 30 to 60 minutes for each deli worker. Neither of these observations will require interactions between the data collector and the deli managers or deli workers. Because there is no burden to either the deli manager or worker, the observation hours are not included in the total annualized response burden estimate of 249 hours (See Table A.12-1).

Table 12-1- Estimated Annualized Burden Hours

Respondents	Form Name*	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)
Retail deli managers	Manager Telephone Recruiting Script	429	1	3/60	22
Retail deli managers	Manager Informed Consent and Interview	300	1	22/60	110
Retail deli managers	Manager Survey	300	1	10/60	50
Retail deli workers	Worker Recruiting Script and Informed Consent	334	1	3/60	17
Retail deli workers	Worker Interview	300	1	10/60	50
Total					249

^* All form names are preceded by, “EHS-Net Listeria Retail Deli Study.”

12-2- Annualized Cost to Respondents

The maximum total annualized cost of this data collection to respondents is estimated to be $3,731 (See Table 12-2). This figure is based on an estimated mean hourly wage of $16.64 for retail food service managers and $10.48 for retail food service workers. These estimated hourly wages were obtained from the U.S. Department of Labor Bureau of Labor Statistics 2009 national occupational employment and wage estimates report (http://stats.bls.gov/oes/current/oes351012.htm; http://stats.bls.gov/oes/current/oes352021.htm).

12.2- Estimated Annualized Burden Costs

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Retail deli managers	182	$16.64	$3,029
Retail deli workers	67	$10.48	$702
Total			$3,731

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There are no other costs to respondents or record keepers.

14. Annualized Cost to the Federal Government

Costs to the government include a portion of the annual cooperative agreement to the EHS-Net sites that will collect the data and the costs of CDC personnel working on the data collection (A.14.1). The EHS-Net sites participating in this study receive equal funding, and we estimate that the sites will use approximately 20% of their cooperative agreement funds to conduct this data collection. We also estimate that one CDC staff member will spend approximately 20% of her time on this data collection and one ORISE fellow will spend approximately 75% of her time on this data collection.

Table 14.1

Expenditure	Cost
Grants to States	$203,500
Salary (1 staff member)	$20,000
Contract costs (1 ORISE Fellow)	$60,000
Total	$283,500

15. Explanation for Program Changes or Adjustments

This is a new data collection associated with an existing generic clearance.

16. Plans for Tabulation and Publication and Project Time Schedule

Table 16.1 provides the data collection activity schedule.

16.1 – Project Time Schedule

Activity	Time Frame
Train EHS-Net sites on data collection	Within 1 month of OMB approval
Data collection	Within 1.5 months of OMB approval
Data entry and quality assurance	Within 1.5-2 months of OMB approval
Data cleaning	Within 7 months of OMB approval
Data analysis	Within 8 months of OMB approval
Manuscript development	Within 10 months of OMB approval

Analysis Plan

The first stage of analysis will involve data cleaning, editing, and recording. The data will be checked for accuracy and examined for inconsistencies. Where appropriate, a frequency response will be done for each variable to examine item non-response and extraneous responses. Variables with high item non-response or of poor quality will be discarded. The second stage of the analysis will differ for USDA-FSIS and EHS-Net. The two analysis plans are described below.

USDA-FSIS Analysis Plan

Background

L. monocytogenes is an important food safety concern (see Background section for more information). The FSIS has taken several steps over the past decade to further prevent listeriosis from ready-to-eat meat and poultry products. Efforts include: 1) identifying which RTE foods pose the greatest risk of listeriosis (i.e., deli meats) through the conduct of a well-known quantitative risk-ranking of RTE foods in 2003; and 2) identifying which mitigations during the processing of RTE foods are most effective in preventing listeriosis from RTE foods and implementing a new rule (9 CFR 430) based on these findings. In addition, FSIS developed a risk-based sampling program where establishments that had less effective Lm controls and produced higher risk RTE meat and poultry products were inspected more frequently based on public health risk (http://www.fsis.usda.gov/PDF/RBVS_Risk_Assess_Jun07.pdf). The result of using these actives was an industry adoption of better Lm controls during processing and an observed dramatic decrease of the percentage of FSIS-regulated RTE meat and poultry products testing positive for Lm (Figure 1). While declines in listeriosis cases attributed to RTE meat and poultry products were initially observed, over time declines in listeriosis cases hit a plateau. This indicated that there were other factors contributing to listeriosis from RTE meat and poultry products.

Figure 1. FSIS L. monocytogenes Verification Testing of RTE Meat and Poultry Products Versus the Incidence of Listeriosis Cases Attributed to RTE Meat and Poultry Products

Recent research strongly suggests that these other factors are associated with L. monocytogenes cross contamination of the deli environment. This research is summarized below.

• A recent study by Cornell University (Wiedmann and Olivier (2011)) found that approximately 30% of stores examined had persistent non-food contact surface (NFCS) contamination, and the levels were sometimes quite high: 10² – 10⁶ cfu/sponge (Figure 2).

Figure 2. Retail Deli store indicating long term environmental Lm contamination. Source: Wiedmann & Oliver 2011. Understanding L. monocytogenes transmission in retail delis to inform risk assessments and risk management. Cornell University & Purdue University.

• During a recent study (Luchansky et al.) of L. monocytogenes in RTE foods at retail (the 2011-2012 Market Basket Survey) by the Agricultural Research Service, in partnership with UC Davis and Drexel University, unsanitary practices were observed during deli operations. A few are show in Figure 3 below. These include poorly cleaned slicers, utensils stored directly with deli salads, a cleaning cloth used to hold a chub, and the absence of gloves.

Figure 3. Unsanitary conditions observed during deli operations. Source: Luchanksy ARS. Upper left: unsanitary slicer; upper right: utensil in direct contact with deli salad; lower left: chub wrapped in unsanitary towel; lower right: slicing directly on hand.

• Industry data (Ecosure, 2007) showed that approximately 33% of deli cases held RTE foods at improper cooling temperatures that support the growth of Lm (above 5 degrees Celsius; recommended Food Code).

• National surveys by National Food Processors Association (Gombas, 2003) and by National Alliance for Food Safety and Security (Draughon, 2006) have found that retail sliced product is approximately 7 times more likely to be contaminated than product sliced and packaged at a food processing (Figure 4; Figure 5).

Figure 4. Lm Prevalence of retail-sliced versus prepackaged deli meats.

Figure 5. Concentration distributions for retail-sliced and prepackaged products. Source: Endrikat et al. 2010 J. Food Protection 73(4):612-619.

• Risk assessments using these national survey data indicated that approximately 80% of the listeriosis cases associated with deli meats result from deli meats sliced at retail, despite the fact that retail-sliced deli meats accounted for only 53% of consumption (Endrikat et al. 2010; Pradhan et al. 2010). While FSIS’ testing programs showed a decline in the percentage of RTE meat and poultry products positive for L. monocytogenes, a corresponding decline in listeriosis was not observed (Figure 1). Retail surveys (Gombas et. al., 2003; Draughon, 2006) of L. monocytogenes in RTE foods found a higher incidence of L. monocytogenes –positive samples for those sliced and packaged in-store compared to those packed by the manufacturer. Using data from these surveys, FSIS published another risk assessment (Endrikat et al., 2010) that indicated that, of the listeriosis cases attributed to deli meat, most (approximately 83%) were associated with deli meats sliced at retail. Cornell University conducted a separate comparative risk assessment (Pradhan et al., 2010) the provided similar findings – most listeriosis cases were associated with deli meats sliced at retail.

Clearly, these data suggest that cross-contamination is a likely contributor to L. monocytogenes contamination at retail, but little is known about how this occurs. Retail practices may result in either cross-contamination from one product to another or through contamination from the retail environment, or both. To better understand how Lm is transmitted in the retail environment and to evaluate which changes in retail practices are most effective in protecting public health, FSIS and FDA developed an interagency risk assessment to guide food safety practices at retail. These studies indicate that retail practices (incoming L. monocytogenes on RTE foods, improper temperature controls, sanitary practices, cross-contamination, etc.) within the retail grocery environment contribute to listeriosis. Industry and consumer group provided risk management questions to be evaluated through the conduct of “what if” scenarios as part of a new risk assessment. Findings from the risk assessment would be used by industry and others to further improve retail food safety.

Specific risk management questions to be answered by the interagency cross contamination model:

1) What would be the impact on the prevalence of products sold in deli department and on the corresponding mean risk of invasive listeriosis per serving of a more frequent or more efficient cleaning of the slicers, other food contact surfaces and/or, the non-food contact surfaces, the reference being the Food Code 2009?

2) What is the impact of the use of gloves in the retail environment?

3) What if scale touch pads, refrigerator and deli case handles and other frequently touched non-food contact surfaces were considered food contact surfaces (i.e., were required to be cleaned and sanitized every four hours and, as a result could then be touched by gloved hands without requiring a decontamination action afterwards)?

4) What if practices were in place so that no cross-contamination occurred in delis (i.e. no additional L. monocytogenes added to incoming products)?

5) What if display cases were not touched with gloved or bare hands (i.e., used tissues or had automatic door open/shut; reduce cross-contamination)?

6) What would be the impact (under the modeling hypotheses) if the level/frequency of L. monocytogenes is reduced in products coming into the retail deli?

7) What would be the impact of “pre-slicing” all products vs. “slicing to order” (hypothesis: less cross-contamination occur in morning prior to other cross-contamination events).

8) What would be the impact of using separate slicers and/or separate counters for products that permit growth and for products that do not?

9) What would be the impact of lower environmental contaminations?

10) What if food workers do not slice products on their gloves?

11) What if stores could be re-designed to refrigerate deli (i.e., no temperature abuse)? What is the impact of bacterial growth in stores?

12) What would be the impact of a full compliance to the ≤40°F storage recommendation?

13) What would be the impact of providing a specific (short) “sell by” and “use by” date for the products sliced to order in deli department?

14) What would be the impact (under the modeling hypotheses) if all (or no) products (e.g. RTE meat and poultry products, RTE salad) coming into the deli were formulated with growth inhibitors?

Support for the conduct of this risk assessment comes in the form of partnership academia (Univ. of MD, Cornell University, Purdue University, and VA Tech), stakeholders (Food Marketing Institute, American Meat Institute, Consumer Science in the Public Interest, Grocery Manufacturers Association, and others), federal partners (FSIS, FDA, CDC and ARS), and working with the states through CDC/EHS-Net.

EHS-Net Data Collection

USDA-FSIS will use the data collected from this study in three ways. First, the data will be used by USDA-FSIS to inform their risk assessment model. The objectives of the risk assessment mode are to characterize: 1) the exposure to L. monocytogenes from consuming ready-to-eat foods prepared in retail delis; 2) the key processes that increase ready-to-eat food contamination in retail delis, and 3) how much the relative risk per serving is reduced according to specific retail deli risk management options.

The notational observation data collected for this study describing sequential activities of deli workers performing work-related tasks will be used as direct inputs into the model and are considered the backbone of USDA-FSIS’s cross-contamination risk assessment module of the model. These model inputs will be used to characterize the potential probability of occurrence of particular cross contamination events. USDA-FSIS will also link some of the data from the manager and worker interviews and structured observation to direct model input values for the risk assessment cross-contamination module. For example, data from the manager interview regarding the ease of breaking down and cleaning a food slicer will be used to characterize thel probability of the formation of a microbial niche.

It is important to note that the data collected in this EHS-Net study will be placed into the model in combination with data from other studies (detailed above). The combination of these data will allow us to estimate the probability of L. monocytogenes cross contamination. For example, data on the frequency with which food workers engage in actions that have the potential to lead to direct cross contamination (e.g., if a data collector observes a food worker cutting lettuce with a knife that was also used to cut raw meat and the knife was not washed in between the two uses, this is evidence of potential direct cross contamination) will be combined with data on the frequency with which L. monocytogenes is found in the retail deli environment to estimate the probability of occurrence of L. monocytogenes cross contamination.

Second, some of the manager and worker interview and structured observation data will be used to evaluate the risk assessment conceptual model. For example, some of the structured observation data will be used to determine if additional pathways or sources of cross-contamination need to be included in the risk assessment.

Third, some of the data (e.g., disposal of deli products and deli salads) will help link the results of the cross-contamination model to an ongoing USDA-FSIS risk ranking project that is designed to identify which ready-to-eat foods pose the greatest risk of listeriosis following ingestion. The following section provides more detail on the risk assessment model.

Model details

To evaluate the impact of retail operating practices and design appropriate risk mitigation strategies, a discrete event mode for retail cross contamination has been developed. An overview of the conceptual model is presented in Figure 6 and the possible cross-contamination activities and sites are shown in Figure 7. The model develops a time-series of concentrations at multiple sites in the deli, and accounts for transfers as two sites (e.g. gloves and scale, or chub and slicer) come in contact. The resulting concentrations in individual sales are then allowed to grow during consumer handling. Doses at consumption are calculated and the risks are evaluated based on the FAO/WHO dose-response model for listeriosis.

Figure 6. Conceptual cross-contamination model.

Figure 7. Detailed conceptual model of cross contamination events within the model.

The current cross-contamination requires that direct contact occur between two sites for bacterial transfer to occur. (The only exception is contamination from a niche or the environment (see Figure 6) as described above.) The quantitative number of bacteria transfer during a contact is controlled by transfer coefficients for each type of contact. These are input data to the model and based on published literature. Example transfer coefficients used within the model are given in Table 16.2.A. Sites that do not come in contact do not have transfer coefficients.

Table 16.2.A. Mean (standard deviation) of the log10 of the transfer coefficients.

	To Meat	Cheese	Salad	Floor	Sink	Handle	Case	Utensil	Utensil Handle	Slicer	Scale	FCS	NFCS	Glove	Hand
From Meat					-0.28 (0.2)			-0.28 (0.2)		-0.28 (0.2)		-0.28 (0.2)	-0.28 (0.2)	-1.69 (0.81)	-1.69 (0.81)
Cheese					-0.28 (0.2)			-0.28 (0.2)		-0.28 (0.2)		-0.28 (0.2)	-0.28 (0.2)	-1.69 (0.81)	-4.96 (0.37)
Salad								-0.28 (0.2)		-0.28 (0.2)		-0.28 (0.2)	-0.28 (0.2)	-1.69 (0.81)	-4.96 (0.37)
Floor														-1.84 (0.87)	-1.84 (0.87)
Sink	-0.28 (0.2)	-0.28 (0.2)												-1.84 (0.87)	-1.84 (0.87)
Handle														-1.84 (0.87)	-1.84 (0.87)
Case														-1.84 (0.87)	-1.84 (0.87)
Utensil	-0.28 (0.2)	-0.28 (0.2)	-0.28 (0.2)											-1.84 (0.87)	-1.84 (0.87)
Utensil Handle														-1.84 (0.87)	-1.84 (0.87)
Slicer	-0.28 (0.2)	-0.28 (0.2)	-0.28 (0.2)											-1.84 (0.87)	-1.84 (0.87)
Scale														-1.84 (0.87)	-1.84 (0.87)
FCS	-0.28 (0.2)	-0.28 (0.2)	-0.28 (0.2)											-1.84 (0.87)	-1.84 (0.87)
NFCS	-0.28 (0.2)	-0.28 (0.2)	-0.28 (0.2)											-1.84 (0.87)	-1.84 (0.87)
Glove	-4.96 (0.37)	-4.96 (0.37)	-4.96 (0.37)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-3.43 (0.79)	-3.43 (0.79)
Hand	-1.69 (0.81)	-4.96 (0.37)	-4.96 (0.37)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-1.84 (0.87)	-3.43 (0.79)	-3.43 (0.79)

The manner in which the contacts are used within the model is illustrated by the stochastic decision tree shown in Figure 8. The Yes/No decision points are determined stochastically from the observed frequencies based on the small observational study summarized in Table 16.2.A. The proposed research will observe if there is any direct contact between deli workers hands and milk crates or deli workers and raw products. This would be a sine qua non requirement for cross contamination.

Figure 8. Illustration of stochastic decision tree within the discrete event model. Note that this is only an illustrative example of a part of the time sequence of serving a customer.

Table 16.2. B. Sequence of events when serving deli meat or deli cheese.

Event	Number of times observed / total observations
Wipe the slicer	7/83
Wash Hands and Change Gloves OR Do not Wash Hands and Change Gloves OR Do not Wash Hands and Do not Change Gloves	33/83 22/83 28/83
Touch a NFCS	4/83
Open the Case	68/83
Close the Case	if had opened it
Touch the Refrigerator Handle	2/66
Open a new chub	if the mass of the chub < mass to be sold
No contact Contact new Chub - Sink Contact new Chub – FCS Contact new Chub - Slicer	If open a new Chub 6/17 4/17 1/17 1/17
Pick up a Chub	83/83
Change Gloves	1/83
Touch the Knob of the Slicer	18/83
Slice on Gloves OR Slice on Deli Tissue	82/83 1/83
Touch the Scale	83/83
Contact Chub - FCS	1/83
Open Case	If had opened/closed it previously
Put Chub in Case	83/83
Close Case	If had opened it previously
Wipe the Slicer	if had not done it at the beginning 15/68

Table 16.2.C provides a detailed description of how USDA-FSIS plans to use the data collected for this study and the data collection variables that will be employed for each of these uses. This table is followed by descriptions of why we are collecting specific variables.

Table 16.2.C-USDA-FSIS data use and data collection variables that will be employed for these uses

General purpose	Specific purpose	Data collection variables
Inputs for risk assessment model	Worker action sequences Identify key processes that have the potential to increase contamination of ready-to-eat food in retail delis Establishment demographics Probability of each type of store Estimate the time needed to serve each customer Number of workers operating the deli Retail deli policies The probability that gloves are worn for each customer sale Frequency and likely effectiveness of sanitation  Retail deli practices Estimate the number of days that a chub can be held in the store before being sold or discarded Identify types of food sliced on each slicer Timing and potential effectiveness of cleaning Potential likelihood of a microbial niche forming Potential timing and effectiveness of cleaning Estimated probability of a niche forming Probability that the product is sold in the deli area of the store Number of sinks and task performed in each Refrigerator and deli case temperatures Glove use Sanitation effectiveness Probability of contamination from non-food contact surfaces         Adjust the probability of a niche associated with a slicer	Worker action sequences data from the notational observation MI-1: Independent or chain MI-2-5: Number of customers served MI-6-8: Number of workers/shifts MI-23-27: Hand hygiene policies MI-28-33: Wiping cloths/sanitizing solution MI-40-46; MI-48-57: Cleaning policies MI-13-19: Chub practices WI-6: Type of foods sliced on slicers WI-6a-6c: Frequency and method of cleaning slicers WI-6d: How easy is it to clean slicer WI-6e: Frequency of slicer cleaning WI-7-10: Frequency and method of cleaning slicers WI-11b: Where are deli salads sold SO-3: Identify tasks performed in sinks SO-4: Refrigerator and deli case temperatures SO-7: Estimate % of workers handling ready-to-eat food with bare hands SO-11-12c: Sanitization effectiveness SO-13-14: Are milk crates and hand trucks in deli are used in other depts. SO-15: Make, model, year of each slicer

MI=Manager Interview; MS=Manager Survey; WI=Worker Interview; SO=Structured Observation

General purpose	Specific purpose	Data collection variables
Evaluate conceptual model	Determine if these are potential sources of contamination          Determine if these are potential sources of contamination Raw animal and deli product storage in relation to one another Direct cross contamination of raw animal products with ready-to-eat foods Determine if practices in areas adjacent to the deli need to be added to the risk assessment model Wet wiping cloths use Determine if practices in areas adjacent to the deli  need to be added to the risk assessment model	MI-34-39a: Cleaning policies of floors, drains, walls MI-59-63: Are milk crates and hand trucks in deli are used in other depts. SO-5-5b: Raw animal and deli product storage in relation to one another SO-8:  Evidence of potential direct cross-contamination         SO-9-9a: Are raw foods cooked in another area SO-10-10a: Use of wet wiping cloths SO-16: Barriers separating deli Sketch of deli layout
Inform ready-to-eat foods risk ranking project	Inform starting Listeria concentrations for deli salads Deli salad preparation	MI-20-22a: Deli product use WI-11-11b: Deli salad prep and sale

MI=Manager Interview; MS=Manager Survey; WI=Worker Interview; SO=Structured Observation

Use of milk crates and hand trucks: Milk crates and hand trucks are not classified as food contact surfaces under the Food Code, yet they could be potential sources of contamination- they could be bringing in materials from other departments. Additionally, published literature reported frequent L. monocytogenes on milk crates. To evaluate if there is any non-zero probability of contamination from these non-food contact surfaces, we need to determine if there is direct contact among them and deli workers.

Cleaning policies for floors, drains, and walls: These items have been found to harbor L. monocytogenes. Thus, it is important to assess whether and how often cleaning policies for these items meet FDA Food Code regulations.

Slicers (e.g., age of, cleaning policies, etc.): Discussions with deli equipment manufacturers, particularly those of slicers, indicate that recent versions of the equipment are easier to disassemble and clean than older versions. The inability to properly clean equipment leads to possible niches or harborage sites, which can contaminate a food contact surface. The portions of the data collection focused on slicers will be used to characterize the potential distribution of ages of equipment in the retail deli. If most delis have relatively modern equipment, the likelihood of a niche should be lower than if they retail older equipment. This analysis will be conducted by type of deli (chain versus independent) to determine if there are significant differences between store types. Contamination from niches is currently evaluated by a sensitivity analysis based on the stochastic frequency of site contamination from niches and the stochastic number of bacteria transferred from the niche to the site.

Cleaning policies for food contact surfaces (e.g., cutting boards, food slicers): We will assess cleaning policies on food contact surfaces to determine if there is any non-zero probability of contamination from these food contact surfaces. If the cleaning policies do not meet FDA Food Code recommendations, there is non-zero probability of contamination from these surfaces.

Evidence of potential direct cross-contamination: Food workers can and do engage in actions that could lead to direct cross contamination. For example, if a food worker cut lettuce with a knife that was also used to cut raw meat and the knife was not washed in between the two uses, this could lead to direct cross contamination- the potential for direct cross contamination. These data will be used to estimate the probability of potential direct cross contamination.

Areas adjacent to the deli: Currently the model concentrates solely on what happens inside the deli. We need to determine if areas adjacent to the deli potentially contribute to the risk of cross contamination and thus, need to be included in the model. This is conceptual model question, not a quantitative simulation question. For example, observations have indicated that raw products (e.g. uncooked chickens for rotisserie cooking) are processed adjacent to some delis. Another example is the storage and handling of milk crate behind the deli area. We do not know if these are common occurrences, if they vary by store type, and if the same retail workers are present in both areas. The proposed study will help answer these questions. If the practices are common and if the same workers are moving back and forth between these areas then, based on stakeholder judgments, the existing conceptual model may need to be expanded to include interactions with adjacent areas.

Wet wiping cloth use: If wet wiping cloth use does not meet FDA Food Code recommendations, it could lead to potential cross contamination.

USDA and CDC note the limitations of the EHS-Net sample to provide a population representative distribution of such characteristics as the age of deli equipment in the U.S. or a national estimate for the probability that workers do or don’t engage in specific behaviors. As such, USDA and CDC commit to transparently characterizing such limitations in their reports and presenting sensitivity analyses for variables that drive its models.

EHS-Net Analysis Plan

EHS-Net will use the data from the manager and worker interviews and the structured observation for the second and third objectives of the study: 1) to identify gaps in retail deli manager and worker food safety knowledge and training and retail deli policies and practices that may contribute to cross-contamination of equipment and ready-to-eat foods; and 2) assess the relationships among retail deli manager and worker food safety knowledge and training; retail deli food safety policies; retail deli establishment, manager and worker characteristics; and retail deli food safety practices.

To identify gaps in retail deli manager and worker food safety knowledge and training and retail deli policies and practices that may contribute to cross-contamination of equipment and ready-to-eat foods, EHS-Net will conduct descriptive analyses on data concerning these topics. Analyses will include frequencies (for categorical variables), and central tendency measures, such as means and medians (for continuous variables). Food safety guidelines found in the FDA Food Code will guide these analyses. For example, the Food Code specifies that to prevent cross-contamination, raw animal products should not be stored or placed above ready-to-eat foods; thus, we will examine the frequency with which this practice is observed in retail delis.

Table 16.2.B describes the descriptive research questions designed to address the study’s descriptive objective and the data collection variables designed to answer these research questions. Note that we will likely create composite variables based on the individual variables listed in the table. Table 16.2.C is a table shell that illustrates how we might analyze and present the descriptive data on retail deli policies collected from this study.

MI=Manager Interview; MS=Manager Survey; WI=Worker Interview; SO=Structured Observation

Table 16.2.B- Descriptive research questions and the data collection variables designed to answer those questions
Research question	Data collection variables
What are the gaps in retail deli manager and worker food safety knowledge and training that may contribute to cross-contamination of equipment and ready-to-eat foods?	MI-66-66b: Food safety training MS-1-MS-8: Food safety knowledge WI-12-12a: Food safety training WI-13a-13e: Food safety knowledge
What are the gaps in retail deli policies that may contribute to cross-contamination of equipment and ready-to-eat foods?	MI-23-27: Hand hygiene policies MI-28-33: Sanitizing solution/wet wiping cloth policies MI-34-57: Cleaning policies MI-58: Actions taken when workers do not follow the policies
What are the gaps in retail deli practices that may contribute to cross-contamination of equipment and ready-to-eat foods?	MI-13-19: Chub use MI-20-22a: Combination products MI-59: Potential cross contamination from push carts/hand trucks MI-60-63: Potential cross contamination- raw foods to ready-to-eat foods MI-64-64a: Are refrigerator temperatures recorded; how often WI-6-10: Slicer use and cleaning SO-6: Temperature recording SO-5-5b; 7-9a: Potential cross contamination- raw foods to ready-to-eat foods SO-10-11: Potential cross contamination- wiping cloths/sanitizing solution SO-12-15e: Potential cross contamination-equipment

Table 16.2.C- Table Shell: Descriptive data on retail deli food safety policies

	n	%
Hand wash policy (MI-23-24)
Good/Safe	xx	xx
Bad/Unsafe	xx	xx
Glove policy (MI-25-27)
Good/Safe	xx	xx
Bad/Unsafe	xx	xx
Policy for wiping cloths/sanitizing solution (MI-28-33)
Good/Safe	xx	xx
Bad/Unsafe	xx	xx
Policy for cleaning cutting boards (MI-43-44)
Good/Safe	xx	xx
Bad/Unsafe	xx	xx
Policy for cleaning food prep slicers (MI-45-46)
Good/Safe	xx	xx
Bad/Unsafe	xx	xx
Policy for cleaning food tables (MI-48-49)
Good/Safe	xx	xx
Bad/Unsafe	xx	xx

MI=Manager Interview

To assess the relationships among retail deli manager and worker food safety knowledge and training, retail deli food safety policies, retail deli establishment, manager and worker characteristics and retail deli food safety practices, EHS-Net will conduct tests for association and logistic regression models. Analysis will involve bivariate tests for association between each individual explanatory (independent) variable and the outcome (or dependent) variable. Odds ratios will be calculated to assess the strength and direction of the bivariate relationships. For those bivariate associations found to be statistically significant at p<.30, the explanatory variables will be used as candidate “predictors” to examine their multivariate relationships with the outcome variable. Multivariable logistic regression will be used to model for the effects that these explanatory variables have in explaining the variations observed in the outcome variable.

Explanatory variables in these analyses include those measuring retail deli manager and worker food safety knowledge and training, retail deli food safety policies, and retail deli establishment, manager and worker characteristics. Outcome variables include those measuring selected retail deli practices. Analyses will focus on practices data collected through observation. Table 16.2.D describes the explanatory research questions designed to address this objective of the study and the data collection variables designed to answer these research questions. Note that we will likely create composite explanatory and outcome variables based on the individual variables listed in the table. Table 16.2.E is a table shell that illustrates how we might analyze and present the data examining the relationships between retail deli policies and the practice variable of whether any evidence of potential direct cross contamination from raw animal products to ready-to-eat foods was observed in the retail delis.

Table 16.2.D- Explanatory research questions and data collection variables designed to answer those questions

Research question	Explanatory variables	Outcome variables
How are retail deli manager and worker food safety knowledge and training related to practices that may contribute to cross contamination of equipment and ready-to-eat foods?	MI-66-66b: Manager food safety training MS-1-MS-8: Manager food safety knowledge WI-12-12a Worker food safety training WI-13a-13e Worker food safety knowledge	SO-5-5b; 7-9a: Potential cross contamination- raw foods to ready-to-eat foods SO-10-11: Potential cross contamination- wiping cloths/sanitizing solution SO-12-15a: Potential cross contamination- equipment
How are retail deli policies related to practices that may contribute to cross contamination of equipment and ready-to-eat foods?	MI-23-27: Hand hygiene policies MI-28-33: Sanitizing solution/wet wiping cloth policies MI-34-57: Cleaning policies MI-58: Actions taken when workers do not follow the policies	SO-5-5b; 7-9a: Potential cross contamination- raw foods to ready-to-eat foods SO-10-11: Potential cross contamination- wiping cloths/sanitizing solution SO-12-15a: Potential cross contamination- equipment
How are retail deli establishment, manager, and worker characteristics related to practices that may contribute to cross contamination of equipment and ready-to-eat foods?	MI-1: Estab.- Ownership MI-2-5: Estab.- No. of customers served MI-6-8: Estab.- No. of employees/shifts MI-9: Estab.- Age of building MI-65: Estab.- Number of managers MI-67-68: Estab.- Mgmt training reqs MI-10-11: Mgmt. experience MI-12: Mgmt. duties WI-1: Wkr. job responsibilities WI-2-2a: Wkr. works in other depts WI-3-3a: Wkr. cleaning/sanitizing duties WI-4-5: Wkr. experience	SO-5-5b; 7-9a: Potential cross contamination- raw foods to ready-to-eat foods SO-10-11: Potential cross contamination- wiping cloths/sanitizing solution SO-12-15a: Potential cross contamination- equipment

MI=Manager Interview; MS=Manager Survey; WI=Worker Interview; SO=Structured Observation

Table 16.2.E- Table Shell: Retail deli manager and worker knowledge and training explanatory variables associated with the outcome variable of whether evidence of potential direct cross contamination from raw animal products to ready-to-eat foods was observed in the retail delis, bivariate analyses

Explanatory variables	Evidence of potential direct cross contamination from raw animal products to ready-to-eat foods was observed (SO-8)
	OR (95% CI)	P
Manager knowledge (MS-1-8)
Bad/Unsafe	x.xx (ref)	.xxx
Good/Safe	x.xx
Manager training (MI-66-66b)
Bad/Unsafe	x.xx (ref)	.xxx
Good/Safe	x.xx
Worker knowledge (WI-13a-13e)
Bad/Unsafe	x.xx (ref)	.xxx
Good/Safe	x.xx
Worker training (WI-12)
Bad/Unsafe	x.xx (ref)	.xxx
Good/Safe	x.xx

OR=Odds Ratio, P=probability level, MI=Manager Interview; MS=Manager Survey; WI=Worker Interview, SO=Structured Observation

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

We are not requesting an exemption to the display of the expiration date.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

There will be no exceptions to certification for Paperwork Reduction Act.

References for Part A

Batz, M., S. Hoffman, and J. Morris. 2011. Ranking the Risks: The 10 Pathogen-Food

Combinations with the Greatest Burden on Public Health. Available at:

http://www.epi.ufl.edu/sites/www.epi.ufl.edu/files/RankingTheRisksREPORT.pdf.

Acceessed 20 May 2011.

Centers for Disease Control and Prevention (CDC). 2008. Preliminary FoodNet data on the

incidence of infection with pathogens transmitted commonly through food--10 states, 2007, 2008. Morbidity and Mortality Weekly Report. 57:366-370.

Endrikat, S., D. Gallagher, R. Pouillot, H. Hicks-Quesenberry, D. LaBarre, C. M. Schroder, and J. Kause. 2010. A comparative risk assessment for Listeria monocytogenes in prepackaged versus retail sliced deli meat. Journal of Food Protection. 73:612-619.

U.S. Food and Drug Administration, Center for Food Safety and Applied Nutrition; U.S. Department of Agriculture, Food Safety and Inspection Service; and Centers for Disease Control and Prevention. 2003. Quantitative assessment of relative risk to public health from foodborne Listeria monocytogenes among selected categories of ready-to-eat foods. Available at: http://www.fda.gov/Food/ScienceResearch/ResearchAreas/RiskAssessmentSafetyAssessment/ucm183966.htm. Accessed 26 April 2011.

U.S. Food and Drug Administration. 2009. FDA Food Code. Available at: http://www.fda.gov/Food/FoodSafety/RetailFoodProtection/FoodCode/FoodCode2009. Accessed 15 September 2011.

Friedman, C., R. Hoekstra, M. Samuel, R. Marcus, J. Bender, B. Shiferaw, S. Reddy, S. Ahuja, D. Helfrick, F. Hardnett, M. Carter, B. Anderson, and R. Tauxe, for the Emerging Infections Program FoodNet Working Group. 2004. Risk factors for sporadic Campylobacter infection in the United States: A case-control study in FoodNet sites. Clinical Infectious Diseases. 38:S285–296.

Gombas, D.E., Y. Chen, R. S. Clavero, and V. N. Scott. 2003. Survey of Listeria monocytogenes in ready-to-eat foods. Journal of Food Protection. 66:559-569.

Green, L.R., C.A. Selman, V. Radke, D. Ripley, J.C. Mack, D.W. Reimann, T. Stigger, M

Mostinger, and L. Bushnell. 2006. Food worker hand washing practices: an EHS-Net

observation study. Journal of Food Protection. 69:2417-23.

Jones, T., B. Imhoff, M. Samuel, P. Mshar, K. McCombs, M. Hawkins, V. Deneen, M. Cambridge, & S. Olsen for the Emerging Infections Program FoodNet Working Group. 2004. Limitations to successful investigation and reporting of foodborne outbreaks: An analysis of foodborne disease outbreaks in FoodNet catchment areas, 1998-99. Clinical Infectious Diseases. 38:S297-S302.

Kassenborg, H., K. Smith, D. Vugia, T. Rabatsky-Ehr, M. Bates, M. Carter, N. Dumas, M. Cassidy, N. Marano, R. Tauxe, and F. Angulo, for the Emerging Infections Program FoodNet Working Group. 2004. Fluoroquinolone-resistant Campylobacter infections: Eating poultry outside of the home and foreign travel are risk factors. Clinical Infectious Diseases. 38:S279–S284.

Kirkland, E., L.R. Green, C. Stone, D. Reimann, D. Nicholas, R. Mason, R. Frick, S. Coleman,

L. Bushnell, H. Blade, V. Radke, C. Selman, and The EHS-Net Working Group. 2009.

Tomato handling practices in restaurants. Journal of Food Protection. 72:1692-8.

Lee, R., M. Beatty, A. Bogard, M. Esko, F. Angulo, C. Selman, and The EHS-Net Working

Group. 2004. Prevalance of high-risk egg-preparation practices in restaurants that prepare

breakfast egg entrees: An EHS-Net study. Journal of Food Protection. 67:1444-50.

Lubran, M. B., R. Pouillot, S. Bohm, E. M. Calvey, J. Meng, and S. Dennis. 2010. Observational study of food safety practices in retail deli departments. Journal of Food Protection. 73:1849-1857.

Mylonakis, E., M. Paliou, E. L. Hohmann, S. B. Calderwood, and E. J. Wing. 2002. Listeriosis during pregnancy: a case series and review of 222 cases, 2002. Medicine (Baltimore). 81:260-269.

Olsen, S., L. MacKinon, J. Goulding, N. Bean, and L. Slutsker. 2000. Surveillance for foodborne disease outbreaks—United States, 1993-1997. Morbidity and Mortality Weekly Report. 49:1–51.

Pradhan, A., R. Ivanek, Y. Gröhn, R. Bukowski, I. Geornaras, J. Sofos, and M. Wiedmann. 2010. Quantitative risk assessment of listeriosis-associated deaths due to Listeria monocytogenes contamination of deli meats originating from manufacture and retail. Journal of Food Protection. 73:620-630.

Scallan, E., R. M. Hoekstra, F. J. Angulo, R. V. Tauxe, M. A. Widdowson, S. L. Roy, J. L. Jones, and P. M. Griffin. 2011. Foodborne Illness Acquired in the United States—Major Pathogens. Emerging and Infectious Diseases Journal. 1:7-15.

Sumner, S., L.G. Brown, R. Frick, C. Stone, L.R. Carpenter, L. Bushnell, D. Nicholas, J. Mack,

H. Blade, M. Torbin-D’Angelo, K. Everstine, and The EHS-Net Working Group. 2011.

Factors associated with food workers working while experiencing vomiting or diarrhea.

Journal of Food Protection. 74:215-20.

Attachments

Attachment 1- EHS-Net-Authorizing Legislation

Attachment 2- EHS-Net Listeria Retail Deli Study-Industry Partners

Attachment 3- EHS-Net-Publications

Attachment 4- EHS-Net Listeria Retail Deli Study-Manager Informed Consent and Interview

Attachment 5- EHS-Net Listeria Retail Deli Study Manager Survey

Attachment 6- EHS-Net Listeria Retail Deli Study-Worker Recruiting Script and Informed Consent

Attachment 7- EHS-Net Listeria Retail Deli Study-Worker Interview

Attachment 8- EHS-Net Listeria Retail Deli Study-Structured and Notational Observations

Attachment 9- EHS-Net Listeria Retail Deli Study- Interagency Agreement

Attachment 10- EHS-Net Listeria Retail Deli Study-Research Partners

Attachment 11- EHS-Net Listeria Retail Deli Study-Manager Telephone Recruiting Script

Attachment 12-EHS-Net Listeria Retail Deli Study- Data Collection Manual

37