Ssb

Supporting Statement Part B

EHS-Net Outbreak Prevention & Response Practices Study

EHS-Net Generic Information Collection Request

OMB No. 0920-0792

OMB Exp. Date: 01/31/2025

Supporting Statement - B

Submitted: January 18, 2024

12. Program Official:

Laura Brown, Ph.D.

Safe Food Section Lead

Centers for Disease Control and Prevention

National Center for Environmental Health

Emergency and Environmental Health Services

Environmental Health Services Branch

4770 Buford Highway, NE F-58

Atlanta, GA 30341-3724

[email protected]

770-488-4332

Table of Contents

13. 1. Respondent Universe and Sampling Methods

2. Procedures for the Collection of Information

3. Methods to Maximize Response Rates and Deal with No Response

4. Test of Procedures or Methods to be Undertaken

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Attachments

Attachment 1 – Authorizing Legislation 

Attachment 2 – EHS-Net Publications 

Attachment 3 – Manager Informed Consent and Manager Interview 

Attachment 4 – Food Worker Informed Consent and Food Worker Interview  

Attachment 5 – Restaurant Environment Observation  

Attachment 6 – Recruiting Script 

Attachment 7 – Manager Informed Consent and Interview Worksheet

Attachment 8 – Food Worker Informed Consent and Interview Worksheet

Attachment 9 – Restaurant Environment Observation Worksheet

Attachment 10 – Recruiting Script Worksheet

B. Collections of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

EHS-Net is a collaborative project of CDC, FDA, USDA, and eight state and local public health departments (Franklin County, Ohio; Minnesota; New York City; New York State; Rhode Island; Tennessee; Southern Nevada Health District, NV; and Harris County, TX). The respondent universe is comprised of retail food establishments in selected geographical areas within the EHS-Net catchment areas. While the number of areas included in EHS-Net is small, they are demographically diverse and provide good geographical coverage of the U.S. (northeast, mid-west, south, and west). When the sampling methods outlined here for ensuring a representative sample in the current study are used, the results of the collection can be used to generalize to the population of retail food establishments in the given EHS-Net site(s).

The respondent universe is all retail food establishments (hereafter referred to as restaurants) in the EHS-Net catchment area. Restaurant lists will be obtained from the restaurant databases maintained by the EHS-Net sites. CDC will use these restaurant lists to generate the sampling frame used to draw the sample for this study.

Each EHS-Net site will enroll 50 restaurants in the study (Table B.1). Since there are no previously published (population) studies that have examined norovirus outbreak prevention and response practices in conjunction with organizational practices in restaurants, we are unable to determine whether this sample size will be able to support at least an 80% study power to detect statistical differences between study groups. Thus, data on expected prevalence of knowledge, attitudes, and practices between different groups of restaurants are not available as inputs for proper calculation of study sample size and power. Enrollment of 50 restaurants per EHS-Net site, totaling 400 restaurants for the entire study, is a reasonable sample size and follows the precedent of previous EHS-Net studies (Green et al., 2006; Kirkland et al., 2009; Sumner et al., 2011). Experience from prior EHS-Net studies also indicates that a sample size of 400 should be sufficient for the analytic purposes outlined below, since the analytic parameters are not likely to be considered rare (in distribution) events. Data collected from this study will provide the necessary information for sample size and power calculations for future studies.

Table B.1

Strata (EHS-Net Sites)	Entity	Number of Entities
Franklin County, Ohio	Restaurants	50
Minnesota	Restaurants	50
New York	Restaurants	50
New York City	Restaurants	50
Rhode Island	Restaurants	50
Tennessee	Restaurants	50
Southern Nevada Health District	Restaurants	50
Harris County, Texas	Restaurants	50

The design is cross-sectional and uses a stratified random sampling plan in which each EHS-Net site serves as its own mutually exclusive stratum. There are two primary reasons for stratifying by EHS-Net site. The first is that food safety regulations vary by jurisdiction. For example, Tennessee state food safety regulations differ from New York State food safety regulations. These regulations greatly influence restaurants’ food safety practices and policies. Therefore, site poses the largest source of variability from a study design perspective. Thus, it is a critically important factor for stratification. The second reason for stratifying by EHS-Net site only is due to practical concerns that limit our ability to stratify on other variables of interest. EHS-Net sites participate in EHS-Net through a cooperative agreement. The nature of this agreement is such that one site cannot be expected to do a disproportionate amount of work in comparison to other sites (because each site receives relatively equal funding amounts). If we did not stratify by EHS-Net site but by some other factor such as ownership (independently owned or belonging to a corporate regional chain), it is likely that some sites would have to carry a greater burden than other sites in term of recruiting and collecting data in a larger number of restaurants. However, we will be collecting data on these factors of interest and will account for their heterogeneity through statistical modeling. Finally, the need for each site to share an equal burden in data collection is the reason why a fixed-sample allocation method was used for each site (50 establishments per site), instead of a proportionate-sample allocation.

Restaurants will be randomly selected, with equal probability, within their respective EHS-Net site, independent of other sites. This process will give each restaurant in a particular sampling frame the same probability of being selected for study participation. There are three reasons for employing this sampling strategy: reducing sampling error, maintaining equal representation by site, and ensuring generalizability. First, as stated previously, the total target population of restaurants from all EHS-Net sites combined constitutes a highly heterogeneous group. To control for such heterogeneity in the total sample, restaurants will be stratified by EHS-Net site so they can be grouped into more homogeneous strata and then sampled within stratum independently. This reduction in heterogeneity of the total sample will lead to reduction in sampling error, which can improve representativeness of the selected sample and provide estimates (e.g., means) that tend to have less variability than estimates produced from samples that were drawn using the unstratified, simple random sampling method. Second, with equal allocation of samples, each EHS-Net site will have equal representation in the parameter estimates of the combined sample. An additional benefit is that even sites with small sampling frames will have sufficient data points to support their site-specific analyses. Third, by ensuring that the sampling of restaurants is done by an entity (CDC) separate from the data collectors (EHS-Net sites) and employing a random selection method, we are able to minimize the potential for selection bias. Parameter estimates or study findings obtained from an unbiased study sample could be generalized to the entire EHS-Net target population.

The average response rate across EHS-Net studies that used methods similar to the proposed study is 45% (Brown et al., 2016; Brown et al., 2014; Radke et al., 2016). We expect a similar response rate for the proposed study.

2. Procedures for the Collection of Information

As indicated earlier, each EHS-Net site will provide CDC with a list of all restaurants in their catchment area. This list will serve as the sampling frame for the site. CDC will use a random number generator in SAS 9.3 to produce a random sample of restaurants for each site. As we expect some restaurants will refuse to participate and some will be ineligible to participate, we will select more than the needed number of restaurants--100 restaurants for each site. Once they receive their sample list from CDC, personnel in each site will contact restaurants by telephone to recruit their participation in the study. If the manager is willing to participate, the EHS-Net specialist will arrange a mutually convenient time to conduct the data collection.

In instances where an EHS-Net site is unable to recruit 50 restaurants from the first list of 100 restaurants, CDC will randomly select another group of 50 restaurants for the site to use to recruit additional respondents. Recruitment will be considered complete once data are collected in 50 restaurants. EHS-Net sites will recruit via the telephone and will keep a log of each contact with the restaurants to document participation rates and reasons for refusal and/or ineligibility.

CDC will not know which restaurants on the sample lists participated in the study, and thus will not be able to link restaurant names with study data. The restaurant identifying information will be maintained by the respective EHS-Net site to facilitate the site visit but will be destroyed when it is no longer needed. Additionally, on all forms only the specific coded restaurant identifier will be used to minimize a risk of someone inadvertently seeing a completed form and being able to associate it with a specific restaurant.

The data collectors are experienced and knowledgeable in environmental health and food safety and will have received training from CDC on data collection for this study. The EHS-Net administrator in each EHS-Net site and CDC staff will perform quality assurance procedures to check for data entry errors.

Managers’ concerns about the safety practices of their restaurants may result in selection bias- a lower rate of study participation among restaurants with worse or non-existent safety practices compared to restaurants with better safety practices. We have conducted studies using methods similar to those proposed for use in this study in the past, and these studies have found a wide range of food safety practices, including poor ones (Bogard et al., 2013; Brown et al., 2013; Brown et al., 2012; Coleman et al., 2013; Green et al., 2006; Kirkland et al., 2009; Lee et al., 2004; Sumner et al., 2011). While the potential for selection bias exists, these studies indicate that these biases may be minimal. Additionally, the study protocol incorporates procedures to minimize the potential for and to detect any indication of selection bias. For example, EHS-Net specialists will be trained in the recruitment process in order to keep non-response rate as low as possible, which will help minimize selection bias.

The interview data collected for this study may be influenced by social desirability bias- the tendency for people to report greater levels of socially desirable behavior (such as not working while ill) than they actually engage in, or to report their best behavior rather than their typical or worst behavior. Although it is difficult to eliminate this bias altogether, it can be limited by ensuring respondents that the information they report will be anonymous, which we will do (Leary, 2004).

Interview forms are available in both English and Spanish. The CDC Multilingual Services Team will translate the English forms into Spanish at a one-time fee of $919.79. Two of the EHS-Net sites are able to use the Spanish interview forms if needed.

Any presentation of data from this study will acknowledge these potential biases and include a discussion of how they impact data interpretation.

3. Methods to Maximize Response Rates and Deal with Nonresponse

We will engage in several activities designed to maximize response rates. First, all recruiters will receive training on the recruiting process that will be locally developed by EHS-Net sites. Second, multiple attempts will be made to contact potential respondents. Specifically, recruiters will make 10 attempts over 5 days to get a participation response from establishments they have not been able to contact, and 5 attempts over 5 days to get a participation response from restaurants that have not provided a response (e.g., ‘call back later’). Third, recruiting scripts will emphasize two issues that have been shown to increase response rates—the anonymous nature of the data collection and the importance of the respondents’ participation in the study.

4. Test of Procedures or Methods to be Undertaken

The data collection materials and methods are based on those used in other previous, successful EHS-Net studies (Bogard et al., 2013; Brown et al., 2013; Brown et al., 2012; Coleman et al., 2013; Green et al., 2006; Kirkland et al., 2009; Lee et al., 2004; Sumner et al., 2011). All data collection materials were reviewed and evaluated by key EHS-Net specialists who are experienced with collecting data for EHS-Net studies. They were also reviewed by CDC EHS-Net specialists with extensive experience in developing and conducting EHS-Net studies. Additionally, all data collection materials were evaluated in pilot tests with 9 retail food establishments. Given that we are experienced in collecting data from retail food establishments with these types of instruments and methods (this will be the eleventh multisite study we have conducted in retail food establishments using similar data collection instruments and methods), we are confident that the study is designed well and do not anticipate the need to make significant changes to the data collection instruments.

Data Analysis Plan. The purpose of this study is two-fold: 1) assess food establishments’ practices regarding outbreak prevention and response; and 2) assess relationships between establishments’ characteristics and food safety management systems and practices concerning norovirus prevention and response. To accomplish the first purpose, we will first conduct analyses (frequencies, means, etc.) that describe study restaurants’ prevention and response practices. We will also conduct analyses that describe the study restaurants and their food safety management systems. Table B.4.1 is a table shell that illustrates how we may analyze and present the descriptive data collected from this study.

Table B.4.1-Table Shell: Descriptive data on outbreak prevention and response practices

	n	%
Have you had any kind of food safety training?
Yes
No
Is this restaurant independently-owned or part of a chain?
Independent
Chain
Does this restaurant have procedures in place specifically for cleaning up after a vomiting or diarrheal event?
Yes
No
How are staff trained to clean tables?
Handbook
Peer training
Manager training
Computer training
Chemical company training
Other

To accomplish the section purpose of assessing relationships between establishments’ characteristics and food safety management systems and practices concerning norovirus prevention and response, we will conduct multivariable regressions. These regressions will identify restaurant characteristics and food safety management system components that are related to good norovirus prevention and response practices.

Table B.4.2 lists key explanatory variables and key outcome variables. Table B.4.3 is a table shell that illustrates how we might analyze and present the data examining the relationships between these explanatory and outcome variables.

Table B.4.2 Key explanatory and practice outcome variables included in multivariable regressions

Explanatory variables

Outcome variables

Restaurant characteristics Restaurant ownership Customers served daily Manager experience Type of establishment Food safety management systems Policies Existence of ill worker policies Existence of cleaning policies Training Manager training and certification Worker training

Employee health/ill workers Worker has worked with vomiting or diarrhea Manager asks employees their about their symptoms when they are sick Establishments uses appropriate criteria for determining when ill workers can return to work Employee hand hygiene Handwashing resources available (soap, method to dry hands, warm water) Cleaning Appropriate chemicals are used for cleaning high touch areas Cleaning after contamination event Restaurant has a kit for cleaning up contamination events

Table B.4.3 Table Shell: Key restaurant and food safety management system characteristic explanatory variables associated with the practice outcome of the presence of a contamination clean-up kit, bivariate analyses

Explanatory variables	Restaurant has a kit for cleaning up contamination events
	OR (95% CI)	P
Restaurant characteristics
Restaurant ownership
Independent
Chain
Customers served daily
> [low end]
< [high end]
Food safety management systems
Existence of cleaning after contamination policies
Yes
No
Manager is certified
Yes
No
Worker food safety training
Yes
No
Use effective disinfectants after contamination events
Yes
No

OR=Odds Ratio, P=probability level

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following people were primarily responsible for the design, including the statistical aspects, of the data collection and will be primarily responsible for data analysis. Laura Brown is the primary contact for statistical aspects and data collection.

Laura Brown, Ph.D.

Behavioral Scientist

Centers for Disease Control and Prevention, National Center for Environmental Health

[email protected]

770-488-4332

Adam Kramer, Sc.D., M.P.H, R.S.

Environmental Health Officer

Centers for Disease Control and Prevention, National Center for Environmental Health

[email protected]

404-498-1228

Rick Hoover, Ph.D.

Statistician

Centers for Disease Control and Prevention, National Center for Environmental Health

[email protected]

706-765-8857

Personnel in the eight EHS-Net sites will be responsible for data collection (See table below).

Site	Number of Personnel
Franklin County, Ohio	1 full-time
Minnesota	1 full-time
New York	1 full-time
New York City	1 full-time
Rhode Island	1 full-time
Tennessee	1 full-time
Southern Nevada Health District	1 full-time
Harris County, Texas	1 full-time

References for Part B

Bogard, A. K., Fuller, C. C., Radke, V., Selman, C. A., & Smith, K. E. (2013). Ground Beef Handling and Cooking Practices in Restaurants in Eight States. Journal of Food Protection, 76(12), 2132-2140.

Brown, L., Hoover, E., Ripley, D., Matis, B., Nicholas, D., Hedeen, N., & Faw, B. (2016). Retail Deli Slicer Cleaning Frequency — Six Selected Sites, United States, 2012. MMWR Morb Mortal Wkly Rep 65, 306-310.

Brown, L., Khargonekar, S., & Bushnell, L. (2013). Frequency of Inadequate Chicken Cross-Contamination Prevention and Cooking Practices in Restaurants. Journal of Food Protection, 76(12), 2141-2145.

Brown, L., Le, B., Wong, M. R., Reimann, D., Nicholas, D., Faw, B., Davis, E., & Selman, C. A. (2014). Restaurant manager and worker food safety certification and knowledge. Foodborne pathogens and disease, 11(11), 835-843.

Brown, L., Ripley, D., Blade, H., Reimann, D., Everstine, K., Nicholas, D., Egan, J., Koktavy, N., & Quilliam, D. N. (2012). Restaurant Food Cooling Practices. Journal of Food Protection, 75(12), 2172-2178.

Coleman, E., Delea, K., Everstine, K., Reimann, D., & Ripley, D. (2013). Handling Practices of Fresh Leafy Greens in Restaurants: Receiving and Training. Journal of Food Protection, 76(12), 2126-2131.

Green, L. R., Selman, C. A., Radke, V., Ripley, D., Mack, J. C., Reimann, D. W., Stigger, T., Motsinger, M., & Bushnell, L. (2006). Food worker hand washing practices: an observation study. Journal of Food Protection, 69(10), 2417-2423.

Kirkland, E., Green, L. R., Stone, C., Reimann, D., Nicholas, D., Mason, R., Frick, R., Coleman, S., Bushnell, L., Blade, H., Radke, V., Selman, C., & The, E. H. S. N. W. G. (2009). Tomato Handling Practices in Restaurants. Journal of Food Protection, 72(8), 1692-1698.

Leary, M. R. (2004). Introduction to behavioral research methods. Pearson/Allyn and Bacon.

Lee, R., Beatty, M. E., Bogard, A. K., Esko, M.-P., Angulo, F. J., Selman, C., & Ehs-Net Working, G. (2004). Prevalence of High-Risk Egg-Preparation Practices in Restaurants That Prepare Breakfast Egg Entrées: An EHS-Net Study. Journal of Food Protection, 67(7), 1444-1450.

Radke, T. J., Brown, L., Hoover, E. R., Faw, B. V., Reimann, D., Wong, M. R., Nicholas, D., Barkley, J., & Ripley, D. (2016). Food Allergy Knowledge and Attitudes of Restaurant Managers and Staff: An EHS-Net Study. Journal of Food Protection, 79(9), 1588-1598.

Sumner, S., Brown, L., Frick, R., Stone, C., Carpenter, L. R., Bushnell, L., Nicholas, D., Mack, J., Blade, H., Tobin-D'Angelo, M., & Everstine, K. (2011). Factors Associated with Food Workers Working while Experiencing Vomiting or Diarrhea. J Food Prot, 74(2), 215-220.



12