1 - SSA Ass Ill Wrkr Plcy 201228

Assessment of Ill Worker Policies Study

OMB Control No. 0920-1227 (Expiration Date 05/31/2021)

Revision Information Collection Request

Supporting Statement - A

Submitted: December 2020

Program Official:

Adam Kramer Sc.D., M.P.H., R.S.

Environmental Health Officer

Centers for Disease Control and Prevention

National Center for Environmental Health

Emergency and Environmental Health Services

Environmental Health Services Branch

4770 Buford Highway, NE S106-5

Atlanta, GA 30341-3724

404-498-1228 (Phone)

770-488-3635 (Fax)

E-mail: [email protected]

Table of Contents

1. Circumstances Making the Collection of Information Necessary

2. Purpose and Use of the Information Collection

3. Use of Improved Information Technology and Burden Reduction

4. Efforts to Identify Duplication and Use of Similar Information

5. Impact on Small Businesses or Other Small Entities

6. Consequences of Collecting the Information Less Frequently

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

9. Explanation of Any Payment or Gift to Respondents

10. Assurance of Confidentiality Provided to Respondents

11. Justification for Sensitive Questions

12. Estimates of Annualized Burden Hours and Costs

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

14. Annualized Cost to the Government

15. Explanation for Program Changes or Adjustments

16. Plans for Tabulation and Publication and Project Time Schedule

17. Reason(s) Display of OMB Expiration Date is Inappropriate

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Goal of the study: This study has two goals 1) to assess restaurant ill worker management practices and plans; and 2) examine the effectiveness of an educational intervention for restaurants to develop enhanced ill worker management practices.

Intended use of the resulting data: Data from this study can be used to develop educational materials, trainings, and tools that are targeted towards improving retail food establishment ill worker management practices. If the intervention is successful, it will be provided to state and local food safety regulatory agencies as a model practice for working with restaurants to develop ill worker management plans and enhance their practices of keeping ill workers from working with food.

Methods to be used to collect data: The study proposes using a quasi-experimental non-equivalent group pre- post-test design. It will involve interviews with a restaurant manager and an observation of the restaurant practices. The study will have an intervention and a control group that will be assessed pre- and post-intervention. If the intervention is successful, it will be provided to the control restaurants and an additional follow up assessment will occur for these facilities.

The subpopulation to be studied: The population to be studied will be voluntarily participating restaurants in the Environmental Health Specialists Network (Minnesota; New York City; New York; Tennessee; Rhode Island; Franklin County, Ohio; Southern Nevada Health District; and Harris County, Texas).

How data will be analyzed: Initial analysis of the success of the intervention will be performed with a chi-square analysis.

A. Justification

1. Circumstances Making the Collection of Information Necessary

The National Center for Environmental Health (NCEH), Centers for Disease Control and Prevention (CDC), is requesting Paperwork Reduction Act (PRA) clearance for a three-year revision information collection request (ICR) entitled “Assessment of Ill Worker Policies Study” (OMB Control No. 0920-1227, expiration date 05/31/2021).

Although approved in 2018, NCEH and its program partners needed to prioritize other data collections (EHS-Net Food Safety Practices & Beliefs, OMB Control No. 0920-0792) over this study, and then delayed the study due to the COVID-19 pandemic. NCEH partners provided feedback to refine this research protocol, to revise the ICR, and to begin this study in 2021. NCEH is requesting approval for revisions which fall into three categories:

1. Changes to strengthen the study, based on recent experience and stakeholder feedback;

2. Changes to respond to the COVID-19 pandemic; and

3. Change to one funded partner due to a new cooperative agreement having been issued.

NCEH is requesting a revised PRA clearance for 820 responses per year and for a time burden of 261 hours per year. These changes result in a decrease of 1,307 annualized responses and 91 annualized hours relative to the 2018 PRA clearance. The changes are detailed in Section A.15.

The EHS-Net

This ICR will be conducted in partnership with the CDC’s Environmental Health Specialists Network (EHS-Net), under CDC Cooperative Agreement No. CDC-RFA-EH20-001. EHS-Net is a collaborative project including representatives from the U.S. Food and Drug Administration (FDA), the U.S. Department of Agriculture (USDA), and eight state and local public health departments (Franklin County [OH], Minnesota, New York, New York City [NY], Rhode Island, Tennessee, Southern Nevada Health District [NV], and Harris County [TX]). CDC’s funding to these state and local health departments, enables them to collaborate with CDC on study design, collecting study data, and co-analyzing study data. The other federal partners provide additional funding and input into study design and data analysis. Data will be collected by personnel in the eight state and local health departments participating in EHS-Net. Industry partners that support this initiative, its goals and research by collaborating on study design and data analysis are in Appendix C. A summary of the research conducted through by EHS-Net has resulted in 30 publications (Appendix D).

Summary of the Literature and Motivation for this Study

Ill food workers are an important cause of foodborne illness outbreaks; they are responsible for a quarter of all restaurant-related outbreaks (Angelo, 2016). And 20% of food workers work while ill with foodborne illness symptoms annually (Sumner, 2011). Reducing the rate at which ill workers work would lower the burden of foodborne illness outbreaks in the United States. The FDA Food Code, a model food code that serves as the basis for all state and local regulatory food codes in the U.S., includes provisions for restaurants aimed at preventing ill workers from working while ill with foodborne illness. These provisions include excluding an employee that is symptomatic with vomiting and/or diarrhea or diagnosed with certain illnesses (e.g., Norovirus, Salmonella typhi, etc.). The Food Code also includes provisions aimed at minimizing the spread of illness by ill workers, such as by limiting bare hand contact with ready-to-eat food items.

Recent research shows that the existence of ill worker policies is linked with lower rates of workers working while ill (Sumner et al., 2011) and lower foodborne illness outbreak rates (Kambhampati et al., 2016), indicating that they are an important component of a restaurant’s food safety plan. Yet, 40% of states have not adopted this provision of the Food Code (Kambhampati et al., 2016). And even in states that have adopted this provision, not all restaurants implement them (Norton, 2015).

Given these data showing the importance of these policies to food safety and the lack of them, the jurisdictions funded through our EHS-Net cooperative agreement wished to develop and assess the effectiveness of an educational intervention designed to motivate restaurant managers to develop, implement, and enforce comprehensive ill worker management plans, plans that would include relevant Food Code provisions. The intervention focuses on ensuring that the ill worker management plans address barriers to workers staying home when ill, as identified in previous research (Sumner et al., 2011). Finally, the intervention focuses on improving restaurant policies and practices (e.g., wearing gloves while handling food) that would minimize the spread of illness by ill workers. While FDA mandates that ill food workers do not work with food, this study goes further to examine the specific policies and to encourage development of a plan that addresses the reasons that employees have reported for continuing to work while sick.

This data collection supports the U.S. Department of Health and Human Services’ Healthy People 2020 goal to “Improve food safety and reduce foodborne illnesses.” Specifically, these new data can be used to prepare educational materials, trainings, and tools that are targeted towards improving retail food establishment ill worker management practices. This data collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. 241) (Appendix A). The 60-day Federal Register Notice (FRN) was published on 09/14/2020 (Appendix B) and is further discussed in Section A.8.

2. Purpose and Use of the Information Collection

The purpose of this ICR is to evaluate the effectiveness of an educational intervention for restaurants, which will be used to encourage restaurant managers to develop ill food worker management plans. This data collection focuses on staff practices concerning ill food workers in retail food establishments.

This study will answer the following questions about the intervention:

1. Does the intervention lead to the development or enhancement of comprehensive ill worker management plans, that include Food Code provisions and that address barriers to workers staying home while ill?

2. Does the intervention improve restaurant policies and practices to minimize the spread of illness by ill workers?

Detailed Purpose and Use

The following table describes the measures that will be used to answer these questions.

Question	Measures
Does the intervention lead to the development or enhancement of comprehensive ill worker management plans, plans that include Food Code provisions?	Increase after intervention delivery in: Frequency of restaurants with ill worker management plans Frequency of restaurants with specific Food Code provisions (e.g., exclusion or restriction of ill employees) in their ill worker management plans Frequency of restaurants with provisions that address barriers (e.g., maintaining a list of available employees) in their ill worker management plans
Does the intervention improve restaurant policies and practices to minimize the spread of illness by ill workers?	Increase after intervention delivery in the frequency of restaurants with policies and practices that minimize the spread of illness by ill workers including: Use of gloves when preparing food Existence of policy on how to clean and sanitize after a vomiting incident in the restaurant Increased hand hygiene practices

Because restaurant policies and practices are influenced by a number of restaurant characteristics, such as number of meals served daily, ownership (independent vs. chain), and manager and worker food safety characteristics, knowledge, attitudes, and practices, we are also collecting data on these variables. They will be included in our analyses, where appropriate.

Intervention

The intervention will be delivered in restaurants by EHS-Net site staff, who are food safety experts. The intervention contains three components. First, EHS-Net staff will talk with the kitchen manager and will cover the following topics:

1) the number of illnesses and outbreaks caused by ill restaurant workers,

2) the importance of preventing ill workers from working,

3) the importance of ill worker management plans to help reduce ill worker rates,

4) provisions included in comprehensive ill worker management plans,

5) known barriers to workers staying home while ill and potential solutions to those barriers,

6) provisions concerning minimizing the spread of illness.

Second, to emphasize the importance of preventing customers from getting sick from ill workers, EHS-Net staff will show the manager a video testimonial of how foodborne illness has affected victims and their families (videos are publicly available on the FDA website).

Third, EHS-Net staff will provide the manager with a guide that contains comprehensive information and policy templates than can be used to develop their own ill worker management plans (Attachment 5a).

Implementation

Intervention restaurants. Twenty restaurants in each EHS-Net site will be randomly selected to receive the intervention. After the restaurants have agreed to participate in the study (Attachment 2), EHS-Net site staff will visit these restaurants, collect pre-intervention data (Attachments 3-4) and then deliver the intervention to the restaurant managers. Approximately 3-6 months later, EHS-Net site staff will visit the restaurants again and collect post-intervention data on the measures of interest. The timeframe for the follow up is based on individual site logistical constraints, and EHS-Net site staff indicated that this timeframe would be needed.

Control restaurants. During the same timeframe, twenty restaurants in each EHS-Net site will be randomly selected to serve as controls for the intervention restaurants. This control group will allow us to control for events that happen during the study period (other than our intervention) that may affect restaurant policies and practices (e.g., a large foodborne outbreak with elevated media coverage). Restaurants will be recruited by telephone (Attachment 2). After the restaurants have agreed to participate in the study, EHS-Net site staff will visit these restaurants, collect the same pre-intervention data collected in the intervention restaurants (Attachments 3-4), but they will not deliver the intervention. Approximately 3-6 months later, EHS-Net site staff will visit the restaurants again and collect post-intervention data on the measures of interest. If the intervention shows preliminary success, it will be provided to the control restaurants at the follow-up and a third telephonic visit will occur in these restaurants. Initial success for the intervention will be measured by if three or more intervention restaurants either develop a written ill worker management plan (if they did not have one at the pre-intervention evaluation) or enhanced their policies (e.g., added provisions addressing reasons why ill workers reported working while ill).

Data collection. Both pre- and post-intervention restaurant visits, for both intervention and control restaurants, will be comprised of the following:

• Manager interview- will collect data on restaurant characteristics and existing ill worker management plans, and manager characteristics (Attachment 3).

• Restaurant environment observation- will collect data on restaurant characteristics and practices to minimize the spread of illness by ill workers (Attachment 4).

Analysis of these data will determine if the measures of interest increased as expected. For example, we will determine if the frequency of restaurants with ill worker management plans increased pre- to post- intervention.

Data collection will include voluntarily participating restaurants in selected geographical areas as stated in Section A.1. While the number of areas included is small, they are demographically diverse and provide good geographical coverage of the U.S. (northeast, mid-west, south, and west). When the statistical methods outlined here for ensuring a representative sample in the current study are used, the results of the collection can be used to generalize to the population of retail food establishments that are a part of the network.

The data collected by this study can be used to identify if an educational intervention can be used to change restaurant processes. If it is effective, it can be shared with state and local food safety regulatory programs. The study will also inform how well (or not) prepared restaurants are to manage ill food workers, by the existence of initial practices, and if an educational intervention can help prepare them to handle ill workers and prevent the spread of illness. The goal of this information collection is to assist CDC and other federal, state, and local food safety programs to develop food safety prevention, intervention recommendations, and tools for food safety programs and the restaurant industry. For example, if the intervention is successful, CDC can disseminate the information and encourage food safety programs and the restaurant industry to implement these programs. CDC can also disseminate information on the knowledge, attitudes, and practices of the restaurant workers and encourage food safety programs and the restaurant industry to address these gaps. Ultimately, these types of actions can contribute to a decrease in the number of incidents of foodborne illness caused by an ill food worker.

3. Use of Improved Information Technology and Burden Reduction

The primary burden to respondents of participation in this study involves their participation in interviews (Attachment 3). It is less burdensome for respondents to provide interview responses verbally than to have to type their responses into an electronic reporting system. Thus, we have chosen not to collect interview data electronically, but rather, collect the data through face-to-face verbal interviews with respondents. Study personnel will record responses on paper-and-pencil forms.

Participation in this data collection is voluntary, and every effort was made to keep the data collection as short as possible and still meet the needs of the data collection.

4. Efforts to Identify Duplication and Use of Similar Information

This data collection will not be a duplication of effort. We have searched relevant scientific bibliographical databases (e.g., PubMed, Ovid, Agricola), attended national meetings (e.g. National Environmental Health Association, International Association of Food Protection), and consulted with other organizations (e.g., FDA, USDA-FSIS) concerning research on this topic. Few studies exist on this topic; the research that exists has been conducted in small geographical regions or with convenience samples. Consequently, data are needed from a random sample of a geographically and demographically diverse population of restaurants. This study is designed to do this.

5. Impact on Small Businesses or Other Small Entities

We expect that about half of the restaurants contacted for participation in this study will be small businesses. Given that small businesses are likely to have different experiences and practices than larger businesses, it is important that small businesses be included in this data collection. Short forms for small businesses will not be developed. We plan to ask the same questions to both large and small restaurants. Small businesses may not have similar written plans as larger businesses, however both size restaurants have a similar issue of employees potentially working while ill and increasing the risk to the public of foodborne illness. The surveys developed will strive to hold the number of questions to the minimum needed for the intended use of the data collection.

6. Consequences of Collecting the Information Less Frequently

All participating restaurants in the study will be interviewed a minimum of two times pre- and post-intervention (Attachment 3), if the intervention is proving successful, it will be provided to the control restaurants on the second site visit and a third telephonic visit will be conducted in these establishments to further gauge the effect of the intervention. If this data collection is not conducted, it will be more difficult for CDC, other federal, state, and local food safety programs, and the food service industry to address the development of ill worker management plans along with gaps in restaurant ill worker knowledge, attitudes, and practices. In turn, it will be more difficult to decrease the number of incidents of foodborne illness caused by ill food workers and for CDC to fully address the U.S. Department of Health and Human Services’ Healthy People 2020 Goal to “Improve food safety and reduce foodborne illnesses.”

There are no legal obstacles to reduce the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for this data collection. This request fully complies with 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A. The 60-day Federal Register Notice was published on 09/14/2020 in Vol. 85 No. 178 pp. 56615 (Appendix B). CDC received one non-substantive comment (https://www.regulations.gov/document?D=CDC-2020-0097-0002). No response from CDC is necessary.

B. Personnel from the EHS-Net sites worked with CDC to develop this data collection in 2017, and to develop the revisions in 2019. Additionally, FDA, USDA, and EHS-Net partners, were also consulted on the data collection.

CDC EHS-Net Sites
Nichole Lemin Assistant Health Commissioner Franklin County Public Health [email protected] 614-525-6130	David Nicholas Research Scientist NY Dept. of Health [email protected] 518-402-7600
Bailey Matis Senior Environmental Project Manager NYC Dept. of Health bmatis@health.nyc.gov 646-632-6515	Nicole Hedeen Epidemiologist MN. Dept. of Health [email protected] 651-201-4075
Brendalee Viveiros RI EHS-Net Coordinator RI Dept. of Health [email protected] 401-222-4774	Lauren DiPrete Senior Coordinator Southern Nevada Health District [email protected] 702-759-1504
JoAnn Monroy Food Safety Program Manager Harris County Health Department joann.monroy@hcphes.org 713-274-6319	Danny Ripley Food Inspector II TN Dept. of Health [email protected] 615-340-5620
Federal Partners
Laurie Williams Consumer Safety Office Office of Food Safety FDA/CFSAN [email protected] 240-402-2938	Stephanie Mickelson Epidemiologist USDA [email protected] 703-305-2894

9. Explanation of Any Payment or Gift to Respondents

There will be no payments or gifts to respondents.

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

The project has been reviewed by the NCEH Information Systems Security Officer (ISSO) who has determined that the Privacy Act does not apply to this collection (Appendix E). Information in identifiable form (IIF) will not be collected at the research sites. Therefore, a system of records notice (SORN) is not applicable.

The individual sites will assign a code number to the restaurant and only that code number will be entered, stored, and transmitted. Subsequent to the follow-up visits, the restaurant name and address will no longer be needed and will be destroyed by the sites. The only information to be collected is business name, street address, business phone numbers, business email address, and any other information that the site uses to contact the restaurant such as an alternate phone number. No personal IIF for restaurant managers will be documented in Attachment 6.

No paper files will be delivered to CDC. Instead, data collectors will enter all paper-and-pencil responses into the REDCAP data management system. Data will be reported to CDC through a web-based information system, the REDCAP data management system. All electronic data will be stored on secure CDC networks. Access to the data will be limited to those with a bona fide need-to-know in order to perform job duties related to the project. User accounts will be issued to the specialists who will serve as the administrator of the system for his or her own site. Through these password protected accounts, users will be granted privileges including entering and accessing data, and correction and deletion of records capabilities. All data records are owned by the site entering the data. Each site possesses ownership of its records and must grant permission to other sites or agencies who would like to use the data. Each site’s data will be stored for twelve years.

Verbal consent will be obtained from respondents. As a part of the informed consent, respondents will be made aware of their ability to retrieve a summary of the study’s findings by contacting their health department 12 months following data collection. The manager’s informed consent script can be found at the beginning of the manager interview (Attachment 3).

Participation in this data collection is voluntary, and respondents are informed of the voluntary nature of the data collection during telephone recruiting (Attachment 2) and in the informed consent script (Attachment 3).

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

This project has been classified as research not involving identifiable human subjects. CDC institutional review board (IRB) approval is not required. This research study centers around restaurant food safety policies and practices and is not about human subjects (Appendix F). There are no sensitive questions in this data collection.

12. Estimates of Annualized Burden Hours and Costs

The goal for the three-year program period is to include 320 restaurants (half to be assigned to the intervention and half to be assigned to the control restaurant groups). The table below shows how the annual number of respondents is derived.

Manager recruiting script (Attachment 2)	712/3 yrs. = 237 per year
Total number of participating restaurants	40 restaurants per site* 8 sites = 320 for 3 yrs.
Total number of intervention or control restaurants	320/3 yrs./2 respondent groups = 53 per year
Manager interviews (at each assessment) for intervention or control restaurants	(same as above) 53 per year
Health departments observations (at each assessment) for intervention or control restaurants	53 per year
NOTE: Control restaurants may have an additional telephonic site visit depending upon the success of the intervention which only includes a manager interview

Estimated annualized burden hours, averaged over the three-year period, are presented for each study respondent group and in total. Eight sites will collect data for this study; each site will collect data in 40 restaurants (20 that receive the intervention initially and 20 that serve as controls). If the intervention is successful, the control restaurants will also receive the intervention and an additional telephonic assessment visit to further determine if the intervention was effective. Thus, there will be a total 320 restaurant manager respondents (40 restaurants * 8 sites) for three years. We expect a manager response rate of 45%; thus, we will need to contact 712 restaurant managers (356 intervention and 356 control, or 119 per year for each group) via telephone in order to meet our goal of 320 respondents (160 intervention and 160 control, or 53 per year for each group). Each respondent to the script will respond only once, and the average burden per response will be approximately 3 minutes (6 annualized hours per group, 12 annualized hours total).

In the intervention restaurants, all participating restaurants in this group will have two visits. In the first visit, the study staff will collect baseline data and the second visit will assess the success of the intervention. The time burden for each manager interview visit is estimated as 20 minutes per response. For the intervention group, an educational intervention will be provided at the first visit estimated as 30 minutes, this is to inform restaurant managers about the risks posed by ill food workers and to help restaurants develop enhanced ill worker management plans. In the second visit; the manager interview will be repeated, incurring similar burden hours as described above (63 hours for two visits for the group).

For the control restaurants, they will have a maximum of three visits. In the initial visit, baseline manager interviews will be conducted, but without the introduction of the educational intervention. If the intervention is successful, it will be introduced to the control restaurants during the second site visit estimated as 30 minutes, and the third telephonic visit will be conducted with the same manager interview as conducted previously to see if a change has occurred. The burden hours for each subsequent visit will be similar to the initial visit (80 hours for three visits for the group).

The health department data collectors will also conduct up to two observational visits to examine the restaurant environment which will take approximately 30 minutes at each visit. These will be conducted at both intervention and control restaurants. These observations will not require interactions between the study personnel and restaurant staff and is estimated to be 53 annualized burden hours for two visits for each group (106 hours for both groups of restaurants). (See Table A.12-1).

Table 12.1- Estimated Annualized Burden Hours

Type of Respondents	Form Name	No. of Respondents	No. of Responses per Respondent	Avg. Burden per Response (in hrs.)	Total Burden (in hrs.)
Restaurant Managers (Intervention and Control Restaurants)	Manager Recruiting Script	237	1	3/60	12
Restaurant Managers (Intervention Restaurants)	Manager Informed Consent and Interview	53	2	20/60	36
	Intervention Log	53	1	30/60	27
Restaurant Managers (Control Restaurants)	Manager Informed Consent and Interview	53	3	20/60	53
	Intervention Log	53	1	30/60	27
Health Department Workers (Intervention and Control Restaurants)	Restaurant Observation Form	106	2	30/60	106
Total					261

The maximum total annualized cost of this data collection to respondents is estimated to be $5,341.81 (See Table 12-2). These estimated hourly wages were obtained from the U.S. Department of Labor Bureau of Labor Statistics 2019 national occupational employment and wage estimates report for food supervisors (http://stats.bls.gov/oes/current/oes351012.htm and environmental health specialists https://www.bls.gov/oes/current/oes194091.htm ).

12.2- Estimated Annualized Burden Costs

Type of Respondents	Form Name	No. of Respondents	No. of Responses per Respondent	Avg. Burden per Response (in hrs.)	Hourly Wage Rate	Total Respondent Costs
Restaurant Managers (Intervention and Control Restaurants)	Manager Recruiting Script	237	1	3/60	$17.77	$213.24
Restaurant Managers (Intervention Restaurants)	Manager Informed Consent and Interview	53	2	20/60	$17.77	$639.72
	Intervention Log	53	1	30/60	$17.77	$479.79
Restaurant Managers (Control Restaurants)	Manager Informed Consent and Interview	53	3	20/60	$17.77	$941.81
	Intervention Log	53	1	30/60	$17.77	$479.79
Health Department Workers (Intervention and Control Restaurants)	Restaurant Observation Form	106	2	30/60	$24.41	$2,587.46
Total						$5,341.81

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no other costs to respondents or record keepers.

14. Annualized Cost to the Federal Government

Costs to the government include a portion of the annual cooperative agreement to the EHS-Net sites that will collect the data and the costs of CDC personnel working on the data collection (A.14.1). The sites participating in this study receive equal funding, and we estimate that the sites will use approximately 20% of their cooperative agreement funds to conduct this data collection. We also estimate that one CDC staff member will spend approximately 50% of their time on this data collection.

Table 14.1-Estimated Annualized Cost to the Federal Government

Expenditure	Cost
Awards to sites	$102,667
CDC Salary (1 staff member)	$16,667
Total	$119,333

15. Explanation for Program Changes or Adjustments

In summary, the proposed revisions fall into three categories:

1. Changes to strengthen the study, based on recent experience and stakeholder feedback.

The EHS-Net affiliated state and local health departments recently completed a study using methods similar to those originally proposed for the current study. Based on this recent experience, the EHS-Net sites recommended several adjustments to improve the current study and improve the recruitment process and participation rate. Adjustments include:

• Deletion of a short survey for restaurant workers (recent experience indicated that this may increase the study participation rate)

• Provision of template language for the EHS-Net affiliated health departments to use to advertise the study (recent experience indicated that this may increase the study participation rate

• Deletion of 13 study questions from the restaurant manager interview (recent experience indicated that these questions were poorly understood by restaurant managers and the desired information was captured in other questions

• Changes in phrasing and order of questions to increase use of plain language and improve flow (e.g., changing “ill” to “sick”).

• Deletion of the onsite observation at the third visit, if needed, at the control restaurants that receive the intervention on the second visit. If the intervention is successful, the data collectors will conduct the manager interview telephonically (Attachment 3) but will not conduct an onsite observation. This will provide further information on whether the policies have changed within the restaurant.

• To streamline the burden worksheets, the burden per visit was reduced to the burden for the data collection and a new form (Attachment 5) was created to account for the burden that will be incurred in receiving the intervention. Previously this had been reported as Educational Intervention with no associated form.

2. Changes to respond to the COVID-19 pandemic.

Although COVID-19 cannot be transmitted through food, it can be transmitted from worker to worker. Thus, ill worker policies may have changed in response to the COVID-19 pandemic. We added five questions to the data collection to assess this change.

3. One change in sites funded by the EHS-Net cooperative agreement.

These changes result in a decrease of 1,307 annualized responses and 91 annualized hours.

Per Year	2018	2021	Net Change from 2018 to 2021
No. Responses	2,127	820	-1,307
No. Hours	352	261	-91

Changes to strengthen the study, based on recent experience and stakeholder feedback

The NCEH EHS-Net Program delayed the start of the study until its cooperative agreement partners completed the data collection for the 2017-2018 GenIC titled “EHS-Net Food Safety Practices and Beliefs Study” (OMB Control No. 0920-0792, expiration date 09/30/2018). The study provided useful information on the proposed methods in the current protocol. Thus, the EHS-Net partners recommended changes to the protocol for the “Assessment of Ill Worker Policies Study,” which we are now submitting under this current revision ICR.

Change	Change location	Change Justification
Provided template language for partners to use in advertising the study to potential restaurant participants	Attachment 1 (added)	Recent experience indicated that this might improve study participation rates.
Delete Food worker informed consent and survey; Delete Research Question: Does the intervention improve food employee knowledge and attitudes towards working while ill?	Food worker informed consent and survey (deleted) This deletion results in a reduction of 1,350 annualized respondents (from 1,350 to 0) and reduction of 115 annualized burden hours (from 115 hours to 0 hours).	Recent experience indicated that not surveying food workers might improve study participation rates; Research question required surveying food workers.
Adjusted follow-up timeframe from 6 months to 3-6 months	Attachments 2, 3	Recent experience suggested that partners need flexibility in scheduling follow-up visits.
Changed follow-up Visit 3 from in-person to telephone; Delete observation data collection from Visit 3	Attachments 2, 3 This deletion results in a reduction of 54 annualized respondents (from 270 respondents to 216 respondents) and reduction of 27 annualized burden hours (from 135 annualized hours to 108 annualized hours).	Recent experience suggests this might increase participation rates; The key information can be obtained through telephone interview.
Changed information systems used to collect study data (from a decommissioned IT system to CDC’s REDCAP)		Our original IT system was unexpectedly decommissioned; REDCAP has a high level of security and is approved for this use.
Deleted 13 manager interview questions	Attachment 3	Recent experience indicated that the questions were poorly understood.
Added 1 manager interview question (restaurant ownership)	Attachment 3	The question was inadvertently omitted from the original instrument.
Updated phrasing to incorporate plain language, improve comprehension and clarity; remove references to surveying food workers	Throughout Protocol body and Attachment 3	Recent experience indicated that these kinds of language changes were needed to improve data collection experience and data quality.
Questions requiring data collectors to review or verify something were moved from manager interview (Att. 3) to observation (Att. 4)	Throughout Attachment 3 Attachment 4	Recent experience indicated that this would improve the data collection process.
Questions order was modified to streamline the interview and clarify skip patterns.	Throughout Attachment 3	Recent experience indicated that this would improve the data collection process.

Changes to respond to the COVID-19 pandemic

Although COVID-19 cannot be transmitted through food, it can be transmitted from ill worker to ill worker. Thus, it seems likely that ill worker policies may have changed as a result of the COVID-19 pandemic. We added five questions to the manager interview to assess these changes (Attachment 3: questions 19, 19a, 23, 23a, 27, 27a).

Each of the EHS-Net affiliated health departments have developed their own COVID-19 plans for determining which and how health department services should be implemented. In order to protect both the data collector and restaurant employees we will rely on each site to determine when it is safe and appropriate to begin data collection following their existing plans and using appropriate personal protective practices.

Changes to participating sites

The EHS-Net cooperative agreement had reached the end of its five-year funding cycle and a new agreement was issued. California did not re-apply for the new agreement and a new site (Franklin County, OH) successfully applied. Thus, California will be replaced by Franklin County, OH for this study.

16. Plans for Tabulation and Publication and Project Time Schedule

Table 16.1 provides the data collection activity schedule.

16.1 – Project Time Schedule

Activity	Time Frame
Train EHS-Net sites on data collection	Within 1 month of OMB approval
Recruitment of restaurants	Within 3 months of OMB approval
Initial data collection	Within 6 months of OMB approval
Follow up data collection	Within 18 months of OMB approval
Follow up data collection (if needed)	Within 30 months of OMB approval
Data entry and quality assurance	Within 36 months of OMB approval
Data cleaning	Within 42 months of OMB approval
Data analysis	Within 48 months of OMB approval
Manuscript development	Within 60 months of OMB approval

A detailed analysis plan can be found in Supporting Statement B (B.4).

17. Reason(s) Display of OMB Expiration Date is Inappropriate

We are not requesting an exemption for the display of the expiration date.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification for Paperwork Reduction Act.

References

Angelo, K., Nisler, A., Hall, A., Brown, L., & Gould, L. (2016). Epidemiology of restaurant-associated foodborne disease outbreaks, United States, 1998–2013. Epidemiology & Infection, 1-12.

Kambhampati, A., Shioda, K., Gould, L. H., Sharp, D., Brown, L. G., Parashar, U. D., & Hall, A. J. (2016). A State-by-State Assessment of Food Service Regulations for Prevention of Norovirus Outbreaks. Journal of Food Protection, 79(9), 1527-1536. doi:10.4315/0362-028X.JFP-16-088

Norton DM, Brown LG, Frick R, Carpenter LR, Green AL, Tobin-D’Angelo M, et al. Managerial practices regarding workers working while ill. pdf icon[PDF – 100 KB] J Food Prot. 2015;78(1):187-95.

Sumner S, Brown LG, Frick R, Stone C, Carpenter LR, Bushnell L, et al. Factors associated with food workers working while experiencing vomiting or diarrhea. pdf icon[PDF – 190 KB] J Food Prot. 2011;74(2):215-20.