Survey FSPs_Supporting Statement B _Clean

Supporting Statement Part B

Supporting Statement (Part B) for

OMB Review and Approval of

Centers for Disease Control and Prevention

Survey of Food Safety Programs

OMB No. 0920 NEW

Submitted: May 2014

Program Official:

Laura Green Brown, Ph.D.

Behavioral Scientist

National Center for Environmental Health

Emergency and Environmental Health Services

Environmental Health Services Branch

4770 Buford Highway, NE F – 58

Atlanta, GA 30341-3724

770-488-4332 (Phone)

770-488-7310 (Fax)

E-mail: [email protected]

2. Table of Contents

B. Collections of Information Employing Statistical Methods
1. Respondent Universe and Sampling Method
2. Procedures for the Collection of Information
3. Methods to Maximize Response Rates and Deal with Nonresponse
4. Tests of Procedures or Methods to be Undertaken
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

List of Attachments

Attachment 1 - Authorizing Legislation

Attachment 2 - 60-day Federal Register Notice

Attachment 3 - Food Safety Program Survey (paper)

Attachment 4 - Food Safety Program Survey (electronic) Screenshots

Attachment 5 - NCEH/ATSDR Research Determination Form

Attachment 6 - Survey of Food Safety Programs Participation Request Email

Attachment 7 - Survey of Food Safety Programs Fact Sheet

Attachment 8 - Survey of Food Safety Programs Telephone Script for Recruited

Health Departments

Attachment 9 - Survey of Food Safety Programs Follow-up 1 Reminder Email

Attachment 10-Survey of Food Safety Programs Follow-up 2 Reminder Email

Attachment 11-Survey of Food Safety Programs Follow-up 3 Reminder Email

1. Respondent Universe and Sampling Methods

Respondent Universe

The respondent universe is all local and state health departments implementing food safety programs (FSPs) in the United States (U.S.). A list of health departments will be obtained from the Centers for Disease Control and Prevention (CDC) Morbidity and Mortality Weekly Report (MMWR) website on state health departments (http://www.cdc.gov/mmwr/international/relres.html) and from the National Association of County and City Health Officials (NACCHO) directory of local health department websites (http://www.naccho.org/about/LHD/). CDC will use this list to generate the sampling frame that will be utilized to select respondents to participate in the Food Safety Program survey. According to these lists, there are 49 state health departments and 3,502 local health departments in the U.S.

Sampling Methods

The goal of the sampling method is to select a sample of health departments with characteristics that are representative of all state and local health departments in the U.S.

We will use a multi-stage cluster design. We will stratify local health departments by size (population served=small or medium/large) and governance structure (centralized or decentralized).Thus, we will have 4 local health department sampling clusters (See Table B.1). We will randomly select health departments to survey within each of these 4 local health department clusters. Because there are only 49 state health departments (Delaware does not have a state health department; Washington, DC will be categorized as a local health department), we will survey all state health departments.

Table B.1. Description of Local Health Department Sampling Clusters

Cluster	Size	Governance
Cluster 1	Small	Centralized
Cluster 2	Small	Decentralized
Cluster 3	Medium/Large	Centralized
Cluster 4	Medium/Large	Decentralized

Sample Size

To obtain a survey sample from which results will be generalizable to the population with a 5% margin of error and a 95% confidence interval, we will need a sample size of 346 local health departments. We expect a 69% response rate; thus, we will need to oversample local health departments by 31% to obtain our desired sample size. Thus, we will contact 502 local health departments (approximately 86 in each cluster). Because of the smaller number of state health departments; we will survey all of them. We expect 69%, or 34, of them to respond. In total, we will contact 551 health departments (502 local and 49 state), and expect responses from approximately 380 (34 state+346 local).

The design and sampling strategy utilized are intended to minimize selection bias and ensure that the sample selected for this survey is representative of all local and state health departments in the U.S.

Response Rate. The most recent NACCHO study (NACCHO, 2012) of local health departments which used methods similar to that of this FSP survey yielded a response rate approaching 69 percent. We expect a similar response rate for this proposed survey.

2. Procedures for the Collection of Information

Data Collection

Once health departments have been selected, CDC staff will send an email to them soliciting their participation in the survey (Attachment 6). The email will describe the survey and its importance, and will also provide informed consent information. It will also contain a link to the electronic version of the survey along with a password to access it. Finally, the email will tell respondents to respond to the email if they prefer to complete a paper version of the survey. CDC staff will mail a paper survey (Attachment 3), and the Survey of Food Safety Programs Fact Sheet (Attachment 7), a self-addressed stamped envelope to those that request a paper version of the survey.

Two weeks after the initial contact by email, CDC staff will follow-up via telephone (Attachment 8)with all respondents who did not respond (either by completing the electronic survey or by requesting a paper survey) to the initial recruitment email. They will also contact recruited local and state health department respondents via email to verify receipt of mailed out survey packets and sent emails. We expect this follow-up contact to occur with approximately 85 percent of respondents (294 local and 28 state health departments).

Four weeks after the telephone follow-up, CDC staff will send a second follow-up email to respondents who have not yet completed the survey (Attachment 10). We expect this follow-up to occur with approximately 40 percent of respondents (138 local and 13 state health departments).

Two weeks after the second email follow-up, CDC staff will follow up via telephone with all respondents who did not yet respond (Attachment 11). We expect this follow-up contact to occur with approximately 10 percent of respondents (35 local and 3 state health departments).

In order to conduct follow-ups with respondents, an identification code will be assigned to respondents completing a paper survey and a password will be assigned to each respondent completing an electronic survey. Each respondent’s code will be printed on the survey sent to them. While this method requires the maintenance of a list of respondents’ names, codes, and passwords, this list will only be used to follow-up with non-respondents. Respondents’ names and the name of the health departments they represent will not be linked with their survey responses. The follow-up list will be stored on a secure computer and only authorized CDC staff will have access to it. We will destroy the list after follow-up activities have been completed.

Participation in the survey will be voluntary. The survey will take up to 2 hours to complete. While this may seem like a relatively long time for data collection, respondents will have up to six weeks to complete the various sections of the survey.

Quality Control Procedures

Respondents who complete the survey electronically will enter their own data. However, data from paper surveys will be entered by CDC staff. Paper survey data will be entered by a CDC staff member, and randomly checked by another CDC staff member to ensure accuracy of data entry.

Potential Biases

Three potential biases exist with the FSP survey. The first potential bias is non-response bias. For example, under-staffed health departments or those in geographic areas where federal collections are less welcome may be less likely to respond to the survey. In this case, our data would not be representative of all health departments. We will address this bias in two ways. First, we will engage in several activities designed to maximize response rates, including recruiter training, extensive follow up with potential respondents, and inclusion of content in recruiting materials that has been shown to increase response rates (see next section for more information). Second, we will attempt to determine if respondents differ systematically from non-respondents. To do this, we will compare respondents and non-respondents on two important characteristics- population size served and governance structure. If significant differences are found in these characteristics, any presentation of the data from this survey will include a discussion of these differences and how they may impact data interpretation.

The second potential bias is selection bias, which occurs when some members of a population are less likely to be included in a survey. For this survey, this bias is minimal for two reasons. First, the population (all local and state health departments in the U.S.) is well-defined and not difficult to identify or contact. Second, our cluster-designed study will minimize selection bias by ensuring that health departments within each of the 5 clusters (4 local health department clusters and 1 state health department cluster) are represented in our study sample.

The third potential source of bias is introduced by concerns that the degree of baseline services offered in a local area and/or the background of the survey respondent influences how the questions on the survey are interpreted.

Any presentation of data from the survey will acknowledge these potential biases and include a discussion of how they impact data interpretation.

Missing Data

This survey will collect retrospective data. Specifically, we will ask respondents to provide data on specific topics for the years 2007-2012. However, we do not expect our data to be impacted by recall bias, because we are not asking respondents to provide us data from memory. Instead, we are asking them to review their records to obtain the data we are asking for. Survey development was informed by local and state health department staff, who helped determine which types of retrospective data would likely be available to survey respondents. Additionally, our pilot respondents also indicated that the data were available to them.

However, we will closely examine our missing data rates during data collection and analysis. We will address missing data concerns through analysis techniques where appropriate and acknowledge these concerns and discus how they impact data interpretation in any presentation of survey data.

3. Methods to Maximize Response Rates and Deal with Nonresponse

We will engage in several activities to maximize survey response rates. First, all recruiters will receive training on the recruiting process. This training will focus on techniques for increase response rates. Second, we will use aggressive follow-up techniques, which have to been shown to lead to higher responses rates (Rea and Parker, 1997) survey. These follow-up techniques include multiple attempts to contact and recruit potential respondents. As described in the Data Collection section above, we will follow up with respondents in attempts to get them to agree to participate in the survey to complete the survey three times. This follow up will be through both telephone and email. Third, recruiting scripts will emphasize an issue that has been shown to increase response rates—the importance of the respondents’ participation in the study.

Additionally, recent NACCHO data collections conducted on local health departments used these techniques and their response rates approached 69% (NACCHO 2012). We hope to obtain the same rate of responses

4. Tests of Procedures or Methods to be Undertaken

All data collection materials were reviewed by the CDC and evaluated in a pilot study by eight key specialists implementing FSPs in the U.S. Results from the pilot study together with other information obtained were used to refine and strengthen the FSP survey. These partners greatly enhance CDC’s experience in collecting data on food safety with this type of instrument and method.

Data analysis plan

The three main objectives of this data collection are to:

1. Describe the current status and activities of local and state FSPs.

2. Describe changes in status and activities that have occurred within FSPs from 2007 to 2012.

3. Describe the relationship between food safety program funding and food safety program status and activities.

To address the first objective of this data collection, we will conduct descriptive analyses (frequencies, means, etc.) on data about:

• Food safety activities

• Workforce capacity and competency

• Workforce staffing

• Financial resources

• Demographics

It is understood that the associations documented must be interpreted with caution, as survey responses will likely be influenced by local variability in the baseline of services offered and differences in background among respondents.

Table B.4.1 contains the survey questions included in these descriptive analyses. Table B.4.2 is a table shell that illustrates how we might analyze and present the descriptive data for Objective 1.

Table B.4.1 Survey of FSPs Survey Questions for Objective 1

FSP Survey Question

Food Safety Activities

1. Is your agency the only agency providing food safety services in your jurisdiction?

1a. If no, which other entities provide food safety services? (e.g., Department of Agriculture, etc.)

2. Does your food safety program follow-up on foodborne illness complaint calls? (examples include

complaints about getting ill from food eaten at a restaurant)

2a1. If yes, about how many foodborne illness calls did you receive in each of the years listed?

2a2. If yes, how many foodborne illness calls required follow up activities in each of the years listed?

2b. What criteria does your program use to determine if a foodborne illness complaint call requires follow up?

3. Does your food safety program follow-up on food safety complaint calls? (examples include complaints about a dirty restaurant)

3a1. If yes, about how many food safety complaint calls did you receive in each of the years listed?

3a2 How many of the food safety calls required follow up activities?

3b. What criteria does your program use to determine if a food safety complaint call requires follow up?

4. Does your food safety program issue permits/licenses?

4a. If yes, which of the following establishments does your food safety program provide permits /licenses to?

5. Does your food safety program conduct inspections?

6. Does your food safety program utilize a risk-based inspection system? (e.g., do high-risk establishments receive more inspections per year than low risk establishments?)

6a. If yes, how does your program define high risk and low risk?

7. Does your food safety program conduct foodborne illness outbreak investigations?

7a. If yes, in which of the following establishments does your food safety program conduct outbreak

investigations?

7c. What activities does your food safety program undertake when investigating a potential foodborne illness

outbreak?

9. Does your food safety program contract out any of its food protection services?

9a. If yes, which of the following services are contracted out?

9b. Why does your food safety program contract out food protection services?

10. Which of the following activities does your food safety program do?

11. Of the following activities, which three do you think are the most important for your food safety program to do?

Workforce Capacity & Competency

12. How many current professional staff members in your food safety program have educational levels in each of the categories listed below?

12. Does your food safety program require any field staff to obtain a food safety license?

13a. If yes, what type of license?

Workforce Staffing

15. What environmental health service(s) besides food services does your food safety program provide?

16. Does your food safety program have the type of staff it needs to provide food protection services?

16a. If no, what type of staff does your food safety program need?

Financial Resources

17. Does your food safety program receive funding from an external funding source? (such as grants,

extramural funding, cooperative agreements)

18. Which of the following services does your food safety program charge food establishments a fee for ?

Demographics

27. Which level of government is your food safety program?

28. What is the total population served by your food safety program?

Table B.4.2- Table Shell: Descriptive data on FSP activities (Objective 1)

	n	%
Does your food safety program take action in response to foodborne illness complaint calls?
Yes	xx	xx
No	xx	xx
Does your food safety program take action in response to food safety complaint calls?
Yes	xx	xx
No	xx	xx
Does your food safety program issue permits/licenses?
Yes	xx	xx
No	xx	xx
Does your food safety program conduct inspections?
Yes	xx	xx
No	xx	xx
Does your food safety program conduct foodborne illness outbreak investigations?
Yes	xx	xx
No	xx	xx

To address the second objective of this data collection, we will conduct analyses that will describe changes from 2007-2012 in the following FSP status and activities:

• Food safety activities

• Workforce staffing

• Financial resources

• Community health

• Demographics

Specifically, we will calculate totals, frequencies and means and the changes in these totals, frequencies and means over time. Table B.4.2 contains the survey questions included in these analyses. Table B.4.4 is a table shell that illustrates how we might analyze and present the descriptive data for Objective 2.

Table B.4.2. FSPs Survey Questions for Objective 2

FSP Survey Question

Food Safety Activities

2a3. Among the foodborne illness calls that required follow up, how many did you have the resources to address in each of the years listed?

3a3. Among the food safety calls that required follow up, how many did you have the resources to track in each of the years listed?

7b. How many foodborne illness outbreak investigations were performed by your food safety

program in each of the years listed below?

7. Are there any services that you believe are important to the effectiveness of your food safety

program that have been reduced or eliminated in the last five years?

8a. If yes, which services?

Workforce Staffing

12. Did changes in staff size affect your food safety program’s ability to provide food protection services in each of the years listed below?

14a. If yes, how did these changes affect your food safety program’s ability to provide food

protection services in each of the years listed below?

Financial Resources

17a. If yes, what percentage of your food safety program’s budget was from sources listed in each of the years below? Please indicate the percentage for each source and for each year in the table below.

19. What was the total budget of your health department in each of the years listed below?

21. What percentage of your food safety program’s budget was spent on staff salaries and benefits in each of the

years listed below?

22. What percentage of your food safety program’s budget was allocated to emergencies in each

of the years listed below?

Community Health

23. How many foodborne illnesses were reported in your jurisdiction in each of the years listed below?

24. How many foodborne illnesses related to outbreaks were reported in your jurisdiction each of the years listed

below?

25. How many foodborne illness outbreaks were reported in your jurisdiction in each of the years listed below?

26. How many people visited your food safety program’s website in each of the years listed below?

Demographics

29. How many full time employees did your food safety program have in each of the years listed

below?

30. How many contract employees did your food safety program have in each of the years listed below?

31. How many seasonal or temporary employees did your food safety program have in each of the years listed

below?

32. How many support staff did your food safety program have in each of the years listed below?

Table B.4.4- Table Shell: Descriptive data on changes in FSP activities (Objective 2)

Food safety activities	N and % of programs with decrease	N and % of programs with increase	N and % of programs with no change
Number of routine inspections conducted by food safety program	xx (xx%)	xx (xx%)	xx (xx%)
Number of foodborne illness outbreak investigations performed by food safety program	xx (xx%)	xx (xx%)	xx (xx%)
Number of food safety and foodborne illness complaint calls performed by food safety program	xx (xx%)	xx (xx%)	xx (xx%)

To address the third objective of this data collection, we will conduct analyses that will describe the relationship between food safety program funding and food safety program status and activities. Specifically, we will examine the relationship between changes in funding and changes in status and services with correlational and odds ratio analyses. Table B.4.5 lists the survey questions included in these analyses. Table B.4.6 is a table shell that illustrates how we might analyze and present the data for Objective 3 examining the relationship between funding and services.

Table B.4.3. FSPs Survey Questions for Objective 3

FSP Survey Question

Changes in Funding

19. What was the total budget of your health department in each of the years listed below?

20. What was the total budget of your food safety program in each of the years listed below?

Changes in Status and Services

Food Safety Activities

2a3. Among the foodborne illness calls that required follow up, how many did you have the resources to address in

each of the years listed?

3a3. Among the food safety calls that required follow up, how many did you have the resources to track in each of

the years listed?

5a. If yes, please list how many establishments your program had the authority to inspect and how many routine

inspections your food safety program conducted in each type of establishment in each of the years listed

below?

7b. How many foodborne illness outbreak investigations were performed by your food safety

program in each of the years listed below?

11a. What could help your food safety program provide better services to the public?

Community Health

23. How many foodborne illnesses were reported in your jurisdiction in each of the years listed below?

24. How many foodborne illnesses related to outbreaks were reported in your jurisdiction each of the years listed

below?

25. How many foodborne illness outbreaks were reported in your jurisdiction in each of the years listed below?

Demographics (Workforce Staffing)

29. How many full time employees did your food safety program have in each of the years listed below?

30. How many contract employees did your food safety program have in each of the years listed below?

31. How many seasonal or temporary employees did your food safety program have in each of the years listed

below?

32. How many support staff did your food safety program have in each of the years listed below?

Table B.4.6- Table Shell: Funding changes associated with whether food safety services decreased from 2007-2012, bivariate analyses (Objective 3)

	Funding from 2007-2012:			Status/Service decreased from 2007-2012
Status/Service	Decreased N (%)	Increased N (%)	Did not change N (%)	Odds ratio (95% CI)*	P
Food Safety Services
The number of routine inspections conducted from 2007-2012	xx (xx%)	xx (xx%)	xx(xx%)	x.xx (x.xx – x.xx)	<x.xxx
The number of foodborne illness outbreak investigations from 2007-2012	xx (xx%)	xx (xx%)	xx(xx%)	x.xx (x.xx – x.xx)	<x.xxx
The number of food safety and foodborne illness complaint calls on which action was taken from 2007-2012	xx (xx%)	xx (xx%)	xx(xx%)	x.xx (x.xx – x.xx)	<x.xxx

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following people are primarily responsible for the design of the survey of food safety programs and will be responsible for data collection and analysis. Elizabeth Armstrong-Mensah and Laura Green Brown are the primary contacts for the data collection, while Brenda Le is the primary contact for the statistical aspects of the survey.

Elizabeth Armstrong-Mensah, PhD.

ORISE Evaluation Fellow

Centers for Disease Control and Prevention, National Center for Environmental Health

[email protected]

770-488-3464

Laura Green Brown, PhD.

Behavioral Scientist

Centers for Disease Control and Prevention, National Center for Environmental Health

[email protected]

770-488-4332

Brenda Le, M.S.

Statistician

Centers for Disease Control and Prevention, National Center for Environmental Health

[email protected]

770-488-3756

References

Kirkland E, Green L, Stone C, Reimann D, Nicholas D, Mason R, et al. 2009. Tomato handling practices in restaurants. J. Food Protect. 72:1692–1698.

Mersdorf S. 2010. 10 Tips to Maximize Survey Response Rates. Available at http://survey.cvent.com/blog/cvent-web-surveys-blog/10-tips-to-maximize-survey-response-rates. Accessed December 14, 2012.

NACCHO. 2012. Local Health Department Job Losses and Program Cuts: State-Level Tables from January/February 2012 Survey.

Sumner S, Brown L, Frick R, Stone C, Carpenter L, Bushnell L, et al. 2011. Factors associated with food workers working while experiencing vomiting or diarrhea. J. Food Protect. 74:215–220.

20