SHI ICR Part A_final_10_12_18

Consumer Research on the Safe Handling Instructions Label for Raw and Partially Cooked Meat and Poultry Products and Labeling Statements for Ready-to-Eat and Not-Ready-to-Eat Products

OMB No. 0583-NEW

Supporting Statement

A. Justification

A.1. Circumstances Making Collection of Information Necessary

Safe handling instructions are required if the meat or poultry component of a product is raw or partially cooked (i.e., not considered ready-to-eat [RTE]) and if the product is destined for household consumers or institutional uses (9 CFR 317.2(l) [meat] and 9 CFR. 381.125(b) [poultry]). The U.S. Department of Agriculture, Food Safety and Inspection Service (USDA, FSIS) established the Safe Handling Instructions (SHI) label for raw and partially cooked meat and poultry products in 1994 (54 FR 14528). Consumer focus groups were conducted to inform the design of the SHI label (Teague & Anderson, 1995; Teague & Anderson, 1993). Since that time, the required design of the SHI label has not been changed.

In response to inquiries from consumer groups and other stakeholders for more information about potential changes to safe handling instructions requirements, FSIS gathered input from members of academia, industry, and consumer stakeholders in November 2013. FSIS presented these suggestions to the National Advisory Committee on Meat and Poultry Inspection (NACMPI) in January 2014. When the SHI label was developed in 1994, minimal internal temperature requirements for determining doneness varied by product. Given product and label size limitations and varying endpoint temperatures, FSIS concluded that “Cook Thoroughly” was the only simple, single statement appropriate to use for all products (54 FR 14538). FSIS now recommends on its website four minimum internal temperatures: one for all poultry (165°F), one for ground meat (160°F), one for all whole-muscle meat (145°F and hold for 3 minutes), and one for fish (145°F). With only four temperature recommendations, the information could be more easily incorporated into the SHI label. Other possible changes to the SHI label include incorporating updated icons and providing a web link or phone number for more information (NACMPI, 2014; Murphy-Jenkins, 2014).

The NACMPI Subcommittee on Food Handling Labels recommended that FSIS pursue changes in the existing SHI label and conduct consumer research to determine the effectiveness of any revisions to the SHI label (NACMPI, 2014). In November 2014, FSIS conducted a strategic planning session to elicit input from FSIS senior leadership on potential revisions to the SHI label, the impact any revisions may have on consumers and industry, and pitfalls to consider. The findings from this session underscored the need to conduct consumer research to determine consumers’ reactions to the current SHI label and potential revisions.

In 2015, FSIS conducted six consumer focus groups (OMB No. 0583-0166; 11/30/2017) to evaluate understanding of the current SHI label and responses to possible revisions. The focus groups revealed that consumers would find certain revisions to the SHI label useful. Participants suggested changes to improve comprehension and adherence to recommended safe handling practices (e.g., add recommendation to use a food thermometer and endpoint temperatures for different cuts of meat and poultry) (Cates et al., 2016).

Additionally, although FSIS has issued guidance to the industry on the modifications that are necessary for the labeling of uncooked boneless, breaded chicken products that may appear RTE because of their cooked appearance (USDA, FSIS, n.d.), there have been reports of illnesses associated with these products even when the labels follow the guidance. In May 2016, the National Chicken Council (NCC) submitted a petition requesting that FSIS establish regulations for the labeling and validated cooking instructions for not-ready-to-eat (NRTE) stuffed chicken breast products. In their petition, the NCC also suggested that research be conducted to examine consumers’ handling of NRTE stuffed chicken breast products as well as their understanding of relevant labeling statements and validated cooking instructions. The American Frozen Food Institute, an industry trade association, and the Safe Food Coalition, a coalition of consumer advocacy organizations, submitted comments in support of the petition (NCC, 2016; American Frozen Food Institute, 2016). Prior to this petition and comments, during the March 2016 NACMPI meeting, the committee reviewed and discussed whether FSIS should pursue proposing mandatory features on the label of processed NRTE products that may appear to be fully cooked (e.g., are breaded or have grill marks). The committee recommended that FSIS require statements such as “Raw, “Uncooked,” or “Ready to Cook” on the labels of raw products that may appear ready to eat so it is clear that these products require cooking to a proper internal temperature before eating (USDA, FSIS, 2016). The committee also recommended that FSIS conduct consumer research to understand the optimal messaging and design of packaging to ensure consumers properly understand that NRTE products that may appear to be fully cooked need to be cooked for lethality. The committee stated that such labeling may help consumers properly distinguish between NRTE products, which require a lethality step, and RTE products, which do not require a lethality step; thus, the committee stated that this labeling may help consumers safely prepare NRTE products. Specifically, the committee suggested that FSIS conduct consumer research to evaluate the effectiveness of possible locations for point of purchase labeling information and various color options, fonts, and other display options.

To assess whether revisions are needed to the SHI label required on all raw and partially cooked products and to evaluate the ability of consumers to properly discern between NRTE and RTE products and how labeling for these products can be improved, FSIS is requesting approval for a new information collection to conduct consumer behavior research. This research will include a web-based experimental study and a behavior change study, which includes three components: an observational meal preparation experiment, eye-tracking study, and in-depth interviews (IDIs). The research will help inform whether potential revisions to the current SHI label are needed and assess whether a label revision would be likely to improve consumer behaviors related to safely preparing raw and partially cooked meat and poultry products. The study will also collect information on consumer use and understanding of labeling for RTE and NRTE meat and poultry products.

A.2. How, by Whom, and Purpose Information Is to Be Used

F SIS has contracted with RTI International and its subcontractor, North Carolina State University (NCSU), to conduct the consumer behavior research. The primary objective of this research is to collect information to determine possible revisions to the current SHI label (see insert) that will improve consumers’ adherence to recommended food safety practices for raw and partially cooked meat and poultry products. A secondary objective is to evaluate the ability of consumers to properly discern between NRTE and RTE products and how labeling for these products can be improved. To address the study’s objectives, the study team will undertake the following activities:

• Conduct a web-based experimental study to assess consumer salience or attention to alternative SHI labels, including alternative icons and messages designed for specific safe handling instructions. The results of the web-based experimental study will be used to identify alternative SHI labels to test in the subsequent behavior change study.

• Conduct a behavior change study to assess and compare consumer behavior in response to the current SHI label (control) and three alternative SHI labels. Participants in the behavior change study will take part in the following three activities:

  • An observational meal preparation experiment while wearing a mobile eye-tracking device in a test kitchen to evaluate the effectiveness of alternative SHI labels relative to the current SHI label on consumers’ adherence to recommended safe handling instructions.

  • An eye-tracking study to obtain quantitative data to measure visual salience in response to the current and alternative SHI labels using mock food packages (i.e., stimuli).

  • IDIs to gather information on participants’ knowledge and perceptions regarding their handling of RTE and NRTE meat and poultry products, in particular their ability to properly discern between RTE and NRTE products and to ensure that NRTE products that may appear to be ready-to-eat are thoroughly cooked.

Web-Based Experimental Study

RTI will subcontract with Lightspeed (www.lightspeedresearch.com) to program the survey instrument and administer the data collection for the web-based experimental study. Respondents will be selected from Lightspeed’s consumer panel, which consists of approximately 1.5 million adults who are double opted-in. The double opted-in process is as follows. First, a prospective respondent clicks on a link from a panel ad and is directed to the panel registration survey. Second, the prospective panelist must complete the panel registration survey (which collects demographic and household information) and must pass several validation checks (e.g., verify postal address) and agree to the website’s Terms and Conditions and Privacy Policy to become panel members.

Lightspeed will set inbound quotas to obtain a sample of adults that mirrors the U.S. population with respect to education, age, race, and English vs. Spanish speaking. Selected panelists will receive an email invitation (Appendix A) and interested panelists will be screened for eligibility. The study will include 3,600 participants with approximately n = 133 exposed to each of 27 SHI labels created by fully crossing the three primary study features—label shape, safe handling instruction text, and safe handling icons—each of which will have three options. The survey will take up to 20 minutes to complete and be available in English and Spanish (see Appendix B). To encourage response, up to three e-mail reminders will be sent to nonrespondents (see Appendix C).

The primary aim of the experimental study is to test 27 mock SHI labels that vary visual design elements to determine which labels are most salient to consumers. Label salience (i.e., participant degree of attention to the label) will be assessed using a limited-time exposure approach with cued recall questions. The data from the experimental study will be analyzed to identify the five SHI labels that best attract respondents’ attention; from these, three labels will be selected for further testing in the behavior change study.

Behavior Change Study

To provide geographic diversity, data collection will be conducted at test kitchens, each similar in design and layout, in four locations across the country: (1) Wake, Orange, Durham, and/or Johnston Counties, NC; (2) Brazos County, TX; (3) Yolo County, CA; and (4) Providence County, RI. Within each location, study participants will be randomly assigned to one of four conditions: three alternative SHI labels or the current SHI label as a control. A total of 480 adults will participate in the behavior change study; of these, 360 will participate in North Carolina and 120 will participate in the other three locations (40 per location). Completing all components of the behavior change study (observational meal preparation experiment, eye-tracking study, and IDIs) will take between 2 and 2.5 hours.

Study participants will be recruited in the four locations using convenience sampling by posting ads in social media outlets (see Appendix D), sending emails to Expanded Food and Nutrition Education Program participants to reach low-income consumers (North Carolina location only) (see Appendix E), and posting notices about the study in various locations (see Appendix F). Additionally, the study team will work with local market research facilities in each location to use outbound recruiting to recruit individuals with specific demographics that may be challenging to recruit using social media (e.g., individuals with a high school education or less and older adults). Recruitment materials will direct prospective participants to either call or email a study team member to be screened for eligibility or access a web link that will host the screening questionnaire (see Appendix G). Inbound quotas will be used to ensure a diverse sample of participants with respect to age, race, ethnicity, education level, and presence of a child in the household. Study enrollment will include contact by phone (see Appendix H) to schedule an appointment with individuals who meet the eligibility criteria followed by a confirmation email or letter (see Appendix I) and a reminder call 1 or 2 days before the scheduled appointment (see Appendix J).

Upon arrival at the test kitchen, participants will provide informed consent (Appendix K) and be fitted with the mobile eye-tracking device. Participants will then watch an instructional video with information on the study and listen to additional instructions provided by a study team member including a walk-through of the test kitchen (see Appendix L). Participants will be provided with the recipes and ingredients needed to prepare three dishes: (1) gluten-free pasta and meatballs using frozen, preformed raw meatballs (under the guise that the dish is for an individual who is on a gluten-free diet), (2) gluten-free pasta and meatballs using raw ground beef, and (3) cherry tomato garnish. Both the packages of the raw ground beef and the raw, frozen, preformed meatballs will bear the assigned SHI label. Video recording of food handling and meal preparation will begin as soon as the participant enters the test kitchen and will end after the participant leaves the test kitchen. Participants’ cleaning and sanitizing of equipment and environment before and after meal preparation will also be recorded. Meal preparation is expected to take 50 to 80 minutes to complete.

Trained coders will watch the videos and code behaviors using an observation rubric (see Appendix M) to evaluate participants’ adherence to the safe handling instructions listed on the SHI label (e.g., use a food thermometer and wash hands after handling raw meat). The coded data will be used to calculate a label adherence score. The videos captured by the mobile eye-tracking device will be reviewed to determine (1) the frequency of viewing the SHI label and other labeling information during meal preparation and (2) if a thermometer was used, the measured endpoint temperature of the meat product.

In addition to wearing the eye-tracking device during the meal preparation experiment, participants will complete an eye-tracking study after meal preparation is complete. The eye-tracking study will provide quantitative data on several visual and attentional processes related to consumer interaction with and use of labeling on RTE and NRTE products. The eye-tracking study will address two primary research questions:

• Which version of the SHI label is most often attended to when consumers look at a busy food package?

• Can consumers properly distinguish between RTE and NRTE products?

The eye-tracking study will collect data from all 480 individuals participating in the meal preparation experiment, with 120 participants viewing each of the SHI labels (current SHI label or one of three alternatives as assigned for the meal preparation experiment). Participants will view four NRTE products (two of which appear RTE) that bear the assigned SHI label (and other required labeling statements), as well as two RTE products bearing the required labeling statements (for a total of six mock products or stimuli). Data collectors will use a script to direct participants’ attention to each product and to complete several tasks to determine which version of the SHI label is most often attended on a meat and poultry package and to assess whether participants can properly distinguish between RTE and NRTE products that appear to be ready to eat (see Appendix N). This study component will take up to 30 minutes to complete. The eye-tracking data will be used to measure several potential outcomes, including visual observations of the area of interest (AOI), such as time to first view and total view time, distribution of attention, and percentage of attention.

The final component of the behavior change study will be the IDIs, which will provide context to the quantitative data and elaborate the process underlying the role of labeling and food safety messaging on cooking practices. Data collectors will use a structured interview guide to conduct the interviews, which will take up to 30 minutes to complete (see Appendix N). The interview will include follow-up questions to the meal preparation experiment to assess reasons for following or not following the recommended safe handling instructions and a set of questions to understand how participants determine whether a meat or poultry product needs to be cooked for lethality.

Statistical analyses comparing the label adherence scores for the control (i.e., current SHI label) and treatment groups (three alternative SHI label versions) will be conducted to identify the label that may most effectively lead to consumers following the safe handling practices on the label. The results of this analysis, along with findings from the eye-tracking study and the IDIs, will provide information on whether revising the SHI label would improve consumers’ safe handling practices when preparing raw or partially cooked meat or poultry products. The Agency will use the findings of this consumer research, along with findings from a cost-benefit analysis, to determine if revisions to the current SHI label are needed.

A.3. Use of Improved Information Technology

Web-Based Experimental Study

The experimental study will use web-based data collection in lieu of in-person data collection, which will greatly reduce the burden on participants because they will not be required to travel to a central location to complete the study. This approach will also expedite the timeliness of data collection because a web-based study will take several weeks to administer versus several months for an in-person study. Furthermore, the use of web-based data collection with participants located throughout the United States will allow the study to reach a more diverse study population than would otherwise be possible using an in-person approach and be significantly less costly to implement.

Behavior Change Study

Most participants will be recruited via social media and have the option to complete a web-based questionnaire for screening, which is less burdensome and more cost-effective than requiring all prospective participants to call research staff to be screened for eligibility. Prospective participants who complete the web-based questionnaire and who meet the eligibility requirements for study participation will still need to be contacted via phone by research staff to schedule an appointment for completing the study.

As part of the behavior change study, all participants will wear a mobile eye-tracking device to collect information on consumers’ attention to product labels during the observational meal preparation experiment. All participants will also wear the eye-tracking device while interacting with the study stimuli (mock meat and poultry products) to collect quantitative data on visual salience for the current vs. alternative SHI labels. This technology is nonintrusive and allows participants to interact with the experimental product stimuli freely.

A.4. Efforts to Identify and Avoid Duplication

FSIS is currently conducting an observational study (OMB No. 0583-0169: In-Home Food Safety Behaviors and Consumer Education: Annual Observational Study) to evaluate its communication and outreach efforts on consumer behaviors. Although the methods for this study are similar to the proposed observational meal preparation experiment component of the behavior change study, the outcomes of the two studies are different. Thus, the findings from the annual observational study cannot be used to answer the research questions of interest in the proposed information collection. Based on a review of the current literature, the Agency concluded that the existing knowledge base on the SHI label does not meet the Agency’s informational needs.

A.5. Methods to Minimize Burden on Small Business Entities

No small businesses will be involved in this collection.

A.6. Consequences of Less Frequent Data Collection

This is a one-time data collection. Without this study, FSIS will not have the needed information to assess whether revisions are needed to the SHI label required on all raw and partially cooked meat and poultry products. The lack of information would impede the Agency’s ability to provide consumers with more useful and actionable information on how to safely prepare raw and partially cooked meat and poultry products at home. Such information could lead to improved consumer practices, thus potentially help reduce foodborne illness in the United States.

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 that Would Cause the Information Collection to be Conducted in a Manner:

• requiring respondents to report information to the Agency more often than quarterly;

• requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• requiring respondents to submit more than an original and two copies of any docu­ment;

• requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than 3 years;

• in connection with a statistical survey, that is not designed to produce valid and reli­able results that can be generalized to the universe of study;

• requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• requiring respondents to submit proprietary trade secret or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

This information collection fully complies with 5 CFR 1320.5(d) (2). There are no special circumstances associated with this information collection that would be inconsistent with the regulation.

A.8. Consultations with Persons Outside the Agency

In accordance with the Paperwork Reduction Act, FSIS published a 60-day notice requesting comments regarding this information collection request (83 FR 34101; 07/19/2018) and received a total of two comments. The comment from the Center for Foodborne Illness Research & Prevention (CFI) and the Consumer Federation of America (CFA) was generally supportive of the information collection and identified specific information to include on a revised SHI label. Many of the modifications requested by CFI and CFA will be considered in the design of the alternative labels to be tested in the web-based experimental study (e.g., providing end-point temperatures, including a website address for more information, and providing instructions to use a thermometer to verify the product has reached the recommended internal temperature). The results of the web-based experimental study and the behavior change study will be used to inform the final redesign of the SHI label. Additionally, the CFI and the CFA encouraged FSIS to quickly design and implement rules that effectively prevent consumers from confusing raw and cooked products. The proposed study will also collect information on consumer understanding of labeling on RTE and NRTE products and assess whether consumers understand the difference between RTE and NRTE products to ensure proper cooking of NRTE products. The Agency will use the results of this research to determine whether modifications are needed to current labeling requirements for NRTE products, in particular products that are raw but appear to be fully cooked.

The comment from the North American Meat Institute (NAMI) was generally supportive and identified two concerns with the information collection. First, NAMI noted that if the study shows negligible change in consumer behaviors, a labeling change is not justified and the current SHI label should not be amended. Following the completion of the consumer research, the Agency will use the study results in a cost-benefit analysis to assess whether the benefits from revising the SHI label (i.e., the potential reduction in foodborne illness) exceeds the cost to industry to revise the label. The second concern noted was that the Agency should allow flexibility, especially with regard to incorporating minimum internal temperatures, and consider the feasibility of labeling options when developing the consumer research. The label designs to be tested in the consumer research will be limited to options that are considered realistic for industry to implement so as to not place undue burden on industry.

The National Agricultural Statistics Service also reviewed the information collection and made supportive comments.

A.9. Payments to Respondents

Web-Based Experimental Study

To encourage panelists to participate in surveys, Lightspeed offers its panel members reward points. Upon completing a survey, points are deposited immediately into a panelist’s account. The number of points awarded is based on survey length, complexity, and incidence rate. For this study, respondents will receive 100 points. Panelists may redeem their accumulated points for online gift certificates, merchandise, and PayPal gift card deposits.

Behavior Change Study

We understand that the OMB guidance about incentives for participation in research is based on the principles of the 2006 memo “Guidance on Agency Survey and Statistical Information Collections.” We propose providing each participant a $100 gift card and a small gift (food thermometer valued at $5.38 and magnet valued at $0.23) to maximize the show rate for the behavior change study and to improve data quality. Additionally, participation in the behavior change study will require substantial commitment and investment of time on the part of the participant, in that they must make a commitment to attend the study at a certain time on a specific date. Participation also requires participants to travel to a designated location, with the average commute in the United States metropolitan areas estimated at about 26.1 minutes (U.S. Census Bureau, 2017) and may also require that the participant obtain child care for a fee. Thus, providing incentives has long been considered a standard practice in conducting consumer research.

Table A-1 provides a breakdown of the cost to participate in the behavior change study by whether the participant has a child(en) requiring child care for a paid fee. Although the cost to participate varies depending on whether paid child care is needed (from $28.45 to $85.53), we propose to offer all participants the same incentive amount ($100) to avoid introducing selection bias that might occur by offering different incentive amounts to individuals with and without children in their households.

The proposed $100 incentive amount is in line with the industry standard. These industry-standard stipends help ensure that respondents can be recruited efficiently and ensure their arrival and participation in the study. These standards also exist to provide fair compensation for costs incurred by participants while participating in the study (i.e., travel and child care expenses). In addition to covering reasonable costs of participation, payment to participants is necessary to ensure that enough respondents from the target population participate in the study. Payment to participants encourages potential participants to agree to allocate their time to the study and maintain that commitment on the day of the research.

Table A-1. Estimated Cost to Participants of Taking Part in the Behavior Change Study by Households with and without Children

Cost Component	Estimated Number of Units	Unit Cost	Total Cost
Households with Children
Cost to travel to/from test kitchen	52.2 milesa,b	$0.545/milec	$28.45
Cost of child care during travel time (1 hour round trip) and attending study (15 minutes before appointment to park, up to 2.5 hours for the study, and 15 minutes after study to checkout and return to vehicle)	4.0 hours	$14.27/hourd	$57.08
Total			$85.53
Households without Children
Cost to travel to/from test kitchen	52.2 milesa,b	$0.545/milec	$28.45
Total			$28.45

^a Source: https://www.census.gov/library/visualizations/interactive/travel-time.html

^b The average commute in a U.S. metropolitan areas is an estimated 26.1 minutes to a designated location. Assuming participants travel 60 miles per hour, the total number of roundtrip miles is 52.2 miles.

^c Source: http://www.gsa.gov/portal/content/100715

^d Source: https://www.care.com/c/stories/2423/how-much-does-child-care-cost/

Offering no incentive or a smaller incentive could potentially exclude sections of the population who cannot participate in the study, either because of the cost of child care and/or travel or the cost of missing work. Excluding sections of the population would limit the information that would be gained through the study and potentially bias the information needed to address the research questions of interest, thus negatively affecting data quality. Moreover, the $100 incentive payment proposed is consistent with what OMB has approved for other consumer food safety studies, when adjusted for the estimated participant burden, for example, OMB No. 0583-0169: In-Home Food Safety Behaviors and Consumer Education: Annual Observational Study; OMB No. 0583-0166: Professional Services to Support Requirements Gathering Sessions for Safe Food Handling Instructions (SHI); OMB No. 0583-0141: Consumer Research, Assessing the Effectiveness and Application of Public Health Messages Affecting Consumer Behavior Regarding Food Safety; and OMB No. 0584-0173: Food Safety Behaviors and Consumer Education: Focus Group Research.

We anticipate that without the gift card incentive and gift, we would need to screen more people to achieve the desired cooperation rate. The current estimated annualized burden for the participant screening is about 8 minutes (0.133 hours) for the study. Without any incentive, we expect that twice the number of individuals would need to be screened so that the total burden for screening would be doubled (452 vs. 226 hours). The cost to respondents and the federal government would increase accordingly.

A.10. Assurance of Confidentiality

The privacy of study participants will be ensured by using an independent contractor to collect the information, by enacting procedures to prevent unauthorized access to participant data, and by preventing the public disclosure of the responses of individual participants.

Web-Based Experimental Study

As part of their registration process, Lightspeed requires panelists to agree to their privacy policy, which includes privacy standards, rights, and information usage. A link to Lightspeed’s privacy policy is always included in study email invitations, is accessible via their panel website, and can be found at http://www.lightspeedresearch.com/privacy-policy/. Lightspeed complies with the research industry standards from the following organizations: the European Society for Opinion and Market Research, Insights Association (formally the Council of American Survey Research Organizations and MRA), Advertising Research Foundation, American Marketing Association, Market Research Society, and Association of Market and Social Research Organisations.

Lightspeed has in place physical, electronic, and managerial procedures to ensure its networks and applications are highly secure and client data are protected. Physical security features include entrances requiring security clearances, secure smart card access, on-site security officers, video surveillance, generator-backed power supply, fire suppression systems with early warning smoke detection, and an HVAC-controlled environment.

Lightspeed uses several layers of network security to prevent unauthorized network access to systems and data. Antivirus software is installed on all servers and workstations. Internet security is provided by the following layered network access architecture: multilayer firewall architecture; data center systems managed via private, firewalled, backend access; a variety of threat monitoring, detection, and intrusion prevention systems (IPS) measures deployed throughout the network; and automated monitoring of network and server performance for WAN, LAN, and production servers.

All data are secured on database servers that only reside on private, backend servers that are behind layered firewall architecture. Data are never stored on a public network or outside the data tier. Relational database management systems (RDBMS) access is strictly controlled and limited to only a few authorized users whose access is limited to the minimum necessary to accomplish administrative tasks. Web and application servers communicate with the RDBMS only via a private network segment with a multilayer firewall architecture in place. Access control is provided to secure data directories. All client-specific data are stored in restricted access data directories controlled by access control lists.

Lightspeed will not share personal information regarding panel members with any third party without the participant’s permission unless it is required by law to protect their rights or to comply with judicial proceedings, court orders, or other legal processes. RTI and FSIS will not have access to panel members’ personal information. No identifying information will be included in the data files delivered to the Agency.

Information regarding informed consent, including assurances of data privacy and security, will be provided on the first screen of the survey (see Appendix B). RTI’s Institutional Review Board reviewed and determined the study is exempt from IRB review (see Appendix O). A Privacy Impact Analysis (PIA) is not required by FSIS or RTI International (the primary contractor on this project).

Behavior Change Study

The only information in identifiable form (IIF) that will be obtained are the participants’ names, phone numbers, and email or mailing addresses for scheduling the appointment for the behavior change study, mailing confirmation letters, and making reminder phone calls. NCSU will maintain this IIF information. These personal identifiers will not be linked to participant data and will not be shared with FSIS or RTI.

Participation in the behavior change study is voluntary, and participants will be advised that their responses will be treated in a secure manner and will not be linked to their names. The digital video and audio tapes will be stored on a password-protected share drive, accessible only to project staff.

Assurances of data privacy and security are documented in the informed consent form (see Appendix K). NCSU’s and RTI’s Institutional Review Boards reviewed and approved the study protocol and instruments (see Appendix P). A PIA is not required by FSIS or RTI International (the primary contractor on this project).

A.11. Justification for Questions of Sensitive Nature

Participants in the web-based experimental study and behavior change study will not be asked any questions that are personal or sensitive in nature. For both the web-based experimental study and the behavior change study, participants will be asked if they or any household members have been diagnosed with cancer, diabetes, or other conditions that weaken the immune system. Individuals will not be asked for a specific diagnosis. Immunocompromised individuals are considered at risk for foodborne illness; thus, it is important to collect information on their or their caregivers’ food handling behaviors

A.12. Estimates of Respondent Burden

The total estimated burden for the web-based experimental study is 3,623.3 hours, and the total estimated burden for the behavior change study is 1,491.9 hours, for a total of 5,115.2 hours (see Table A-2). The annualized cost to all respondents for the proposed information collection is $92,687.42 (5,115.2 x $18.12 per hour) (the 2017 U.S. median hourly wage rate¹).

Web-Based Experiment Study

The total burden for the web-based experimental study is 3,623.3 hours. To achieve 100 completed surveys during the pretest, Lightspeed will send email invitations to 1,700 randomly selected panel members. To achieve 3,600 completed surveys during the full-scale study, Lightspeed will send email invitations to 70,000 randomly selected panel members. The invitation email for the pretest and the full-scale survey is expected to take 2 minutes to read (0.033 hour). The survey is expected to take 20 minutes to complete (0.333 hour).

Table A-2. Estimated Annual Reporting Burden

Portion of Study	Appendix(s) for Data Collection Instrument or Form		No. of Respondents					Annual Frequency per Response				Total Annual Responses		Hours per Response			Total Hours
Web-Based Experimental Study
Pretest invitation	A		1,700					1				1,700		0.033 (2 min.)			56.7
Pretest	B		100					1				100		0.333 (20 min.)			33.3
Survey invitation	A		70,000		1				70,000				0.033 (2 min.)			2,333.3
Survey	B		3,600		1				3,600				0.333 (20 min.)			1,200.0
Total																3,623.3
Behavior Change Study
Recruitment information	D, E, F		—		—				—				—			—
Screening questionnaire	G		1,695		1				1,695				0.133 (8 min.)			226.0
Appointment phone script, confirmation emails, reminder phone script	H, I, J		565		1				565				0.117 (7 min.)			65.9
Consent form and video		K, L		480		1				480					0.167 (10 min.)			80.0
Meal preparation, eye-tracking, and IDI		L, N		480			1				480				2.333 (140 min.)			1,120.0
Total																		1,491.9
Total																		5,115.2

Behavior Change Study

The total burden for the behavior change study is 1,491.9 hours. The study will be advertised via social media, emails, and postings in grocery stores and supplemented with outbound recruiting. Prospective participants will complete a screening questionnaire by phone or via a web link to determine eligibility. We estimate that 1,695 individuals will complete the screener and 565 (33%) will be eligible and subsequently contacted by phone to schedule an appointment. Of these, we estimate that 480 (85%) will participate in the behavior change study. Each screening is expected to take 8 minutes (0.133 hour). It is expected to take each participant a total of 7 minutes (0.117 hour) to read or listen to each appointment call/confirmation email/reminder call. It is expected to take each participant 10 minutes (0.167 hour) to read the informed consent form and watch the instructional video and up to 140 minutes (2.333 hours) to complete the behavior change study, which includes an observational meal preparation experiment (50 to 80 minutes), an eye-tracking study (30 minutes), and an IDI (30 minutes).

A.13. Capital and Start-Up Costs and Subsequent Maintenance

No capital, start-up, operating, or maintenance costs are associated with this information collection.

A.14. Annual Cost to Federal Government

The estimated total cost to the federal government for this information collection is $1,478,415. The costs arise from the time spent by the contractor to develop the study design and materials, collect the data, analyze the data, and prepare and deliver a final report.

A.15. Reasons for Changes in Burden

This is a new information collection.

A.16. Tabulation, Analysis, and Publication

The planned schedule for the information collection survey is shown in Table A-3. Once OMB approval is received, we will begin the data collection activities for the web-based experimental study. The contractor will provide FSIS a report that summarizes the study methods and results within 45 days of completion of the data collection. Appropriate statistical analyses will be used to analyze the survey data and identify the three label designs that best capture consumers’ attention for testing in the behavior change study. Within 30 days of providing the web-based experimental study summary report, we will begin the data collection activities for the behavior change study. The contractor will provide FSIS a report that summarizes the study methods and results within 90 days of completing the data collection. For the meal preparation experiment, the contractor will conduct statistical analyses comparing the label adherence scores for the control (current SHI label) and three treatment groups to identify the label that may most effectively lead to consumers following the safe handling practices on the label. All eye-tracking data will be reviewed, coded, and processed to produce eye-tracking metrics for each AOI, including total time spent viewing each AOI, which will be used in additional statistical analyses and to create heat maps and gaze plots. Finally, data from the IDIs will be analyzed using QSR International’s NVivo 11 qualitative data analysis using a thematic content analysis approach.

Table A-3. Project Schedule

Date	Activity
Within 15 days following OMB approval	Begin data collection for web-based experimental study
Within 60 days following OMB approval	Complete data collection for web-based experimental study
Within 90 days following OMB approval	Complete summary report on web-based experimental study
Within 30 days following report on web-based experimental study	Begin data collection for behavior change study
Within 180 days following report on web-based experimental study	Complete data collection for behavior change study
Within 240 days following report on web-based experimental study	Complete summary report on behavior change study

Using the findings from the behavior change study, we will construct logistic regression models to examine the association between attention to SHI labels (from the eye-tracking study) and proper execution of each of the safe food handling behaviors (from the meal preparation experiment). Poisson regression models will also be used to examine the relationship between attention to SHI labels and the number of properly executed safe food handling behaviors. These data will provide empirical evidence on the SHI label option that is most effective at encouraging consumers to follow recommended safe handling practices for raw and partially cooked meat and poultry products.

Dissemination of the study results may include internal briefings, presentations, and reports and posting on FSIS’s website and potentially a manuscript for publication in a peer-reviewed journal.

A.17. OMB Approval Number Display

The OMB approval and expiration date will be displayed on all materials associated with the study. No exemption is requested.

A.18. Exceptions to the Certification

There are no exceptions to the certification.

References

American Frozen Food Institute. (2016, August 17). Letter to USDA, FSIS on Petition Number 16-03; Petition to Establish Regulations for the Labeling and Validated Cooking Instructions for Not-Ready-to-Eat Stuffed Chicken Breast Products That Appear Ready-to-Eat. Retrieved from https://www.fsis.usda.gov/wps/wcm/connect/802587a4-f31f-4293-bad6-5eafa120ef49/16-03-Support-Ltr-081716.pdf?MOD=AJPERES

Cates, S. C., Kosa, K. M., & Muth, M. K. (2016). Professional services to support requirements gathering sessions for safe food handling instructions: Focus group research. Final Report (Contract No. AG-3A94-13-0003, Order AF3A94-K-14-0060). Prepared for U.S. Department of Agriculture, Food Safety and Inspection Service, Office of Public Affairs and Consumer Education.

Murphy-Jenkins, R. (2014, January). Food safety handling labels. Washington, DC: USDA, FSIS. Retrieved from http://www.fsis.usda.gov/wps/wcm/connect/9adeea60-4d52-4eb6-a15a-8e15b6873192/NACMPI-Food-Safety-Handling-Labels-010714.pdf?MOD=AJPERES

National Advisory Committee on Meat and Poultry Inspection. (2014). Report and recommendations based on FSIS proposed questions. Subcommittee on Food Safety Handling Labels. Washington, DC: NACMPI. Retrieved from http://www.fsis.usda.gov/wps/wcm/connect/70d59a26-32bb-4d25-9050-462b0b0cacb5/Food-Safe-Handling-Labels-NACMPI.pdf?MOD=AJPERES

National Chicken Council. (2016, May 24). Letter to USDA, FSIS on Petition to Establish Regulations for the Labeling and Validated Cooking Instructions for Not-Ready-to-Eat Stuffed Chicken Breast Products That Appear Ready-to-Eat. https://www.fsis.usda.gov/wps/wcm/connect/e63c4755-2965-4872-b834-fcd798f58488/16-03-National-Chicken-Council.pdf?MOD=AJPEREShttps://www.fsis.usda.gov/wps/portal/fsis/topics/regulations/advisory-committees/nacmpi-reports

Teague, J. L., & Anderson, D. W. (1993). Consumer focus groups on safe-handling labels for the pathogen reduction effort. Final report prepared for USDA, Food Safety and Inspection Service. Research Triangle Park, NC: RTI International.

Teague, J. L., & Anderson, D. W. (1995). Consumer preferences for safe handling labels on meat and poultry. The Journal of Consumer Affairs, 29(1), 108–127.

U.S. Census Bureau. (2017). Average One-Way Commuting Time by Metropolitan Areas. Retrieved from https://www.census.gov/library/visualizations/interactive/travel-time.html

U.S. Department of Agriculture, Food Safety and Inspection Service. (n.d.). Labeling Policy Guidance Uncooked, Breaded, Boneless Poultry Products. Retrieved from https://www.fsis.usda.gov/wps/wcm/connect/6d7b7f70-e11b-4861-adc8-6f3269c3eeec/Labeling_Policy_Guidance_Uncooked_Breaded_Boneless_Poultry_Products.pdf?MOD=AJPERES

U.S. Department of Agriculture, Food Safety and Inspection Service. (2016). NACMPI Reports – 2016. Retrieved from https://www.fsis.usda.gov/wps/portal/fsis/topics/regulations/advisory-committees/nacmpi-reports

1 Bureau of Labor Statistics, U.S. Department of Labor, Occupational Employment Statistics, Accessed 5/31/2018, [https://www.bls.gov/oes/current/oes_nat.htm]

27