CNPP OMB Cover Memo 0584-0523_Task 2_quant 8 19 PRAO recommendations 8-27-15 RGauclean

Memorandum

Date: August 28, 2015

To: Steph Tatham, OMB Desk Officer

Through: Rachelle Ragland Greene, Food and Nutrition Service, Information Clearance Officer; Ruth Brown OCIO Desk Officer

From: Jessica Larson, Senior Nutritionist, Center for Nutrition Policy and Promotion (CNPP)

USDA – Food, Nutrition and Consumer Services

Re: Under Approved Generic OMB Clearance No. 0584-0523 Request for Approval to Perform Formative Research to Test Messages Developed to Help Communicate 2015 Dietary Guidelines Recommendations

Center for Nutrition Policy and Promotion (CNPP) of the USDA Food, Nutrition, and Consumer Services is requesting approval for formative research under Approved Generic OMB Clearance No. 0584-0523.

This request is to acquire clearance to conduct formative research in order to develop message concepts to communicate recommendations stemming from the Dietary Guidelines for Americans 2015.

The following information is provided for your review:

1. Title of the Project: CNPP Message Concept Web Experiment

2. OMB Control Number and Expiration Date: 0584-0523, January 31,2016

3. Public Affected by this Project: Individuals and Households. The research will be conducted with Americans over the ages of 18 and will include both English- and Spanish-speaking individuals. See section 7, Project Purpose, Methodology & Formative Research Design for a description of the number of participants necessary for the research methodology. The definitions of the public affected by this project are identified in the screening instruments (see Attachments B-1 and B-2).

4. Number of Respondents:

CNPP, along with our contractors, will contact approximately 1,577 individuals and households to recruit:

• 1,200 participants completing the survey in English or Spanish.

  • 300 will take a survey in Spanish

  • 900 will take a survey in English

Among the 1,577 individuals initially contacted, we estimated that 315 (20%) will not participate in any phase of research, whereas 1,262 (80%) will go on to participate in the screener. We further estimate that among 1,262 participants who would a respond to a screener, 62 (5%) will not be eligible to complete the survey and 1,200 (95%) will be eligible to complete the survey.

Table 1: Number of respondents

Instrument	Total Number of Participants	Number of respondents	Proportion of Responders	Number of non-respondents	Proportion of non-respondents
Invitation
Invitation for English Language Survey	1183	947	80%	236	20%
Invitation for Spanish Language Survey	394	315	80%	79	20%
Screener
Invitation for English Language Survey	947	947	100%	0	0%
Invitation for Spanish Language Survey	315	315	100%	0	0%
Survey
English Language Survey	947	900	95%	47	5%
Spanish Language Survey	315	300	95%	15	5%
	1577	1200		377

5. Time Needed Per Response:

Participants will engage in a 15 minute web-based survey (http://dimensions.edgeresearch.com/e.asp?p=APT1501T&S=C) addressing topics outlined in the survey instrument (Attachments C-1 and C-2 and C-1A for screen shot of the survey).

The table summary of estimated burden hours is presented below.

Table 2: Burden Hours

6. Total Burden Hours on Public: 492.4

7. Project Purpose, Methodology, and Formative Research Design:

Background

This year, the U.S. Departments of Agriculture and of Health and Human Services will release the Dietary Guidelines for Americans 2015 policy document, which is the cornerstone of Federal nutrition programs. Updated every five years, the Dietary Guidelines for Americans (DGA) serve as the nation’s authoritative recommendations regarding recommended calorie consumption, making informed food choices, and being physically active in order to attain and maintain a healthy weight, reduce risk of chronic disease, and promote overall health. Recommendations from the DGA are intended for Americans ages 2 years and older, including those at increased risk of chronic disease.

The USDA Center for Nutritional Policy and Promotion (CNPP) is collaborating with HHS’s Office of Disease Prevention and Health Promotion (ODPHP) to develop communication strategies to promote healthy eating among Americans based on the upcoming DGA. Specifically, CNPP is tasked with developing communication strategies for the general population to encourage dietary practices recommended by the DGA, as well as physical activity.

As is the case with the release of the DGA, CNPP must ensure that the specific consumer recommendations (i.e., message concepts) that stem from the DGA communicate information clearly, as intended. In other words, it must minimize the possibility of misinterpretation. Previous research to inform the development of such consumer-facing messaging focused on each message, individually. For example, research participants were asked to evaluate several ways to communicate the idea, “Make half your plate fruits and vegetables” to inform the final wording. This year, however, represents an important additional goal for the DGA recommendations.

In 2015, CNPP and ODPHP determined that, in addition to communicating DGA recommendations clearly, this concept messaging must also communicate the notion of holistic dietary patterns—the idea that consumers should use the information from all of the DGA recommendations to help them make better nutrition decisions now, and into the future. In addition, CNPP wants to gauge how best to depict this notion visually, as well as with words and phrases.

The approval of this information collection is necessary for CNPP to meet these aforementioned objectives:

• Identify what message concepts can best summarize the idea of a dietary pattern and encourage consumers to think about nutrition decisions, writ large.

• As with previous Dietary Guidelines-related research, this information collection must also identify how best to communicate new message concepts to encourage the adoption of healthy food and beverage consumption decisions.

Qualitative research conducted by CNPP has addressed these topics and led to the refinement of messaging strategies. Following the best practices of consumer messaging research, the web experiment as a follow-up to the qualitative research will help CNPP to determine message concepts and image concepts that most effectively communicate the information as intended among key audiences.

Purpose

Strong message concepts are a function of two key factors: clear expression of the necessary information and an understanding of the audience that needs to hear and internalize it. USDA and HHS have a clear plan to develop the Dietary Guidelines recommendations and determine the most crucial information to share with consumers, but without a better understanding of how consumers might hear or interpret the message concepts, it may fall on deaf ears. Consumer-facing research will provide the remaining intelligence needed to increase the likelihood of success.

Qualitative research has already provided invaluable insight into the development of effective strategies to inform and motivate Americans to eat healthier in the context of the DGA. As a follow-up, this research builds on the findings of the qualitative analyses and is necessary to guide further development and refinement of consumer-facing messaging and other collaterals. The primary advantage of the survey is that it provides an opportunity to statistically compare the audience’s reaction to different messages and material. Furthermore, it relates the potential differences in audience receptivity to the messages to their psychographic and demographic characteristics. A survey approach also allows for cost-effective message testing research to be conducted with a broader spectrum of American public than would be possible through qualitative research alone.

The research that the CNPP plans to conduct falls under the rubric of efficacy research. Efficacy research seeks to measure how potential audiences assess different campaign message and/or collateral material under control conditions. It emphasized internal validity of the findings over external validity in order to accurately compare different messages. Efficacy research can be contrasted with effectiveness research that seeks to measure the impact of the messages in the population and is typically considered evaluation research. Given that this effort falls outside the realm of evaluation research, the research that the CNPP plans to conduct will be a web experiment rather than population-based survey.

The information collected during this phase of the research will provide the research and communication teams with understanding of:

• Which messages are most likely to resonate with the target audiences

• What specific language in key messages that audiences prefer

• Comprehension and receptivity to the construct of dietary pattern.

• Audience reaction to graphics conveying the idea of a dietary pattern

Measures of message receptivity will be related to the audience’s demographic and psychographic characteristics shown by qualitative research as possible factors affecting the audience’s assessment of the messages. Furthermore, analyses will explore the possible relationship between participants’ familiarity with MyPlate and their assessment of the DGA messaging. Thus, as the last research stage of formative research, message testing research will help the CNPP to finalize the communication material and dissemination strategies by offering statistically robust analyses of audiences’ preferences.

These research activities directly meet the Secretary of Agriculture’s directive to develop innovative approaches to inform Americans about 2015 Dietary Guidelines and ultimately positively impact the health of all Americans.

Methodology/Research Design

Sampling

Participants will be recruited from an existing online panel. Two samples will be used:

• English-speaking participants

• Oversample of Spanish-speaking participants (both Spanish dominant and bi-lingual)

The primary sampling criteria is the age of participants (over 18). The English Language sample will be balanced on age, geographic region, gender, income, and race to reflect U.S. population. The Spanish Language sample will be balanced on age, geographic region, gender and income.

The sampling strategy intends to obtain a sample of participants to achieve sufficient statistical power to:

• Provide robust comparisons between DG messages and graphics

• Analyze potential influence of demographic and psychographic characteristics on the audience’s receptivity to DG messaging

It is not the intent of the sampling strategy to be able to estimate any population parameters. Therefore, the sampling approach represents the needs of a web experiment designed to measure message efficacy, as opposed to population-based survey design to investigate message effectiveness and impact.

Recruitment

Invitations to participate in the survey (see Attachment A1 and Attachment A2) will be sent only to participants who fit the study criteria as based on the panel member characteristics kept by the panel providers. We should note that pre-screening of the participants is done using existing electronic records and therefore does not impose additional burden on the respondents.

Screening

After participants consent to the study (see section 8), a brief screener will be administered (Attachments B1 and B2). The screener will:

• Confirm age criteria for study participation

• Screen out participants who work in nutrition-related fields (e.g., health care providers, dietitians, those working in the wellness industry) and participants who work in marketing, advertising or market research.

• For Hispanic participants, measure the level of acculturation. Hispanic participants who were recruited as part of the English Language sample and were determined to be bi-cultural or Spanish dominant will be reassigned to the Spanish Language sample. Hispanic participants who were recruited as a part of Spanish Language sample and were determined to be acculturated will be reassigned to English Language sample.

Web experiment

After completing the screener, participants who qualify will proceed to the main survey (Attachment C-1 and C-2) in the appropriate language. The survey will contain several modules described below:

Familiarity with and understanding of MyPlate

The survey will measure the participants’ familiarity with MyPlate and recognition of the MyPlate logo. Those familiar with MyPlate will be asked their understanding of the concepts conveyed by MyPlate.

Dietary Pattern definition

Participants will be presented with two sentences that define the concept of dietary pattern. Follow-up questions will measure the participants’ receptivity of the concept. Participants will also be asked to select a phrase which they think best captures the concept of dietary pattern.

Message testing

The survey will test two umbrella messages encouraging healthier eating behavior consistent with the 2015 Dietary Guidelines. The messages will be presented in random order. Subsequent tasks and measures will be identical for both messages.

Once a message is presented, participants will be asked to engage in a “highlighting task”. Using an electronic “highlighter,” participants will be asked to mark words or phrases in the message that they liked. Once the “highlighting task” is complete, participants will be asked an open-ended question to capture elements of the message they did not like. They will also receive a battery of questions about their assessment of the message and the extent to which the message encouraged them to engage in healthier eating behaviors and additional information seeking about nutrition.

Graphics testing

Graphic representation of the idea of a dietary pattern will be presented to participants in random order. A brief assessment of the participants’ reactions to each graphic will be administered after each presentation.

Psychographics

The module will administer the measures of attitudes and behaviors that may impact receptivity to CNPP messages, including:

• Participants concerned with eating healthier

• Assessment of their own eating pattern

• Self-efficacy to engage in healthier eating behaviors

• Perceived social influence to eat healthier

Demographics

The final module will collect demographic measures including income, educational attainment and household size. In addition, participants will be asked about their role in the household regarding food purchasing decisions. Participants’ height and weight information will also be collected in order to calculate BMI.

Incentives

It is customary for panel providers to offer points for participation in the surveys. Accumulated points over many surveys can typically be traded for small monetary gifts, merchandise, or entry into a sweepstakes. The monetary value of points for participation in a 15 minute survey is typically less than one dollar. Point systems are part of the numerous other panel management activities conducted by the panel providers in order to keep panelists engaged. However, the price of the sample mostly depends on incidence rates and therefore does directly correlate to the panel management cost incurred by the sample provider. Thus, when using an online sample, the Federal government does not directly provide incentives to the participants, rather Federal government receives access to the sample that may or may not be incentivized by the panel provider.

8. Confidentiality:

CNPP and contractors will follow procedures for maintaining safeguards consistent with the Privacy Act during all stages of data collection. A system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports in the Federal Register on March 31, 2000, Volume 65, Number 63, and is located on pages 17251-17252 discusses the terms of protections that will be provided to respondents.

Using data collection instruments in Attachments D-1 and D-2, all participants will receive a privacy statement along with the consent form. The confidentially statement will inform the participants that all opinions they provide will not be shared with anyone outside this study, except as otherwise required by law, and none of their responses will be associated with personally identifiable information (PII). Participants will also be informed that findings will be presented as a summary. The consent forms will be the first screen on the on-line survey and require participants’ electronic acknowledgement prior to proceeding to the rest of the survey. Furthermore, the privacy act statement for this data collection will be displayed as the first screen upon participants’ entry into the survey (see Attachment F 1). Since the privacy act statement will be a part of the survey instrument, no additional burden hours are associated with it.

Additional steps employed to notify the respondents about their privacy are as follows:

• As part of the conditions of employment, all trained staff working on the project have signed a non-disclosure agreement (NDA) (Attachment E-1) prohibiting them from discussing any information about the research studies they work on with anyone not involved with the project, especially discussing or releasing personally identifiable information (PII).

9. Federal Costs: $998,319.67

10. Research Tools/Instruments:

• Attachment A-1: Survey Invitation (English)

• Attachment A-2: Survey Invitation (Spanish)

• Attachment B-1: Screener (English)

• Attachment B-2: Screener (Spanish)

• Attachment C-1: Survey Instrument (English)

• Attachment C-1a Screen shot of survey instrument

• Attachment C-2: Survey Instrument (Spanish)

• Attachment D-1: Consent form (English)

• Attachment D-2: Consent form (Spanish)

• Attachment E-1: Confidentiality Agreement (English)

• Attachment F-1: Privacy Act Statement

3