NF2 SSB_111215_Updated

Experimental Study on Consumer Responses to Nutrition Facts Labels with Various Footnote Formats and Declaration of Amount of Added Sugars

0910-NEW

SUPPORTING STATEMENT

B. Statistical Methods

1. Respondent Universe and Sampling Methods

The respondent universe of this study is adult participants in Synovate’s online Consumer Opinion Panel (“ePanel”); Synovate is the Agency’s contractor for this study. U.S. consumers who are 18 or older are invited to join the ePanel primarily through an affiliate marketing program. Select web sites, portals and Internet Service Providers partner with Synovate to promote ePanel membership through targeted email campaigns as well as placement of banner and pop-up advertisements. Consumers may also join ePanel through referrals from existing ePanel members and re-enlistment of former members. Currently, ePanel has over 2.5 million participants.

Respondents for the cognitive interviews will be recruited from a commercial database of residents in the Washington, D.C. metropolitan area. We will recruit approximately 15 respondents to make sure at least 9 of them will show up for the interviews (see Appendix C for screener).

The target sample size for the experimental study is 10,000 respondents. A quota will be developed prior to the study so that the overall sample of panelists who participate in the study will be balanced against the U.S. Census in gender, age, education, and race/ethnicity, i.e., inbound-balanced. The planned balancing categories are: (a) gender: female and male, (b) age: 18-34, 35-54, and 55+, (c) education: high-school graduate or less and one year or more college education, and (d) race/ethnicity: non-Hispanic white and other.

As discussed in Section A2, we will test hypotheses related to between-label differences as well as interactions among label condition, food category, and nutrition profile with respect to perceived levels of nutrients, product and labeling perceptions, and likelihood of product selection in a comparison task. We will impose no a priori direction of differences, if any (i.e., we assume all tests are two-tailed). The target sample size will yield approximately 178 observations for each of 56 experimental conditions (36 conditions for evaluating effects of the footnotes plus 20 conditions for evaluating effects of the added sugars declaration). We expect that this will provide adequate power to identify 3-way interactions of a medium size.

The Agency does not intend to generate nationally representative results or precise estimates of population parameters from the experimental study. The study will use a convenience sample rather than a probability sample. Despite the attempt to match between the study’s sample and the respondent universe in four demographic characteristics, matching is used solely to produce a sample with a reasonable degree of diversity in key demographic characteristics.

Rather, the strength of the proposed experimental study lies in its internal validity, on which meaningful estimates of differences across experimental conditions can be produced and generalized. As discussed in the following sections, the Agency has taken commonly accepted measures to enhance internal validity of the study. Examples of these measures include random assignment of respondents and conditions, use of control groups, and use of comparison conditions and relevant covariates.

2. Procedures for the Collection of Information

The contractor will use a telephone invitation (Appendix C) to recruit respondents for the cognitive interviews. The recruitment will target for diversity in respondents’ gender, age, race, and education. Respondents will also be required to meet other eligibility conditions (also in Appendix C). Eligible respondents will be asked to complete the draft questionnaire (Appendix D) independently. Then, a moderator will interview each participant about how he or she interpreted certain questions and the process by which the participant selected his or her responses.

For the experimental study, adult members of the ePanel will be invited by email to a dedicated Website to complete the study online (Appendix E). We estimate that it will take respondents about 15 minutes to complete the study.

All experimental conditions will involve a sequence of multiple tasks, described in Table 3. Participants will be randomly assigned to perform a task involving the comparison of two products (either two yogurts or two cereals) that may or may not include a declaration for added sugars (Section A). After completion of the comparison task, participants will be randomly assigned to view a single label for a food from a different product category for completion of the single-product tasks (Sections B and C). The food category will depend on whether the participant is randomized to an experimental condition aimed at evaluating a footnote or added sugars modification. Those assigned to a footnote experimental group will view either a frozen meal or crackers; those assigned to an added sugars experimental group will view either yogurt or cereal, whichever category they did not see in the comparison task. The label formats are shown in Appendix A and Appendix B. After completion of the single-product tasks, participants will go on to view a label for a randomly selected beverage product (Section D). Finally, participants will rate the label format itself, as applicable, in terms of qualities such as helpfulness and usefulness (Section E). Participants who view an added sugars label format in Section A and are subsequently assigned to an added sugars experimental condition for the single-product tasks will view the same added sugars label format across all tasks, including Sections B through E. Participants who view a control label format in Section A and are subsequently assigned to an added sugars experimental condition for the single-product tasks will view the same control label format across all tasks, including Sections B through E.

Table 3. Structure of experimental study

Section	Topic
A	Two-product comparison task – which product in a pair would be identified as a healthier product
B	Single-product evaluation task to compare (1) how products are rated when labeled with different footnotes; (2) how products are rated when labeled with and without an added sugars declaration; and (3) how products are rated when labeled with different amounts of added sugars – perceived overall healthfulness, perceived relationship between a food and selected health conditions, perceived nutritional content. Participants will be assigned to view one label format (e.g., one footnote format or one added sugars format) for this section according to their randomly selected experimental condition assignment; participants in all conditions will complete the same set of measures.
C	Comprehension of percent Daily Value, including measures of ability to determine when foods can be considered a good or excellent source of specific nutrients
D	Comprehension of the sugars declaration in the presence or absence of added sugars, including measures of ability to determine the amount of sugars and carbohydrates in the product
E	Label evaluations – to what extent the footnote messages or added sugars declarations are perceived as understandable, useful, believable, and helpful
F	Consumption and purchase of foods included in the study and typical food label use; ability to identify added sugars in a list of ingredients
G	Dietary awareness and interests
H	Health status and demographics

In all tasks, participants will view Nutrition Facts label images accompanied by a product identity caption (e.g., “Cereal X” or “Frozen meal Y”), but no front panels or brand names (either fictitious or real) will be included. Within each category of product, respondents may be assigned to evaluate a nutrition profile that reflects better or worse characteristics overall in terms of calories, fat and saturated fat, sugars, fiber and vitamins and/or minerals. The nutrition profiles are provided in Appendix F. In addition, the two-product comparison task (in Section A) will vary in its difficulty. The easier comparison involves a scenario where the product with fewer calories and/or less fat contains lower amounts of total sugars and added sugars than the comparator. The harder comparison involves a scenario where the product with fewer calories and less fat has an amount of total sugars lower than that of the comparator, but a higher amount of added sugars.

Dependent measures will include responses in a two-product comparison task (Section A in Table 3), ratings of a product’s overall healthfulness, perceptions about a product’s expected relationship with selected health conditions, and perceived product nutrient levels (Section B). Responses to questions about percent Daily Values will be compared across the footnote experimental conditions (Section C). Responses to identification questions concerning the amounts of total carbohydrates and sugars present in the three beverage products will be compared to examine differences that may arise in response to the amount or presence of declared added sugars (Section D). Label ratings (Section E) will also be compared.

Auxiliary measures will be collected and used to help understand participants’ responses to the label (Sections F, G, and H). The planned measures include consumption and purchase of the product categories included in the experimental conditions; label use behavior; familiarity with types of added sugars; and health status and demographics.

3. Methods to Maximize Response Rates and Deal with Non-response

Our experience with online experimental studies suggests that about 25% of panel members who are sent invitations will complete an FDA-commissioned study. The Agency will implement several procedures to maximize participation. We will conduct cognitive interviews and pretests to help improve understandability of the questionnaire, to reduce participant burden, and to enhance interview administration (see the cognitive interview invitation in Appendix C; the cognitive interview will use the questionnaire included in Appendix D). We will keep the study questionnaire at a reasonable length to minimize non-completion.

In addition, the contractors will (1) identify FDA as the sponsor of the study and state the purpose of the study in their invitation and reminder to encourage participation (see Appendix G for reminder); (2) provide an email address and a toll-free number for prospective participants to inquire about the authenticity of the interview and other questions; and (3) monitor all interviews and sample assignment and solve any problems daily throughout the course of the collection of information.

4. Test of Procedures or Methods to be Undertaken

FDA plans to perform two tests to minimize collection burden on respondents and improve quality of collected information. The first test consists of cognitive interviews; the primary purpose of these interviews is to understand the thinking processes that respondents use to answer the survey questions.

The second test is field pretests focusing more on the length of the questionnaire and respondent burden. The contractor who is responsible for the data collection will administer the full questionnaire to 150 adult members of Synovate’s web-based consumer panel shortly after OMB approval of the collection of information. This is the same web-based consumer panel from which the control groups will be selected.

Some fine-tuning of the data collection activity may result from the cognitive interviews or the pretests, but substantive changes are not expected. This proposed information collection requests OMB approval for these described cognitive interviews and pretests in combination with the main collection of information. We will inform OMB of any changes to the survey procedures or data collection instruments with a final version before actual data collection begins.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The contractor, Synovate, Inc. will collect the information on behalf of the Agency. Valerie Fuller DiPaula, Ph.D. is the Senior Study Director and project lead at Synovate. Analysis and dissemination of the data will be led by Serena Lo, Ph.D., telephone 240-402-2443.

5