Added Sugars Study SS Part B

Experimental Study on Consumer Responses to Nutrition Facts Labels with Various Footnote Formats and Declaration of Amount of Added Sugars

0910-NEW

SUPPORTING STATEMENT

B. Statistical Methods

1. Respondent Universe and Sampling Methods

The sampling frame for this study is adult participants in an online consumer panel maintained by Ipsos; Ipsos is the Agency’s contractor for this study. U.S. consumers who are 18 or older are invited to join the panel primarily through an affiliate marketing program. Select web sites, portals and Internet service providers partner with Ipsos to promote panel membership through targeted email campaigns as well as placement of banner and pop-up advertisements. Consumers may also join the panel through referrals from existing panel members and re-enlistment of former members. Currently, the panel has over 800,000 U.S. participants.

Respondents for the cognitive interviews will be recruited from a commercial database of residents in the Washington, D.C. metropolitan area. We will recruit approximately 15 respondents to make sure at least 9 of them will show up for the interviews.

The target sample size for the experimental study is 10,000 respondents. A quota will be developed prior to the study so that the overall sample of panelists who participate in the study will be balanced against the U.S. Census in gender, age, education, and race/ethnicity, i.e., inbound-balanced. The planned balancing categories are: (a) gender: female and male, (b) age: 18-34, 35-54, and 55+, (c) education: high-school graduate or less and one year or more college education, and (d) race/ethnicity: non-Hispanic white and other.

As discussed in Section A2, we will test hypotheses related to between-label differences as well as interactions among label condition, food category, and nutrition profile with respect to perceived levels of nutrients, product and labeling perceptions, and likelihood of product selection in a comparison task. We will impose no a priori direction of differences, if any (i.e., we assume all tests are two-tailed). The target sample size will yield approximately 130 to 140 observations for each of 73 experimental conditions (28 conditions for evaluating effects of the footnotes plus 45 conditions for evaluating effects of added sugars declarations). We expect that this will provide adequate power to identify 3-way interactions of a medium size.

The Agency does not intend to generate nationally representative results or precise estimates of population parameters from the experimental study. The study will use a convenience sample rather than a probability sample. Despite the attempt to match the study’s sample with the U.S. Census in four demographic characteristics, matching is used solely to produce a sample with a reasonable degree of diversity in key demographic characteristics.

Rather, the strength of the proposed experimental study lies in its internal validity, on which meaningful estimates of differences across experimental conditions can be produced and generalized. As discussed in the following sections, the Agency has taken commonly accepted measures to enhance internal validity of the study. Examples of these measures include random assignment of respondents and conditions, use of control groups, and use of comparison conditions and relevant covariates.

2. Procedures for the Collection of Information

The contractor will use a telephone invitation to recruit respondents for the cognitive interviews. The recruitment will target for diversity in respondents’ gender, age, race, and education. Eligible respondents will be asked to complete the draft questionnaire independently. Then, a moderator will interview each participant about how he or she interpreted certain questions and the process by which the participant selected his or her responses.

For the experimental study, adult members of the contractor’s consumer panel will be invited by email to a dedicated Website to complete the study online. We estimate that it will take respondents about 15 minutes to complete the study.

All experimental conditions will involve a sequence of multiple tasks, described in Table 3. Participants will be randomly assigned to an experimental condition aimed at evaluating a Nutrition Facts modification related to the footnote or to added sugars declarations. Those assigned to a footnote experimental group will view a Nutrition Facts label for a frozen meal or crackers and will be asked a series of questions about the information shown on the label (Section B).

Those assigned to an added sugars experimental group will view two Nutrition Facts labels for two yogurts, two frozen meals, or two cereals and will be asked a series of questions about how the two products compare with one another based on information shown on the label (Section A). The added sugars experimental groups will then view a Nutrition Facts label for a second product in whichever category (yogurt, frozen meal, or cereal) they did not see in the comparison task (Sections B and C).

Finally, participants in both the footnote and added sugars experimental conditions will rate the label format itself, as applicable, in terms of qualities such as helpfulness and usefulness (Section E). All participants will view the same assigned label format across all sections of the questionnaire. The label footnotes are shown in Appendix A and label formats including added sugars declarations are shown in Appendix B. Nutrition profiles for all of the hypothetical products are shown in Appendix C.

Table 3. Structure of experimental study

Section	Topic
A	(Added sugars experimental conditions) Two-product comparison task – which product in a pair would be identified as a healthier product, which one has fewer calories, and which one has less added sugar
B	(All experimental conditions) Single-product evaluation task to compare (1) how products are rated when labeled with different footnotes; (2) how products are rated when labeled with and without an added sugars declaration; and (3) how products are rated when labeled with different amounts of added sugars – perceived overall healthfulness, perceived relationship between a food and selected health conditions, perceived nutritional content. Participants will be assigned to view one label format (e.g., one footnote format or one added sugars format) for this section according to their randomly assigned experimental condition; participants in all conditions will complete the same set of measures.
C	(Footnote experimental conditions) Measures of ability to determine when foods can be considered to be low in specific nutrients or an excellent source of specific nutrients
D	(Added sugars experimental conditions) Measures of ability to determine the amounts of sugars, total carbohydrate, and added sugars in a product when added sugars are either included or not included in the Nutrition Facts.
E	(All experimental conditions) Label ratings – to what extent the footnote messages or added sugars declarations are perceived as understandable, useful, believable, and helpful
F	(All experimental conditions) Consumption and purchase of foods included in the study and typical food label use; ability to identify added sugars in a list of ingredients
G	(All experimental conditions) Dietary awareness and interests
H	(All experimental conditions) Health status and demographics

In all tasks, participants will view Nutrition Facts label images accompanied by a product identity caption (e.g., “Cereal X” or “Frozen meal Y”), but no front panels or brand names, either fictitious or real, will be included. Within each category of product, respondents may be assigned to evaluate nutrition profiles that reflect better or worse characteristics overall in terms of calories, fat and saturated fat, sugars, fiber and vitamins and/or minerals (see Appendix C for the nutrition profiles). In addition, participants who complete the two-product comparison task (in Section A) and who are assigned to view labels that include added sugars declarations will view one of two pairs of nutrition profiles within each product category. One pair involves a scenario where the product with fewer calories and/or less fat in the pair also contains a lower amount of added sugars than the comparator. The other pair involves a scenario where the product with fewer calories and/or less fat in the pair contains a higher amount of added sugars than the comparator. Participants who complete the two-product comparison task and who are assigned to view labels that do not include added sugars declarations (i.e., control labels) will view corresponding pairs of nutrition profiles without added sugars information.

Dependent measures will include responses in a two-product comparison task (Section A in Table 3), ratings of a product’s overall healthfulness, likelihood of consuming a product when trying to reduce the risk of selected health conditions, and perceived product nutrient levels (Section B). Responses to questions about product attributes (e.g., whether a product provides low or excellent levels of selected nutrients) will be compared across the footnote experimental conditions (Section C). Responses to identification questions concerning the amounts of total carbohydrates and sugars present in a product will be compared to examine differences that may arise in response to the amount or presence of declared added sugars (Section D). Label ratings (Section E) will also be compared.

Auxiliary measures will be collected and used to help understand participants’ responses to the label (Sections F, G, and H). The planned measures include consumption and purchase of the product categories included in the experimental conditions; label use behavior; familiarity with types of added sugars; and health status and demographics.

3. Methods to Maximize Response Rates and Deal with Non-response

Our experience with online experimental studies suggests that about 25% of panel members who are sent invitations will complete an FDA-commissioned study. The Agency will implement several procedures to maximize participation. We will conduct cognitive interviews and pretests to help improve understandability of the questionnaire, to reduce participant burden, and to enhance interview administration. We will keep the study questionnaire at a reasonable length to minimize non-completion.

In addition, the contractor will (1) identify FDA as the sponsor of the study and state the purpose of the study in their invitation and reminder to encourage participation; (2) provide an email address and a toll-free number for prospective participants to inquire about the authenticity of the interview and other questions; and (3) monitor all interviews and sample assignment and solve any problems daily throughout the course of the collection of information.

4. Test of Procedures or Methods to be Undertaken

FDA plans to perform two tests to minimize collection burden on respondents and improve quality of collected information. The first test consists of cognitive interviews; the primary purpose of these interviews is to understand the thinking processes that respondents use to answer the survey questions.

The second test is field pretests focusing more on the length of the questionnaire and respondent burden. The contractor, who is responsible for the data collection, will administer the full questionnaire to 150 adult members of the contractor’s web-based consumer panel shortly after OMB approval of the collection of information.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The contractor, Ipsos, will collect the information on behalf of the Agency. Valerie Fuller DiPaula, Ph.D. is the Senior Study Director and project lead at Ipsos. Analysis and dissemination of the data will be led by Serena Lo, Ph.D., telephone 240-402-2443.