ALP Panel Members

Document Identifier: OS-0990-XXXX

Examining Consumer and Producer Responses to Restaurant Menu Labeling Requirements: Survey Protocol

Background and Design Overview

This section provides the background of this survey for the research team and ASPE. It is not part of the materials to be presented to respondents.

Objectives

This survey collects data on consumers’ potential responses to the Food and Drug Administration’s (FDA’s) new restaurant menu labeling regulation. Respondents of the survey are randomly assigned to different menus with and without calorie labeling. The primary goal is to estimate how calorie labeling differentially affect choices in different type of food outlets and consumers with different individual characteristics. Secondary goals are to estimate how consumers trade off prices and calories, and to calculate the welfare gains from labeling.

Overall survey structure

The survey is to be administered through the American Life Panel, an online panel that is representative of the U.S. adult population. The number of participants invited should result in 2,000 completed responses. The budget is for 40,000 survey minutes, which comes to 2000 completes at 20 minutes.

Each respondent will be presented with nine different menus (see list in Table 1). He/she will be asked to make food choices from each menu, followed by the final section of the survey where respondents will answer attitudinal and behavioral questions.

Table 1: The 9 types of food outlets

Fast food burger chains

Fast casual Asian restaurants

Ice cream parlors

Movie theatre snack bars

Pizza-by-the-slice stands

Organic, locally sourced restaurants

Fast casual Mexican restaurants

Salad/sandwich restaurants

Coffee shops

The design will be pretested to take 20 minutes or less. Aside from budgetary constraints, this time limit will avoid problems related to respondents becoming disengaged or fatigued in the later part of the survey.

What do we randomize?

1. For each individual the order of food outlets (fast food, Asian, ice cream, etc) will be randomly assigned. This will prevent any potential bias due to the order of appearance of the menus/food outlets.

2. For each food outlet the labeling of the menu shown is also randomly determined. This is the primary experiment, with the following treatments:

• Treatment A: no calorie labels (this will serve as the “control”)

• Treatment B: with calorie labels which meet the requirements of the FDA’s new regulation, i.e. the size of the calorie declaration must be no smaller than the size of the name or the price of the menu item it refers to, whichever is smaller. In general such calorie declarations must be in the same color, or a similar color as that used for the name of the associated menu item. The contextual statement about recommended daily caloric intake is shown. This is the “do minimum” treatment in which the new regulation will be met just barely.

• Treatment C (for only four of the food outlets): We allow an alternative labeling design for the fast casual Asian restaurant, the salad/sandwich restaurant, the pizza-by-the-slice stand and the organic, locally sourced restaurant. The design will meet the requirements of the new regulation and use fonts that are more pronounced than Treatment B (e.g. through the use of a heavier font and/or colors that stands out from the background). While many restaurants will use minimal requirements, some are likely to feature calories more prominently (as Subway has done for a long time). This design will allow separating visibility from other restaurant effects (e.g. intentional health halo).

Because we have nine food outlets, five of them have two treatments (A and B), and four of them have three treatments (A to C), it is not possible to ensure that each individual will be shown equal numbers of Treatment A, B, and C. The extent that this may or may not introduce respondent bias will be determined empirically.

Examples of Treatment B will be shown in the survey protocol chapter; examples of Treatment C will be shown in the Appendix A.

Other possible treatments can be implemented quickly if ASPE wants to pursue them, but they are currently not in the plan to be fielded:

• Calorie labels presented as in Treatment B, but with sodium information as well

Other potential treatments that we considered but decided not to include:

• With or without contextual information (e.g. recommended daily caloric intake);

• With or without interpretive information (e.g. traffic light symbols).

3. For some food outlets (fast-food burger, ice-cream, movie theatre, fast casual Mexican and coffee), the menus shown with have varying sets of prices. The options are:

• Default prices

• Lower calorie choices are approximately 20% cheaper (a “healthy dining subsidy”)

• High calorie choices are approximately 20% more expensive (a “fat tax”).

This price manipulation breaks the perfect collinearity between prices and calories and allows the study team to estimate the price sensitivity and eventually the consumer gain from better choices. Details of the price levels are included in Appendix B.

Potential limitations

This study focuses on the choice of main entree, but generally not additional selections over the course of their meals, e.g. appetizers, drinks, desserts, and so on. In a few food outlets, multiple choices are made possible (e.g. popcorn and drinks in movie theater), but in restaurants respondents are about a main item only.

Survey Protocol

[The survey starts here.]

Introduction

The following questions will ask you about your preferences among different types of foods available away from home.

We encourage you to answer this survey on your desktop or laptop, to ensure that the images in the survey are clearly visible. Thank you.

Section 0

Section 1

[Section numbers are for internal reference only. Each section corresponds to a food outlet and the order will be randomized.]

[Page title] At a fast-food restaurant

[Question 1-1] Imagine you’re at fast-food restaurant for lunch. It’s a new fast-food restaurant chain, similar to McDonald’s, Burger King, and Wendy’s. What would you order from the following menu?

[If the respondent chooses a combo, ask Question 1-1a]

For your combo, would you order a regular or diet soft drink?

[Price manipulation will be implemented in this food outlet.]

[Next page]

You ordered [insert order here].

[Question 1-2] On a scale of 1 (least satisfied) to 5 (most satisfied), how satisfied are you with your choice?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 1-3] On a scale of 1 (poorest value) to 5 (greatest value), how do you rate the value of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 1-4] On a scale of 1 (least healthy) to 5 (most healthy), how do you rate the healthfulness of the menu?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 1-5] In a typical month, how often do you eat food from a fast food restaurant of this type?

Section 2

[Page title] At a Fast Casual Restaurant

[Question 2-1] Now, imagine you’re at a new Asian restaurant that has an atmosphere like Panera Bread, Culvers, or Zaxby’s. What would you order from the following menu?

[This is a fixed price menu, so no price manipulation will be implemented.]

[Next page]

You ordered [insert order here].

[Question 2-2] On a scale of 1 (least satisfied) to 5 (most satisfied), how satisfied are you with the choice you made?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 2-3] On a scale of 1 (poorest value) to 5 (greatest value), how do you rate the value of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 2-4] On a scale of 1 (least healthy) to 5 (most healthy), how do you rate the healthfulness of the menu?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 2-5] In a typical month, how often do you eat food from restaurants like Panera Bread, Culvers, or Zaxby’s?

Section 3

[Page title] At an ice cream parlor

[Question 3-1] After dinner, you are out with friends at this ice cream parlor. What would you order from the menu?

First, pick a style. Then, pick a flavor (up to three).

[Price manipulation will be implemented in this food outlet.]

[Next page]

You ordered [insert order here].

[Question 3-2] On a scale of 1 (least satisfied) to 5 (most satisfied), how satisfied are you with the choice you made?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 3-3] On a scale of 1 (poorest value) to 5 (greatest value), how do you rate the value of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 3-4] On a scale of 1 (least healthy) to 5 (most healthy), how do you rate the healthfulness of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 3-5a] In a typical summer month, how often do you eat ice cream or frozen yogurt from an ice cream parlor or frozen yogurt place?

[Question 3-5b] In a typical winter month, how often do you eat ice cream or frozen yogurt from an ice cream parlor or frozen yogurt place?

Section 4

[Page title] At the movie theater

[Question 4-1] Imagine you are at the movie theater. You are standing in front of the snack bar of the movie theatre. What would you order?

[Price manipulation will be implemented in this food outlet.]

[Next page]

You ordered [insert order here].

[Question 4-2] On a scale of 1 (least satisfied) to 5 (most satisfied), how satisfied are you with the choice you made?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 4-3] On a scale of 1 (poorest value) to 5 (greatest value), how do you rate the value of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 4-4] On a scale of 1 (least healthy) to 5 (most healthy), how do you rate the healthfulness of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 4-5a] In a typical month, how often do you go to the movies?

[Question 4-5b] When you go to the movies, how often do you order something from the snack bar?

(1) Every time I go

(2) Every other time

(3) 1 in 3 times

(4) Less than 1 in 3 times

Section 5

[Page title] At a Pizza-by-the-Slice Stand

[Question 5-1] Imagine you’re at a pizza stand for lunch, where you can purchase pizza by the slice. What would you order from the following menu? Select as many slices as you like for a meal.

[This is a fixed price menu, so no price manipulation will be implemented.]

[Next page]

You ordered [insert order here].

[Question 5-2] On a scale of 1 (least satisfied) to 5 (most satisfied), how satisfied are you with the choice you made?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 5-3] On a scale of 1 (poorest value) to 5 (greatest value), how do you rate the value of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 5-4] On a scale of 1 (least healthy) to 5 (most healthy), how do you rate the healthfulness of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 5-5] In a typical month, how often do you eat pizzas from restaurants like Pizza Hut or Dominoes? Please include take out or delivery as well as eating at the restaurant.

Section 6

[Page title] At an Organic, Locally Sourced Restaurant

[Question 6-1] Imagine you’re at a pizza restaurant for lunch, where fresh, organic, locally sourced and other premium ingredients are served. You can purchase pizza by the slice. What would you order from the following menu? Select as many slices as you like for a meal.

[This is a fixed price menu, so no price manipulation will be implemented.]

[Next page]

You ordered [insert order here].

[Question 6-2] On a scale of 1 (least satisfied) to 5 (most satisfied), how satisfied are you with the choice you made?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 6-3] On a scale of 1 (poorest value) to 5 (greatest value), how do you rate the value of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 6-4] On a scale of 1 (least healthy) to 5 (most healthy), how do you rate the healthfulness of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 6-5] In a typical month, how often do you eat food from restaurants where fresh, organic, locally sourced and other premium ingredients are served?

Section 7

[Page title] At a Mexican Fast Casual Restaurant

[Question 7-1] Imagine you’re having lunch at a Mexican fast casual restaurant, similar to Chipotle or Qdoba’s Mexican Grill. What would you order from the following menu?

[Next page]

You ordered [insert order here].

[Question 7-2] On a scale of 1 (least satisfied) to 5 (most satisfied), how satisfied are you with the choice you made?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 7-3] On a scale of 1 (poorest value) to 5 (greatest value), how do you rate the value of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 7-4] On a scale of 1 (least healthy) to 5 (most healthy), how do you rate the healthfulness of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 7-5] In a typical month, how often do you eat food from restaurants like Chipotle or Qdoba’s Mexican Grill?

Section 8

[Page title] At a Salad/Sandwich Restaurant

[Question 8-1] Imagine you’re having lunch at a fast-casual sandwich/salad restaurant like Potbelly’s, Subway, Firehouse Subs, and Jimmy John’s Gourmet Sandwiches. What would you order from the following menu?

[Next page]

You ordered [insert order here].

[Question 8-2] On a scale of 1 (least satisfied) to 5 (most satisfied), how satisfied are you with the choice you made?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 8-3] On a scale of 1 (poorest value) to 5 (greatest value), how do you rate the value of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 8-4] On a scale of 1 (least healthy) to 5 (most healthy), how do you rate the healthfulness of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 8-5] In a typical month, how often do you eat food from restaurants similar to Potbelly’s, Subway, Firehouse Subs, and Jimmy John’s Gourmet Sandwiches?

Section 9

[Page title] At a coffee shop

[Question 9-1] Imagine you’re at coffee shop, similar to Starbucks or The Coffee Bean & Tea Leaf. What would you order from the following menu?

[Question 9-1a]

[Next page]

You ordered [insert order here].

For your [drink name], what kind of milk would you like?

• 2% milk

• Whole milk

• Skim milk

• Soy milk

[Question 9-2] On a scale of 1 (least satisfied) to 5 (most satisfied), how satisfied are you with your choice?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 9-3] On a scale of 1 (poorest value) to 5 (greatest value), how do you rate the value of the available choices?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 9-4] On a scale of 1 (least healthy) to 5 (most healthy), how do you rate the healthfulness of the menu?

[“I don’t know” is also provided as an option, on the far right of the scale]

[Question 9-5] In a typical month, how often do you consume drinks from a coffee shop similar to Starbucks or The Coffee Bean & Tea Leaf?

Section 10

[Page title]

[This is where we ask the attitudinal and behavioral questions. Socio-demographics of the respondents are already available from ALP and do not need to be asked. There should also be data on literacy, numeracy, and long term planning, so we can skip those. We have time for more than 10, but probably less than 20 questions.]

[Question 10-1] How hungry are you at the moment on a scale of 1 (very hungry) to 10 (not at all hungry)?

[show scale here]

[Question 10-2] How important are the following characteristics for making a food selection?

	Very important	Important	Somewhat important	Not very important	Not at all important
Large portion					
Good value for money					
Low price					
Vegetarian					
Is low in calories					
Keeps me healthy					
Helps me control my weight					
Need to taste good					
Is something new					

[Question 10-3] When you shop at a supermarket, do you look at nutritional information when choosing between similar foods?

Always	Most of the time	About half the time	Sometimes	Never
				

[Question 10-4] When calorie information is available in the restaurant, how often do you use this information to decide what to order? [Source: BRFSS 2011, Module 4, 279-280]

Always	Most of the time	About half the time	Sometimes	Never
				

[Question 10-5] Which of these statements best describes the food eaten in your household in the last 12 months: 

• Enough of the kinds of food we want to eat

• Enough but not always the kinds of food we want

• Sometimes not enough to eat

• Often not enough to eat

• Don’t know

[Question 10-6] What is your estimate of how many calories are in a standard sized can of Coca-Cola (12 fl oz, or 355 ml)?

[Respondent will be given a box to fill in any numeric values]

[Question 10-7] Suppose you have to choose between $100 in cash right now, or $x in cash in 1 year? What value of money x would make you exactly indifferent between the two options?

$100 now = $[number box] in one year

[Question 10-8a]

Suppose you can choose between two future scenarios:

Future A	Future B
All chain restaurants provide calorie information on their menus.	Chain restaurants do not provide calorie information on their menus but you'll be able save $40 per year on your meals outside the home.
Which future would you choose? Future A Future B I don’t know

[Question 10-8b]

[If the respondent chooses Future A previously] You said you would choose Future A. i.e. all chain restaurants provide calorie information on their menus. What if the savings in Future B is increased to $60 per year instead, which future would you choose?

[If the respondent chooses Future B previously] You said you would choose Future B, i.e. no calorie information on menus of chain restaurants. What if the savings in Future B is reduced to $20 per year instead, which future would you choose?

Future A	Future B
All chain restaurants provide calorie information on their menus.	Chain restaurants do not provide calorie information on their menus but you'll be able save $Y per year on your meals outside the home.
Which future would you choose? Future A Future B I don’t know

[Question 10-8c]

[Follow the same structure as Question 10-8b. A higher or lower saving will be shown in this question depending on the answer in Question 10-8b]

[See diagram in Appendix C which illustrates the logic and the dollar values to be shown.]

[Question 10-9] Do you consider yourself to be …{Source: WHQ030 from NHANES}

• Underweight

• About the right weight

• Overweight

[Question 10-10a] About how tall are you without shoes?

Your height: [box] feet and [box] inches

[Question 10-10b] About how much do you weigh without shoes?

Your weight: [box] pounds

Appendix A – Menus with more pronounced calories

This appendix shows the menus of the four food outlets (fast casual Asian, salad/sandwich, pizza-by-the-slice, and organic and locally sourced restaurant) where Treatment C, i.e .bolder font, is tested.

At a Fast Casual Asian Restaurant (bolder font)

At the Salad/Sandwich Shop (bolder font)

At a Pizza Restaurant (bolder font)

At the Pizza-By-The Slice Restaurant (bolder font)

Appendix B – Price levels

This appendix shows the price levels of the food items in the four food outlets (fast-food burger, ice-cream, movie theatre, fast casual Mexican and coffee) where price variation is implemented.

Prices used in the fast food burger chain

Prices used in the ice-cream parlor

Prices used in the movie theatre snack bar

Prices used in the fast casual Mexican restaurant

Prices used in the coffee shop

Appendix C – Contingent Valuation Questions Logic

Essentially, each respondent gets a contingent valuation question with three parts (10-8a to 10-8c). Part (a) starts at $40, and it will be followed by (b) and (c) at $0, $20, $30, $40, $50, $60 or $80 depending on the previous answer. The logic that determines the dollar amount to be displayed in (b) and (c) is illustrated in Figure 1.

Figure 1: CV questions sequencing

* WTA stands for willingness to accept. In this context, WTA is the respondent’s implicit value of information.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0990-NEW. The time required to complete this information collection is estimated to average of 150 hours/ minutes per response, including the time to review instructions, search existing data resources, gather the data needed and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to:  U.S. Department of Health & Human Services, OS/OCIO/PRA, 200 Independence Ave., S.W., Suite 336-E, Washington D.C. 20201,   Attention: PRA Reports Clearance Officer