Form Appendices A - # Appendices A - # Protocol, Scenarios, Debriefing Questions

Appendix A: Three recall formats

Participants will be randomly assigned to one of the three recall formats. The assigned format will be used for the participant throughout the entire recall portion of the task.

Control Format

This format is a simplified, self-administered version of the current CEQ. Participants will be prompted with section-level information (household furnishings and related items) and then detailed questions (sofas, living room chairs, living room tables). Each item will be presented on a screen by itself. The participant will move on to the next screen to receive the next question.

Plus Format

This format duplicates the Control format with the addition of a field for reporting merchant.

Outlets Format

This format provides section-level structure but does not ask about specific items. Participants are asked to report merchants first, and then the expenses at those merchants. The merchant name is pre-filled by default but participants may edit the field.

Appendix B: Distractor Task

A distraction in between learning items and recalling those items facilitates the clearing of working memory and the encoding of the learned items into long-term memory. In this task, participants will be shown images and asked to make judgments unrelated to the main task. These images will include items learned in the main task alongside images of items that were not presented during learning. This process of distraction increases the effectiveness of the break between learning and recall, which may otherwise be too short for effective testing of recall from memory.

Below is an illustration of the instructions and response scale. Participants will be shown 15 images, ordered randomly. These are images that were not presented during the learning phase.

---page break---

You’re done shopping!

And now for your next task, please consider the items presented in the following images. Please rate each item according to its level of quality.

[insert image]

Very high quality

Somewhat high quality

Neither high nor low quality

Somewhat low quality

Very low quality

---page break---

Images to be used:

Appendix C: Protocol

1. Introduction

---page break---

Welcome!

Thanks for your interest in our study. You’re here because we have asked you to participate in our study on recall of past purchases from memory. In this task, you’ll go through a game where you’ll make choices and shop for items, and then later tell us about what you bought. You won’t be asked to share any information about your own shopping.

Unlike some surveys or online tasks you may be familiar with, we ask that you complete this survey all at one time and that you only start once you are in a quiet place where you won't be disturbed. The survey takes about 30 minutes, on average. Only share information you're comfortable with - nothing too personal - but please be honest and follow the instructions.

 Please do not use your browser's back button. 

This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141. We will use the information you provide for statistical purposes only. Your participation is voluntary, and you have the right to stop at any time. This survey is being administered by Qualtrics and resides on a server outside of the BLS domain. The BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response. By proceeding, you give your consent to participate in this study.

---page break---

2. Instructions

---page break---

Instructions

We’ll first ask you to read a first-person story in which you’ll make choices about what you want to do next. It may be different from your real life but try to immerse yourself in the character and imagine that it is you.

Please read through each screen completely before continuing on to the next screen. Please don’t write anything down or otherwise record what’s on the screen – just focus on the story!

Afterwards, we’ll ask you about your life in the game and a few questions about your life in the real world.

First, let’s learn about your character. Continue on to the next screen to get started.

---page break---

Your Profile

Here is the backstory on your character:

Where do you live?

You live in the suburbs of a city called Springfield.

You have family that lives nearby and you see them often.

What do you do?

You have a steady office job at a small company in the city. You’ve been working on updating your home and trying your hand at some DIY repairs. In your spare time, you like to watch movies and hang out with friends.

Even if this doesn’t sound like your life, try to imagine that this is you and that the events you’ll read about are part of your life.

Ok, let’s get started.

---page break---

3. Learning Phase (see Appendix D)

4. Distractor Task (see Appendix B)

5. Recall phase, random assignment to one of three formats (see Appendix A)

6. Debriefing Questions (see Appendix E)

7. Thank you

---page break---

Thank you for participating in this study.

A survey like the one that you just completed is conducted regularly nationwide to create a picture of how much money US households are spending so that we can calculate inflation and the cost of living.

We need people to answer the survey accurately and give us a complete picture of their spending, but we also need to make the survey easy to answer and as short as possible.

If you have any thoughts about this study or how we can improve the way that we ask about household expenditures, please leave them in the box below. Thank you!

[open ended text entry box]

Appendix D: Learning Phase Narrative

All scenarios will follow a standard format that comprises an expenditure narrative, a choice related to the expense, and a summary of the transaction that includes all critical expenditure information. The instrument will force participants to spend at least three seconds on each of the screens, to encourage attentive reading of the narrative.

Condition	Screen 1: Expenditure Narrative	Screen 2: Choice Related to Expense	Screen 3: Summary of Transaction
Outlet is salient and item is not: Store name is memorable and described at length Item is not described	While waiting for friends to meet you for dinner, you find yourself standing outside of a few shops. One store catches your eye: it has a sign in the shape of a boat that reads “Tie-Tanic.” You wander in to find the store is full of tchotchkes. You pick out a gift for your brother’s birthday.	At the cash register, they offer you a choice of wrapping paper. Which do you choose?	That was a great store – you plan to come back to Tie-Tanic for a present for your brother next year, too. Here’s a summary of your purchases: Merchant: Tie-Tanic Total spent (not including tax): $12.99 [Fill selection]
Item is salient and outlet is not: Item is memorable Multiple similar stores that may lower recall accuracy for outlet	You come home to find out your dog has ripped up the bottom of your living room curtains. What’s worse, your parents are coming to visit next week and you don’t want your mom saying “I told you not to get that dog.” You need to get new curtains today!	Where do you go?	You end up finding some curtains that look like the ones your mom bought for you and that the dog destroyed. Here’s a summary of your purchases: Merchant: [Fill selection name] [Fill selection logo] Total spent (not including tax): $54.99
Outlet is visited repeatedly: -A single outlet is visited multiple times -The purchased items are unrelated and from different expenditure categories	You want to decorate your house in preparation for the next time you have guests over. A nice, but cheap, picture frame would be a good start – maybe you’ll frame that photo of your dog that you took in the park. You head to Walmart to pick up a frame.	Which one do you choose?	Once you’ve made your selection, you check out at the front of the store. You pick up a few extra items and add them to your shopping basket, too. Here’s a summary of your purchases: Merchant: Walmart Total spent (not including tax): $32.97 [Fill selection]
	Friends and family are coming over this weekend for a cookout. When you invited people over, the date seemed so far away… but now it’s only a few days until the party. Looks like you need to get yourself a grill! You stop by Walmart to buy one.	Which one do you choose?	Getting the grill is enough for the day – you’ll get the rest of the party supplies later. Here’s a summary of your purchases: Merchant: Walmart Total spent (not including tax): $85.99 [Fill selection]
An item is purchased repeatedly from different outlets: -Scenario is similar to routine or habitual purchases -Outlet information may not be strongly associated with the items	The new movie in the theaters this week is “Furious 7”. It’s playing at the Uptown Cinema - your favorite place to see a movie. Your friend takes a little bit of persuading to agree to come with you, but when you offer to buy the tickets – it’s a deal.	Which show time would you prefer? FURIOUS 7 Showtime: Fri 9:15p FURIOUS 7 Showtime: Fri 5:45p FURIOUS 7 Showtime: Sun 6:15p FURIOUS 7 Showtime: Sun 4:15p	You’ve got the tickets before the movie sold out. Now you just have to wait in line to get good seats… better stock up on some snacks. Here’s a summary of your purchases: Merchant: Uptown Cinema Total spent (not including tax): $22.00 [Fill selection] [Fill selection]
	You still haven’t had a chance to see the latest “Avengers” movie, even though you’ve heard a lot about it. It got decent reviews, so you’ll give it a shot.	When do you want to go? AVNGRSAGEO Showtime: Tue 5:45p AVNGRSAGEO Showtime: Thu 5:45p AVNGRSAGEO Showtime: Fri 8:15p AVNGRSAGEO Showtime: Fri 9:45p	It’s definitely a movie for getting popcorn at the concession stand! Here’s a summary of your purchases: Merchant: Cinema City Total spent (not including tax): $15.50 [Fill selection]
	Looking over the movie listings, you see that “Jurassic World” is going to be playing at The Screen on the Green. You get a couple of friends together and you plan to see it this week.	Which version do you want to see? Standard	Hope it’s a good movie! Here’s a summary of your purchases: Merchant: The Screen on the Green Total spent (not including tax): $12.00 [Fill selection]
	Cinema City is showing some classic movies this week. There’s a double feature of classic Alfred Hitchcock movies – Rear Window and Vertigo.	When do you want to see it? HITCHDOUB Showtime: Sat 12:45p HITCHDOUB Showtime: Sat 3:30p HITCHDOUB Showtime: Sun 3:30p	If these are good, maybe you’ll have a chance to check out some of the other classics being shown? Here’s a summary of your purchases: Merchant: Cinema City Total spent (not including taxes): $16.00 [Fill selection]
Outlet crosses expenditure categories: -The purchased items are unrelated to each other -Some items may not be memorable (storage bin) but the association with the other items from the outlet may improve recall accuracy	On your way home, you stop by Target to pick up a few things you need. The toaster broke and you have not had waffles for over a week now. Speaking of waffles - you remember that you need to get a birthday present for her.	Sweaters are on sale- she’s always wearing sweaters. Which do you pick?	She may or may not like it, but it’s the thought that counts, right? Here’s a summary of your purchases: Merchant: Target Total spent (not including taxes) $49.98 [Fill selection]
Goal-driven purchases: -Items are from different expenditure categories and different outlets but are all related to a single goal	The do-it-yourself approach to updating your bathroom is taking over your life. In order to complete the part of the project you’re working on, you realize that you have to stop what you’re doing and run to Ace Hardware around the corner to pick up a socket wrench.	The store has 4 different kinds. Which do you choose?	You’ll need a bigger tool box soon… but that’s a purchase for another day. Here’s a summary of your purchases: Merchant: Ace Hardware Total spent (not including taxes): $26.49 [Fill selection]
	If you never have to see your bathroom again, it will be too soon. The long process of fixing it up continues. Today, you’re picking out lighting. Home Depot is in a shopping center with other stores you need to run errands at, so you head there.	You pick out four that are in your price range. Which one do you choose?	Luckily, it comes with the hardware you need for installation. Here’s a summary of your purchases: Merchant: The Home Depot Total spent (not including taxes) $39.99 [Fill selection]
	Another weekend coming up? Another day to spend fixing up your bathroom. This time, you’re installing new towel rods. You’ve heard that Lowe’s has a good selection, so you head there.	You narrow it down to just four options. Which do you choose?	Good – it will match the rest of the bathroom fixtures. Here’s a summary of your purchases: Merchant: Lowe’s Total spent (not including taxes): $36.99 [Fill selection]

The presentation of these scenarios will be randomized to the extent possible, in order to minimize order effects. For conditions with multiple expenditures that should be distributed across months (e.g., going to the movies), the expenditures will be fixed to appear in separate months but the order within month will be randomized.

In addition to the expenditure information, each scenario will include the month and date of the expenditure, to be assigned after randomization so that the scenarios will appear to occur in chronological order. For example, whichever scenario is presented first will be labeled for all participants as occurring on June 5^th; the second scenario will be labelled as occurring on June 17^th; the third on June 23^rd; and the fourth on June 29^th.

Appendix E: Debriefing Questions

Were there any other items that you remember but have not yet entered?

[empty table in the same format as used throughout the survey]

---page break---

For each item reported above, ask the following:

What was the main reason for why you had not entered [Fill Description of the Expense]?

It did not fit into any of the categories that were asked about.

I remembered a little bit about the item but not enough to enter it.

I forgot about the item and only remembered after I should have entered it.

Other, please specify [text entry]

---page break---

And now we’d like to hear your thoughts about the task you just did. There will be a chance to write in your general comments at the end, but first we’d like to ask a few questions.

1. Were the items that you bought as part of the story easy or difficult to remember?

Very easy

Somewhat easy

Neither easy nor difficult

Somewhat difficult

Very difficult

2. Were the merchants and places that you shopped at as part of the story easy or difficult to remember?

Very easy

Somewhat easy

Neither easy nor difficult

Somewhat difficult

Very difficult

3. Were the expense amounts ($) easy or difficult to remember?

Very easy

Somewhat easy

Neither easy nor difficult

Somewhat difficult

Very difficult

4. Were the months of the expenses easy or difficult to remember?

Very easy

Somewhat easy

Neither easy nor difficult

Somewhat difficult

Very difficult

5. If you have anything to add about what it was like to remember the expenses, please write those here.

[open ended text entry box]

---page break---

6. Thinking about the part of the task where we asked you to tell us about your expenses – how burdensome would you say answering the questions felt?

Not at all burdensome

Slightly burdensome

Somewhat burdensome

Very burdensome

Extremely burdensome

7. Compared to other HITs that you’ve done on mTurk, which would you say was more burdensome?

It was less burdensome than others HITs

It was more burdensome than other HITs

It was about the same amount of burden

8. When we asked you to tell us about the expenses from the story, how clear did you think the questions were?

Not at all clear

Slightly clear

Somewhat clear

Very clear

Extremely clear

9. When we asked you to tell us about the expenses from the story, did the questions we asked make it easy or difficult to report the expenses?

Very easy

Somewhat easy

Neither easy nor difficult

Somewhat difficult

Very difficult

10. Think about your own life – if you were asked to recall your own expenditures from the last three months, what would help you to remember everything that you bought? Please rank the options below from Most Helpful (1) to Least Helpful (4).

(randomized order)

• A list of specific examples of things that people buy – like TVs, sports equipment, sofas, and mugs.

• A list of general categories of things that people buy – like home furnishings and clothing.

• A place where I could list the stores that I shopped at.

• A calendar where I could make notes about events and things that I did.

11. If there is something else that would help that we haven’t listed above, please write it in below.

[open text entry box]

---page break---

1. During the first half of the task when you read the story and made choices, did you do any of the following? Please mark all that apply.

(randomized order)

• Think about the similarities and differences between the story and my life

• Make choices as if they were real life choices

• Take notes or otherwise kept a record of the story

• Memorize as much detail as I could

• Stop paying attention after a while

2. How focused did you feel while you read the story and made choices? Please be truthful - your response will not be used against you.

Not at all focused

Somewhat focused

Slightly focused

Very focused

Extremely focused

---page break---

And now a few final questions about yourself.

1. What is your sex?

Male

Female

2. In what year were you born?

[open text box; numeric]

3. Are you Hispanic or Latino?
   Yes
   No

4. What is your race? (Please select one or more)

American Indian or Alaska native

Asian

Black or African American

Native Hawaiian or Pacific Islander

White

5. What is the highest degree you received?

No schooling completed

Elementary school diploma

High school diploma or the equivalent (GED)

Associate degree

Bachelor’s degree

Master’s degree

Professional or doctorate degree

17