Note to Reviewer - Autobiographical Retrieval Study

December 15, 2015

NOTE TO THE REVIEWER OF:	OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research”
FROM:	Erica Yu Research Psychologist Office of Survey Methods Research
SUBJECT:	Submission of Materials for the Autobiographical Retrieval Strategy Probes follow-up study

Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.

The total estimated respondent burden hours for this study is 206 hours.

If there are any questions regarding this project, please contact Erica Yu at 202-691-7924 or Scott Fricker at 202-691-7390.

1. Introduction and Purpose

The retrieval of autobiographical information -- a combination of personal experiences (episodic memory) and general knowledge (semantic) -- is fundamental to the design of many survey questions and yet remains poorly understood. At the Bureau of Labor Statistics (BLS), the Consumer Expenditure survey (CE) is one such survey that relies on recall of autobiographical information, in the form of past purchases. The BLS has completed several projects with the goal of improving data quality for the CE surveys. In support of the upcoming redesign of the Consumer Expenditure Quarterly Interview Survey (CEQ), this study will investigate whether interviewer prompting of retrieval strategies may lead to increased expenditure reporting.

Recall from memory is widely believed to be a process driven by the activation of items in memory based on the strength of association with a “cue.” The cue can come from any source – a previously recalled memory item, a new survey question, or an interviewer probe, for example. As a result of this associative process, different cues may lead to the activation and retrieval of different items. Furthermore, it is possible that additional cues may lead to the activation and retrieval of additional items.

A previous study conducted by the BLS Office of Survey Methods Research (OSMR) found that participants asked to recall past purchases used seven distinct recall strategies. The three most commonly used strategies were based on a shared activity, a goal, or a shopping episode. For example, a participant retrieves one item and then another because the items were both purchased in preparation for a trip to the beach (shared activity). The study also found that different retrieval strategies were associated with different recalled expenditures.

The findings from this previous OSMR study suggest that the use of more than one retrieval strategy may result in additional, different reported items. The proposed study will investigate whether interviewer prompting of additional retrieval strategies increases expenditure reporting. If the results from this study show that probing of retrieval strategies increases reporting, then further research will be needed for developing a possible future application of new interviewer training on probing with retrieval strategies during recall.

2. Research Design

The objective of this study is to explore whether respondent recall of expenditure information is aided by probes that encourage specific retrieval strategies. The probes to be used in this study have been designed based on the findings from the previous OSMR study on retrieval of autobiographical information from memory that showed three retrieval strategies were used frequently: recalling a shared goal (e.g., purchasing two items that are both used for the goal of improving fitness), activity (purchasing two items to be used in a do-it-yourself project), or shopping episode (purchasing two items during a single trip to a shopping center). Each probe is designed to encourage participant use of one of these retrieval strategies. The scripting for each retrieval strategy probe is described below.

Participant task

In conventional recall studies that collect participants’ expenditures reports, it is not possible to evaluate the effectiveness of probing because a participant’s actual expenditures are unknown. To address this limitation, in the present study participants will first read a series of fictional vignettes that contain information about a character’s expenditures, complete an intervening “distractor” task (see Appendix A), and then they will be administered an abridged version of the current CEQ and asked to recall the expenditures they learned in the vignettes. At the completion of the expenditure recall survey, participants will be asked debriefing questions to evaluate their use(s) of and reactions to the probes (Appendix B).

The same expenditures will be shown to all participants; the only between-participant manipulation will be the type of probing asked. This design enables analysis of probe effectiveness and comparisons across participants. Paradigms similar to this are regularly used in experiment-based studies of recall from memory in the psychology literature.

Learning phase

During this portion of the task, the to-be-remembered information will be presented as a narrative, embedded with narrative choice points designed to increase the autobiographical nature of the memories. All participants will undergo the same learning phase before being randomly assigned to one of the three probing conditions being tested.

The narratives include expenditure scenarios designed to test the effectiveness of the goal, activity, and shopping episode retrieval strategies. For example, several expenditures are narrated as being purchased during a single shopping trip. The implementation of the shopping episode probe will evaluate whether prompting participants to use that retrieval strategy results in any additional reports. These narratives are in Appendix C.

Recall phase

Participants will complete a self-administered, abridged version of the current CEQ, focusing on sections related to the expenses from the narratives (from the production CAPI instrument Version 14.43, 05-24-2013; OMB information stored at http://www.reginfo.gov/public/do/PRAViewIC?ref_nbr=201304-1220-004&icID=46041). The protocol is included in Appendix D. All participants will receive the same questions but the probes will be tailored by experimental condition.

Probe Administration

Participants will be randomly assigned to receive one of three probe treatments. The conventional implementation of interviewer probing is to administer probes only after an initial recall question has been asked. In this study, we test two different versions of this post-question probing, administered at the end of each cluster of related expenditure questions. Participants in Group 1 will be administered standard probes that simply restate the broad expenditure category name (e.g., “Any other clothing?”). Participants in Group 2 also will be administered follow-up probes, but rather than simply restating the broad expenditure category name, the probes will mention a specific retrieval strategy. And we test a third treatment in which the retrieval-strategy cue is provided to participants as an instruction at the start of a broad expenditure category section, prior to asking any specific expenditure questions (Group 3). More details about each treatment group follows:

• Group 1: Follow-up, expenditure category probe

Participants will be administered the survey questions and respond to the questions as they normally would. Upon reaching the end of a broad expenditure category section of related expenses (e.g., vehicle operating expenses), the instrument will administer a probe that repeats the name of the general expenditure category encompassing the group of related expenses (e.g., “Any more expenses for vehicle operating expenses?”). This group serves as a control group to indicate whether the act of probing, rather than the retrieval strategy suggestion, increases reporting of expenditures.

• Group 2: Follow-up, retrieval strategy probe

As in Group 1, participants in Group 2 will read the survey questions and respond to the questions as they normally would. Upon reaching the end of a broad expenditure category section, the instrument will probe with a suggestion for a retrieval strategy and give the respondent an additional opportunity to report expenditures. At the time of the first administration of the probe, the instrument will briefly explain the purpose of the retrieval strategy probe. The first administration of the probe will be read as follows: “Now, we’d like you to consider the last group of questions that we just asked you – about major household appliances. Are there any expenditures you had but might have forgotten to report? Some folks who have answered these questions before say that you might be able to remember even more items if you think about each of the items you bought and consider what you were doing on the day that you bought it, why you bought it, or what you used the item for. So, thinking back to the items you reported - does thinking about the dates that you bought the items, why you bought the items, or what you used the items for, remind you of any other [category name] expenditures that you haven’t reported yet?” A shortened version of this probe will be administered at the end of each subsequent broad expenditure category section: “Again, thinking about the shopping trip when you bought [insert expenditure category name] items, or the reason(s) you bought that/those item(s), can you think of any other [category name] expenses that we might have missed?”

• Group 3: Retrieval strategy probe, before recall begins

In this group, participants are probed with a suggestion for a retrieval strategy at the start of the CEQ survey, immediately following the introduction to the recall interview concept. At this time, the instrument also gives a brief explanation of the purpose of the retrieval strategy probe. Then, the instrument will proceed with the survey questions as normal. The instructions to be read at the start of the instrument are as follows: “Some folks who have answered these questions before say that you might be able to remember even more items if you think about each of the items you bought and consider what you were doing on the day that you bought it, why you bought it, or what you used the item for. Please keep these ideas in mind to help you while you are answering the survey questions.”

Differences between the three conditions have been limited as much as possible to only the key differences described above regarding the administration of the probes. All other design features and procedures will be held constant across the three groups.

3. Measures of interest

The instrument will record the implementation of the probes and the items reported by the respondents both before and after probing. Debriefing questions will collect participant assessments of the probes, including whether the participants felt the probes were helpful or burdensome. Measurement of the time added for administering the probes will also be captured by the Qualtrics software. Together, these measures will be qualitatively analyzed to conclude whether probing of retrieval strategies warrants further research.

4. Participants

This online study is designed to explore the impact of administering retrieval strategy probes during recall in the CEQ. The research design requires a large sample of 494 participants to be randomly assigned to one of the three experimental groups, with additional participants recruited to account for break-offs.

The task will be administered on-line using the Qualtrics survey website. Participants will be recruited using a convenience sample from Amazon Mechanical Turk; this study is focused on internal validity rather than representativeness of any population.

5. Burden Hours

Our goal is to obtain feedback from 494 participants. We anticipate that the online task will last no longer than 25 minutes. Burden hours are expected to total 206 burden hours.

6. Payment to participants

A $2.50 incentive will be provided to all participants.

7. Data Confidentiality

Recruiting of participants will be handled by Amazon Mechanical Turk. Participants will be informed as to the voluntary nature of the study. Participants will also be informed that the study will be used for statistical purposes to understand recall from memory. No pledge of confidentiality will be given to survey respondents.

Appendices:

Appendix A: distractor task

Appendix B: debriefing questions

Appendix C: narratives

Appendix D: protocol

Appendix A:

A distraction in between learning items and recalling those items facilitates the clearing of working memory and the encoding of the learned items into long-term memory. In this task, participants will be shown images and asked to make judgments for an unrelated task. These images will include items learned in the main task alongside images of items that were not presented during learning. This process of distraction increases the effectiveness of the break between learning and recall, which may otherwise be too short for effective testing of recall from memory.

Below is an illustration of the instructions and response scale. Participants will be shown 15 images, ordered randomly.

---

You’re done shopping!

And now for your next task, please consider the items presented in the following images. Please rate each item according to its level of quality.

[insert image]

Very high quality

Somewhat high quality

Neither high nor low quality

Somewhat low quality

Very low quality

---

Images to be used:

Appendix B: Debriefing questions

Thanks. That completes this portion of the study. Now, we’d like to spend a bit of time getting a better sense of your experiences.

What was your overall impression of today’s tasks?
[text box]

Did you find anything particularly difficult or confusing?
[text box]

At the very beginning of the study, you read through a number of descriptions about shopping and were asked to select among different expenditure options. What was your reaction to this task?
[open-ended text box]

How difficult would you say that task was?
Not at all difficult
Slightly difficult
Moderately difficult
Very difficult
Extremely difficult

Okay, now I’d like to ask you some specific questions about the last part – where we asked you to recall the expenses you’d read about and selected earlier. What were your reactions to this part of the task?
[text box]

How difficult would you say that task was?
Not at all difficult
Slightly difficult
Moderately difficult
Very difficult
Extremely difficult

Can you tell us a bit more about how you were able to recall the different expenses? Can you give an example, walk through how you recalled one of the expenses?
[text box]

Was that basically the way you remembered and reported all of the expenses, or did you have to use different types of recall strategies/things to recall other types of expenses?
[text box]

For Group 1 participants:

At the end of the different sections of the survey, we would ask you “any other purchases in [name category].”

• What were your reactions to this question? [open-ended text box]

• How helpful would you say that question was?
  Not at all helpful
  Slightly helpful
  Moderately helpful
  Very helpful
  Extremely helpful

For Group 2 participants:

At the end of the different sections of the survey, we would ask you “Thinking back to the items you reported - does thinking about the dates that you bought the items, why you bought the items, or what you used the items for, remind you of any other expenditures that you haven’t reported yet?”

• What were your reactions to this question? [open-ended text box]

• How helpful would you say that question was?
  Not at all helpful
  Slightly helpful
  Moderately helpful
  Very helpful
  Extremely helpful

For Group 3 participants:

At the start of the interview, we said “Some folks who have answered these questions before say that you might be able to remember even more items if you think about each of the items you bought and consider what you were doing on the day that you bought it, why you bought it, or what you used the item for. Please keep these ideas in mind to help you while you are answering the survey questions.”

• What were your reactions to this information? [open-ended text box]

• How helpful would you say that question was?
  Not at all helpful
  Slightly helpful
  Moderately helpful
  Very helpful
  Extremely helpful

Appendix C: Narratives

It’s time to start thinking about renewing your gym membership. If you want to keep improving your fitness, you’ll have to exercise more often. You didn’t really like your old gym (what was that smell in the locker room?) so you want to choose a new one.

Which one do you pick?

They are running a deal – no initiation fees and $129.99 for the first six-months if you pay upfront.

What better way to keep you on your fitness track than to commit yourself to a race? You’ll have no choice but to train for it! There is a race for charity scheduled in a few months and they are offering several different race lengths.

Which do you choose?

5k 10k Half marathon Full marathon

Commitment made! And the $35.00 registration fee will go to a good cause.

Some workmates have volunteered you to participate in an office sports team. They persuaded you to join because they know you’ve gotten a lot faster at running away from things now that you’ve been exercising more. There are four sports to choose from.

Which one do you pick?

You write your name on the sign-up sheet and make a check out for $25 for the club fees.

It’s official: you have lost enough weight that your pants are now too big for you. Time to celebrate! By buying storage bins to toss all of your old clothes into. You head to the store and find that there are lots to choose from.

Which one do you pick?

You head to check out and it rings up on sale at $6.50. Goodbye old clothes, hello new wardrobe!

You’ve noticed your running shoes are wearing out and it’s time to replace them. You head to the shopping center with the sporting goods store. Once you pull into the parking garage you see that it is slim pickings. You see four spots available.

Which space do you choose?

The space next to the person who parked crookedly

The space with the concrete pillar blocking the left side

The space on the top level exposed to the hot sun

The space that is as far away as possible from the elevator up to the shopping center

Darn! They don’t have the running shoes you want in your size. Looks like you made the trip out here for nothing. An unfortunate waste of $7.00 for parking.

You’ve ignored the sticker on your windshield urging you to change your car’s oil every 3,000 miles for long enough. Now the light on your dashboard is also indicating you need to take your car in to get the oil changed. You’ll have time to take it to a garage today while you run other errands.

Which place do you want to go to?

The oil and the oil change come to a total of $76.00.

A catalog from the local community center comes in the mail and you flip through it. Based on the pictures, it seems like the classes they are offering are full of people your age having fun. You decide to sign up for one.

Which class do you pick?

Great! The class meets every Wednesday evening for two weeks. Including the tuition and all the materials fees, it costs $40.00.

An email from Target announces that their patio furniture is on sale this week. It would be nice to have some furniture for your little patch of outdoor space. You browse online to see what’s available. You narrow it down to four sets that are within your price range.

Which do you pick?

Good choice! After the discount, the price of the patio furniture came to $94.00.

You have been buying a lot of stuff recently… maybe it’s time for a quiet night in. You stop by the grocery store on your way home and pick up a RedBox movie. Not a lot of choices, but there are at least a few to pick from.

Which movie do you choose?

You’ve already got popcorn at home. Total cost for a night of entertainment: $1.50.

The extended family is getting together, which means finding a way to entertain the grandparents, the little kids, and everyone in between. It sounds like at least one afternoon will be spent at the zoo.

It’s a big place – where do you want to start?

Good pick – that’s bound to be the favorite. Your admission ticket to the zoo costs $11.00.

You recently purchased a small condo in the city, but it requires some upgrading. In order to save money, you decide to take on some of the remodeling projects yourself, starting in the kitchen. You’ve spent the last two weeks tearing out the cabinets, replacing countertops, and painting, and now you want to find a new, energy-efficient dishwasher. You’ve narrowed the selection to these four.

Which one do you pick?

Nice selection – functional, nice looking, and on sale for $285 with an extended warranty. Don’t forget to sign up for the free delivery!

The renovation is going well, but your electric drill has stopped working. Time to buy a replacement (you still have quite a bit of work left to do, after all). You research different makes and models of new drills, and settle on these four. Each model offers slightly different options, but they are similarly priced at $64.99 at your local hardware store.

Which do you choose?

Remodeling is hard work, and HOT! To help beat the heat you decide to purchase a small area fan. The store has four models that should do the trick – they’re all powerful, portable, and not too pricey.

Select one of these fans, each only $19.99:

Wonderful – you can feel the cool breezes already!

Three weeks into the remodeling, and the new kitchen is starting to look really good. Unfortunately, the same can’t be said for your right knee, which is swollen and sore due to all of the kneeling and crouching you’ve had to do. You look online and find four knee braces that seem like they’d help and are well reviewed by others. They are all around $20.

Which one do you pick?

Your online order for the brace is $22.35, including taxes and shipping. It will arrive tomorrow afternoon, in time for you to tackle the rest of the kitchen!

The kitchen remodel is nearly complete – just some minor touch-up tasks left to do - but the entire first floor is covered in dust. Cleaning is not your favorite activity, so you decide to hire professional cleaners. These four companies get good ratings, are available, and within budget. Which do you pick?

The cleaners did a good job - the house looks great! $100 well spent.

Your niece’s 6^th birthday is next month. She really wants a princess dress, so you drive to the Disney store in the mall and find these options.

Which one do you pick?

Cute choice – she’ll love it. And at $19.95, it doesn’t break your bank.

Since you are already at the mall shopping, you decide to stop by Footlocker to get a new pair of running shoes. You like these four, all of which are on sale for $69.99.

Which one do you pick?

On your drive back from the mall, you stop at a little jewelry store which your neighbor said was having a great sale. Good thing you did – they have a lot of earrings that your mom would like at very reasonable prices. You narrow the selection down to these four – which do you choose as a present for your mom?

Excellent choice. They’re on sale for $45.

Driving back from the mall and jewelers, you remember that you need food and litter for your cat Whiskers. The pet store you stop at has a different selection of food than you’re used to buying at your local grocery store. Each of these brands is $1.29 a can.

Which do you choose?

Good choice - Whiskers is lucky to have a loving owner like you!

Running errands has made you thirsty. Next door to the pet store is a 7-11, so you walk over and grab a bottle of water. At the check-out counter, you also decide to buy a $2 lottery scratch-off card (you can’t win if you don’t play!). Which of these scratch offs do you pick?

Good luck!

Appendix D: Protocol

Introduction to the Study

Welcome!
Thanks for your interest in our research. You're here because we've asked you to participate in research on how memory works. The purpose of this study is to improve the Consumer Expenditure Survey, which is a survey that collects information about goods and services that Americans purchase. This survey is one our most important, since its data are used in research, policy decisions, and to help determine the national and local inflation rate.

Unlike some surveys or online tasks you may be familiar with, we ask that you complete this task all at one time and that you only start once you are in a quiet place where you won't be disturbed for about 25 minutes. Only share information you're comfortable with - nothing too personal - but please be honest and follow the instructions.

Please do not use your browser's back button.

This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141. We will use the information you provide for statistical purposes only. Your participation is voluntary, and you have the right to stop at any time. This survey is being administered by qualtrics.com and resides on a server outside of the BLS Domain. The BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response. By proceeding with this study, you give your consent to participate in this study.

---page break---

We will begin today by having you read through a series of brief scenarios – they are descriptions of situations in which you are making a purchase of some sort. These are obviously fictional scenarios – we just made them up. Your job is to read each description, and at the end of each one, you will be asked to make a choice between different purchase options.

For example, there may be a scenario where it says that you are shopping for a new cellphone, and then at the end it lists several possible types of cellphones you could choose to buy.

Your task simply is to choose one of the options. There are no “right” or “wrong” answers to these scenarios. You are free to choose whichever item you like for whatever reason. Now, because these are short, fictional descriptions, there obviously is going to be a lot of information that we just couldn’t include in the write-ups. Usually there will be a picture of the different options, and some information about the costs, but otherwise you’ll just make your decisions based on your personal preference. Finally, although it probably goes without saying, these are only hypothetical purchases – we aren’t going to be expecting you to write us a check for the items you select, and unfortunately we cannot actually give them to you. This is just an exercise that will help us to better understand consumer expenditure decisions and reporting.

Learning Task Administration

Administer the learning task (see Appendix C).

Distractor Task Administration

Administer the distractor task (see Appendix A).

Expenditure Survey Administration

Administer the abridged version of the CEQ. The sections that will be included are: Section 6 Appliances, household equipment and other selected items; Section 8 Home furnishings, and related household items; Section 9 Clothing and clothing services; Section 12 Vehicle operating expenses; Section 15 Medical and health expenses; Section 16 Educational expenses; Section 17 Subscriptions, memberships, books and entertainment expenses; Section 19 Miscellaneous expenses. The follow-up questions for each reported expenditure will be limited only to a brief description of the item and its cost; other follow-up questions such as who the item was purchased for will be excluded due to time considerations. The household roster and sections unrelated to the scenarios used in the task will also be excluded, as they are not needed for evaluation of the research questions. The instrument will also administer probes, as described in Section 2. Research Design on pages 4-5.

For example, participants assigned to receive the retrieval strategy probe at the end of each section (Group 2) will be asked:

Since the 1^st of May, have you or anyone in your household had any expenses for

1. Sofas?

2. Living room chairs?

3. Living room tables?

4. Ping-pong tables, pool tables, or other similar recreation room items?

5. Other living, family, or recreation room furniture?

6. Living room furniture combinations?

7. Dining room or kitchen furniture?

8. Mattresses or box springs?

9. Bedroom furniture other than mattresses or box springs?

10. Infants’ furniture?

11. Infants’ equipment?

12. Patio, porch, or outdoor furniture?

13. Barbecue grills or outdoor decorative items?

14. Office furniture for home use?

15. Lamps, lighting fixtures, or ceiling fans?

16. Other household decorative items?

Thinking back to the items you reported buying - does thinking about the dates that you bought the items, why you bought the items, or what you used the items for, remind you of any other household furnishings and decorative items expenditures that you haven’t reported yet?

Debriefing Questions

Administer the debriefing questions (see Appendix B).

Thank You

Thank you for participating in this research.

The aim of this research is to study how memories are recalled - we hope to understand what strategies people use to recall this kind of information and how these memories are organized. We will use your answers along with the answers of hundreds of other people to try to find common patterns that might tell us more about how memory works.

If you have any comments that you'd like to share with us about your answers or about the research in general, please leave them here.

[text box]