Note to Reviewer - Cognitive Testing of Consumer Expenditure Diary Survey

August 9, 2024

NOTE TO THE REVIEWER OF:	OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research”
FROM:	Erica Yu Office of Survey Methods Research
SUBJECT:	Submission of Materials for Testing of Changes to the Meals, Snacks, and Drinks Away From Home category of the Consumer Expenditure Survey Diary

Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.

The total estimated respondent burden for this study is 27 hours.

If there are any questions regarding this project, please contact Erica Yu at 202-691-7924.

1. Introduction

The Consumer Expenditure surveys (CE) Diary survey tasks respondents with writing down everything the household spends money on each day during a reference period. One of the four categories of expenses collected in the diary, Meals, Snacks, and Drinks Away from Home, is being targeted for revisions due to a change in data requirements. Currently, respondents are asked to enter “the name of the restaurant, vendor, or cafeteria where you made this purchase”. Moving forward, the Office of Prices and Living Conditions (OPLC) has changed a data requirement and now needs to collect both the name of the restaurant and the name of the delivery service or app, if applicable. The proposed change is intended to provide outlet data for pricing. OPLC has requested that OSMR assist with pre-testing of revised diary form and instructions wording. This study will iteratively test different diary forms and instructions to identify effective wording.

2. Methodology

The proposed diary designs will be tested using cognitive interviewing. Cognitive interviews provide an in-depth understanding of the participant’s thought processes and reactions to the questions. In particular, this method enables us to conduct probing into how participants think about expenses for their household as well as have participants use the experimental versions of the diary.

The interviews will be conducted remotely via Microsoft Teams with participants from anywhere in the United States. We anticipate that each session will average no more than 60 minutes. The interviews will be administered by OSMR staff; observers from CE may attend interviews.

The proposed diary forms and instructions will be mocked-up and presented to participants as PDFs that they can look at on screen.¹ At the start of the study, there will be three versions of the diary form and instructions, one for each of two initially proposed designs along with one version of the current diary form and instructions.² Participant feedback on the current version will provide insight into the effect of any changes to instructions or labels, which will help us to understand what additional revisions might be effective. The protocol focuses on the Meals, Snacks, and Drinks Away from Home section, but permits collection of feedback on the Food and Drinks for Home Consumption section as well; previous feedback from research participants indicates that there can be confusion between these categories, which may be relevant for the current research questions. Screenshots of the diaries are shown in Appendix A.

Each participant will be randomly assigned to use one version of the diary. As this is pretesting, we expect that modifications may be made during the course of the study based on initial results. Although the goals of the testing, and overall design, will remain the same, findings from preliminary results may be used to improve the materials, tasks and debriefing probes. Modifications are likely to range from changes to the diary instructions to the inclusion of additional probing questions.

Each session will follow the cognitive interview protocol outlined in Appendix B. For all participants, the interview will begin with an introduction to the CE Diary Survey. Then, participants will be shown their assigned diary and given instructions on how expenses should be entered, similar to the instructions given in a CE Diary Survey placement interview. Participants will be asked to report several of their own Meals, Snacks, and Drinks Away from Home expenses into the diary as the interviewer shares their screen over the Teams video call. The interviewer will make the notations on the diary on-screen. The interviewer will take note of any difficulties the participant has with the reporting their expenses, such as knowing what information belongs where in the form, but will not intervene.

The interviewer will then ask the participant to enter several fictional expenses for scenarios designed to understand whether the instructions and form design were effective. The final portion of the interview will be spent probing for feedback, including the participant’s preferred diary form and instructions.

3. Participants

Up to 24 participants will be recruited for cognitive interviews. During recruitment, participants will be screened for behaviors relevant to the study: frequency of dining out and ordering delivery or carryout. Efforts will be made to recruit participants with a mix of demographics, particularly different education levels (as a proxy for income levels) and household sizes, which are highly predictive of household spending. See Appendix C for the screener interview script and Appendix D for the recruitment advertisement. Recruitment may use up to four different strategies:

• Flyers on bulletin boards at grocery stores, libraries, and other community locations

• Emails to community listservs

• Advertisements on Craigslist and similar internet bulletin boards

• Emails to previous OSMR research study participants

4. Burden Hours

Total expected burden for this study is 27 hours, as shown in the table below. Participants may spend up to five minutes answering screening questions. Recruited participants may spend up to 60 minutes in the interview itself.

Number Contacted	Time spent Recruiting	Screening Burden	Number of Participants	Session Length	Session Burden	Total Burden
30	5 minutes	2.5 hours	24	60 minutes	24 hours	27 hours

5. Payment to Participants

A $50 incentive will be used to recruit participants ($1200.00 total), to cover the costs associated with participation including internet access and child care.

6. Data Confidentiality

During the interviews, participants will be given a pledge of confidentiality. Remote interview participants will be shown the following information through screen sharing on Microsoft Teams, and asked for their verbal consent prior to beginning the study. Data from remote interviews will be recorded in text documents stored on BLS servers, without participant identifying information.

Do you have any questions before we proceed?

Do you agree to participate?

7. Appendixes

Appendix A: Screenshots of diaries

Appendix B: Cognitive Interviewing protocol

Appendix C: Interview recruitment and eligibility script

Appendix D: Recruitment advertisement

1 These diary designs are intended to be printed but they will be tested on a computer screen in the current study. Because the research questions are about whether respondents understand the instructions and what information to enter, rather than the usability of the forms, we are using digital versions in remote interviews to take advantage of the online mode, including increased geographic diversity and interview scheduling convenience.

2 https://www.bls.gov/cex/research_papers/pdf/2024-ced-diary-form.pdf

3