Note to Reviewer - Pretesting 2019 CE Changes

December 7, 2017

NOTE TO THE REVIEWER OF:	OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research”
FROM:	Erica Yu Research Psychologist Office of Survey Methods Research
SUBJECT:	Submission of Materials for the Pretesting of proposed revisions to the CE Quarterly Interview Survey and Diary Survey 2019

Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.

The total estimated respondent burden hours for this study is 154 hours.

If there are any questions regarding this project, please contact Erica Yu at

202-691-7924.

1. Introduction and Purpose

The Bureau of Labor Statistics has completed several projects with the goal of improving the Consumer Expenditure surveys (CE). Every two years, the questions are evaluated by the program office and questions in need of revision are identified. Before the revisions are made, they are reviewed and tested by the Office of Survey Methods Research (OSMR). This year, in addition to those revisions, the program office has requested OSMR test additional changes being proposed as a part of the new Policy Analysis and Research section and the integration of outlet questions into the Interview and Diary surveys.

Several of the revisions being proposed are to keep current the market basket of goods and services, which provides item weights for the Consumer Price Index (CPI). Given the rapid changes in the marketplace, changes need to be made to the Interview and Diary to ensure that the data accurately reflect consumer buying habits. Based on input from CPI commodity analysts and economists, areas of the Interview and Diary surveys were revised to maintain the most up-to-date market basket possible and to collect outlet information for CPI’s frame of outlets for pricing.

To improve CE data quality, the program office has proposed revisions to reduce respondent burden, update or remove outdated questions, and address general instrument issues.

Additional revisions to the CE are being made to incorporate questions to collect outlet information for CPI. The majority of the revisions are simple translations from the Telephone Point of Purchase Survey to the CE, with proposed question wordings and help screens as replications of current question wordings with a few modifications to account for procedural changes. These revisions to the Interview and the Diary are being tested for comprehension of the questions in the context of the CE and difficulty in reporting. Additional questions with extended reference periods that are different from the standard three months are also being added, and these are being tested for comprehension and difficulty.

The Policy and Analysis Research section questions were developed to inform the upcoming redesign of the survey. Record use was identified as an area of concern for which the program could learn new insights by adding a small number of questions at the end of the final wave of the survey. These questions are being tested for comprehension and to assess the value that the responses can provide to the redesign.

2. Research Design

The purpose of this study is to evaluate whether the proposed revisions to the 2019 CE Surveys and supplementary materials (Infobook, Attachment 1; Help screens, Attachment 2) are clear and understandable to respondents, and are interpreted by respondents as intended, and to explore refinement of the revisions as needed. A complete list of the revisions to be tested is included in Appendix A.

Because the number of proposed revisions exceeds what can be tested in a single session, the full list of items is broken down into modules of related revisions and associated debriefing questions that will be administered together. The set of questions (modules) presented to each participant and their order will vary. Participants will be assigned modules based on their experience with and knowledge of the subject matter, as determined by self-report. Although the content for each participant will vary, the length of the protocol will be fixed for all participants. The full protocol is shown in Appendix B.

Given the breadth of the revisions being tested, the study design uses a mixture of modes and protocols. Modes include in-lab cognitive interviewing, at-home diary completion, and online testing. Each mode is chosen to take advantage of its features:

• In-lab interviewing – to gain insight into the respondent’s thought processes and reactions through interviewer probing. These interviews will be done one-on-one in the OSMR laboratory. Interviews will be conducted by OSMR staff who are experienced in conducting these types of interviews.

• At-home diary completion – to understand diary behavior, which is not only a solitary activity unlike telephone or in-person interviewing, but also intended to capture daily expenses in (close to) real time.

• Online testing through Amazon’s Mechanical Turk (mTurk) service – to identify individuals with expenses that typically have low reporting rates and for which it is difficult to recruit for lab-based studies. These participants will complete an online form which asks them the survey questions of interest and then several debriefing questions, including questions to probe the accuracy of their response to the CEQ questions and about the experience of answering those questions. This online testing is a type of unmoderated cognitive interview. Feedback can be gained from a larger number of participants that are more diverse geographically and in terms of their expenditures than would be possible in the OSMR research lab in Washington, DC.

Protocols for in-lab sessions will be either one or two visits. The one-visit protocols will follow standard cognitive interview format, whereby the interviewer will administer the survey questions followed by a debriefing to understand the responses given and collect the participant’s feedback. Two-visit protocols, which include diary feedback, will follow this format:

• Visit 1: Participants will be asked a shortened version of the CE interview immediately followed by debriefing questions. They will then be introduced to the diary form and the interviewer will explain what should be entered in the different sections of the diary and how to enter it. This is similar to diary placement that is conducted by field interviewers. If a second visit has not been scheduled, one will be made at that time.

• At-Home Diary Completion: Participants will be asked to take the diary home with them and enter every expense made by them or by other members of their household for one week. The interviewer will call the participant approximately 3 days after Visit 1 to see if the respondent has any questions about filling out the diary or if they have had any issues with filling it out. These questions and issues will be recorded.

• Visit 2: The participant will return to the lab between 6 and 10 days after their first visit and bring the diary with them. Participants will be asked additional interview questions, including record use questions. Afterwards, they will be debriefed about their diary-keeping experience.

The findings will be evaluated qualitatively. For diary and online testing, quantitative analyses will be considered but statistical significance is not a goal. Rather, the debriefing feedback from all modes is the primary outcome to be used to evaluate the proposed revisions and make any recommendations.

As this is pretesting, we expect that modifications may be made during the course of the study based on initial results. Although the goals of the testing, and overall design, will remain the same, findings from preliminary results may be used to improve the interview questions. Modifications are likely to range from slight changes to question wording (e.g., naming examples of the expense category) or additional instructions (e.g., inclusion or exclusion instructions).

3. Participants

In-Lab Interviewing. Thirty-five participants will be recruited from a participant database maintained by OSMR. Efforts will be made to select participants with varying levels of education, income, and occupation, and familiarity with the expenditures being tested based on self-reported information provided during the initial recruitment process.

Online Testing. Three hundred participants will be recruited using a convenience sample from Amazon Mechanical Turk of adult U.S. citizens (18 years and older).

4. Burden Hours

The total burden hours for this study will be 154.

In-Lab Interviewing. The number of participants needed for testing a given survey question is dependent upon the results and the point at which saturation is reached and no further information is being gained from additional interviews. Given that variability, our goal is to interview no more than 35 participants in the OSMR lab, with 20 of these participants completing the diary and returning for a second visit. We anticipate that each session will last 45 minutes but no longer than one hour, for a total of 55 burden hours for the interviews (55 sessions X 1 hour). Scheduling an appointment is expected to take no longer than 5 minutes and participants will not be compensated for time spent on scheduling. We estimate we will need to contact approximately 45 people in order to get 35 interviews. This will add an additional 3.75 burden hours (45 participants X .0833 hours) to the study, for a total of 58.75 burden hours for cognitive interviews.

At-home Diary Completion. It is estimated that participants will spend 1 hour on at-home diary completion, including a mid-week call to discuss any questions about the diary. This will add a total of 20 burden hours (20 participants X 1 hour) for diaries.

Online Testing. Each of the revisions that will be tested online will be treated as an independent module (see Appendix B). Our goal is to obtain feedback from 100 participants for each module. We anticipate that the task will take no longer than 15 minutes, including screening, demographics, target questions, and debriefing questions for a total of 75 burden hours (3 modules X 100 participants X 0.25 hours).

5. Payment to Participants

A total of $2,650 will be spent on payments to participants.

In-Lab Interviewing. The 20 participants completing the two-visit protocol will each be paid a total of $80 for their participation. Because it is vital that the participant returns for the second interview, we will compensate the participant $30 for their first visit and $50 at the time of their second visit. The 15 participants completing the one visit protocol will each be paid $40 for their participation. A total of $2,200 will be paid to participants for cognitive interviews.

Online Testing. Each of the 300 participants will be paid $1.50 for participating in the study, a typical rate provided by Mechanical Turk for similar tasks; a total of $450.00 will be paid directly to Amazon Mechanical Turk for participant payments.

6. Data Confidentiality

In-Lab Interviewing. Participants will be informed as to the voluntary nature of the study. Participants will also be informed that the study will be used for internal purposes to improve the design of a national Consumer Expenditure interview survey. Participants coming into the lab will be given a consent form to read and sign (See Appendix C). Information related to this study will not be released to the public in any way that would allow identification of individuals except as prescribed under the conditions of the Privacy Act Notice.

Online Testing. Recruiting of participants will be handled by Amazon Mechanical Turk. Once participants are recruited into the study, they will be provided a link to the survey, which is hosted by Survey Monkey. The data collected as part of this study will be stored on Survey Monkey’s servers. Using the language shown below, participants will be informed of the voluntary nature of the study and they will not be given a pledge of confidentiality.

This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141 (Expiration Date: April 30, 2018). You are not required to respond to this collection unless it displays a currently valid OMB control number. We will use the information you provide for statistical purposes only. Your participation is voluntary, and you have the right to stop at any time. This survey is being administered by Survey Monkey and resides on a server outside of the BLS Domain. The BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response. By proceeding with this study, you give your consent to participate in this study.

Appendices and Attachments

Appendix A: List of revisions to be tested

Appendix B: In-lab and Online Protocol

Appendix C: Consent form

Attachment 1: Infobook pages

Attachment 2: Help screens

Attachment 3: Diary

Appendix A: List of Revisions to be Tested

Below is a list of the proposed revisions and the mode of testing to be used in the study. Several of the revisions are planned for both lab and online testing.

In-Lab Interviewing

• Add Interview questions to collect pet food expenses

• Add Interview questions about record use

• Modify the Diary to collect outlets

• Change the Interview questions collecting swimwear expenses; collect activewear in existing categories

• Change the Interview questions collecting coats and jackets expenses

• Add Interview questions collecting outlets

Online Testing

• Combine campers into a single item code

• Change the Interview questions collecting tire repair expenses

• Add Interview questions to capture Bike-Sharing expenses

• Change the Interview questions collecting swimwear expenses; collect activewear in existing categories

• Change the Interview questions collecting coats and jackets expenses

• Add Interview questions collecting outlets

• Add questions collecting prepaid cellular minutes expenses

Appendix B, Part 1: In-Lab Protocol

The protocol includes the questions that will be tested, additional questions that provide context for the tested question, and debriefing questions. The goal is to identify the most effective wording for questions and response choices by understanding the associations that participants have with terms used in the question wording and how they interpret question meanings. The interviewer may ask unscripted follow-up questions (e.g., “Can you tell me more about that?”) if a participant’s answer suggests additional, pertinent information can be gained. Note that the survey questions and the debriefing questions are listed together in this document, but all survey questions will be asked first and all debriefing questions will be asked at the end of the interview.

Introductory Material

• Hi! Thank you for coming in today.

• I am …... [This is my colleague ( ) who will be taking notes for us today]

• Have you participated in any of our studies before? (if yes, Which ones?)

• [Sign Consent Form/Permission to audiotape] (see Appendix C)

• Explanation:

  • We are going to be working with the Consumer Expenditure Survey, an ongoing survey that provides information about the buying habits of American consumers.

  • Our purpose in bringing you in today is to have you provide feedback on the survey - we’re trying to find out what you think a question is asking for and how hard it is to answer. We are not here to evaluate you, we are looking to improve the survey, so there are no wrong answers. When you tell us that you find something about the survey to be difficult to understand or do, then that helps us to know that we need to change the survey. All the information you give us will be kept completely confidential, and will be used to improve the survey.

• Any questions before we begin?

• The Consumer Expenditure Survey is a household survey so I am going to ask you to report expenses that you or anyone else in your household have made during the last three months. To get started I’d like to get some information about you and the people in your household.

• How many people are in your household, including yourself?

• [if more than 2 in HH] OK, and who is the next person? I don’t need their name, just how they are related to you.

Name	Relationship	Age	Gender
	Participant		M F
			M F
			M F
			M F
			M F

Two visit protocol

Visit 1

Next we’re going to be going through a short version of the interview. Typically, when we interview people we go through lots of questions and it takes about an hour for just the interview itself. What we’re going to do today is much shorter – I will leave out a lot of the questions – and then we’ll go over what it was like to answer the questions we did ask you. So, the first part of what follows will be a shortened interview, and then the next part will be about getting feedback on what that experience was like. After that, we’ll talk about the diary that you’re going to be taking home with you and filling out throughout the week.

Now I am going to ask about expenses for the purchase or rental of selected household items. Please include any shipping and handling charges with the cost of any item that was shipped.

1. Camping equipment

2. Hunting and fishing equipment, including all guns

3. Winter sports equipment

4. Water sports equipment

If yes to any –

• What did you purchase (or rent)?

  • [Verbatim]

• Was this item

  • Purchased for someone inside the household?

  • Rented

  • Purchased for someone outside your household?

• [If not rented] When did you purchase it?

  • January

  • …

  • December

• What did it cost?

  • [Numeric]

• Did this include sales tax?

  • Yes

  • No

• Was this expense combined with anything else?

  • [If yes] What other item is the (description) combined with?

• What is the name of the business where this was purchased?

  • [Verbatim]

• Any other [FILL]?

Now I am going to ask about clothing, footwear, and accessories, for infants, children, and adults. You may find it helpful to refer to receipts, credit card statements or other records to answer the questions. Please include any shipping and handling charges with the cost of any item that was shipped.

Since the first of (reference month), have (you/you or any members of your household) purchased any of the following items either for members of your household or for someone outside your household?

1. Outerwear

2. Sport coats, tailored jackets, or blazers

3. Shirts, sweaters, blouses, or tops

4. Pants, jeans or shorts

5. Dresses

6. Skirts

7. Accessories

8. Swimwear, swim cover-ups, or swimwear accessories

9. Watch or jewelry repair

If yes to any –

• What did you buy?

  • [Verbatim]

• Was this (were these) purchased for someone inside or outside of your household?

  • Inside your household

  • Outside your household

  • Both inside and outside your household

• Was this (Were these) purchased for...

  • Man, age 16 and over

  • Woman, age 16 and over

  • Boy, age 2 to 15

  • Girl, age 2 to 15

  • Child under 2 years old

• When did you purchase it?

  • January

  • …

  • December

  • Same amount each month

• (What is your monthly expense?/How much did it cost?)

  • [Numeric]

• Did this include sales tax?

  • Yes

  • No

• Was this expense combined with anything else?

  • [If yes] What other clothing is (description) combined with?

• For 1-8: What is the name of the business where this was purchased?

• For 9: What is the name of the business where you had this expense?

  • [Verbatim]

• Was this paid for -

  • Online?

  • In person?

  • By mail or telephone order?

  • Other

• If in person: In what city and state is [CLASTORE] located?

Debriefing:

• Was there anything that you considered reporting but didn’t? [repeat back to the participant the list of expenses reported]

  • Did you not report these initially because you weren’t sure if we wanted them, you forgot, or something else?

• Did you shop at more than one place? How did you decide which store name to report?

Loop through for each category:

• When we asked you about [fill], what does that mean to you?

• Take a look at this list. Look at each item and mark whether the item would definitely be included in the category or definitely not included in the category. If you don’t know what an item is or are not sure whether it should be in the category, mark “not sure”

Water sports equipment	Definitely IN	Definitely OUT	Not sure
Swim cap
Goggles
Swimsuit*
Swimsuit cover-up/wraparound*
Water shoes

(Present together with Coats, Jackets, and Furs)

Outerwear	Definitely IN	Definitely OUT	Not sure
Ski boots*
Winter scarf*
Sweater*
Sportsjacket/Blazer*
Snow suit
Hunting jacket
Fur coat
Hooded sweatshirt/Hoodie*
Leather jacket
Down vest

Now, please do the same exercise for the category “Coats, jackets, and furs”. I realize that this might sound similar to “outerwear” so your answers might be similar, but please go ahead and fill this out anyway.

Coats, jackets, and furs	Definitely IN	Definitely OUT	Not sure
Ski boots*
Winter scarf*
Sweater*
Sportsjacket/Blazer*
Snow suit
Hunting jacket
Fur coat
Hooded sweatshirt/Hoodie*
Leather jacket
Down vest

Shirts, sweaters, blouses, or tops	Definitely IN	Definitely OUT	Not sure
Graphic t-shirts
Hooded sweatshirt/Hoodie
Running gear like reflective shirts
Sportsjacket*
Workout tank tops
Leotard for ballet
Active wear

Pants, jeans, or shorts	Definitely IN	Definitely OUT	Not sure
Skirts*
Yoga pants
Slacks/Trousers
Rompers
Running gear like running pants
Leggings
Bike shorts
Snowsuits
Active wear

Swimwear, swim cover-ups, or swimwear accessories	Definitely IN	Definitely OUT	Not sure
Board shorts
Swim cap*
Goggles*
Beach dress cover-up
Wet suit*
Water shoes*
Flip flops*

• Did seeing that list bring to mind any related expenses that you haven’t already told me about?

• [For relevant “Not sure” items, probe for reason.]

• [If participants marked out-of-scope marked with *) items as “Definitely In” or in-scope items as “Definitely Out”, probe to find out more.]

• Now please take a look at this page (show proposed Infobook pages). This page is what we will show to respondents during the survey, to provide examples of the kinds of things we are looking for. Does seeing these lists change what you thought about these categories?

Since the first of (reference month), have (you/you or any members of your household) had expenses for any of the following, either for (you/your household) or for someone outside your household?

1. Fresh flowers or potted plants?

2. Pet food?

3. Purchase of pets, pet supplies, or medicine for pets?

4. Pet services?

5. Veterinarian expenses for pets?

If yes to any –

• What was the expense for?

  • [Verbatim]

• In what month did you have this expense?

  • January

  • …

  • December

  • Same amount each month

• Was this expense for someone inside or outside your household?

  • For household

  • For someone outside your household

• What was the total amount of this expense?/What is your monthly expense?

  • [Numeric]

• Was this expense combined with anything else?

  • [If yes] What other expense is the [description] combined with?

• What is the name of the business where this was purchased?

  • [Verbatim]

Debriefing:

• Did your household have any expenses for the care of any animals, whether you consider them “pets” or not? [Collect list of all animal types]

• Was there anything that you considered reporting but didn’t? [repeat back to the participant the list of expenses reported]

  • Did you not report these initially because you weren’t sure if we wanted them, you forgot, or something else?

• About how many expenses for pet food did you have since the first of (reference month)? Was it easy or difficult to add the dollar amounts together? [Probe for difficulty in coming to a sum total expense]

• Did you shop at more than one place? How did you decide which store name to report?

• When we asked you about “pet food”, what does that mean to you?

• Take a look at this list. Look at each item and mark whether the item would definitely be included in the category or definitely not included in the category. If you don’t know what an item is or are not sure whether it should be in the category, mark “not sure”

  Pet food	Definitely IN	Definitely OUT	Not sure
  Pill pockets
  Greenies
  Peanut butter given to a dog that works on a farm*
  Prescription cat food bought from the vet’s office
  Heartworm medication*
  Dog daycare meal fees
  Wild bird seed blend*
  Training reward dog treats
  Food for a cow being raised to be sold*

• Did seeing that list bring to mind any related expenses that you haven’t already told me about?

• [For relevant “Not sure” items, probe for reason.]

• [If participants marked out-of-scope items as “Definitely In”, probe to find out more.]

Diary placement at Visit 1

Now I will describe how to use the diary; what information we want you to enter and where it goes.

• The purpose of the diary is to capture everything your household spends money on during the course of the week. Big things, like rent, and small things, like a pack of gum, should all get recorded here.

• The diary is divided up by day and by type of item. Each day of the week will have its own set of pages. These days refer to the day the purchase was made, not the day you are entering it. So if you buy some groceries on Monday, you would flip to this page and enter them.

• In addition to days of the week, the diary is also broken into categories; Food and Drink Away from Home (or restaurant-type meals); Food and Drink for Home Consumption (things that you eat or prepare at home); Clothing, Shoes, Jewelry, and Accessories; and All Other Products and Services (which is everything that doesn’t fit into the other three categories.
  The reason we divide up the world of purchases this way is because we’re interested in slightly different information depending on what category an item falls under.

• Let’s start with Food and Drink Away From Home. For this category, enter:

• What meal it was purchased for

• The name of the restaurant or vendor

• We also want you to check one of these boxes describing what type of place you purchased the food or drink from; fast food, a full service restaurant, a vending machine, or a cafeteria.

• You would then enter the total cost of the food or drink, including tax and tip.

• If alcohol was purchased, you would check what types of alcohol were purchased and enter the price of the alcohol in this box here

• For Food and Drink Away From Home, you enter the whole meal as one line. For everything else (groceries, clothes, bills, paper towels) we need each item entered separately.

• In Food and Drink for Home Consumption, for example, if you spent $20 at the grocery store, we need to know that you spent $3.99 on milk, $2.50 on bread, $1.99 on frozen peas, and so on.

• Also for grocery items we need to know what type of packaging the item came in…whether it was fresh, frozen, bottled or canned, or something else.

• We need the price for that individual item without tax, and

• We need to know if you purchased it for someone outside of your household, such as a gift or a donation.

• And we need to know the name of the store or website where it was purchased. If you are listing more than one item from the same place, you can just draw a line down like this to show that it’s the same.

• The next category is Clothing, Shoes, Jewelry, and Accessories. Accessories, here, means clothing accessories; things like hats, belts, wallets, or purses. Like with groceries, we need each item entered separately. So if you buy a shirt and a pair of pants, each of those should be on a separate line.

• We need a description of what you bought

• The cost of that item, without tax

• The gender and age category of the person it was purchased for

• And whether it was purchased for someone outside your household

• Our last category is All Other Products, Services, and Expenses. This is a catchall category for anything that doesn’t fit into the other three categories. This will include things like dry cleaning, movie tickets, gasoline. These too need to be itemized.

• For these items, we need a description of what was purchased

• The cost, without tax

• Whether it was purchased for someone outside your household, and

• The name of the store or website where it was purchased.

• If you have any questions about what to enter into the diary or where to enter it, there are examples at the top of the page as well as in the front.

• Do you have any questions?

• Throughout the week, I’d like to ask you to do two things.

  • Please hold on to any receipts or bills for things that you paid and recorded in the diary. I’ll ask you to bring those in with you to your next appointment so we can use them to talk about the diary.

  • Please also take home this list with you and spend a few minutes gathering the information you would use to help you talk about these expenses. For example – if you have a mortgage, then that’d probably be your monthly mortgage bill. I don’t need to see the actual expenses but I want to talk about the documents – what bills, account statements, etc – that you would use in an interview.

    • Housing

    • Education

    • Vehicle Maintenance and Repair

    • Income

• Have you scheduled your visit for next week?

• [if No] OK, can we schedule you now?

• If you have any questions during the week, my phone number is written on the diary. If I’m not there, please leave a message and I’ll get back to you as soon as possible.

• Is it alright if I call you on [THREE DAYS AFTER MEETING] to see how things are going?

• [if Yes] What’s the best number to reach you at?

• [if No] That’s OK. I’ll see you at our meeting next week and see how things went.

Mid-week Phone call

• Hi. This is ______ from BLS. I’m calling to see how things are going with the expenditure diary. [Record participant’s open-ended answer]

• Have you entered any expenses into the diary? Yes, No

• Do you have any questions? Yes, No

  • [Record participant’s questions]

• Thank you for your participation so far. Please remember to bring the bills, receipts, account statements, or any other documents that you think would be helpful in an interview, as well as any receipts from expenses from this week. I will see you on ______.

Visit 2

Welcome back. Today we will be testing some additional interview questions and also talking about the diary.

[Sign consent forms and give payment to participant.]

Now I am going to ask about household expenses. Please refer to any billing statements or other records you have when answering these questions. Please remember to include any bills you receive or pay online or have automatically deducted. Report any bill you have received, even if the bill has not been paid.

• Since the first of the reference month not including this month, have you/you or any members of your household paid for internet or wireless internet access away from home, at places such as cafes, hotels, or airplanes?

• Yes

• No

• What was the total amount paid?

  • [Numeric]

• If Yes, then ask: What is the name of the business where you had this expense?

  • [Verbatim]

• Was this service paid for –

• Online?

• In person?

• By mail or telephone order?

• Other

• (If answered in person): In what city and state is (business name) located?

Debriefing:

• [If no] You reported that you did not have an expense for internet or wireless internet access away from home, at places such as cafes, hotels, or airplanes, since the first of the reference month. Did you have this expense in the last year? Yes, No

• If yes, ask outlet questions

• If no, skip to final feedback question

• You reported that you had an expense for internet or wireless internet access away from home. Please briefly describe in your own words the expense you had for internet away from home. For example, you were on an airplane and you paid for internet access. [open]

• What is the name of the business where you had this expense? [open]

• You reported that the name of the business where you had this expense is [FILL].

How confident are you that the business you reported is the business that you paid for the internet service?

• Very confident

• Moderately confident

• Somewhat confident

• Not at all confident

How confident are you that the name that you reported could be used to identify the business, such as in the phonebook or on the internet?

• Very confident

• Moderately confident

• Somewhat confident

• Not at all confident

• Did you have more than one expense since the first of the reference month? Yes, No

  • [If yes] How did you decide which business name to report? [open]

• Did you have more than one expense during the last year? Yes, No

• [If yes] How did you decide which business name to report? [open]

• You reported that you paid for this expense [FILL].

How confident are you that the payment method you reported is the method you used?

• Very confident

• Moderately confident

• Somewhat confident

• Not at all confident

• You reported the location as [FILL]. How confident are you that the location you reported is the location from which the business operates?

  • Very confident

  • Moderately confident

  • Somewhat confident

  • Not at all confident

• Do you have any feedback about the way that this question is worded?

What is the name of the business where you had this expense? [open]

Now I am going to ask some questions about medical payments and reimbursements.

Since the first of the reference month, have you or any members of your household made payments for the following?

1. Eye examinations, treatment, or surgery

2. Purchase of eye glasses or contact lenses

3. Dental care

4. Hospital room or hospital services

5. Services by medical professionals other than physicians

6. Physician services

7. Lab tests or x-rays

8. Other medical care and services

9. Hearing aids

• If yes ITEM 2, 9 then ask:

  1. What is the name of the business where this was purchased?

  2. Was this paid for -

     • Online?

     • In person?

     • By mail or telephone order?

     • Other

  3. (If answered in person): In what city and state is (business name) located?

• If yes ITEM 1, 3, 5, 6, or 8, then ask:

  1. What is the name of the business that provided this service?

If ITEM = 6: *If response includes the word “hospital”, probe for specific physician’s name. If unable to obtain physician’s name, enter hospital name.

2. Was this paid for -

   • Online?

   • In person?

   • By mail or telephone order?

   • Other

3. (If answered in person): In what city and state is (business name) located?

Debriefing:

• [If no] You reported that you did not have an expense for [FILL] since the first of the reference month. Did you have this expense in the last year? Yes, No

  • If yes, ask outlet questions

  • If no, skip to final feedback question

• You reported an expense for [FILL]. Please briefly describe the expense in your own words. For example, you went to a doctor’s office and paid a co-pay. [Open]

• You reported the name of the business as [FILL]. Was it easy or difficult to report the business that provided the service?

  • Very confident

  • Moderately confident

  • Somewhat confident

  • Not at all confident

• How confident are you that the name that you reported could be used to identify the business, such as in the phonebook or on the internet?

  • Very confident

  • Moderately confident

  • Somewhat confident

  • Not at all confident

• Did you have more than one expense?

  • [If yes] How did you decide which business name to report?

• Was it easy or difficult to report the expense amount? Tell me more about that.

  • Did you receive any bills after the date of service?

  • Did you or will you get reimbursed for any amount?

  • How confident are you that this is the final out-of-pocket amount that you paid?

    • Very confident

    • Moderately confident

    • Somewhat confident

    • Not at all confident

• When you heard the question “Was this paid for online, in person, or by mail or telephone order?”, tell me what that made you think of.

  • What information did you think we were trying to collect?

  • Was it easy or difficult to report the payment method?

  • Tell me more about that.

  • Did you ever need to use more than one method of payment?

  • How confident are you that this is the method of payment that you used?

    • Very confident

    • Moderately confident

    • Somewhat confident

    • Not at all confident

• At the time of this service, did you have health insurance that covers some or all of the cost of medical expenses? Did you account for any benefits or reimbursements that this insurance provides?

• Do you have any feedback about the way that this question is worded?

What is the name of the business where you had this expense? [open]

We just went through a short version of the Consumer Expenditure Survey interview. I asked you whether you had expenses, and if so, how much you paid for them, what month you had the expense, the name of the business or store that you paid, and whether that expense was online or in-person. At the end of that survey, we then go on to ask a few more questions – and now I want to test out those questions with you.

Please keep in mind your experience having answered some of these interview questions and also what you learned during the week from thinking about and gathering records like bills, receipts, and account statements. These questions are about records in the interview and what you think would be useful for others to know about how best to prepare for an interview. Just like the questions asking what you bought, these are questions that we are testing out before putting in our national interview.

Now I have a few final questions for you about the bills, receipts, and account statements that you keep or have access to. During the survey, I mentioned that you would find it helpful to have these kinds of records - like bills, receipts, or statements - available to help you to report your expenses accurately. We want to encourage respondents to use more records during the interview and need your help answering the following questions.

1. Now that you have completed this interview, which would you say is more useful for helping you report your expenses: itemized bills and receipts or bank account and credit card statements, or something else?

   1. Itemized bills and receipts

   2. Bank account and credit card account statements

   3. [do not read aloud] Other—specify [ ]

   4. [do not read aloud] Don’t know

2. If you had received in the mail a list of all of the expenses that you would be asked about, would you have used it to prepare for the interview?

   1. Yes

   2. No

2a) If Q2=Yes, THEN ASK: How would you have used it?

SELECT ALL THAT APPLY

3. Get an idea of what the survey was about

4. Find bills or account statements for some of the expenses

5. Ask other household members for their expenses

6. [do not read aloud] Other—specify [ ]

2b) If Q2=No, THEN ASK: Please explain why you would not have used the list.

[record verbatim]

3. For each of the following types of records, please tell me how you would use it in an interview: would you hand it to an interviewer to look at, look at it for yourself during an interview but not show it to the interviewer, or not bring it to an interview at all?

DO NOT READ ALOUD ITALICIZED TEXT

USE “OTHER” ROWS TO RECORD EXAMPLES GIVEN BY THE RESPONDENT

VERSION A for half of participants:

	Hand to interviewer	Look at privately	Not bring to interview	Don’t have these
Electricity bills
Mortgage bills
Receipts from a night out
Receipts for clothes purchases
Paystub
Your personal credit card or checking account statements
[Other household member credit card or checking account statements]
Other – [specify]
Other – [specify]
Other – [specify]

VERSION B for half of participants:

	Hand to interviewer	Look at privately	Not bring to interview	Don’t have these
Itemized detail about household expenses like mortgages
Itemized detail about personal purchases like clothing
Itemized bills from restaurants and bars
Paystub
Personal credit card or checking account statements, which show total amounts spent
[Other household member credit card or checking account statements]
Other – [specify]
Other – [specify]
Other – [specify]

Debriefing:

• What does “itemized bills and receipts” mean to you? Can you give some examples from real life?

• What does “personal credit card or checking account statements” mean to you? Can you give some examples from real life?

• Are there any other types of record that you think might use in an interview?

• What does “hand to an interviewer” mean to you?

• What does “look at privately” mean to you?

• Tell me more about why you’d be ok sharing [some records] but not [others].

• How did it feel to be asked a question about whether you would hand something to an interviewer when the interviewer was the one asking you the question?

• In one of the test questions that I asked you, I referred to a list that we might mail out ahead of the interview that shows what is going to be asked about. The question was - If you had received in the mail a list of all of the expenses that you would be asked about, would you have used it to prepare for the interview? And how?

SELECT ALL THAT APPLY

• Get an idea of what the survey was about

• Find bills or account statements for some of the expenses

• Ask other household members for their expenses

• [do not read aloud] Other—specify [ ]

Are there any other ways that you can think of that we should list?

• Did you gather any records at home? Tell me what you considered useful – why you chose those records. [Record the types of records, whether itemized]

Diary Pick-up at Visit 2

Ok, now let’s switch gears and talk about the diary.

• What was it like filling out the diary this week?

• When did you fill out the diary?

• At the time that you had the expense

• On the day that you had the expense

• At the end of the week

• Did you have any issues while entering items into the diary?

  • Yes

  • No

• [if Yes] Can you tell me more about that? [Record participant’s open-ended answer]

• Did you use the instructions?

  • Yes

  • No

• [if Yes] Can you tell me more about that? [Record participant’s open-ended answer]

• Did you look at the examples?

  • Yes

  • No

• [if Yes] Can you tell me more about that? [Record participant’s open-ended answer]

• Were there any expenses that you did not enter into the diary?

  • Yes

  • No

• [if Yes] Please describe those expenses and why you didn’t enter them. [Record participant’s open-ended answer]

• We’re particularly interested in the names of the store, website, and restaurant that you went to. How confident are you that the business names that you recorded are the actual places where you had the expense? Would we be able to look these businesses up, say, in the phone book or on the internet, or do they go by a different legal name?

• Did you have any difficulty deciding what to report or how?

• Did you bring in your receipts? Review receipts with participant and assess whether they match the recorded diary entries. Probe for sources of error.

• Review diary entries for errors and probe for sources of error. Focus on outlet accuracy and level of detail.

One visit protocol

Next we’re going to be going through a short version of the interview. Typically, when we interview people we go through lots of questions and it takes about an hour for just the interview itself. What we’re going to do today is much shorter – I will leave out a lot of the questions – and then we’ll go over what it was like to answer those questions. So, the first part of what follows will be a shortened interview, and then the next part will be about getting feedback on what that experience was like.

First, let's talk about the construction or alteration of property you own or rent. You should not include jobs that have been or will be totally reimbursed by someone outside your household such as a landlord.

• Since the first of the reference month, have you or has any member of your household had any expenses for-

  1. Homes under construction including a vacation home or second home?

  2. Building an addition to the house or a new structure, such as a porch, garage, or new rooms?

  3. Finishing a basement or an attic or enclosing a porch?

  4. Remodeling one or more rooms in the house?

  5. Landscaping the ground or planting new shrubs or trees?

  6. Building outdoor patios, walks, fences, or other enclosures, driveways, or permanent swimming pools or hot tubs?

  7. Repairing outdoor patios, walks, fences, driveways, or permanent swimming poos?

  8. Painting, either interior or exterior, or wallpapering?

  9. Plastering or paneling?

  10. Plumbing or water heating installations or repairs?

  11. Electrical work?

  12. Heating or air conditioning jobs?

  13. Flooring installation, repair, or replacement including carpeting, wood, vinyl, and tile?

  14. Insulation?

  15. Roofing, gutters or downspouts?

  16. Siding?

  17. Installation, repair, or replacement of window panes, screens, storm doors, awnings, and the like?

• What work was done? What other work was included in this job?

• What is the name of the business that provided the service contract, maintenance, or repair?

• In what city and state is (business name) located?

Since the first of (reference month), excluding this month, have (you/you or any members of your household) purchased any materials or supplies for JOBS NOT YET STARTED that include Painting/wallpapering?

• What was the total cost of these materials and supplies?

• What is the name of the business where the materials were purchased?

• Were they paid for -

  • Online?

  • In person?

  • By mail or telephone order?

  • Other

• (If answered in person): In what city and state is (business name) located?

Now I am going to ask about expenses for the purchase or rental of household appliances and other selected items. Please include any shipping and handling charges with the cost of any item that was shipped.

Since the first of (reference month), have (you/you or any members of your household) purchased or rented any of the following items?

1. Lawn mowing machinery, or other yard equipment

2. Power Tools

3. Non-power tools

4. General sports equipment (exclude athletic shoes for sports related use, such as football, baseball, soccer or bowling)

5. Health and exercise equipment

6. Camping equipment

7. Hunting and fishing equipment, including all guns

8. Water sports equipment

9. Outboard motors

10. Bicycles or bicycle equipment

11. Tricycles or battery powered riders

12. Playground equipment

13. Other sports or recreation equipment

14. Video game hardware or accessories

15. Toys or games?

16. Arts or crafts kits?

• What did you purchase or rent?

• Was this item…

  • Purchase for someone inside your household?

  • Rented?

  • Purchased for someone outside your household?

• [if purchased] When did you purchase it?

  • Jan

  • …

  • Dec

• What did it cost?

  • [Numeric]

• Did this include sales tax?

  • Yes

  • No

• Was this expense combined with anything else?

  • What other item is the [FILL] combined with?

• Did you purchase any other [FILL]?

Now I am going to ask about expenses for home furnishings and related household items. Please include any shipping and handling charges with the cost of any item that was shipped.

Since the first of (reference month), have (you/you or any members of your household) purchased for (you/your household) or for someone outside of your household any –

1. Sofas?

2. Living room chairs?

3. Living room tables?

4. Ping-pong, pool tables or other similar recreation room items?

5. Other living room, family, or recreation room furniture including desks, wall units, and shelving?

6. Living room furniture combinations?

7. Dining room or kitchen furniture?

8. Mattresses or box springs?

9. Bedroom furniture other than mattresses or box springs?

10. Infants furniture?

11. Infants equipment

12. Patio, porch, or outdoor furniture

13. Barbeque grills or outdoor decorative items

14. Rugs or other non-permanent floor coverings, including carpet squares

15. Curtains or drapes

16. Blinds, shades or other window coverings

If yes to any –

• What did you purchase? [Verbatim]

• In what month did you purchase it?

  • Jan

  • …

  • Dec

• Was this purchased for your household or for someone outside your household?

  • For use by household

  • For someone outside the household

• What was the purchase price? [Numeric]

• Did this include sales tax? Yes, No

• Was this expense combined with anything else?

  • What was combined with [FILL]?

• Did you purchase any other [FILL]? Yes, No

Now I am going to ask some questions about medical payments and reimbursements.

Since the first of the reference month, have you or any members of your household made payments for the following?

1. Eye examinations, treatment, or surgery

2. Purchase of eye glasses or contact lenses

3. Dental care

4. Hospital room or hospital services

5. Services by medical professionals other than physicians

6. Physician services

7. Lab tests or x-rays

8. Other medical care and services

9. Hearing aids

• If yes ITEM 2, 9 then ask:

  1. What is the name of the business where this was purchased?

  2. Was this paid for -

     • Online?

     • In person?

     • By mail or telephone order?

     • Other

  3. (If answered in person): In what city and state is (business name) located?

• If yes ITEM 1, 3, 5, 6, or 8, then ask:

  1. What is the name of the business that provided this service?

If ITEM = 6: *If response includes the word “hospital”, probe for specific physician’s name. If unable to obtain physician’s name, enter hospital name.

2. Was this paid for -

   • Online?

   • In person?

   • By mail or telephone order?

   • Other

3. (If answered in person): In what city and state is (business name) located?

Now, as I mentioned earlier, we’re skipping a lot of questions that are in the original survey, just for time’s sake. Normally, when we reach close to the end of the interview, we would ask you about expenses for food, beverages and other items (you/your household) (have/has) purchased since the first of (reference month). We would cover expenses for groceries, dry cleaning, hair cuts, bank service fees, taxis, public transportation, that sort of thing. But, for time’s sake, let’s just say we just finished asked you about those things and now let’s continue with the survey.

The next few questions are about your expenses during the last five years, that is from [2014] to today. We have used the responses you’ve already given us to shorten this part of the interview, so I’ll only be asking you about expenses that you told me that you did not have in the last three months.

During the last five years since 2014, have you or any members of your household had any expenses for….

Read one at a time

The purchase of any outboard motors or powered sports vehicles?	If no to (ITEM =33 in Section 6B) and (ITEM = 6 in Section 11)
The purchase of any unpowered boats or trailers?	If no to (ITEM = 3; and ITEM = 4,and, ITEM = 7, and, ITEM = 8 in Section 11)
The purchase of any power tools?	If no to (ITEM = 17 in Section 6B)
The purchase of any nonpowered tools or miscellaneous hardware?	If no to (ITEM = 18 in Section 6B)
Service contracts, maintenance, or repairs for major appliances?	If no to (ITEM = 1 and 2 in Section 7)
Repairing, refinishing, or reupholstering furniture?	If (REPFURN = No in Section 8B)
The purchase of any materials or supplies for any paint, wallpaper, tools or related supplies?	If no to (ITEM = 8 in Section 5)
Inside Home Maintenance and Repair?	If no to (ITEM = 3, 4, 8, 9, 10, 11, 12, and 17 in Section 5)
The purchase of any video game hardware, games or accessories?	If no to (MINAPPLY = 700, ITEM = 12 in Section 6B) and (ITEM = 23 in Section 17)
The purchase of any bicycles or bicycle accessories?	If no to (ITEM = 34 in Section 6B)
The purchase of any toys, games, hobby supplies, or playground equipment?	If no to (MINAPPLY = 550 and 560, ITEM 35, and 36 in Section 6B) and (MISSCODE = 330 and 340, ITEM = 14, and 15 in Section 19A)
The purchase of any hunting, fishing or camping equipment?	If no to (MINAPPLY = 490 and 500, ITEM = 29, and 30 in Section 6B)
The purchase of any general sports equipment	If no to (ITEM = 27, 28, 31, 32, and 37 in Section 6B) and (ITEM = 4 in Section 8A)

If item 1, 2, 3, 4, 9, 10, 11, 12, 13= Yes

1. What is the name of the business where this was last purchased?

2. How much did you spend?

3. Was this paid for -

   1. Online?

   2. In person?

   3. By mail or telephone order?

   4. Other

4. (If answered in person) In what city and state is (business name) located?

If item 5 = Yes

1. What is the name of the business that provided the last service contract, maintenance, or repair?

2. What was the total cost?

3. Was this paid for -

1. Online?

2. In person?

3. By mail or telephone order?

4. Other

4. (If answered in person) In what city and state is (business name) located?

If item 6 = yes

1. What is the name of the business that last repaired, refinished, or reupholstered the furniture?

2. What was the total expense?

3. Was this service paid for -

1. Online?

2. In person?

3. By mail or telephone order?

4. Other

4. (If answered in person) In what city and state is (business name) located?

If item 8 = yes

1. What is the name of business that provided the last service contract, maintenance, or repair?

2. How much did you spend?

3. In what city and state is (business name) located?

If item 7 = yes

1. What is the name of the business where the materials were last purchased?

2. How much did you spend?

3. Were they paid for –

1. Online?

2. In person?

3. By mail or telephone order?

4. Other

4. (If answered in person) In what city and state is (business name) located?

During the last five years since 2014, have you or any members of your household had any expenses for….

The skip logic in the table refers to the production instrument questionnaire numbering but the items being referred to will be used by the interviewer to identify responses from earlier in the interview for use in the skip logic in the study.

The purchase of any floor coverings such as hard surface tiling, carpets or scatter rugs	If no to (ITEM =13 in Section 5A and ITEM = 26 in Section 8A)
The purchase of any window coverings such as curtains, drapes or blinds?	If no to (ITEM =27 and 28 in Section 8A)
The purchase of any infants’ furniture?	If no to (ITEM = 10 in Section 8A)
The purchase of any bedroom furniture including mattresses or springs?	If no to (ITEM = 8 and 9 in Section 8A)
The purchase of any outdoor furniture?	If no to (ITEM = 12 in Section 8A)
The purchase of any living room, dining room or kitchen furniture?	If no to (ITEM = 1, 2, 3, 7, and 25 in Section 8A)
The purchase of any other furniture including entertainment centers, bookcases or desks?	If no to (ITEM = 15 in Section 8A)
Eyeglasses or eye care?	If no to (ITEM = 1 and 2 in Section 15B)
Physicians’ services?	If no to (ITEM = 6 in Section 15B)
Dental services?	If no to (ITEM = 3 in Section 15B)
Services by other medical professionals	If no to (ITEM = 5 and 12 in Section 15B)

If item 1, 2, 3, 4, 5, 6, 7 = yes

1. What is the name of the business where this was last purchased?

2. How much did you spend?

3. Was this paid for -

1. Online?

2. In person?

3. By mail or telephone order?

4. Other

4. (If answered in person) In what city and state is (business name) located?

If item 9 = yes

• What is the name of the doctor’s office where you had the expense?

• How much did it cost?

• Was this paid for -

1. Online?

2. In person?

3. By mail or telephone order?

4. Other

• (If answered in person) In what city and state is (business name) located?

If item 10 = yes

• What is the name of the dentist’s office where you had the expense?

• How much did it cost?

• Was this paid for -

1. Online?

2. In person?

3. By mail or telephone order?

4. Other

• (If answered in person) In what city and state is (business name) located?

If item 8, 11 = yes

• What is the name of the business where you last had this expense?

• How much did it cost?

• Was this paid for -

1. Online?

2. In person?

3. By mail or telephone order?

4. Other

• (If answered in person) In what city and state is (business name) located?

Debriefing:

• What were your impressions of this section of questions with the five year recall period? What did you think we wanted to collect from you?

• Did the questions feel new or familiar like you had been asked them already? Tell me more about that.

• What were your impressions of the introduction?

• What was the recall period that we had been asking about before? [open, DK option] Did you experience any uncertainty about what recall period we were asking about?

• Did you notice any other differences between this session and the earlier set of questions? [Probe if any difficulty in changing the order of the follow up questions]

• Was it easy or difficult to think about expenses from the last five years?

  • Very easy

  • Somewhat easy

  • Somewhat difficult

  • Very difficult

• Please tell us of any particular questions that were difficult: [open]

• Please rate how easy or difficult it was to understand what kind of expenses the questions were asking about.

  • Very easy

  • Somewhat easy

  • Somewhat difficult

  • Very difficult

• Please tell us of any particular questions that were difficult: [open]

• You reported an expense for [FILL]. Please briefly describe the expense in your own words. For example, you went to a doctor’s office and paid a co-pay; or you bought a sofa online and had it delivered.

• You reported the name of the business as [FILL]. Was it easy or difficult to report who provided the service? How confident are you that this is the business that provided the service? Tell me more about that.

• Did you have more than one expense? How did you decide which business name to report?

• Now thinking specifically about the medical expenses questions –

  • Was it easy or difficult to report the expense amount? Tell me more about that. Did you receive any bills after the date of service? Did you or will you get reimbursed for any amount? How confident are you that this is the final out-of-pocket amount that you paid?

  • When you heard the question “Was this paid for online, in person, or by mail or telephone order?”, tell me what that made you think of. What information did you think we were trying to collect? Was it easy or difficult to report the payment method? Tell me more about that. Did you ever need to use more than one method of payment? How confident are you that this is the method of payment that you used?

  • Do you have health insurance that covers some or all of the cost of medical expenses? Did you account for any benefits or reimbursements that this insurance provides?

• Now thinking specifically about the Inside home maintenance question – please take a look at this list and tell me whether you consider each item to be Definitely In, Definitely out of the category, or you’re not sure. [show list of CE items]. Probe “not sure” and incorrect classifications.

• Now I’d like for us to review the questions and get your feedback on the way that they are worded. Please compare your understanding of the question to what is written in these screenshots. This is what we give to our interviewers to help them understand whether items are in or out of the categories. Please give me your honest feedback on whether it is clear to you what should be included or excluded in a category. [Review the question wordings with the participant and probe for comprehension problems]

  • Do these match what you originally thought when you heard the question?

  • Can we add or change anything about these screens to make them clearer?

Online Testing Protocol

For online testing, debriefing questions will focus on assessing the accuracy of responses to the target questions. The goal is to monitor the responses from a large number of online participants to detect irregular responses. Each module will be listed on mTurk as a separate study with an initial screening question. The screening criteria for the study will be listed in the study invitation. Potential participants will see the following information on the mTurk site.

Description:	We'll ask you to recall expenses you have made recently and answer a few follow-up questions - about [rental of computer, telephone, or office machines for non-business use]
Keywords:	memory, survey, research
Full Description:	We are conducting a study to improve the way we ask questions in a government survey. We need to understand whether the questions are clear and easy to understand. Select the link below to complete the survey. At the end of the survey, you will receive a code to paste into the box below to receive credit for taking our survey. Please ONLY participate if you answer yes to this question: [Since the first of [November], have you or any members of your household had any expenses for the rental of computer, telephone, or office machines for non-business use]? Make sure to leave this window open as you complete the survey. When you are finished, you will return to this page to paste the code into the box.

There will be three separate postings using the following screeners:

Since the first of the reference month, have you or any members of your household rented a camper?
Yes*, No
These participants will be asked about campers, tires, and clothes.

Do you or any member of your household currently use a pre-paid (no contract) mobile phone?
Yes*, No
These participants will be asked about pre-paid minutes, bike-sharing, and clothes.

Since the first of the reference month, have you or any members of your household had any expenses for bike-sharing?
Yes*, No
These participants will be asked about pre-paid minutes, bike-sharing, and clothes.

If the participant clicks the link to the study, they will be asked the initial screening question. If the participant’s answer fits the screening criterion for that module, they will be forwarded to a screen with the introductory text on the next page. If they choose to proceed, they will be asked several demographic questions, the focal questions for that module, and then the debriefing questions. If the participant’s answer does not fit the screening criterion, they will see the following text.

Thank you for your time, but you do not meet the qualifications for this study.

---page break---

Welcome! Thanks for your interest in our survey. You’re here because we have asked you to participate in our research. We are asking you and hundreds of other people to tell us about what they think.

Unlike some surveys or online tasks you may be familiar with, we ask that you complete this survey all at one time and that you only start once you are in a quiet place where you won't be disturbed. The survey takes about 10 minutes, on average. Only share information you're comfortable with - nothing too personal - but please be honest and follow the instructions.

 Please do not use your browser's back button. 

This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141 (Expiration Date: April 30, 2018). You are not required to respond to this collection unless it displays a currently valid OMB control number. We will use the information you provide for statistical purposes only. Your participation is voluntary, and you have the right to stop at any time. This survey is being administered by Survey Monkey and resides on a server outside of the BLS Domain. The BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response. By proceeding with this study, you give your consent to participate in this study.

---page break---

As we start, please understand that we ask the same topics of everybody we talk to. We realize some of these questions may not apply to your household. Most questions that we will be asking refer to a specific time period. During this survey, the time period, unless we state otherwise is for the past three months, that is, from the 1st day of (reference month) to today.

We’ll start by asking a few questions about you and your household. And, after we ask about your expenses, we will then ask you some additional questions to better understand your answers.

Let’s get started!

---page break---

What is your age in years?

What is your gender?

• Male

• Female

What is the highest level of school you have completed or the highest degree you have received?

• Grades 1-12/No Diploma

• High School Diploma or Equivalent

• Some College

• Associate’s Degree

• Bachelor’s Degree

• Master’s Degree

• Doctorate or Professional Degree

Are you Hispanic or Latino?

• Yes

• No

What is your race?

• American Indian or Alaska Native

• Asian

• Black or African-American

• Native Hawaiian or Other Pacific Islander

• White

How many people live in your household, including yourself?

• 1

• 2

• 3

• 4

• 5 or more

[if HHSize > 1] How many of those people are under the age of 18?

• 1

• 2

• 3

• 4

• 5 or more

---page break---

[Separate screen for each item, showing the question stem and only one item at a time]

Since the first of (reference month), have you or any members of your household rented any –

1. Automobiles, trucks, vans, minivans, or SUVs

2. Motor homes

3. Campers, including those attached to trucks

4. Motorcycle, motor scooter, or moped (motorized bicycle)

5. Trailers other than a camper, such as for a boat or cycle

• If yes –

Since the first of (reference month), not including (current month), what has been your expense for renting this vehicle?

[numeric]

Debriefing:

• [If item 3 = Yes] You reported that you rented a camper. Please briefly describe that expense; for example, I went on a trip and rented a truck camper.

• [If item 5 = Yes] You reported that you rented a trailer other than a camper. Please briefly describe that expense; for example, I went on vacation and rented a boat trailer that hitches to the back of my car.

• Was there anything that you considered reporting but didn’t?

  • Did you not report these initially because you weren’t sure if we wanted them, you forgot, or something else?

• Please rate how easy or difficult it was to understand what kind of expenses the questions were asking about.

  • Very easy

  • Somewhat easy

  • Somewhat difficult

  • Very difficult

• Please tell us of any particular questions that were difficult: [open]

• Take a look at this list. Look at each item and mark whether the item would definitely be included in the category or definitely not included in the category. If you don’t know what an item is or are not sure whether it should be in the category, mark “not sure”

  Campers, including those attached to trucks	Definitely IN	Definitely OUT	Not sure
  Travel trailer
  Camping trailer
  Camping tent
  Toy hauler
  Towable RV
  Fifth wheel
  Boat trailer
  Utility trailer

• Did seeing that list bring to mind any related expenses that you haven’t already told me about?

Since the first of (reference month), have (you/you or any members of your household) had expenses for any of the following?

1. Patches, plugs, or other repairs for flat tires

2. Tire purchases or mounting

If yes to any –

• What was the expense for?

• Did this expense include labor? Yes, No

• In what month did you have this expense?

  • Jan

  • ..

  • Dec

• What was the total cost?

• Did this include sales tax?

• Was this combined with any other expense?

• Has any of this expense been, or will any of it be, reimbursed? How much?

• Did you have any other expenses for (description)?

Debriefing

• [If item 1 = Yes] You reported that you had an expense for Patches, plugs or other repairs for flat tires. Please briefly describe that expense; for example, I had a flat tire and bought a patch kit and repaired the tire myself.

• [If item 1 = Yes] You reported that you had an expense for Tire purchases or mounting. Please briefly describe that expense; for example, I bought four used tires for my car and paid the garage to install the tires.

• Did you do the work yourself or pay someone else?

  • Myself

  • Paid someone else

• Was there anything that you considered reporting but didn’t?

  • Did you not report these initially because you weren’t sure if we wanted them, you forgot, or something else?

• Please rate how easy or difficult it was to understand what kind of expenses the questions were asking about.

  • Very easy

  • Somewhat easy

  • Somewhat difficult

  • Very difficult

• Please tell us of any particular questions that were difficult: [open]

• Take a look at this list. Look at each item and mark whether the item would definitely be included in the category or definitely not included in the category. If you don’t know what an item is or are not sure whether it should be in the category, mark “not sure”

  Patches, plugs, or other repairs for flat tires	Definitely IN	Definitely OUT	Not sure
  New tire to replace a flat
  Spare tire to keep in the car
  Tire warranty
  Roadside assistance for a flat tire
  Tire balancing service
  Tire valve stem replacement
  Tire sealant
  Patch kit
  Tire pressure gauge

• Did seeing that list bring to mind any related expenses that you haven’t already told me about?

Since the first of (reference month), have (you/you or any members of your household) purchased any of the following items for your household or for someone outside your household?

1. Health clubs, fitness centers, swimming pools, weight loss centers, or other sports or recreational organizations

2. Fees for participating in sports such as golf, bowling, biking, hockey, football, or swimming

3. Bike-sharing

• For item 1: What is the name of the health, fitness, or recreation club?
  For item 2: Ask if not apparent – describe the item
  For item 3: What is the name of the bike-sharing system?

• Was this purchased for your household or someone outside your household?

• In what month did you have this expense? (* Enter 13 for same amount each month of the reference period/ )

• (What was the total amount of this expense?/What is ^YR_YRCUS monthly expense?) ((Include shipping and handling fees./ ))

• Was this expense combined with anything else? What other expense is the (description) combined with?

• [Item 2 = yes] What is the name of the business that manages this organization or event?

• Was this paid for –

  • Online?

  • In-person?

  • By mail or telephone order?

  • Other

Debriefing

• [If item 2 = Yes] You reported that you had an expense for Fees for participating in sports. Please briefly describe that expense; for example, I joined a bowling league.

• [If item 3 = Yes] You reported that you had an expense for bike-sharing. Please briefly describe that expense; for example, I paid for a one-day pass to use a city’s bike-share system when I was on a vacation.

• Did you do the work yourself or pay someone else?

  • Myself

  • Paid someone else

• Was there anything that you considered reporting but didn’t? [repeat back to the participant the list of expenses reported]

  • Did you not report these initially because you weren’t sure if we wanted them, you forgot, or something else?

• Please rate how easy or difficult it was to understand what kind of expenses the questions were asking about.

  • Very easy

  • Somewhat easy

  • Somewhat difficult

  • Very difficult

• Please tell us of any particular questions that were difficult: [open]

• Take a look at this list. Look at each item and mark whether the item would definitely be included in the category or definitely not included in the category. If you don’t know what an item is or are not sure whether it should be in the category, mark “not sure”

Fees for participating in sports such as golf, bowling, biking, hockey, football, or swimming	Definitely IN	Definitely OUT	Not sure
Billiards
Running races
Triathlons
Martial arts
Hunting and fishing licenses
Bike-sharing
Bike rental
Adult kickball
Recreation center fees

Bike-sharing	Definitely IN	Definitely OUT	Not sure
Purchase of a new bike
Dockless bikes
Bike-sharing annual membership
Bike-sharing, one-time use fees
Bike rental

• Did seeing that list bring to mind any related expenses that you haven’t already told me about?

Now I am going to ask about clothing, footwear, and accessories, for infants, children, and adults. You may find it helpful to refer to receipts, credit card statements or other records to answer the questions. Please include any shipping and handling charges with the cost of any item that was shipped.

Since the first of (reference month), have (you/you or any members of your household) purchased any of the following items either for members of your household or for someone outside your household?

1. Outerwear

2. Sport coats, tailored jackets, or blazers

3. Shirts, sweaters, blouses, or tops

4. Pants, jeans or shorts

5. Dresses

6. Skirts

7. Accessories

8. Swimwear, swim cover-ups, or swimwear accessories

9. Watch or jewelry repair

If yes to any –

• What did you buy?

  • [Open]

• Was this (were these) purchased for someone inside or outside of your household?

  • Inside your household

  • Outside your household

  • Both inside and outside your household

• Was this (Were these) purchased for...

  • Man, age 16 and over

  • Woman, age 16 and over

  • Boy, age 2 to 15

  • Girl, age 2 to 15

  • Child under 2 years old

• When did you purchase it?

  • January

  • …

  • December

  • Same amount each month

• (What is your monthly expense?/How much did it cost?)

  • [Numeric]

• Did this include sales tax?

  • Yes

  • No

• Was this expense combined with anything else?

  • [If yes] What other clothing is (description) combined with?

• For 1-8: What is the name of the business where this was purchased?

• For 9: What is the name of the business where you had this expense?

  • [Verbatim]

• Was this paid for -

  • Online?

  • In person?

  • By mail or telephone order?

  • Other

• If in person: In what city and state is [CLASTORE] located?

Debriefing:

• Was there anything that you considered reporting but didn’t?

  • Did you not report these initially because you weren’t sure if we wanted them, you forgot, or something else?

• Did you shop at more than one place? How did you decide which store name to report?

• If yes to item = 9 – Did the expense you report include service, such as paying someone to fix a watch band or replace a battery?

Loop through for each category:

• Take a look at this list. Look at each item and mark whether the item would definitely be included in the category or definitely not included in the category. If you don’t know what an item is or are not sure whether it should be in the category, mark “not sure”

Water sports equipment	Definitely IN	Definitely OUT	Not sure
Swim cap
Goggles
Swimsuit
Swimsuit cover-up/wraparound
Water shoes

Outerwear	Definitely IN	Definitely OUT	Not sure
Ski boots
Winter scarf
Sweater
Sportsjacket/Blazer
Snow suit
Hunting jacket
Fur coat
Hooded sweatshirt/Hoodie
Leather jacket
Down vest

Now, please do the same exercise for the category “Coats, jackets, and furs”. I realize that this might sound similar to “outerwear” so your answers might be similar, but please go ahead and fill this out anyway.

Coats, jackets, and furs	Definitely IN	Definitely OUT	Not sure
Ski boots
Winter scarf
Sweater
Sportsjacket/Blazer
Snow suit
Hunting jacket
Fur coat
Hooded sweatshirt/Hoodie
Leather jacket
Down vest

Shirts, sweaters, blouses, or tops	Definitely IN	Definitely OUT	Not sure
Graphic t-shirts
Hooded sweatshirt/Hoodie
Running gear like reflective shirts
Workout tank tops
Leotard for ballet
Active wear

Pants, jeans, or shorts	Definitely IN	Definitely OUT	Not sure
Gym shorts
Yoga pants
Slacks/Trousers
Rompers
Running gear like running pants
Workout leggings
Bike shorts
Snowsuits
Active wear

Swimwear, swim cover-ups, or swimwear accessories	Definitely IN	Definitely OUT	Not sure
Board shorts
Swim cap
Goggles
Beach dress cover-up
Wet suit
Water shoes
Flip flops

Watch or jewelry repair	Definitely IN	Definitely OUT	Not sure
New watch band you change yourself
Ring resizing
Cleaning and polishing
Watch battery replacement
Replacement of stones
DIY Watch repair kit

• Did seeing that list bring to mind any related expenses that you haven’t already told me about?

• Now please take a look at this page (show proposed Infobook pages). This page is what we will show to respondents during the survey, to provide examples of the kinds of things we are looking for. Does seeing these lists change what you thought about these categories?

Now I am going to ask about utilities including telephone bills. Please refer to any billing statements or other records you have when answering these questions. Please remember to include any bills you receive or pay online or have automatically deducted. Report any bill you have received, even if the bill has not been paid.

Since the first of (reference month), have (you/you or any members of your household) received any bills for telephone, internet, cable TV, or satellite TV services, including cellular and Voice Over IP? Do not include bills for telephones used entirely for business purposes.

(What is the name of the company which provides the service?/"*company name)

How much were you billed for in (last month)? Do not include any unpaid charges from a previous billing period.

• What is your usual billing period for this service?

• Month

• 2 months

• Quarter

• Annual

• Other

• Which of the following telephone service items were included in the bill(s) –

• Telephone service (mobile, residential, or Voiceover IP)?

• Internet service (including broadband, DSL, and dial-up)?

• Cable or satellite television service?

• What types of telephone services did the bill include –

  • Residential Service including Voice over IP?

  • Mobile or Cellular Service including prepaid?

Did (you/you or any members of your household) receive any other bills for telephones, internet, cable TV, or satellite TV not used entirely for business purposes?

Since the first of (the reference month) not including this month, have you or any member of your household purchased any prepaid cellular minutes, not already reported?

If yes –

• What was the total amount paid?

• What is the name of the business where these were purchased?

Debriefing

• You reported that you had an expense for prepaid cellular minutes. Please briefly describe that expense; for example, I bought pay-as-you-go minutes on a card from Best Buy.

• Was there anything that you considered reporting but didn’t?

  • Did you not report these initially because you weren’t sure if we wanted them, you forgot, or something else?

• Please rate how easy or difficult it was to understand what kind of expenses the questions were asking about.

  • Very easy

  • Somewhat easy

  • Somewhat difficult

  • Very difficult

• Please tell us of any particular questions that were difficult: [open]

• Take a look at this list. Look at each item and mark whether the item would definitely be included in the category or definitely not included in the category. If you don’t know what an item is or are not sure whether it should be in the category, mark “not sure”

  Prepaid cellular minutes	Definitely IN	Definitely OUT	Not sure
  Pay-as-you-go daily plan
  Pay-as-you-go monthly plan
  In-store activation fee for a new phone
  Pre-paid phone plan
  Prepaid refill card
  Skype credit
  International call services like Rebtel

• Did seeing that list bring to mind any related expenses that you haven’t already told me about?

Appendix C: Consent Form

OMB Control Number: 1220-0141

Expiration Date: April 30, 2018

The Bureau of Labor Statistics (BLS) is conducting research to increase the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.

We estimate it will take you an average of 45 minutes to participate in this research (ranging from 40 to 60 minutes).

Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. OMB control number is 1220-0141 and expires April, 30, 2018.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974 (DOL/BLS – 14 BLS Behavioral Science Research Laboratory Project Files (81 FR 47418)), as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. Per the Federal Cybersecurity Enhancement Act of 2015, Federal information systems are protected from malicious activities through cybersecurity screening of transmitted data.

Page 52 of 52