Note to Reviewer - Globals

June 13, 2013

NOTE TO THE REVIEWER OF:	OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research”
FROM:	Brandon Kopp Research Psychologist Office of Survey Methods Research
SUBJECT:	Submission of Materials for testing the Effectiveness of Global Questions and Different Length Reference Periods in the CEQ

Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.

The total estimated respondent burden hours for this study is 40 hours.

If there are any questions regarding this project, please contact Brandon Kopp at

202-691-7514 or Jennifer Edgar at 202-691-7528.

1. Introduction and Purpose

The Consumer Expenditure Survey (CE) is currently going through a redesign process. Two of the stated goals of this process are to reduce respondent burden and measurement error. These goals are not always compatible with one another. One may wish to reduce burden by asking more general questions that each cover a range of topic areas (referred to below as global questions), but this may make the respondent’s task of answering a question more difficult, leading to greater measurement error. This, counter-intuitively, may lead to an increase in overall burden if the respondent perceives the additional memory search and answer formulation requirements of the question as more burdensome.

The purpose of the current study is to examine two methods that have been proposed to reduce respondent burden in a redesigned Consumer Expenditure Quarterly Interview Survey (CEQ); (1) lengthening the reference period and (2) asking about expenditure categories at a higher level of aggregation. Using a longer reference period (even if only for certain items) would limit the number of times that those items need to be asked about across a survey panel. Asking a global question about a high-level expenditure category (e.g., furniture) rather than a series of questions about the items that make up the category (e.g., sofas, living room chairs, living room tables, etc.) would reduce the number of questions a respondent has to answer and potentially decrease burden. The challenge is determining the proper length of the reference period or the proper level of aggregation for a given category or item that allows for a decrease in burden while minimizing any associated increase in measurement error.

The current study is meant to provide preliminary information on these issues by asking participants about purchases they’ve made, while varying the reference period or level of aggregation, and determining the measurement error properties of the their answers using cognitive interviewing. If this testing shows that the implementation of global questions and longer reference periods are successful (i.e., burden is decrease while measurement error is held constant or reduced), then these topics would be considered for more intensive, future testing in which measurement error is more thoroughly assessed through financial records collection or some other means.

2. Research Design

Cognitive interviews provide an in-depth understanding of a respondent’s thought processes and reactions to a question. The cognitive interviews for this study will be done one-on-one, in the Office of Survey Methods Research (OSMR) laboratory, and will be audiotaped. Interviews will be conducted by staff from the OSMR who are experienced in conducting these types of interviews. The testing protocol is included in Appendix B.

The interviews will be broken into two blocks; one for the testing of questions with varying reference periods and another for global questions. The order of the blocks will be counterbalanced so that half of the participants receive the reference period section first and the other half receives global questions first.

• Reference Period – Participants will be divided into 3 groups and will be asked to report the incidence and expenditure amounts for sets of items for either the past 3, 6, or 12 months (see Appendix B). Each participant will receive some questions for each reference period. Probing methods will then be used to better understand the accuracy of responses. For questions asked with 3 and 6 month reference periods, participants will be asked about the incidence of expenditures at the longer reference periods (i.e., whether they made any purchases of that type in the last 6 or 12 months). Participants will also be asked about their estimation strategies.

• Global Questions - Participants will be asked to report their overall expenditure for three global expenditure categories (i.e., furniture, computer-related expenses, and household linens) for the past 3 months. Participants will be asked to think aloud while coming up with their answer. They will then be asked to list items that they included in their answer in an open-ended way. They will then be shown a list of items (some of which BLS would consider part of the category, some of which would not be included) and be asked to select which items definitely belong to the category, then separately select which items definitely do not belong to the category (see Appendix C for item lists). Finally, participants will be asked several questions about how easy or difficult it was to report for a category and whether it would be easier for them to report for individual items.

3. Participants

Participants will be recruited from the OSMR participant database. During the recruiting process, we will screen potential respondents to find participants who have recently made particular purchases (see Appendix D for screening questions), because those experiences are the topic of the study. We will aim to balance participants across levels of education, income, and occupation, based on self-reported information that is stored in the OSMR participant database.

4. Burden Hours

The number of participants is dependent upon the results and the point at which diminishing returns is reached and no further information is being gained from additional subjects. Given that, our goal is to obtain no more than 40 participants in the OSMR lab. We anticipate that each session will average 45 minutes and last no longer than one hour, for a total of 40 burden hours. Participants will be compensated $40 for their time.

5. Data Confidentiality

Participants will be informed as to the voluntary nature of the study. Participants will also be informed that the study will be used for internal purposes to improve the design of a national Consumer Expenditure Interview Survey. Participants will be given a consent form to read and sign (see Appendix E). Information related to this study will not be released to the public in any way that would allow identification of individuals except as prescribed under the conditions of the Privacy Act Notice.

Attachment A: Introductory Material

• Hi! Thank you for coming in today.

• I am …... [This is my colleague ( ) who will be taking notes for us today]

• Have you participated in any of our studies before? (if yes, Which ones?)

• Consent Form/Permission to audiotape

• Explanation:

• We are going to be working with some questions from the Consumer Expenditure Survey, an ongoing survey that provides information about the buying habits of American consumers. This information is used for a lot of things but its primary purpose is to help calculate inflation. Are you familiar with the inflation rate?

• What we are going to do today is go through a few questions from the survey and get your reactions to them. The purpose of this session is to help us find out more about how people respond to these questions. Basically, we’re trying to find out what you think a question is asking and how hard it is to answer. We are not here to evaluate you, we are looking to improve the questions, so there are no wrong answers. All the information you give us will be kept completely confidential, and will be used to improve the survey questions.

• Any questions before we begin?

Appendix B: Cognitive Testing Protocol

Reference Periods

	Large (Salient) Items	Small (Less Salient) Items	Recurring Items
Group	Airline Tickets Coats, Jackets, or Furs Televisions, all types Automobile, truck, minivan, van or SUV Washer and/or Dryer	Shirts, Blouses, or Tops Camping Equipment Power Tools Pets, Pet Supplies, or Medicine for Pets Photography	Mortgage or Rent Electricity Automobile or Other Vehicle Insurance Mobile/Cellular Service Accounting Fees, including Tax Preparation
1	3 Months	6 Months	12 Months
2	6 Months	12 Months	3 Months
3	12 Months	3 Months	6 Months

For each item participants will be asked:

• Since the first of [the reference month], have you or any members of your household purchased any (or had expenses for) [expenditure category]?

  • (if Yes) What was the cost of those items?

  • (if Yes) In what month did you purchase them?

After all questions have been asked for expenditure item, participants will be asked follow-up questions:

• I asked you about expenses for [expenditure category] since [the reference month] and you said “[repeat respondents answer, the expenditure amount, and the month of purchase]”

• Please explain how you arrived at your answer?

• (if Yes) How confident are you that this expenditure occurred in the last [X] Months? (Not At All Confident – Extremely Confident)

  • (if Not At All or Extremely Confident) Why are you [Not At All/Extremely Confident]?

• (if Yes) How confident are you that you accurately recalled the price of the item? (Not At All Confident – Extremely Confident)

  • (if Not At All or Extremely Confident) Why are you [Not At All/Extremely Confident]?

• (if Yes) How confident are you that you purchased this item in [month]? (Not At All Confident – Extremely Confident)

  • (if Not At All or Extremely Confident) Why are you [Not At All/Extremely Confident]?

• (if No) How confident are you that you did not purchase [expenditure category] in the last [X] Months? (Not At All Confident – Extremely Confident)

  • (if Not At All or Extremely Confident) Why are you [Not At All/Extremely Confident]?

For items with a 3 or 6 month reference period:

• Thinking back over the last 12 months, how many did you have for [expenditure category]?

• How accurately would you be able to report your purchases for [expenditure category] for a 12 month period? (Not At All Accurate – Extremely Accurate)

Global Questions

• Since the first of [the reference month], have you or any members of your household purchased, for you or your household or for someone outside of your household, any furniture?

  • (if Yes) What was the total cost of these items?

  • (if Yes) In what month did you purchase them?

• Since the first of [the reference month], have you or any members of your household purchased or rented any computer hardware, software, accessories, and service?

  • (if Yes) What was the total cost of these items?

  • (if Yes) In what month did you purchase them?

• Since the first of [the reference month], have you or any members of your household purchased any household linens or window coverings?

  • (if Yes) What was the total cost of these items?

  • (if Yes) In what month did you purchase them?

• Furniture

  • (if Purchased) What items did you purchase? What was the cost of each of those items?

  • What comes to mind when I say “furniture?”

  • [Provide list of expenditure items] Which of these items do you think definitely belongs in the category “furniture?”

  • Which of the items do you think definitely does not belong in the category “furniture?”

  • Did looking at that list remind you of any other things you’ve purchased since the first of [the reference month]?

    • (if Yes) What else did you purchase?

    • (if Yes) What was the cost of those items?

    • (if Yes) Why didn’t you include these items in your earlier answer about furniture? [Forgot about purchase, Didn’t think item would be considered furniture, etc.]

  • Did you think this question referred to the rental of furniture? Why or why not?

    • Have you rented any furniture since the first of [the reference month]?

  • Was the question about how much furniture you’ve purchased since the first of [the reference month] easy or difficult to answer?

    • What made it [easy/difficult] to answer?

  • [Show current question item list] Would you rather be asked if you’ve purchased furniture, in general, or about these different types of furniture separately? Which would be easier for you to talk about? Why?

• Computer hardware, software, and accessories

  • [if Purchased] What items did you purchase? What was the cost of those items?

  • What comes to mind when I say “computer hardware, software, accessories, and service?”

  • [Provide list of expenditure items] Which of these items do you think definitely belongs in the category “computer hardware, software, accessories, and service?”

  • Which of the items do you think definitely does not belong in the category “computer hardware, software, accessories, and service?”

  • Did looking at that list remind you of any other things you’ve purchased since the first of [the reference month]?

    • (if Yes) What else did you purchase?

    • (if Yes) What was the cost of those items?

    • (if Yes) Why didn’t you include these items in your earlier answer about computer hardware, software, accessories, and service? [Forgot about purchase, Didn’t think item would be considered part of this category, etc.]

  • Was the question about how much computer hardware, software, accessories, and service you’ve purchased since the first of [the reference month] easy or difficult to answer?

    • What made it [easy/difficult] to answer?

• [Show current question item list] Would you rather be asked if you’ve purchased computer hardware, software, accessories, and service, in general, or about these different categories of computer items separately?

• Household Linens and Window Coverings

  • [if Purchased] What items did you purchase? What was the cost of those items?

  • What comes to mind when I say “household linens or window coverings?”

  • [Provide list of expenditure items] Which of these items do you think definitely belongs in the category “household linens or window coverings?”

  • Which of the items do you think definitely does not belong in the category “household linens or window coverings?”

  • Did looking at that list remind you of any other things you’ve purchased since the first of [the reference month]?

    • (if Yes) What else did you purchase?

    • (if Yes) What was the cost of those items?

    • (if Yes) Why didn’t you include these items in your earlier answer about household linens or window coverings? [Forgot about purchase, Didn’t think item would be considered part of this category, etc.]

  • Was the question about how much household linens or window coverings you’ve purchased since the first of [the reference month] easy or difficult to answer?

    • What made it [easy/difficult] to answer?

• [Show current question item list] Would you rather be asked if you’ve purchased household linens or window coverings, in general, or about these different categories of computer items separately?

Appendix C: Item Lists

NOTE: Items with grey backgrounds are “lures” or do not belong in the global category. The background color will be removed for testing.

Furniture
Sofa	Stove	Sectional	Office Chair	Rocking Chair
Bean Bag Chair	End Table	TV Stand	Lamp	Shoe Rack
Mirror	Coat Rack	Television	Ottoman	Dinette Set
Bookcase	China Cabinet	Dining Table	Dishwasher	Box Spring
Headboard	Mattress	Bunk bed	Bed Frame	Crib
Coffee Maker	High Chair	Patio Umbrella	Deck Chairs	File Cabinet
Loveseat	Room Divider	Vase	Pool Table	Clock
Refrigerator	Recliner	Drafting Table	Gun Cabinet	Garbage Disposal
Play Pen	Desk	Entertainment Center	Dresser	Card Table

Furniture
Sofas	Living Room Chairs	Living Room Tables	Wall Units, Shelves Or Cabinets	Infant’s Furniture
Other Living Room Furniture	Living Room Furniture Combinations	Dining Room Or Kitchen Furniture	Mattresses Or Box Springs	Bedroom Furniture Other Than Mattresses Or Box Springs
Patio, Porch, Or Outdoor Furniture	Office Furniture For Home Use
\

Computer Hardware, Software, Accessories, and Service
Mouse Pad	Home Computer	Sony eReader	Printer Cartridge	Computer Games
Answering Machine	Modem	Computer Installation	Digital Camera	iPad
Laptop	Mouse	GPS Device	Memory (RAM)	Typewriter
External Hard Drive	Cell Phone Charger	Monitor	Thumb Drive	Blank DVD
USB Flash Drive	Computer Repair	Scanner	CD/DVD Drive	iPhone
Nook	Printer	Playstation Console	Kindle	Netbook

Computer Hardware, Software, Accessories, and Service
Digital Book Readers	Computers, Computer Systems, or Related Hardware	Computer Software, Including Games	Portable Memory	Repair and Service Contract for Computer, Computer Systems, and Related Equipment

Household Linens and Window Coverings
Sheets	Shirt	Furniture Protector	Shower Curtain	Pajamas
Pillow	Mattress Pad	Cloth Diaper	Bedspread	Quilt
Face Cloth	Throw Rug	Bath Mat	Toilet Cover	Scarf
Undergarments	Electric Blanket	Cloth Napkins	Placemats	Air Conditioner
Pillowcase	Chair Pad	Towels	Washing Machine	Decorative Pillow
Blinds	Shades	Curtains	Drapes	Baby Blanket
Duvet	Slip Cover	Comforter	Doilies	Sofa
Sweater	Blanket	Vest	Tablecloth	Dish Towel

Household Linens and Window Coverings
Bedroom Linens	Bathroom Linens	Kitchen or Dining Room Linens	Other Linens	Slipcovers, Decorative Pillows, or Cushions
Room-size rugs or other Non-Permanent Floor Coverings	Curtains or Drapes	Blinds, Shades, or Other Window Coverings

Appendix D: Screening Questions

Must say “Yes” to 2 out of 3 of the questions below to participate.

1. Have you purchased any of the following items in the last year? YES NO

• Airline Tickets

• Coats, Jackets, or Furs

• Televisions, all types

• Automobile, truck, minivan, van or SUV

• Washer and/or Dryer

2. Have you purchased any of the following items in the last year? YES NO

• Shirts, Blouses, or Tops

• Camping Equipment

• Power Tools

• Pets, Pet Supplies, or Medicine for Pets

• Photography

3. Have you purchased any of the following items in the last year? YES NO

• Mortgage or Rent

• Electricity

• Automobile or Other Vehicle Insurance

• Mobile/Cellular Service

• Accounting Fees, including Tax Preparation

Must say “Yes” to the following question to participate.

4. Have you purchased any of the following items in the last three months? YES NO

• Furniture

• Computer Hardware, Software, Accessories, or Services

• Household Linens

Appendix E: Consent Form

CONSENT FORM

The Bureau of Labor Statistics (BLS) is conducting research to increase the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.

We estimate it will take you an average of 45 minutes to participate in this research (ranging from 30 minutes to 60 minutes).

Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. OMB control number is 1220-0141, and expires 2-28-2015.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

OMB Control Number: 1220-0141

Expiration Date: [02-28-2015]

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

15