CE Validation Study

February 26, 2009

NOTE TO THE REVIEWER OF:	OMB CLEARANCE #1220-0141 “Cognitive and Psychological Research”
FROM:	Jennifer Edgar & Scott Fricker Research Psychologists Office of Survey Methods Research
SUBJECT:	Submission of Materials for the CE Validation Study

Please accept the enclosed materials for approval under the OMB clearance package #1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, I am submitting a brief description of the study.

The total estimated respondent burden hours for this study is 78 hours.

If there are any questions regarding this project, please direct them to Jennifer Edgar

(202-691-7528) or Scott Fricker (202-691-7390).

1. Introduction and Purpose

Previous research suggests that data from the Consumer Expenditure Quarterly Interview Survey (CEQ) contains measurement error (i.e., Tucker, 1985; Tucker, Meekins & Biemer, 2005, etc.). The goal of this study is to explore the interview process and the challenges faced by respondents to provide insights into possible sources of error, especially processes and errors that lead to under reporting. In addition, although designed as a personal visit interview survey, about 35 percent (Safir & Goldenberg, 2008) of the interviews are collected over the telephone, and we will look at the possible impact of mode on measurement error.

2. Research Design

This study will include a CEQ interview that will mimic actual field administration as closely as possible, followed by an in-depth debriefing. Participants will be randomly assigned to a mode of administration, either telephone or personal visit. Prior to the interview, the researchers will send an advance letter (Appendix A) to confirm the date and time of the interview, as well as to provide background information about the interview.

The researchers will either call participants or visit them in their homes to conduct the CEQ interview (available at www.bls.gov/cex/capi/2008/cecapihome.htm). They will follow field procedures as closely as possible in terms of interview questions and materials. In phone interviews, the researcher will conduct the interview by cell phone from his/her car, so that the next phase of the data collection, the debriefing, can occur in the respondent’s home without requiring a separate appointment and visit.

After completing the interview, the researchers will inform the participants that they have a 10 to 15 minute break, during which they should collect any records or materials that would help them review their answers to the interview questions. Respondents will also be asked to complete a questionnaire (Attachment B) about the interview after they have had a chance to collect and organize their records. A detailed debriefing will follow the break and completion of the questionnaire.

Researchers will follow a semi-structured interviewing procedure for the debriefing, following an outline of question topics (Attachment C) but pursuing any participant comments or reactions that are relevant and informative to the research objectives. The debriefing will be an open-ended discussion between the researchers and the participant, focusing on several elements:

1. the participant’s experience during the CEQ interview,

2. the effectiveness of the CEQ Information Book,

3. the availability and use of participant records, and

4. the impact of survey mode.

3. Participants

Twenty six participants will be recruited from the OSMR participant database. Screening questions will aid recruitment of participants with the desired target knowledge. Efforts will be made to select participants with varying levels of education, income, and occupation, based on self-reported information provided during the initial recruitment process.

4. Burden Hours

Each CEQ interview is estimated to last 1.5 hours, and each debriefing an additional 1.5 hours, for a total of 3 hours per participants. Therefore, we estimate that the total burden hours will be 78 hours.

5. Payment

For this study we will be paying participants $100, rather than the standard $40. The increase in respondent payments is justified because our participants will have to spend up to 3 hours answering survey and debriefing questions - three times longer than the typical OSMR study. Both the duration of the session and the intrusive nature of the study topic (asking for detailed information and for respondents to access personal records on household expenditures) make recruitment more difficult. Initial recruiting efforts during a small pre-test suggest that it would be particularly difficult to gain the cooperation of several sub-groups in which we are interested (e.g., employed individuals, higher income households) without increased remuneration. The additional payment will be necessary to recruit and gain cooperation from potential participants.

6. Data Confidentiality

Participants will be informed of the voluntary nature of the study. Participants will also be informed that the study will be used for internal purposes to improve the design of a national consumer expenditure interview survey. Participants will be given a consent form to read and sign (Appendix D), either prior to beginning the interview (for face-to-face interviews) or prior to beginning the debriefing (for telephone interviews). Information related to this study will not be released to the public in any way that would allow identification of individuals except as prescribed under the conditions of the Privacy Act Notice.

7. Attachments

Attachment A:	Advance Letters
Attachment B:	Respondent Questionnaire
Attachment C:	Debriefing Outline
Attachment D:	Consent agreement form and Privacy act statement

8. References

Safir, A. & Goldenberg, K. (2008). Consumer Expenditure Survey Program: Telephone Effects in the Consumer Expenditure Quarterly Interview Survey. Internal BLS Report, submitted August 2008.

Tucker, C. (1985). Identifying Measurement Error in Consumer Expenditure Reports, American Statistical Association Proceedings.

Tucker, C., Meekins, B. & Biemer. P. (2005). Estimating the Level of Underreporting of Expenditures among Expenditure Reporters: A Further Micro-Level Latent Class Analysis. Internal BLS Report, submitted August 2005.

Attachment A: Advance Letters

CE-303-PV

(8-2008)

FROM THE OFFICE OF SURVEY METHODS RESEARCH

U.S. Bureau of Labor Statistics

Thank you for agreeing to participate in our study! On (insert date and time) an interviewer will visit you to conduct an interview for your household for the Consumer Expenditure Quarterly Interview Survey. In this interview we will ask about your household expenditures during the past three months.

The data collected in this survey gives us an accurate picture of the seasonal and long-term purchasing patterns of Americans. The Department of Labor uses those data to update the Consumer Price Index, which indicates the rate of inflation and affects interest rates, wages, pensions, and social security benefits for millions of Americans.

To help shorten the interview, you may want to have some records of expenses available for reference. On the back of this letter are examples of records, which may reduce the interview time and improve the quality of the survey results.

All information about you and your household will be kept strictly confidential. We will use this information only for research purposes. Your voluntary participation is extremely important to ensure that the final results are complete and accurate.

Thank you for your participation and assistance in the Consumer Expenditure Quarterly Interview Survey. We appreciate your continued commitment to this important survey. Please contact Jennifer Edgar (202-691-7528) or Scott Fricker (202-691-7390) if you have any questions or need to reschedule.

Sincerely,

Jennifer Edgar, PhD

Scott Fricker, PhD

Primary Investigators

Office of Survey Methods Research

U.S. Bureau of Labor Statistics

2 Massachusetts Ave., NE

Washington DC 20212

Here are some examples of bills/receipts that

may help to reduce the interview time:

Utilities

Insurance Premiums

Medical and Education Expenses

Auto Repair Expenses

Trip Expenses

Home Maintenance and Repair

Household Appliances and Home Furnishings Purchases/Repairs

Sports and Hobby Equipment Purchases

Clothing Purchases

CE-303-TV

(8-2008)

FROM THE OFFICE OF SURVEY METHODS RESEARCH

U.S. Bureau of Labor Statistics

Thank you for agreeing to participate in our study! On (insert date and time) an interviewer will call you to conduct an interview for your household for the Consumer Expenditure Quarterly Interview Survey. In this interview we will ask about your household expenditures during the past three months. Following the telephone interview, the interviewer will visit your home to discuss the interview with you in more detail.

The data collected in this survey gives us an accurate picture of the seasonal and long-term purchasing patterns of Americans. The Department of Labor uses those data to update the Consumer Price Index, which indicates the rate of inflation and affects interest rates, wages, pensions, and social security benefits for millions of Americans.

To help shorten the interview, you may want to have some records of expenses available for reference. On the back of this letter are examples of records, which may reduce the interview time and improve the quality of the survey results.

All information about you and your household will be kept strictly confidential. We will use this information only for research purposes. Your voluntary participation is extremely important to ensure that the final results are complete and accurate.

Thank you for your participation and assistance in the Consumer Expenditure Quarterly Interview Survey. We appreciate your continued commitment to this important survey. Please contact Jennifer Edgar (202-691-7528) or Scott Fricker (202-691-7390) if you have any questions or need to reschedule.

Sincerely,

Jennifer Edgar, PhD

Scott Fricker, PhD

Primary Investigators

Office of Survey Methods Research

U.S. Bureau of Labor Statistics

2 Massachusetts Ave., NE

Washington DC 20212

Here are some examples of bills/receipts that

may help to reduce the interview time:

Utilities

Insurance Premiums

Medical and Education Expenses

Auto Repair Expenses

Trip Expenses

Home Maintenance and Repair

Household Appliances and Home Furnishings Purchases/Repairs

Sports and Hobby Equipment Purchases

Clothing Purchases

Attachment B: Respondent Questionnaire

Please answer the following questions about your experience with the interview.

1. How long do you think the interview lasted? ______ minutes

2. How easy or difficult was it for you to answer the questions that I asked you?

Very Easy

Easy

Neither easy nor difficult

Difficult

Very Difficult

3. Over the course of the interview, do you think your answers became more accurate, less accurate, or did they stay about the same?

More accurate as interview proceeded

Stayed the same throughout the interview

Less accurate as the interview proceeded

4. Here is a list of the types of expenses I just asked you about. There is a 5-point scale shown at the top that goes from very easy to very difficult. For each item, I want you to tell me how easy or hard it was to answer the questions for that type of expense.

   	Very Easy	Easy	Neither easy nor difficult	Difficult	Very Difficult
   Housing, mortgage or rent expenses
   Utilities (i.e. phone, water, electricity, cable)
   Construction or home repair expenses
   Appliances and household equipment (i.e. electric kitchen appliances, computers, musical instruments)
   Household item repairs service contracts (i.e. repair for television, termite treatment)
   Home furnishings and related household items (sofas, lamps, dishes, glassware)
   Clothing
   Costs for rented, leased or owned vehicle (including purchasing, and repair or service for vehicles)
   Non-health insurance (i.e. life insurance, home owners insurance)
   Medical and health expenses (i.e. health insurance, hospital bills, medical services)
   Educational expenses (i.e. lessons, tuition, school related expenses)
   Subscriptions, memberships, books and entertainment expenses (newspaper subscriptions, health clubs, cds, photo processing)

5. Again, here is a list with the expenses covered during the interview. This time please indicate how accurate you think your answers were for each type of expense. In this case the options from very accurate to very inaccurate.

	Very accurate	Somewhat accurate	Neither accurate nor inaccurate	Inaccurate	Very inaccurate
Housing, mortgage or rent expenses
Utilities (i.e. phone, water, electricity, cable)
Construction or home repair expenses
Appliances and household equipment (i.e. electric kitchen appliances, computers, musical instruments)
Household item repairs service contracts (i.e. repair for television, termite treatment)
Home furnishings and related household items (sofas, lamps, dishes, glassware)
Clothing
Costs for rented, leased or owned vehicle (including purchasing, and repair or service for vehicles)
Non-health insurance (i.e. life insurance, home owners insurance)
Medical and health expenses (i.e. health insurance, hospital bills, medical services)
Educational expenses (i.e. lessons, tuition, school related expenses)
Subscriptions, memberships, books and entertainment expenses (newspaper subscriptions, health clubs, cds, photo processing)

Attachment C: Debriefing Outline

Reviewer Note: The debriefing will be conducted using a conversational style of interviewing, the below questions are simply content suggestions.

Introduction

1. Introduce the debriefing portion of the interview, explain purpose and overview of methods

   1. “We’re now going to start the second half of the study. We realize that the survey you just completed can be challenging for some people. It can be difficult to remember everything you’ve purchased [or know about all the expenses other people in your household may have had], and we’re conducting this study to get a better understanding of how you answer the questions.

   2. “To best do this, we want to go back through each question and talk about them in more detail. We’re especially interested in the purchases you reported and any you may have forgotten, [or purchases made by other people in your household that you didn’t know about]. We’ll talk about how complete your original answers may be, and I’d like us to work together to try to get the most complete, accurate information.

   3. “Please don’t hesitate to tell me if you’re not sure of any information, or if you’ve changed your mind and think now that your original answer is incomplete or inaccurate. That’s exactly the kind of thing we’re most interested in here.

   4. I have a list of questions to ask you, but I’m hoping we can have a good, open discussion as we go through the interview. Please don’t hesitate to tell me what’s on your mind.

   5. Any questions before we begin?

Records

1. Does your household keep records of any type to track expenses, for example, do you save receipts in a folder or keep a record of checks you’ve written?

   1. No

   2. Not sure

   3. Yes, but not in a systematic way

   4. Yes, in a systematic way

(if yes) What records do you keep?

1. Check book register

2. Receipts

3. Accounting software

4. Other

2. Do you keep paper records, electronic copies, or both?

1. Paper copies

2. Electronic copies

3. Both

4. Other

3. What type of filing system does your HH use to save records?

a. How up to date is it?

4. Do you generally have records for most of your expenses?

1. What types do you have records for generally?

2. What types don’t you generally have records for?

5. What records did you pull together for this interview?

1. If some: why did you get those records ready?

2. If none: did you think about getting records ready? Why didn’t you?/in retrospect, do you think having records would have been useful?

Information Book

1. (PV) During the interview, we gave you this information book to refer to. How often did you refer to it when the interviewer mentioned a certain page?

   1. Every time I was asked to

   2. Most of the time I was asked to

   3. Some of the time I was asked to

   4. None of the time I was asked to

2. Was it helpful to you in any way?

1. Yes: How so?

2. No: Why not?

6. (T) When we conduct the interview in person, rather than over the telephone, respondents are given this information book to refer to. When questions are asked about a certain type of expense, for example, telephone expenses, home furnishings, appliances, or other types of household equipment, the interviewer will ask you to look at a page with examples of those types of expenses.

3. Do you think this reference book would have helped you during the interview or not? How so?

Section review

1. For selected purchases: Would it be possible to check a receipt or record to confirm the price?

1. If yes: How easy or difficult was it for you to locate and retrieve this record?

2. If no: Why didn’t you check a record during the interview?

1. Would anyone else in your household be able to locate a record for this?

1. For selected sections: let’s look at the information book now and see if there are any (other) items that you may have missed.

2. Review respondent questionnaire, Let’s talk now about the sections you found most difficult

   1. What about those sections made them difficult?

   2. Why were they more difficult than the other sections?

3. Review respondent questionnaire, Let’s talk about the sections you found easiest

   1. What about those sections made them easy?

   2. Why were they easier than the other sections?

4. Review respondent questionnaire: Let’s talk now about the sections where you think your answers were least accurate

   1. What about those purchases makes you think your answers might not be accurate?

   2. Is there anything we could have done to help you arrive at more accurate answers?

5. Review respondent questionnaire: what about the sections where your answers were most accurate, why do you think those answers were most accurate?

Overall Debriefing

1. If you had to describe how burdensome the interview was, what would you say?

1. Could you tell me what the term burden means to you?

2. What do you think about the length of the survey?

   1. How long did you expect it to be?

3. How about the questions asked?

1. Were you able to answer most of them?

2. Any especially difficult?

3. How accurate do you think your answers were overall?

4. Were there any that made you feel uncomfortable?

1. If no: were there any that you could see making someone else feel uncomfortable?

4. How exact did you feel your answers needed to be?

1. Do you think you provided estimates for any of your answers?

1. Which questions?

2. Why?

2. If Felt answers needed to be exact – what was that impression based on?

3. How accurate did you feel your answers were overall?

1. Probe questionnaire results – what made specific sections more or less accurate?

5. [Multi-person CUs] Throughout the interview, the questions ask “Since the first of XXX, have you or anyone in your household purchased….” How well do you think your answers represent the spending done by your entire household?

1. Could you tell me about your knowledge of the spending of other people in your household?

2. How do you know what they spend?

3. Do they know what you spend?

4. Do you think the answers to any of the questions would have been different if someone else in your household had answered the interview questions?

6. Did you feel it was necessary to ask other people in your household about their spending?

1. What gave you that impression?

2. Did you ask other household members about their expenses?

7. (Telephone) Overall, how would you describe the experience of being interviewed over the telephone was?

8. (Telephone) Do you feel that you gave the survey your full attention?

   1. Were you doing anything else while answering the questions (i.e., looking through the mail, washing dishes)

9. If you had been given the choice to complete the survey over the phone or in person, which would you choose? Why?

10. Do you have any other comments or feedback about the interview?

Attachment D: Consent agreement form and Privacy act statement

Consent Form

The Bureau of Labor Statistics (BLS) is conducting research to improve the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.

We estimate it will take you an average of three hours to participate in this research.

Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. OMB control number is 1220-0141, and expires February 28, 2009.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

OMB Control Number: 1220-0141

Expiration Date: 2/28/09

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent