CE Balance Edit Study

April 20, 2011

NOTE TO THE REVIEWER OF:	OMB CLEARANCE #1220-0141 “Cognitive and Psychological Research”
FROM:	Scott Fricker Research Psychologist Office of Survey Methods Research
SUBJECT:	Submission of Materials for the CE Balance Edit Study

Please accept the enclosed materials for approval under the OMB clearance package #1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, I am submitting a brief description of the study.

The total estimated respondent burden hours for this study are 25 hours.

If there are any questions regarding this project, please direct them to Scott Fricker (202-691-7390).

1. Introduction and Purpose

The current Consumer Expenditure Quarterly Interview Survey (CEQ) asks an extensive battery of expenditure questions that takes 65 minutes to administer on average. The length of the interview is thought to increase respondents’ perceptions of burden, and to reduce response rates and survey data quality. One alternative design that has been suggested for the CEQ is to collect detailed expenditure information for only a subset of the CEQ categories, and to supplement data for the remaining categories by asking respondents to report their total expenditures for the those categories. This approach would take less time and be less expensive to administer, and respondents may perceive it as less intrusive and burdensome than the current design. There is evidence in the survey research literature, however, that global questions may lack the definitional clarity and detailed retrieval cues offered by a more exhaustive, decomposed set of questions, and that this can result in accidental omissions or memory lapses that reduce expenditure reporting (e.g., Jolliffe, 2001; Pradhan, 2009).

To address concerns about the potential for poor quality in such global expenditure reports, some surveys have implemented a ‘balance edit’ process in which reported expenditures, income, and changes in assets are reconciled in the field (e.g., the Family Expenditure Survey and the Survey of Household Spending, conducted by Statistics Canada; the American Life Panel survey, conducted by RAND). Households that have expenditures that are out of balance with the reported cash flow (income and assets) are probed to identify and reconcile possible sources of error. Several studies have found the resulting data to be of higher quality than the alternative methods of collection (see, e.g., Bosworth et al., 1991; Brzozowski & Crossley, 2010; Hurd & Rohwedder, 2010). The purpose of the proposed study is to explore whether the CEQ might be able to adopt greater use of global expenditure questions and a balance edit/cash flow reconciliation approach.

2. Research Design and Procedures

The proposed study will be a small-scale, lab-based test that presents participants with a modified version of the CEQ survey that asks global expenditure, income, and asset questions and includes a balance edit procedure for reconciling expenditure-income/asset disparities. The test sessions will be conducted in the Office of Survey Methods Research (OSMR) laboratory. At the start of each session, a researcher will explain the study’s purpose and procedures and obtain informed consent from the participants (Attachment I). Study participants then will be given a face-to-face interview in which they will be asked a brief set of demographic questions about the HH, global expenditure questions for 34 categories (covering all of the CEQ section topics), detailed expenditure questions about clothing (for comparisons to the global clothing expenditure report), and questions about their HH income and assets (see Attachment II). Expenditures and assets will be collected for the preceding month (1-month recall period), and participants will be asked for HH income using a flexible reference period (weekly, by-weekly, monthly, or annual). All data will be recorded in an Excel spreadsheet that will enable researchers to calculate the ratio of participants’ expenditure-to-income/assets over the reference period. When the ratio deviates by 20 percent or more from 1.0, a balance edit reconciliation procedure will be implemented. During reconciliation, participants will be given the opportunity to review their reported expenditures, income and assets, and to make changes to their earlier reports in order to bring their expenditure-income/asset ratio closer to 1.0. Researchers will record any changes made by respondents. Reconciliation will terminate when expenditures are within 10 percent of reported income/assets, or when respondents are unable or unwilling to continue the process. Following data collection and any reconciliation, participants will be administered a short set of debriefing questions to explore their reactions to the survey (e.g., recall/reporting difficulties, reactions to the survey and reconciliation; see Attachment III for debriefing items).

3. Study Objectives

The purpose of the proposed study is to examine the feasibility of implementing global expenditure questions and a reconciliation procedure in a modified CEQ survey. In particular, we will assess whether participants are willing and/or able to provide global expenditure reports across all of the CEQ section categories, as well as HH income and assets – and attempt to determine any mediating factors (e.g., frequency of purchase; topic sensitivity; knowledge; conceptual clarity; cognitive difficulty; etc.). The main focus of this study is on the reconciliation process. We will attempt to measure how long this process takes, its effectiveness in identifying and correcting errors of commission and omission, and respondents’ reactions. In addition, the study design will enable us to examine several factors that may affect the efficacy of the reconciliation process. In particular, we are interested in exploring whether participants have difficulty answering asset-related questions (e.g., changes in loan or saving account balances) for the one-month reference period, and the extent to which certain features of the reconciliation procedure – assumptions necessary for triggering reconciliation (e.g., deriving monthly income values for participants who only can provide weekly or annual income; estimating after-tax income) - impact participants’ reactions or the quality of their reports.

NOTE: study participants will not be asked to bring in their records for this study.  Although some participants may find the reconciliation process difficult without the ability to consult records, we decided against requiring records for three reasons.  First, the present study is a first test of a basic reconciliation procedure. It is designed to assess whether participants understand what is being asked of them and the extent to which they engage in an effortful manner; we will capture respondent reactions to the process as well as indirect measures of data quality (e.g., additional reports elicited during reconciliation, indicators of satisficing or effortful responding, respondents’ assessment of accuracy, etc.).  Second, people differ in the amount of records they maintain, the format/portability/organization of those records, and in their willingness/ability to bring those records to a lab and use them.  That differential availability and use could complicate the interpretability of the study results, and would require a larger study that systematically examined the effects of record usage on reconciliation.  Finally, we were concerned that asking participants to bring in sensitive records that potentially could be lost or disclosed in transit.  However, spontaneously use of records during the test session (e.g., by those who bring in checkbooks, who have smart-phones with access to accounts, etc.) will be noted.  In addition, if during recruitment people ask whether they need to bring in records, we will tell them that they do not need to bring them, but can if they would like to do so. 

4. Participants

Twenty participants will be recruited from the OSMR participant database. We will select participants who vary in their income, family size, education, and employment status.

5. Burden Hours

We anticipate that each test session will last 1.25 hours on average (i.e., 10 minutes for front matter, 35 minutes for administration of the modified CEQ survey and reconciliation items, and 30 minutes for debriefing.). Therefore, we estimate that this study will use a total of 25 burden hours.

6. Payment

For this study we will be paying participants the standard $40.

7. Data Confidentiality

Participants will be informed of the voluntary nature of the study. Participants also will be informed that the study will be used for internal purposes to improve the design of the Consumer Expenditure Quarterly Interview survey. Participants will be given a consent form to read and sign (Attachment I) prior to beginning the test session. Information related to this study will not be released to the public in any way that would allow identification of individuals except as prescribed under the conditions of the Privacy Act Notice.

8. Attachments

Attachment I:	Consent agreement form and Privacy Act statement
Attachment II:	Respondent Questionnaire
Attachment III:	Debriefing Items

9. References

Jolliffe, D. (2001). Measuring Absolute and Relative Poverty: The Sensitivity of Estimated Household Consumption to Survey Design. Journal of Economic and Social Measurement, 27(1), 1-23.

Pradhan, M. (2009). Welfare Analysis with a Proxy Consumption Measure: Evidence from a Repeated Experiment in Indonesia. Fiscal Studies, 30, 391-417.

Attachment I

Consent Agreement Form and Privacy Act statement

Consent Form

The Bureau of Labor Statistics (BLS) is conducting research to improve the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.

We estimate it will take you an average of one and a quarter hours to participate in this research.

Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. The OMB control number is 1220-0141 and expires February 29, 2012.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

OMB Control Number: 1220-0141

Expiration Date: 2/29/12

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

6