Field testing plan

The Census Bureau plans to conduct additional research under the generic clearance for questionnaire research (OMB clearance number 0607-0725). We will be conducting a field test to permit a data quality comparison of two interviewing methods for the Survey of Income and Program Participation (SIPP) – SIPP’s standard “question list” approach, using a 4-month reference period, and a new interviewing methodology (under consideration for use in the re-engineered SIPP several years hence) called event history calendar (EHC) interviewing, using a 12-month, calendar year reference period. Respondent and interviewer debriefings, as well as a form of behavior coding assessment, will also contribute to the evaluation of the two methods.

In essence, the field test is a reinterview study among a sample of former SIPP households who provided information about their calendar year 2007 circumstances in the normal course of interview waves 10, 11, and 12 of the just-completed 2004 SIPP panel. Starting in April, and continuing for a period of approximately six weeks, the field test will collect data for the same calendar year 2007 time period in a single interview using EHC methods. An “un-primed” control group will also be administered the EHC interview – this will provide a means of assessing the “priming” effect on EHC responses of having previously been interviewed about the 2007 time period. The control group consists of cases which were dropped from the 2004 panel in a budget-trimming exercise following their wave 8 interview, conducted in June through September 2006.

The central research question is as follows: Is the quality of the survey reports obtained in the single EHC interview, using a 12-month calendar year reference period, comparable to that of three standard SIPP interviews, each of which collected data about a 4-month reference period? Results of this research will weigh heavily in the final decision about the viability of EHC methods – especially those which would employ a 12-month reference period – in the re-engineered SIPP. A single interview per year, versus the current SIPP’s three interviews, would clearly contribute toward two key goals of the re-engineering effort: the reduction of respondent burden and the reduction of costs. The question is whether the EHC approach can contribute toward those goals without compromising data quality.

The EHC field test interviews will be conducted in two states, Illinois and Texas, by Census Bureau field representatives (FRs) working out of the agency’s Chicago and Dallas Regional Offices (ROs). (In fact, the Texas cases are restricted to four metropolitan areas: Austin, Dallas/Fort Worth, Houston, and San Antonio.) We are restricting the sample in this way in response to several factors, including budget considerations, of course, but also the availability of RO staff – and especially experienced SIPP FRs – during the time period of the field activity. The main reason, however, is to permit the later use of administrative record data, which will permit the most objective assessment of relative data quality for some characteristics. Illinois and Texas are two states where the Census Bureau has (or expects that it soon will have) in place arrangements to obtain record data on state-level programs (e.g., TANF, Food Stamps) for research purposes. Note that as of wave 8 of the 2004 panel, all SIPP responding households were asked to explicitly consent to having their SIPP and administrative record data linked. Only those cases which gave consent will be linked in this research project.

The total sample size for the “un-primed” (wave 8 sample cut) control group is approximately 850 addresses; the main reinterview sample will consist of approximately 1,100 addresses. Each “adult” (age 15+) in responding households will be administered an EHC interview. In addition to statistical analyses examining differences between the two interviewing methods, we will administer a very brief (2-3 minute) respondent debriefing form to one respondent in each household. FRs will also fill out a brief evaluation form for each interview, and will be debriefed concerning the EHC interview as a group at the end of the interviewing period. We also hope to be able to tape record a small number of EHC interviews – with respondents’ explicit consent, of course – for a subsequent behavior coding evaluation.

Respondents will be informed that their involvement is voluntary and that the information they provide is confidential and will be seen only by sworn Census Bureau employees involved in the research project. We expect a relatively high response rate for the field test because the sample consists largely of households which have demonstrated a long-term willingness to cooperate with the SIPP interview. Nevertheless, we will take standard measures to maximize response. For example, we will continue SIPP’s long-standing practice of offering sample households an incentive as an inducement to cooperation (in this case, a $40 debit card), as well as standard Census Bureau follow-up procedures to convert those initially reluctant to respond through the use of highly experienced senior interviewers and other refusal conversion specialists.

A copy of the paper-and-pencil EHC questionnaire (and its accompanying “Control Card” for capturing the household roster and basic demographics) is enclosed. (A special note about the EHC form: We have designed the form to be assembled such that the last page (p. 5) is offset to sit above the other pages, in order to make the “landmark events” visible and accessible to the interviewer throughout the interview. The title of the form and the box containing the OMB clearance information are also included at the top of this page, and thus also visible throughout the interview.) A draft of the respondent debriefing questions is also enclosed.

We estimate that completion of the EHC and respondent debriefing questionnaires will take, on average, approximately 30 minutes per person. Assuming 2.1 adults per household, we estimate the maximum total respondent burden hours for this project to be approximately 2,048 hours

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Jeffrey Moore, Ph.D.

Center for Survey Methods Research

Statistical Research Division

U.S. Census Bureau

Room 5K013

Washington, D.C. 20233

(301) 763-4975

[email protected]