ETL Field Test Plan

The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). The objective of this research is to conduct a field test with respondent debriefings of the Enumeration of Transitory Locations (ETL) form that will be used in the 2010 Census. The ETL will be used to enumerate people staying in places where highly mobile people may be found who do not have a usual home elsewhere. These include marinas, racetracks, carnivals, etc. The questionnaire is an interviewer-administered paper form.

The ETL has a very similar format to the Enumerator Questionnaire (for Nonresponse Followup and Update/Enumerate operations) that has been approved for cognitive and usability testing under the generic clearance. The main difference between these forms is the residence instructions. The ETL is designed to capture very specific residence situations (that of someone in a transitory location who has no other place where they might be counted).

We are concerned with 1) respondents’ understanding of the residence questions and their ability to respond to them correctly, and 2) the ability of field interviewers to use the form correctly. The second aspect is more similar to usability testing than cognitive testing, although we have not planned to use traditional laboratory techniques to test these instruments. It is important to examine the operation of these forms under something approximating normal field conditions, and for this reason, we propose to split the testing into two phases. One phase will examine how well the respondent understands the form, through respondent debriefing. The other phase will be a small-scale usability evaluation designed to assess problems for enumerators in the field. This is an attempt to evaluate the forms through a study larger than a cognitive test, but smaller than a field test. Copies of the draft questionnaire and the information sheet given to respondents are enclosed.

In the first phase of the study, we will arrange to enumerate 2 sites in the Washington, DC metropolitan area (RV parks, marinas, racetracks, or carnivals) where we have permission of the site management to do a small scale mock enumeration. We plan to work in pairs: two teams of two people each will be required. One staff member will enumerate the sites using the enclosed materials from the ETL (the unit verification sheet and the ETL questionnaire). The second staff member will observe the interview and conduct respondent debriefings using the enclosed debriefing questionnaire. We will debrief each respondent to determine whether 1) if a household is enumerated, there is another place where they should have been counted, and 2) if they were not enumerated, there was a place where they would likely have been counted. We will gather enough information to assess whether or not we think that person had a good chance of being counted somewhere else.

The second phase of the study is to see whether actual Field Representatives (FRs) can use the instrument under field conditions. In this phase we will bring in two local FRs who have some interviewing experience. A brief training will inform the participants about the operation and how to use the forms (the unit verification sheet and the ETL questionnaire). The enumeration will take place at an ETL site (RV park) where we have permission to do a small mock enumeration. The FRs will attempt to conduct the full operation with a set of units. Each FR would be accompanied by an SRD observer. The observer will take notes throughout the day on any problems the interviewer seems to be having. Later, the SRD observers will conduct an interviewer debriefing to obtain the FR’s assessment of how the forms work, and how they could be improved. During the debriefing, the SRD researchers will walk through the questionnaires question by question with the interviewers letting the interviewer report any problems or issues they experienced and the researchers probing on any areas they had noted as problematic. In this piece, we're particularly interested in form navigation, so that will be the focus. The researchers will assess the usability of the form from these observations, debriefings, and from opinions of the interviewers.

During July and August, staff from the Center for Survey Methods Research (CSMR) will conduct this enumeration, debriefing and usability testing. The respondent debriefing portion will include a maximum of 80 interviews. The usability portion will yield a maximum of 40 interviews. This yields a maximum of 120 interviews.

The interviews will be tape-recorded, with the participants' permission, to facilitate summary of the results. All participants will be informed that their response is voluntary and that the information they provide is confidential.

The estimated time for completion of each of the interviews is 10 minutes. The respondent debriefing will take an extra estimated 5 minutes. We estimate 120 interviews totaling 20 respondent burden hours and 80 respondent debriefings totaling approximately 7 hours. Thus, the estimated burden for this research is 27 hours.

The contact person for questions regarding data collection and study design is:

Jennifer Hunter Childs

Center for Survey Methods Research

U.S. Census Bureau

Room 5K112F

Washington, D.C. 20233

(301) 763-4927

[email protected]