2010 AQE Reinterview Evaluation Plan

2010 Census Alternative Questionnaire Experiment (AQE) Reinterview Evaluation

Individual Request for Clearance

September 1, 2009

The Census Bureau plans to conduct the 2010 Census Alternative Questionnaire Experiment (AQE) Reinterview Evaluation under the 2010 Census Program for Evaluations and Experiments (CPEX) generic clearance (OMB approval number 0607-0952).

2010 AQE Reinterview – Updates

The overall goal of the AQE and related reinterview effort is to improve a user-friendly mailout questionnaire that can be accurately completed by respondents. Since the race and Hispanic origin item treatments are the main focus of the 2010 AQE, the Census Bureau would like to determine ways to reduce item nonresponse to the race and Hispanic Origin questions, as well as increase reporting of the five race categories defined by the Office of Management and Budget (OMB). A secondary goal is to improve the reporting of detailed Asian groups, Pacific Islander groups, American Indian and Alaskan Native tribes, and specific Hispanic origins in the race and Hispanic origin questions.

In terms of experimental treatments on the AQE mailout questionnaires, we plan to test modified race and Hispanic origin examples, a Hispanic origin question that accepts multiple responses, race questions that clarify OMB categories, and various combined race and Hispanic origin question strategies. The AQE reinterview is designed to be a followup data collection conducted on sampled respondents across the 15 AQE panels devoted to race and Hispanic origin research (as well as two control panels). This reinterview will ask probing questions aimed at getting self-identified race and Hispanic origin measures that are closer to the respondents’ true self-identification compared to that which is obtained from the paper, self-administered instrument. These reinterview measures will be used to assess the relative accuracy of the AQE paper questionnaire treatments.

Universe and Respondent Selection

The main measure from the AQE reinterview results will be response bias. This will be obtained by comparing the respondents’ self-identified races and origins from the telephone AQE reinterviews to those obtained from the paper questionnaires. The AQE reinterview will ask probing questions aimed at getting self-identified race and Hispanic origin measures that are closer to the respondents’ true self-identification compared to that which is obtained from the paper, self-administered instrument. These reinterview measures will be used to estimate bias (i.e., relative accuracy) of the AQE paper questionnaire treatments. Although race/origin does not have a verifiable ‘truth’ in the census self-identification context, a “closer-to-truth” race and origin measure will be ascertained from an initial open-ended question followed by a series of probing yes/no and followup questions. The respondent is exposed to the same OMB race and ethnic reporting categories that are on the paper form in an attempt to maintain similar essential survey conditions. However, the reinterview probes use a more conversational approach to determine how respondents self-identify race and origin without various limitations of the paper instrument (e.g., format and space constraints).

The exact wordings of the reinterview questions were developed in consultation with experts in the sociology of race and ethnic reporting. The questionnaire underwent extensive cognitive testing with a diverse group of participants. The ultimate goal was to arrive at a questionnaire design that is optimal for collecting a race and Hispanic origin classification that is closer to the respondent’s true self-identification than the paper questionnaire items. One automated (WebCATI) reinterview instrument will be used to measure response bias of the AQE paper questionnaire results (and the 2010 Census race and Hispanic origin items via the control panels). The same reinterview will be offered in both English and Spanish. All cognitive testing has been completed.  The attached set of AQE Reinterview questions is the final version.  The final questions reflect results from testing.

The reinterview will be conducted with the mail return questionnaire respondent whenever possible; proxy respondents will be accepted as a last resort only. The mail questionnaire respondent will be interviewed about his/her own race(s)/origin(s) self-identification and the same respondent will be asked to identify the race(s)/origin(s) of one other randomly selected household member.

The AQE has a complex stratified sample design that is intended to reach the appropriate populations of interest for the proposed treatments, while still providing accurate national estimates. The design concentrates on achieving a significant mailout to minority demographic groups such as Hispanic or Latino people, Asian people, and Black people (population group terminology is based on Census 2000 response category names). The universe consists of census tracts likely to contain relatively high proportions of these subgroups. The final sample of housing units will be selected from within these areas, which partition the country into four strata.

The current AQE reinterview sample design utilizes the over-sampling of population groups of interest from the AQE mailout stratified design. A one-in-five sample of mail returns from the AQE strata will be systematically sampled for the reinterview such that the population groups of interest will be over-sampled as a result of the AQE stratified design.

The AQE reinterview has a targeted sample size of 3,780 for each of the 15 race/origin treatment panels and two control panels, which is based on an estimated mail response rate of between 65 to 75 percent for each stratum. The percentage of telephone numbers that are captured (either from the mailout paper questionnaires or from a telephone number look-up operation), as well as the success rate in obtaining completed interviews, will determine the final sample sizes. Since operational processes may result in a larger (or smaller) actual sample size, we assume 4,000 cases per panel for burden estimation.

No updates for the following:

• Procedures for Collecting Information

• Methods to Maximize Response

• Tests of Procedures and Measures

• Contact for Statistical Aspects and Data Collection

Estimate of Burden Hours for 2010 AQE Reinterview Evaluation

Experiment/Evaluation	Total # of Respondents	Estimated Response Time	Estimated Burden Hours
AQE Reinterview	68,000	7 minutes	7,934

2010 AQE AND CONTENT REINTERVIEW SCHEDULE AND COSTS

Project Schedule for 2010 AQE Reinterview and 2010 Content Reinterview Evaluation

The data collection will occur from June 1, 2010 though July 31, 2010.

Costs To Federal Government for 2010 AQE Reinterview and 2010 Content Reinterview Evaluation

The preliminary estimated cost to the Federal Government for the AQE Reinterview and Content Reinterview is $2,800,000. The Census Bureau is the only agency bearing this cost.

Contact(s) for Statistical Aspects and Data Collection

For questions on statistical methods or the data collection described above, please contact Jason Machowski of the Decennial Management Division at the Census Bureau (Phone: 301-763-4173 or email [email protected], or Erin Love of the Decennial Management Division at the Census Bureau (Phone: 301-763-2034 or email [email protected]).

3