Cognitive Testing Plan -- 2010 Census Deadline Messages

The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). The objective is to conduct cognitive testing on additional deadline messages for the 2010 Census mailout materials as part of the 2010 Census Program for Evaluations and Experiments (CPEX). Specifically, we will test two more messages. One of the messages is a revision of the personal visit motivation message cognitively tested earlier (submitted to you on September 17, 2008 and approved on September 25, 2008). The other message is a new message that attempts to motivate respondents to return their form by appealing to a cost savings. We also plan to test a revised return envelope. The testing of this envelope did not take place as anticipated during the last round of cognitive testing.

For the 2010 CPEX, there will now be ten panels, controlling for five different deadline message treatments and two different mailing strategies. The goal of these experiments is to determine whether a particular deadline message and mailout strategy improves response rates. The goal of this additional cognitive testing is to determine whether respondents understand two new messages included in the mailing pieces:

1) a new cost savings motivational message:

“Please complete and mail back the enclosed census form by April 5. Mailing your census form on time saves money that would otherwise be used to follow up with you.”; and

2) the revision of the personal visit motivation message (formerly known as the NRFU Motivation), which no longer uses the word “inconvenience” to describe the personal visit. The new wording is, “Please complete and mail back the enclosed census form by April 5 so that you can avoid a personal visit from an interviewer.”

The mailing strategy for each treatment includes a message on the advanced letter, outgoing envelope for the initial questionnaire, cover letter for the initial questionnaire, and reminder postcard. In this cognitive testing we will only evaluate the new messages on the cover letters for the following reasons:

• the deadline message in the advance letter in both treatments is identical to what was tested in the previous round of cognitive testing;

• the deadline message on the outgoing envelope of both treatments was also tested in the previous round of cognitive testing; and

• while the deadline messages on the reminder postcard for the two treatments are new, they are identical to what is on the cover letters. In the previous testing we did not learn any new information about the messages placed on the reminder postcard beyond that on the cover letter. We will save resources by not printing new reminder cards for testing.

The attachments contain the two new deadline messages on the cover letter. We have also included other pieces of mail (the advance letter and envelope, outgoing envelope, and form) that we need to show to the respondent in order to “set the scene” for the cognitive testing.

We will also test a return envelope. In the last round of testing, we did not test the redesigned return envelope as planned, but instead tested an envelope similar to the one DeMaio, Beck and Schwede tested (and which was submitted to you for approval on September 19, 2006). We confirmed their results. Our respondents too had problems inserting their completed census forms correctly into the return envelope. Based on these results, three changes have been made to the envelope: 1) the instruction on the front of the envelope is in a different, more readable font; 2) the instruction on the back of the envelope is placed lower; and 3) there is a box around the instruction on the back of the envelope. (The instruction text does not differ between the envelope tested earlier and the revised envelope.) The goal of the return envelope testing is to see if these changes make the instructions more noticeable and increase respondents’ likelihood of inserting the census form correctly.

During February and March of 2009, staff from the Center for Survey Methods Research in the Statistical Research Division will conduct 10 cognitive interviews in the metropolitan Washington DC area. Interviews will be conducted in the cognitive laboratory at the U.S. Census Bureau and at locations convenient for the respondents.

All interviews will be tape recorded with the respondents’ permission, to facilitate summarization of results. Participants will be informed that their response is voluntary and that the information they provide is confidential and will be seen only by employees involved in the research project. Participants will be paid $40 for assisting with the research.

Each cognitive interview session is expected to take one hour; thus the total burden is 10 hours.

The contact person for questions regarding data collection and study design is:

Elizabeth Nichols

Statistical Research Division

U.S. Census Bureau

4600 Silver Hill Road

Rm 5K112A

Washington, D.C. 20233

(301) 763-1724

[email protected]