Opt-In GovDelivery Panel Testing Plan

The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We plan to conduct a pilot test using an online questionnaire to gather information about email subject lines and screen designs to collect residential address information. This pilot test is part of an iterative testing strategy that we plan to use to supplement the 2020 Census research program. The goal is to determine the best email subject lines to maximize self-response for the 2020 Census. Email subject lines that fail to generate significant click-through rates in the nonprobability panel are not likely to perform better in a probability panel. Eliminating poorly performing email subject lines before larger-scale testing is the purpose of the nonprobability testing we propose here.

We will send survey invitation emails to a sample of people who opted-in to participate in Census Bureau research studies through the Census Bureau’s email subscription website, run by GovDelivery. There was no incentive to sign up and there is no incentive (other than a copy of the research report) to participate. Currently, the panel has over 7000 emails. Because of the opt-in nature, this panel is considered a nonprobability panel. This test is a pilot test as we have not yet contacted any of the panel to participate in a research study. We will measure the open rates and click through rates to the emails with different subject lines.

There are four panels in this test. Each panel consists of a sample of 250 email addresses for a total sample of 1,000 email addresses. There are of two different versions of the online survey. The only difference in the surveys is the screen that collects residential address information. There are two different initial emails, both containing a link to the survey. The only difference in the emails is the subject line of the initial email. The design is completely crossed. The sample will receive a maximum of three notification emails:

• one of the initial emails on Monday, January 6,

• a reminder email on Thursday January 9 (if they had not yet click on the link to the survey), and

• a final reminder email on Thursday, January 16.

The survey will be closed on Friday, January 17 at midnight.

The survey that the emails link to will collect data on address collection screen designs. The Census Bureau hypothesizes that many people will come to the online 2020 Census without their housing unit ID and thus they will have to enter their address as part of the data collection. We must make sure that the address screen collects accurate address data in order for us to match to our master address file and geocode. Without good addresses, we will not be able to count people at the correct location for the census.

After the address screens, we ask opinion questions as a part of the “debriefing.” The objective of these questions is to gather qualitative data in order to guide future iterations of this test, to gain a sense of how respondents want to be contacted about the census, to answer the census, and what these highly motivated individuals think the census collects. These data will be shared with 2020 Census staff and the communications area of the Census Bureau. The answers to the demographic questions and questions on new technologies will allow us to look at characteristics of respondents.

The pilot test will be conducted from January 6 through January 17, 2014. Staff from the Center for Survey Measurement’s Human Factors and Usability Research Group will select the sample and send the emails through GovDelivery. The survey will be hosted on our secure servers within the Application Services Division of the Census Bureau that hosts all other secure online production surveys. The username needed to enter the survey will be the email address where the email was sent (this is the same email used to sign up to participate in Census Bureau research studies). If the respondent starts the survey but does not complete it, that person will not be allowed to re-enter the site later. The emails and questions were usability tested earlier under a separate generic clearance letter sent on June 12, 2013.

Future iterations of this survey will be conducted under a new generic clearance that is in preparation.

We estimate that users will spend 5 minutes on average completing the survey and approximately 5 minutes reading emails. Thus, the total estimated respondent burden for this study is 167 hours.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Elizabeth Nichols

Center for Survey Measurement

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-1724

[email protected]