OMB August 2016 Memo Revised

The Census Bureau plans to conduct new research under the Generic Clearance for 2020 Census Field Test to Automate Field Data Collection Activities (OMB Control Number 0607-0971). The Center for Survey Measurement in conjunction with the Decennial Statistical Studies Division plans to conduct iterative small-scale mail-out tests to refine messaging in invitations to an online survey. Results from these tests will inform the 2020 Decennial Census testing program for further testing. This submission documents our third test in this series.

This test has two objectives. The first has four treatments and the second has three treatments, which are completely crossed creating a twelve panel test.

The first objective, called the “Data Use Statement” objective, determines which data use statement will maximize response to an online survey. It is assumed that telling respondents how their data will be used will increase response rates. This objective tests several versions of the data use statement to examine the effect on response rates. The variations will appear in the first and fourth mailings. For this test, we use the current standard data use statement as our control. An alternate control will use a modified letter that makes no statements about how the data will be used. The remaining two treatments test the "most compelling data use statement" and "least compelling data use statement" as suggested by prior message testing research using conjoint analysis (Quarles, 2015). We are testing three variations, in addition to the standard letter, to determine whether (1) the assumption that data use statements improve response rates is true; (2) whether different data use statements drive response given everything else in the letter; and (3) whether conjoint analysis results can be used to predict respondent behavior.

The second objective, called the “Postcard Closure” objective, determines the effect of alternative postcard formats on response rates. Our control treatment uses a standard first reminder postcard that does not need to be opened to be read. Two other treatments will test the effect of adding either a tabbed closure or perforated closure to the first reminder postcard. The tabbed closure requires the mail recipient to break a seal on the postcard in order to access the content, while the perforate closure (typically used for confidential communications) requires that the recipient tear off three perforated strips in order to view the content. It is thought that these alternative postcard formats might increase attention to the first reminder postcard, thereby boosting response rates. In this test, we will send these three postcard types in the second mailing. We will send only standard postcards for the third mailing.

We plan to conduct this test with a Census Day of August 1, 2016. There will be a maximum of four mailings to each housing unit. The first mailing is the initial letter on July 25, 2016. The data use statement will differ in the first mailing. The second mailing, occurring one week later on August 1, 2016, is a postcard. The type of postcard closure will differ in the second mailing. The third mailing, on August 8, 2016, is another reminder postcard. All housing units will receive identical third mailings. The fourth mailing is the final letter on August 15, 2016. The data use statement will differ in the fourth mailing. The survey will close out August 26, 2016, approximately six weeks after the first mailing. The third and fourth mailings will not be sent to any housing unit for which a response has been received.

This is an online-only data collection test. There is no paper questionnaire or telephone data collection for this test. Because some sampled households might not be able to respond online, we offer a telephone number with an automated message indicating that we will contact them if we need their information if they cannot respond on the Internet.

We will select a sample of 9,000 addresses from the Census Bureau’s Master Address File. Each Data Use treatment will have a sample of 2,250 housing units and each Postcard Closure treatment will have a sample of 3,000 housing units. The design will be fully-crossed. Given a response rate between 40 and 45 percent, we should be able to detect an estimated 5 percent difference in response using alpha=0.10 and beta=0.80, adjusted for multiple comparisons. The sample will exclude housing units selected for other recent Census tests and the American Community Survey.

The online survey is similar to the 2015 National Content Test online survey used approved under the (OMB number 0607-0985) until 6/30/2018. The survey collects general demographic data.

The first enclosure includes:

• Initial Letter (showing differences for each treatment by colored text)

• Internet card (this card is included with the initial letter and the final letter)

• Postcard Reminder

• Second Postcard Reminder

• Final Letter (showing differences for each treatment by colored text)

The second enclosure includes the survey questions (screenshots where available and text when the screenshot is not available).

We estimate that users will spend 10 minutes on average completing the survey and approximately 5 minutes reading mailing material. Thus, the total estimated respondent burden for this study is approximately 2,250 hours, which assumes everyone reads the mailing material and answers the survey.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Casey Eggleston

Center for Survey Measurement

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-6144

[email protected]

2