OMB Justification Statement

The Census Bureau plans to conduct new research under the Generic Clearance for 2020 Census Field Test to Automate Field Data Collection Activities (OMB Control Number 0607-0971). The Center for Survey Measurement in conjunction with the Decennial Statistical Studies Division plans to conduct iterative small-scale mail-out tests to refine messaging in invitations to an online survey. Results from these tests will inform the 2020 Decennial Census testing program for further testing. This submission documents our first test in this series.

This testing was motivated by the American Community Survey research conducted under contract by Reingold (www.census.gov/acs/www/Downloads/library/2014/2014_Walker_02.pdf). Their qualitative testing indicated that altruistic messages about the local community resonate with the public and might be a motivator for response. Although they recommend a geographical targeting approach to the local community, this is not feasible operationally. A more operationally feasible approach is to target at the state or city level. We plan to test three different “community benefits” messages in the letter invitations. The first treatment is identical to the 2015 National Content Test (NCT) message mentioning the federal funding benefits for “each community” – this will serve as the control panel. The second treatment modifies the message to mention federal funding benefits for the city and state where the household receives mail. The third treatment modifies the message to mention federal funding benefits for the state where the household receives mail. We are interested in determining whether the geographic specificity elicits a higher response rate to the online survey.

Secondarily, we will conduct a split-panel test of the online survey, making a slight modification to the telephone number field format between the two surveys. This research objective was motivated by the findings of usability testing of the 2015 Census Test online instrument on mobile devices. In usability testing, we observed iPhones toggling between the number keypad and the letter keypad as the user tabbed between the telephone fields. As part of this test, one online survey treatment will include the current Census Bureau phone number response fields: the first 3-digit field is for the area code; the next 3-digit field is for the exchange, and the last 4-digit field is for remainder of the telephone number. The other online survey treatment mimics the fields used by the U. S. Postal Service for a request to hold mail. They use one field with an imbedded format [(___)___ -____ ]. We plan to measure data quality and response burden (as measured by time on task and error corrections) between the two online survey treatments.

We plan to conduct this test beginning July 13, 2015. There will be a maximum of four mailings to each housing unit. The first mailing is the initial letter on July 13, 2015. The second mailing, occurring one week later on July 20, is a postcard. The postcard will not differ between treatments. The third mailing, on July 27, is another reminder postcard. This postcard will not differ between treatments either. The fourth mailing is the final letter on August 3, 2015. The survey will close out August 22, 2015, approximately six weeks after the first mailing. The third and fourth mailings will not be sent to any housing unit for which a response has been received by July 20. The final mailing will not be sent to any housing unit for which a response has been received by July 27.

This is an online-only data collection test. There is no paper form or telephone data collection for this test. Because some sampled households might not be able to respond online, we offer a telephone number with an automated message indicating that we will contact them if we need their information if they cannot respond on the Internet.

We will select a sample of 6,000 addresses from the Census Bureau’s Master Address File. Each benefit-message treatment will have a sample of 2,000 housing units, while each online survey treatment will have 3,000 housing units. The design will be fully-crossed. Given a response rate between 40 and 45 percent, we should be able to detect an estimated 5 percent difference in response using alpha=0.10 and beta=0.80, adjusted for multiple comparisons. The sample will exclude housing units selected for other recent Census tests and the American Community Survey.

The online survey is similar to the online survey used for the small-scale email testing approved under the Generic Clearance for Internet Nonprobability Panel Pretesting (OMB number 0607-0978). The survey collects opinion and general demographic data.

The first enclosure includes:

• Initial Letter (showing differences for each treatment by colored text)

• Internet card

• Postcard Reminder

• Second Postcard Reminder

• Final Letter (showing differences for each treatment by colored text)

• Internet card

The second enclosure includes the survey questions (screenshots where available and text when the screenshot is not available).

We estimate that users will spend 8 minutes on average completing the survey and approximately 5 minutes reading mailing material. Thus, the total estimated respondent burden for this study is approximately 1,300 hours, which assumes everyone reads the mailing material and answers the survey.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Elizabeth Nichols

Center for Survey Measurement

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-1724

[email protected]

2