Nonsubstantive Change - Prenotice Letter Effectiveness Test

OMB Non-substantive Change Request

Department: Commerce

Agency: U.S. Census Bureau

Title: American Community Survey Methods Panel Tests

OMB Control Number: 0607-0936

Expiration Date: 12/31/2015

2014 ACS Prenotice Test

Currently, the ACS mailing strategy includes a prenotice letter to let households know they will be receiving instructions in the mail to complete the American Community Survey. We mail this a few days before we mail out the letter inviting response via the Internet. In addition to providing notification, the prenotice letter is intended to encourage the respondent’s cooperation and thereby increase the likelihood of a response once the invitation is received. Use of a prenotice letter is standard practice in many household surveys and past research has consistently shown the benefits of pre-notification. Former Census director Robert Groves posted the following in his February, 2010 Census blog (http://directorsblog.blogs.census.gov/2010/02/18/why-use-advance-letters/), “Years of research have shown that higher percentages of people receiving the mail questionnaire return a completed form after they receive the advance letter compared to those who receive merely the census form with a simultaneous request to return it.” However, much of the past research has dealt with sending a prenotice prior to mailing a paper questionnaire, as was done in the ACS before it adopted the Internet push strategy in 2013. The 2012 National Census Test included research on the impact of a prenotice letter under an Internet push strategy; the results did not indicate significant differences in Internet or overall self-response rates between the groups that received or did not receive a prenotice letter. While this is encouraging, it is hard to generalize these results to the ACS since it is lesser-known than the census.

The Census Bureau is working with Reingold, Inc. to create an integrated messaging campaign for ACS and assess the ACS mail materials. Reingold has recommended eliminating the ACS prenotice mailing because it does not contain a call to action. Before we can consider such action, we must determine the impact this could have on ACS response rates, in particular Internet self-response. Failure to respond by Internet leads to more costly attempts to solicit response, i.e. mailing the paper questionnaire and conducting CATI and CAPI. This research will guide us in determining the impact on response if we drop the prenotice letter and how large that effect could be. Using the September 2014 ACS panel, we will analyze response (at various points in the data collection cycle) with an experimental treatment where the prenotice is not mailed and compare response rates to those obtained through the standard ACS mailing strategy. Using these results, we will estimate the impact to costs at various stages in the data collection cycle.

To field this test, we plan to use ACS production (Clearance number: 0607-0810, expires 6/30/2016). Thus, there is no increase in burden from this test since the treatment will result in approximately the same burden estimate per interview (40 minutes). We have divided the monthly production sample into nationally representative groups of approximately 12,000 addresses each. For the September 2014 panel, we will use one of these groups for the experimental treatment group that will not receive the prenotice letter. The remaining cases in the September 2014 panel will comprise the control and receive all standard ACS mailings. As we are using production cases for the test, the test will run through the complete 3-month data collection.

A key evaluation measure for this test is the Internet self-response rate prior to mailing out the second mail package, which contains the paper questionnaire. Comparing the Internet self-response rate between a treatment group and the remainder of the panel allows us to detect a 0.8 percentage point difference with 80% power and α=0.1.

Contact

Jennifer Tancreto

301-763-4250

[email protected]

References:

Dillman, D.A., Smyth J.D., Christian, L.M. (2009). Internet, Mail, and Mixed-mode Surveys

Reiser, C., Barron S., Bentley, M., Hill, J.M., Meier A., Pape, T. (2013). 2012 National Census Test Contact Strategy Results, forthcoming

Return to Top