NCS_2017AdaptiveStrategyTest

OMB Nonsubstantive Change Request

Department: Commerce

Agency: U.S. Census Bureau

Title: American Community Survey Methods Panel Tests, 2017 Adaptive Strategy Test

OMB Control Number: 0607-0936

Expiration Date: 8/31/2018

Request: The Census Bureau plans to conduct research under the Methods Panel clearance to test an additional mailing strategy for the American Community Survey (ACS), designed to increase self-response in areas that saw a decrease with the addition of the internet data- collection mode in 2013. The Census Bureau proposes to conduct a field test of this new “choice” mailing strategy as part of the production ACS October 2017 panel. This will require a change in the mail materials for the choice mailing method. No additional burden is expected.

Background: In 2013, the Census Bureau added an internet data-collection mode. The goal of this change was to decrease data collection and processing costs and to provide a more convenient way for people to respond to the ACS. The result of adding the internet mode was that overall self-response rates increased, but they also decreased in certain areas of the country (Baumgardner, 2014). The Census Bureau hypothesizes that respondents in these areas may not have internet access or prefer to respond by paper. In response to this decrease, the Census Bureau has devised a test offering a choice in modes (paper and internet) in areas with low internet response in the hopes of increasing self-response.

The Census Bureau conducted similar research to this project in the 2015 National Content Test (Bentley, 2016). The results showed offering a choice method was effective. The research conducted in this project builds on the findings from the 2015 National Content Test.

Timeline: The test is being planned as part of the ACS October 2017 panel, adhering to the same data collection protocols as production ACS.

Method: Currently, for the 50 states and the District of Columbia, the ACS has five mailings:

• An initial mailing package.

• A reminder letter.

• A paper questionnaire package (if a response has not been received).

• A reminder postcard (if a response has not been received).

• An additional reminder postcard (if an internet or mail response has not been received and the address is not eligible for telephone followup).

In the first two mailings, the Census Bureau pushes for a response via internet. Then a paper questionnaire is sent in the third mailing if the Census Bureau has not received a response. The Census Bureau calls this strategy the Internet Push strategy. For the Choice mailing strategy, the Census Bureau would include the paper questionnaire in an earlier mailing so that respondents have a choice between paper and internet. The choice method will follow the mailing schedule that the Census Bureau uses for the Puerto Rico Community Survey (where the Census Bureau does not offer an internet option):

• A prenotice letter.

• An initial paper questionnaire package.

• A reminder postcard.

• A replacement paper questionnaire package (if an internet or mail response has not been received).

• An additional reminder postcard (if an internet or mail response has not been received and the address is not eligible for telephone followup).

Analysis Metrics: The Census Bureau will develop an algorithm to classify census tracts based on their likelihood to respond via the internet and historical observation of mode preference. Once these census tracts are identified, the Census Bureau will assign approximately half of the households within these tracts to receive the Choice mailing materials, and the other half will receive the Internet Push (current production) materials. Households within tracts not identified by the algorithm will receive the Internet Push (i.e., current production) materials.

The main purpose of the analysis is to determine if offering choice mail materials to targeted areas improves response rates while at least maintaining costs. The analysis will answer the question: What is the impact of offering a choice of mode on the self-response rates, final response rates, reliability of the estimates, and cost? Self-response rates will be examined overall and by mode (internet and mail). Final response rates include all modes of data collection. The cost analysis will account for the increased cost related to the mailings using the Choice mailing strategy and will determine if there was sufficient increase in self-response to offset the costs.

Sample: The current proposed algorithm to identify census tracts that would benefit from receiving a paper questionnaire earlier in the mailing process has identified approximately 25,000 census tracts from the more than 72,000 census tracts. The Census Bureau estimates that 100,000 housing units in those tracts will be in the ACS sample in a month. The Census Bureau will randomly assign approximately half of these housing units to the Choice mail methodology and half to the Internet Push methodology. This sample design will allow detectable differences of approximately 1.0 percentage point between the self-response return rates. The power of the test, which is 80 percent and where α=0.1, assumes a 50 percent self-response rate.

Use of Incentives: None.

Burden: There is no change in burden to the public associated with this test. The test is being conducted using production ACS sample. No additional contacts are being made; the sequence of mailing and the content of the mailings (including a paper questionnaire earlier, for example) are being modified.

Public Comments: A Federal Register Notice was published on May 15, 2017 (Federal Register Vol. 82, No. 92, 2017-09727) describing this test. This Federal Register Notice is included in the materials with this Nonsubstantive Change Request. There were no public comments on this Notice.

Attachments:

Attachment A- Current Production Mail Materials

Attachment B- Adaptive Strategy Mail Materials

Attachment C- ACS Research and Evaluation Analysis Plan

Attachment D- 30 Day Federal Register Notice

References:

Baumgardner, S., Griffin, D., & Raglin, D. 2014. “The Effects of Adding an Internet Response Option to the American Community Survey”, 2014 American Community Survey Research and Evaluation Report Memorandum Series, ACS14-RER-21. Retrieved March 6, 2017 from https://www.census.gov/library/working-papers/2014/acs/2014_Baumgardner_04.html

Bentley, M., Mathews, K. 2016. “2015 National Content Test Study Plan for Optimizing Self-Response”, DSSD 2020 Decennial Census R&T Memorandum Series #E-07, U.S. Census Bureau, May 19, 2016.

Contact Information:

Please contact the following individuals for further information about this test:

Data Collection:

Agnes Kee

American Community Survey Office

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-1516

[email protected]

Statistical Aspects:

Anthony Tersine

Decennial Statistical Studies Division

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-1994

[email protected]

3