ACS 2022 Content Test Overview

4

Department: Commerce

Agency: U.S. Census Bureau

Title: American Community Survey 2022 Content Test

OMB Control Number: 0607-0936

Expiration Date: 08/31/2024

Request: The U.S. Census Bureau is requesting approval to conduct the American Community Survey (ACS) 2022 Content Test. The purpose of the content test is to evaluate proposed changes to some of the current questions and to evaluate three new proposed questions.

The total estimated response burden for the 2022 Content Test is 99,600 hours.

Background: The Census Bureau conducts an ACS content test periodically to improve the content of the ACS data collection instruments. Similar to previous tests, the ACS 2022 Content Test will be a field test of the wording, format, and placement of select revised questions and proposed new questions. The results of this test will help determine which revised or new questions will be implemented in the ACS.

Working through the Office of Management and Budget Interagency Committee for the ACS, the Census Bureau solicited proposals from federal agencies to change existing questions or add new questions to the ACS. Existing topics that will be in the field test are:

• household roster

• educational attainment

• health insurance coverage

• disability

• labor force question series

• income and Supplemental Nutrition Assistance Program (SNAP)

For these topics, the 2022 ACS Contest Test will compare the current wording to the proposed test wording. Health insurance coverage, labor force, and income will also test a second version of proposed changes.

The following three new proposed questions will be field tested:

• solar panels

• electric vehicles

• sewage disposal

Two versions of the electric vehicles question will be tested; solar panels and sewage disposal will test one version.

Method: This test will include a control treatment and two experimental treatments. The Control Treatment will include the current ACS (production) version of the questions. New topics will also be included in this treatment. The Test Treatment will include the test question version for all topics except household roster. Because changes to within-household coverage can impact results of person-level questions, the household roster question(s) will be the current production version (for the control and test treatment) in order to avoid confounding. Changes to the roster question(s) will be tested in a separate treatment, called the Roster Treatment. Health insurance coverage, labor force, and income will include a second set of experimental questions to be tested in the Roster Treatment. Table 1 shows the experimental design for the test.

Table 1. Experimental Design

Topic	Control Treatment	Test Treatment	Roster Treatment
Household Roster	Production	Production	Test Version
Educational Attainment	Production	Test Version	Production
Health Insurance Coverage	Production	Test Version 1	Test Version 2
Disability	Production	Test Version	Production
Labor Force	Production	Test Version 1	Test Version 2
Income	Production	Test Version 1	Test Version 2
SNAP	Production	Test Version	Test Version
Solar Panels	Test Version	Test Version	Test Version
Electric Vehicles	Test Version 1	Test Version 2	Test Version 1
Sewage Disposal	Test Version	Test Version	Test Version

Data will be collected via self-response (paper and internet) and interviewer modes. Question wording can be found in Attachment A for internet wording and Attachment B for the paper questionnaires. Wording for the interviewer mode is usually either the same as paper or internet. Deviations for interviewer wording can be found in the analysis plan (Attachment C).

Other than changes to question wording, data collection methods for the test will remain largely consistent with production ACS methods. All mail materials to solicit self-response will be the same as the production ACS mail materials. Data collection protocols that are different are noted in Section 2.3 of the analysis plan (Attachment C).

Analysis Metrics: A variety of metrics will be used to assess the proposed content changes to the ACS data collection instruments. These metrics include item missing data rates, response distributions, comparisons to benchmarks and administrative data, response reliability, and other topic-specific metrics. Details on the analysis metrics for each topic are found in the analysis plan in Attachment C.

Sample: The ACS 2022 Content Test will consist of a national sample of roughly 120,000 housing unit addresses, split evenly across the three treatments. The sample will not include housing units in Alaska, Hawaii, or Puerto Rico due to cost constraints. Group quarters facilities are out of scope for the test.

The sample size provides enough statistical power (0.80) to detect a difference in the gross difference rates of at least two percentage points between the control and test groups (with α=0.1). Gross difference rates measure response reliability and are calculated by comparing responses from the interview to a followup interview. See the analysis plan for more details.

The sample size and power requirements for this test were established based on the questions for electric vehicles, because this topic has the smallest universe of respondents among the topics being tested. We expect statistical testing on the other topics with larger universes to be able to detect smaller differences in the gross difference rates.

Timeline: The test will be conducted in parallel with data collection activities for the September 2022 ACS panel, with the first mailout beginning September 1, 2022. Follow-up interviews are scheduled to be completed by the end of January 2023.

Use of Incentives: None.

Burden: This field test will require a total sample of 120,000 housing units. Responding housing units have an average burden of 40 minutes for the interview (regardless of mode). Approximately 58,800 of these housing units will have eligible responses that are selected for a follow-up interview, which has an estimated burden of 20 minutes. The response burden for the ACS 2022 Content Test is 80,000 hours for the initial interview and 19,600 hours for the follow-up interview, for a total of 99,600 hours.

Attachment A: ACS 2022 Content Test Internet Question Wording

Attachment B: ACS 2022 Content Test Paper Questionnaires

Attachment C: ACS 2022 Content Test Analysis Plan

Attachment D: ACS 2022 Content Test 30-Day Federal Register Notice

Contact Information:

Please contact Elizabeth Poehler for further information about this test:

Elizabeth Poehler

American Community Survey Office

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-9305

[email protected]