OMB Memo on 2020 ACS Respondent Burden Testing

Final 4 January 28, 2020

2020 American Community Survey Respondent Burden Testing

Submitted Under the Generic Clearance for Pretesting

Request: The U.S. Census Bureau, in its continuing effort to reduce respondent burden on the American Community Survey (ACS), plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). For the 2020 American Community Survey (ACS) Respondent Burden Testing, the Census Bureau has contracted with Westat, a statistical survey research corporation headquartered in Rockville, Maryland, to perform cognitive testing.

Purpose: The research objectives of this study are to assess 1) recipients’ reactions to receiving both the ACS and the 2020 Decennial Census, and 2) their impressions of modified ACS mail materials communicating that their household needs to complete both surveys.

Households in sample for the ACS in 2020 are legally required to complete both the ACS and the 2020 Census. Receiving two sets of mailings in one calendar year may be confusing to respondents. Also, being asked to complete both the 2020 Census and the ACS may be burdensome and may decrease the rate of self-response to the ACS. During the 2010 Census, ACS response rates were higher than usual in the first few months of the year but were lower than usual in the spring and summer months (see Chesnut and Davis 2011, Baumgardner 2013).¹ The increase in response early in the year may have occurred because the communications campaign for the 2010 Census brought attention to the U.S. Census Bureau. The decrease later in the year may be because recipients thought the ACS was the census and disregarded the ACS questionnaire because they household already responded to the census.

To mitigate these issues in 2020, a set of modified mail materials have been developed with language that directly addresses the difference between the ACS and the 2020 Census. These materials will be used during the main response period for the 2020 Census, March through September. Frequently Asked Questions (FAQs) that address recipients’ questions on the difference between the ACS and the 2020 Census have been added to mail materials as well. The 2020 ACS language modifications are based on an evaluation conducted by the Census Bureau in 2010 (see Chesnut and Davis 2011) and were cognitively tested in 2018. In this study, we will have an opportunity to speak to people who received these materials in 2020.

Timeline and Location: Forty-eight cognitive interviews will take place with ACS sampled households that are part of the March, April, or May panels. These panels were chosen because ACS data collection for these panels overlaps with the majority of the 2020 Census data collection activities. Cognitive interviewing will take place between approximately April 14, 2020 and August 31, 2020.

Westat will conduct interviews in multiple locations where Westat staff are located. These locations are: Rockville, MD, Raleigh, NC, Ft. Collins, CO, Orange County, CA, Cleveland, OH, and Lancaster, PA. To manage travel costs, interviews will occur within a two-hour driving radius of Westat cognitive interviewing staff at a location convenient to the respondent, such as a local library.

Language: Interviews will be conducted in English only.

Sample and Recruitment: Recruiting will take place between April 7, 2020 and August 31, 2020. For each ACS panel, sampled households that have self-responded to the ACS as well as sampled households that have not self-responded will be recruited.² For each interview, self-respondents will be recruited as soon as possible after responding to the ACS. Non-respondents will be recruited after they have been sent all ACS and 2020 Census mailings. The Census Bureau will provide Westat with response status and available contact information for each panel from which Westat will recruit cognitive interview participants. The proposed schedule is shown in Table 1.

Table 1. Recruitment and interview schedule by panel month and response type

ACS Panel	Response type	Recruitment begins	Interviewing occurs
March	Self-Response households	April 7	April 14 - May 29
	Non-Response households	May 1	May 11 - June 30
April	Self-Response households	May 1	May 11 - June 30
	Non-Response households	June 1	June 8 - July 30
May	Self-Response households	June 1	June 8 - July 30
	Non-Response households	July 1	July 8 - August 31

Because the focus of this study is to understand reactions to the mail materials, participants must have been mailed both the ACS and the 2020 Decennial Census in order to be eligible to participate.³ All recruited respondents must be age 18 and older and must handle the mail for their household, either alone or in combination with other household members. For ACS self-respondents, we will recruit participants with diverse race/ethnicity, age, sex, level of education, and household size to the maximum extent possible. Because gaining interview participation from non-respondents will be difficult, we do not have demographic requirements for ACS non-respondents. Table 2 presents additional recruiting requirements.

Table 2. Target Participant Characteristics

Characteristics	Target number of interviews
March ACS panel	16
April ACS panel	16
May ACS panel	16
Self-respondents to the ACS	42 (at least 5 from each panel who self-responded after the 3rd mailing)
Non-respondents to the ACS	6 (at least 2 from each panel)
Less than a bachelor’s degree	10
Live in a rural area	6

See Attachment A for the full Recruitment Plan.

To recruit 48 respondents who meet the specified recruitment characteristics, Westat will contact households by phone using phone numbers provided by the Census Bureau. Attempts to screen and schedule participants are limited to those cases with a phone number on the Census Bureau file. Westat will use a screener (see Attachment B) to screen potential respondents who are interested and eligible. Eligibility is based on characteristics of interest in ACS self-respondents and non-respondents as described in the Recruitment Plan (see Attachment B) and demographic characteristics such as age, gender, and education level. Westat will then inform those eligible participants that someone will be in touch with them within three business days to schedule the interview. Interviews will be scheduled with participants as soon as possible to keep the recall period between receipt of the mailings and the cognitive interview about those mailings short.

Protocol: Westat will conduct 48 English-language cognitive interviews on participants’ general mail behaviors as well as experience and reactions to receiving ACS and 2020 Census mail materials. Westat will ask probes to participants as needed to determine 1) participants’ reactions to receiving both the ACS and the 2020 census mailings, and 2) their impressions of modified ACS messaging that explained their household needed to complete both surveys.

These interviews will rely primarily on participants’ memories of the materials their household was sent. If needed, participants will be presented with printed copies of the ACS mail materials. Westat will observe participants’ interactions with these mail materials and ask respondents about their reactions to them. Westat will conclude the interviews by asking participants about their attitudes toward the government and taking surveys in general.

Each cognitive interview for participants who are interested and eligible will last approximately 60 minutes. The protocol for the interview is in Attachment D.

Consent: Westat will inform participants that their response is voluntary and that the information they provide is confidential and will be accessed only by employees involved in the research project. The interviews will be audiotaped and may be subject to observation from designated sworn employees of Westat and the Census Bureau. Participants will be asked to sign a consent form (see Attachment C).

Incentive: Respondents who are eligible and complete an interview will receive $40 to offset the cost of participation (e.g., transportation, childcare costs).

Length of interview: We estimate that each of the 48 cognitive interviews will take approximately one hour. This results in a burden of 48 hours.

The screening questionnaire will take approximately five minutes per person. We estimate that we will screen 330 people in order to obtain 48 interviews. Therefore, for the first round of interviewing, we estimate a total of 330 people screened for a total of 28 hours (330 people at 5 minutes each).

Thus, the total estimated burden for this research is 76 hours.

Table 1. Total Estimates Burden

Category of Respondent	No. of Respondents	Participation Time	Burden
Screening	330	5 minutes	28 hours
Cognitive Interviews	48	60 minutes	48 hours
Totals			76 hours

Materials for this Project:

• Attachment A: American Community Survey (ACS) Cognitive Testing of American Community Survey Recipients during a Decennial Year - Research Goals, Recruiting Requirements, Research Questions and 2020 Modified Materials

• Attachment B: ACS Cognitive Testing Recruitment Plan and Screener

• Attachment C: ACS Respondent Burden Testing Consent Form

• Attachment D: ACS Respondent Burden Testing - Cognitive Interview Protocol

• Attachment E: ACS Respondent Burden Testing – Participant Data Sheet

The contact person for questions regarding data collection and study design is:

Elizabeth Poehler

Decennial Statistical Studies Division

Room # 4H475

Washington, DC 20233

(301) 763-9305

[email protected]

1 https://www.census.gov/library/working-papers/2011/acs/2011_Chesnut_01.html

https://www.census.gov/library/working-papers/2013/acs/2013_Baumgardner_01.html

2 The ACS sub-samples non-responding households to be included in an in-person CAPI follow-up. To avoid additional burden on potential respondents and to assist with recruiting, only households that are not included in the CAPI follow-up will be eligible for inclusion in this study.

3 This requirement means housing units that are not sent mail, such as those in remote Alaska or for which the Census Bureau only has a physical location description, are not eligible.