Memo

American Community Survey (ACS) Respondent Comment/Feedback

Request: The Census Bureau plans to conduct additional questionnaire pretesting research under generic clearance (OMB number 0607-0725). We ask to undergo cognitive pretesting of new respondent comment and feedback questions on the ACS, conducted by staff in the Center for Survey Measurement (CSM).

Purpose: The ACS Office is interested in understanding respondents’ perceptions of participation in the survey and, therefore, have paid attention to comments received by respondents. However, there has been concern about the number of telephone calls the Census Bureau receives regarding complaints about the ACS. The respondent advocate answers an estimated 300 telephone calls about complaints on ACS each year. In response, CSM has been asked to conduct cognitive testing on questions asking for respondents’ comments and feedback that will be placed at the end of the survey. The purpose of this testing is to develop questions where respondents believe they can adequately voice their opinion about the ACS and do not feel they need to call the Census Bureau.

Population of Interest: In Round 1, we will interview both previous household respondents of the ACS and members of the general population. For Round 2, we will only look at respondents from the general population.

Timeline: Two rounds of testing will be conducted. The first round will be conducted between March to April 2018 and the second round will be conducted from June to July 2018.

Language: Cognitive testing for this research will be in English only.

Method: CSM staff will conduct 35 cognitive interviews: 20 in Round 1 and 15 in Round 2. In Round 1, this project will be an add-on to Round 2 of the questionnaire pretesting research from the ACS Respondent Perceptions of Burden Cognitive Interviews project submitted on August 18, 2017 and cleared by OMB. Round 2 of this project will not be conducted in conjunction with other research.

Sample: For Round 1, the respondents will be those recruited for the ACS Respondent Perceptions of Burden cognitive testing project. These respondents will be from both the general population and prior respondents who had completed the ACS one to three months prior to the interview. Interviews will be conducted in the Washington DC area and possibly in West Virginia, the Richmond, VA area, and Delaware.

For Round 2, we will recruit respondents from the general population. Interviews will be conducted in the Washington DC area. We will recruit respondents based on a variety of demographic groups: race, ethnicity, age, gender, household income, and household size.

Recruitment: The prior ACS respondents will be recruited via telephone by CSM or the Census telephone contact centers. We will recruit respondents who completed the ACS via Computer Assisted Personal Interview (CAPI), Internet, and mail. If they agree to participate, they will receive an email confirming their interview appointment. The sample, telephone script, and training will be provided to the contact center (these documents were included in the ACS Respondent Perceptions of Burden Cognitive Interviews project OMB submission and are not attached with this submission).

For both rounds, the general population respondents will be recruited by CSM staff using the following methods: advertisement postings on Craigslist, broadcast messages distributed to Census headquarters staff, and fliers posted by local community organizations. See Enclosure 7 for the recruitment advertisement. Any respondent interested in participating in the testing will be administered the Census Bureau generic pretesting screener questionnaire in addition to questions about who handles the mail in their household, as shown in Enclosure 6. If a respondent agrees to participate, an email will be sent confirming their interview appointment.

Participation in this research is voluntary and respondents can choose not to participate. Interviews will be held at the Census Bureau and at off-site locations convenient to interviewees such as libraries.

Protocol: In Round 1, prior respondents and general population respondents will first be administered the protocol from the ACS Respondent Perceptions of Burden Cognitive Interview project (submitted on August 18, 2017 and cleared by OMB) as shown in Enclosure 1.

Following the testing of the burden series, 20 respondents (of the 30 respondents from the ACS Respondent Perceptions of Burden project) will be administered the Round 1 Respondent Feedback and Comment protocol found in Enclosure 2. Respondents will be asked to examine feedback items that would either be included on the ACS paper questionnaire or as part of the ACS internet instrument. Note, the feedback items displayed on the Web would be presented as a hardcopy of the screen design in the cognitive interview. Three versions of the feedback item on the paper questionnaire or two versions of the internet feedback item will be presented. Respondents will answer only one version of the feedback item. All five versions of the feedback item are displayed in Enclosure 5.

For Round 2, the respondents will be asked to complete the ACS when they come in for the cognitive interview. Following the completion of the ACS, they will then examine the feedback items for their thoughts and opinions and then be asked debriefing questions about their experience responding to the survey. See Enclosure 3 for the Round 2 Respondent Feedback and Comment protocol. All 15 respondents in this round will be recruited from the general population. Respondents will complete the ACS questionnaire via paper questionnaire or an internet instrument. See Enclosure 4 for a copy of the ACS questionnaire. Questions on the internet instrument are identical to those on the paper questionnaire.

Consent: The consent form will explain to participants that their participation is voluntary and that they can decline to answer any questions. The form states their interviews are confidential and can only be accessed by researchers that are directly involved in the study. It also mentions that the respondent agrees to have the interview be audio or video recorded. Respondents who do not sign the form to give the interviewer permission to audio or video record the interview will still be able to participate in the study.

Use of Incentive: We will offer $40 to offset the cost of participation, to accommodate for expenses like travel and parking.

List of Materials: Below is a list of what will be used in the ACS Respondent Comment/Feedback study.

1. Round 1 Protocol with ACS Burden Questions (Enclosure 1)

2. Round 1 Protocol with Respondent Comment/Feedback Questions (Enclosure 2)

3. Round 2 Protocol (Enclosure 3)

4. ACS questionnaire on paper (Enclosure 4)

5. Respondent Feedback and Comment Questions (Enclosure 5)

6. Additional Screener Questions (Enclosure 6)

7. Recruitment advertisement text (Enclosure 7)

8. Consent form (Enclosure 8)

Length of interview: Since Round 1 will be in conjunction with the ACS Respondent Perception of Burden Cognitive Interview project, estimated burden for recruiting respondents in Round 1 is not included here. For Round 1, this memo only estimates the number of minutes it will take to administer the comment and feedback questions protocol (Enclosure 2). We estimate 10 minutes per respondent for a total of 4 burden hours for Round 1.

For Round 2, respondents for this study will be recruited using the pre-approved generic screening questionnaire that will take 10 minutes per person to complete and then additional screener questions that will be an extra 2 minutes. We expect to screen three people for one respondent who participates in the study. As a result, the expectation is 45 people will be screened which will sum up to 2 hours of burden. We anticipate the administration of the ACS and the comment and feedback questions to be 60 minutes per respondent. This will total to 15 hours of burden in Round 2. Therefore, the sum of the estimated burden will be 21 hours for this project.

Table 1. Total Estimated Burden

Category of Respondent	No. of Respondents	Participation Time	Burden
Screening for Round 2	45	2 minutes	2 hours
Cognitive Interviews – Round 1 (add-on)	20	10 minutes	4 hours
Cognitive Interviews – Round 2	15	60 minutes	15 hours
Totals			21 hours

The contact person for questions regarding data and the design of this research project is listed below:

Jonathan Katz
Center for Survey Measurement
U.S. Census Bureau
Washington, D.C. 20233
(301) 763-5956
[email protected]