OMB Memo

2020 Census Barriers, Attitudes and Motivators Survey Cognitive Interviews

Request: The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We propose to conduct a cognitive/usability evaluation of the 2020 Census Barriers, Attitudes and Motivators Survey (CBAMS). As part of this current submission, we are seeking approval for cognitive/usability interviews conducted by Census Bureau staff.

Purpose: The purpose of this research is to test added or revised questions and to evaluate the usability of the screen design in the 2020 Census Barriers, Attitudes and Motivators Survey in English and Spanish. In 2010, the results of CBAMS were used to identify various attitudinal mindsets. For this decade, CBAMS will help to add an understanding of households’ attitudes and motivators that can best reach them. The information will help inform an audience segmentation that clusters households by propensity to self-respond, demographics, and attitudes. The results from this testing will inform the final 2020 CBAMS instrument.

Population of Interest: The planned cognitive/usability pretesting evaluation will focus on assessing and improving the questionnaire content and user experience for the general population.

Timeline: There will be one round of cognitive/usability testing that will begin in June 2017 and end in August 2017.

Language: Testing will be conducted in English and Spanish.

Method: From June 2017 to July 2017, staff from the Center for Survey Measurement will conduct 40 cognitive/usability interviews. We will conduct 10 interviews in English and 10 in Spanish using a paper version of the self-administered instrument (See Attachment 1: 2020 CBAMS Paper Questionnaire).

An additional 10 usability interviews will be conducted in English and 10 in Spanish using a web version of the instrument. If participants are testing on mobile devices, they will use their own smartphones or tablets to complete the survey. If participants are testing on desktop or laptop devices, those will be provided by the Census Bureau.

Sample: Our recruiting efforts for English interviews will target respondents with low educational attainment and diverse demographics including race, age, and gender. In Spanish, we will target recruiting to monolingual Spanish speakers with low educational attainment. Usability participants will have at least one year of experience using the Internet for things other than email.

 

Recruitment: Participants will be recruited using a combination of word-of-mouth, fliers posted at local community organizations such as recreation centers, advertisements on Craigslist.com, and broadcast messages distributed through the Census Bureau’s daily online newsletter. Interviews will be conducted at the Census Bureau’s Response Research Laboratory and at locations convenient to interviewees in DC, Maryland, Virginia, Pennsylvania, and West Virginia.

Informed Consent: We will inform participants that their response is voluntary and that the information they provide is confidential and will be accessed only by employees involved in the research project. The consent form will also indicate that the respondent agrees that the interview can be video and audio-taped to facilitate analysis of the results (see Attachment 4: Consent Form). Participants who do not consent to be video or audio-taped will still be allowed to participate.

Protocol: For the paper form, we will conduct interviews using the think aloud method and quasi-retrospective probes (See Attachment 2: 2020 CBAMS Cognitive Interview Protocol Paper). Using quasi-retrospective probes will allow the respondent to respond to sections of the questionnaire uninterrupted before being asked probing questions about the survey questions they just answered. During probing, respondents will be asked about how they came up with their answers for questions, their comprehension of specific words and terms in the questionnaire, and about the level of difficulty of the questions. Participants will be asked to complete the entire survey.

For the web instrument, participants will first be asked to complete a demographic questionnaire (Attachment 5: Demographic Questionnaire). Interviews will use the think aloud method and retrospective probes (See Attachment 3: 2020 CBAMS Usability Interview Protocol). Using retrospective probes in a usability interview is preferred because it allows respondents to answer the entire questionnaire uninterrupted by probing questions that might interfere with the task. To log in to the survey, respondents will be given mailing materials with instructions to log in using an ID and password provided (See Attachment 7: 2020 CBAMS Mailing Materials). After logging in, participants will be asked to complete the entire survey. After completing the web form, each participant will be asked to complete a satisfaction questionnaire (Attachment 6: Satisfaction Questionnaire), and asked probing questions about different screens (For screenshots of the web instrument and retrospective probes see Attachment 8- 2020 CBAMS Usability Debriefing Slides). During probing, respondents will be asked about how they came up with their answers for questions, their comprehension of specific words and terms in the questionnaire, and about the level of difficulty of the questions.

Use of Incentive: Due to the length of the interview and the necessity to travel to test locations, we plan to offer an incentive of $40 to offset the costs of participation, such as travel and parking.

Below is a list of materials to be used in the current study:

1. Attachment 1: 2020 CBAMS Paper Questionnaire

2. Attachment 2: 2020 CBAMS Cognitive Interview Protocol Paper

3. Attachment 3: 2020 CBAMS Usability Interview Protocol Web

4. Attachment 4: Consent Form- CBAMS 2020

5. Attachment 5: Demographic Questionnaire

6. Attachment 6: Satisfaction Questionnaire

7. Attachment 7: 2020 CBAMS Mailing Materials

8. Attachment 8: 2020 CBAMS Usability Debriefing Slides

Length of Interview: We estimate 90 minutes per respondent, overall. (Respondents will be screened using the Census Bureau’s generic screener; thus the burden hours for screenings are covered under a separate request.) The total estimated respondent burden for this request is 60 hours (90 minutes x 40 interviews).

The contact persons for questions regarding data collection and the design of this research are listed below:

Yazmin Argen Garcia Trejo

Center for Survey Measurement

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-3355

[email protected]

Aleia Fobia

Center for Survey Measurement

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-4075

[email protected]

Page 3 of 3