OMB Memo

2020 CPEX Extending the Census Environment Mail Materials Cognitive Testing

Submitted Under Generic Clearance for Questionnaire Pretesting Research

Request: The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We will conduct cognitive interviews to evaluate materials proposed for the 2020 CPEX Extending the Census Environment experiment.

Purpose: The Census Bureau is continually looking for ways to increase self-response rates to the decennial census in an effort to reduce field follow-up activities. Reduced field follow-up activities decreases data collection costs. To this end, the Census Bureau is planning to conduct an experiment (referred to as “Extending the Census Environment”) as part of the 2020 Census Program for Evaluations and Experiments (CPEX) to evaluate whether modified mail materials that are designed to integrate elements of the communications campaign for the 2020 Census increase self-response.

To investigate this research question, the Census Bureau, in collaboration with the communications contractor Team Y&R, has developed a set of experimental letters, postcards, and other mail materials (see Enclosure 1 for a detailed list of materials to be included in the testing as well as indicators of which are experimental). The experimental mail materials differ from production mail pieces in a variety of ways:

• Several mail pieces include icons/graphics that are not present in 2020 production mail materials

• Several mail pieces include slightly modified confidentiality assurances (although note that the required language still appears in the FAQs document, which does not differ from 2020 production)

• Several mail pieces include alternative statements about how the data from the 2020 Census will be used or could benefit the respondent’s community

• One (the Every Door Direct Mailer) is a new mail piece that has no corresponding mailing in 2020 production materials

• One (the wearable insert or sticker) is a promotional item that will only be received by households selected for this experiment

The purpose of the cognitive testing is to help evaluate whether the modifications to the mail materials are appealing/motivating to respondents, whether the modifications are the source of any concerns or confusion, and to identify possible refinements to the icons, text, or design of the experimental materials. We also want to ensure that the experimental mail materials continue to look like official communications from the federal government and that they clearly outline what respondents are being asked to do (respond to the 2020 Census) and how they can do so (by filling out the census questionnaire online or on paper).

Population of Interest: Everyone living in the United States is legally required to respond to the 2020 Census. Mail materials attempting to collect a self-response (via mail of paper) from households will be sent to known residential addresses that have not been identified for group quarters enumeration. To reflect this population, the Extending the Census Environment experiment will select a nationally-representative subset to receive experimental mail materials. The population of interest for this cognitive testing, therefore, is all individuals living in non-GQ housing in the United States.

Timeline: One or two rounds (as needed) of up to 25 cognitive interviews will be conducted from July 1 to August 30, 2019.

Language: Interviews will be conducted in English only.

Method: Staff from the Center for Behavioral Science Methods will conduct one or two rounds of cognitive interviews (as needed) with up to 25 respondents. Interviews will be conducted in-person at the Census Bureau headquarter and at locations convenient to interviewees in the local metropolitan area (DC, Maryland, Virginia, and West Virginia).

Sample and Recruitment: We will recruit respondents using the methods described below. Our recruiting efforts will target respondents who are most likely to handle the mail and react to it in a household; this includes identifying some respondents who receive mail at a post office box. We will recruit respondents who read at different literacy levels, using education as a proxy measure. We will also recruit respondents with diverse demographic characteristics and household composition to the maximum extent possible.

Recruitment: Respondents will be recruited through fliers posted at local community organizations, advertisements on Craigslist.com, and broadcast messages distributed through the Census Bureau’s daily online newsletter. All recruiting materials are attached (see Enclosure 2).

Protocol: We will conduct our cognitive interviews using printed drafts of the experimental mail materials and associated production materials (assembled into mail packages as respondents would receive them in the mail at their address). They will not be asked to complete any part of the questionnaire. Respondents will view a complete set of mail materials in order of production contact (randomly assigned to see either the Internet Choice or Internet First set first). Then they will quickly peruse just the experimental pieces for the remaining materials (these will not be presented in envelopes or along with other supporting materials). They will comment on the experimental versions and we will ask specific probes about the materials (see Enclosure 3).

Researchers will observe respondents’ interactions with these mailings as well as ask respondents about their reactions to them. We will ask probes as needed to determine whether respondents noticed the experimental elements and messaging, whether they liked or had any concerns about these elements, and what they would do after reading it.

Consent: We will inform participants that their response is voluntary and that the information they provide is confidential and will be accessed only by employees involved in the research project. The consent form will also indicate that the respondent agrees that the interview can be audio and/or videotaped to facilitate analysis of the results (see Enclosure 4). Participants who do not consent to be video and/or audio-taped will still be allowed to participate.

Incentive: Participants will receive $40 to offset the costs of participation in this research, such as travel and parking.

Length of Interview: We estimate that each of the up to 25 interviews will take approximately one hour. This results in a burden of 25 hours.

The pre-approved generic screening questionnaire will take approximately ten minutes per person, and there are no additional screening questions specific to this research. Therefore we do not estimate any additional burden due to screening.

Thus, the total estimated burden for this research is 25 hours.

Table 1. Total Estimated Burden

Category of Respondent	No. of Respondents	Participation Time	Burden
Screening	75	0 minutes	0 hours
Cognitive Interviews	25	60 minutes	25 hours
Totals			25 hours

Below is a list of materials to be used in the current study:

Enclosure 1. Experimental 2020 Census Mailing Materials

Enclosure 2. Recruitment materials

Enclosure 3. Cognitive interview protocol

Enclosure 4. Consent form

The contact person for questions regarding data collection and the design of this research is listed below:

Casey M Eggleston

Center for Behavioral Science Methods

U.S. Census Bureau

Room 5K020F

Washington, D.C. 20233

(301) 763-6144

[email protected]

3