OMB Memo

2020 Census Quantitative Copy Testing

Request: The Census Bureau plans to conduct additional research under the Generic Clearance for Internet Nonprobability Panel Pretesting (OMB number 0607-0978). We propose to conduct an experiment of different ad materials using a non-probability web panel. After showing the respondents the ad stimulus, we will present them with a questionnaire to measure their recall, awareness, attitudes, intended behaviors, among other measures. As part of this current submission, we are seeking approval to conduct of the data collection.

Purpose: This experiment will test whether 2020 Census television and radio advertisements perform better than control ads that do not mention the 2020 Census on factors including message recall, census awareness, ad likability and enjoyability, behavioral intention to respond to the census and recommend it to others, attitudes towards the census, and the Census Bureau’s corporate image. The implementation of this experiment will (1) identify whether 2020 Census diverse mass television and radio ads were effective and (2) help the Census Bureau identify whether diverse mass advertisements can be tested using quantitative methods in 2030.

Population of Interest: The planned data collection will focus on the general population.

Timeline: Testing will occur in January 2020.

Language: Testing will be conducted in English only.

Method: The experiment will test five 2020 Census television advertisements against a control, non-census advertisement and three 2020 Census radio advertisements against a control, non-census advertisement. All thirty-second diverse mass television advertisements occurring during the awareness or motivation phases will be tested. Three (of six) thirty-second diverse mass radio advertisements will be tested.

We will compare outcome measurements related to recall, awareness, likability, intention to respond to the census, intention to recommend the census, Census Bureau corporate image, as well as positive attitudes towards the census and efficacy to respond to the census between treatment and control respondents. We hypothesize that treatment groups (i.e., respondents who see a 2020 Census ad) will respond more positively to these measures than the control group (i.e. respondents who do not see a 2020 Census ad). We will compare ads within the same medium (i.e., television or radio) only.

To test television advertisements, we will embed two advertisements in the middle of a five-minute clip from a soap opera. One advertisement will be a thirty-second treatment or control advertisement, and the other will be a thirty-second distractor advertisement. The distractor advertisement will be the same for all advertisements in both treatment and control groups. The control television advertisement will be a fake Prudential insurance advertisement. We will assign randomly distractor and target (i.e., treatment or control) advertisements to be the first or second advertisement shown. In total, there will be five treatment television ads and one control ad tested.

To test radio advertisements, we will embed two advertisements in the middle of a shorted soap opera audio clip. Similar to television, one advertisement will be a thirty-second treatment or control advertisement, and the other will be a thirty-second distractor advertisement. The distractor advertisement will be the same for all advertisements in both treatment and control groups. The control and distractor radio advertisements are yet-to-be-determined. We will assign randomly distractor and target (i.e., treatment or control) advertisements to be the first or second advertisement shown. In total, there will be three treatment radio ads and one control ad tested.

Sample: A sample will be purchased from Dynata, an online non-probability panel provider via Qualtrics, a company with which the U.S. Census Bureau already has a relationship.

Dynata will select members of their online panel via stratified random sampling with unequal probabilities of selection so that the respondents who complete the questionnaire will resemble the nation’s population as closely as possible in terms of age (i.e., 18-24, 25-34, 35-44, 45-54, 55-64, 65 and older), race and ethnicity (i.e., Hispanic, white, black, Asian, and other races), gender (i.e., male, female), education (i.e., less than high school, high school graduate, some college, college graduate or more), region of residence (i.e. Northeast, Midwest, South, West), and income during the last full calendar year beginning January 1 and ending December 31 (i.e., less than $25,000, $25,000 to $49,999, $50,000 to $74,999, $75,000 to $99,999, $100,000 or more).

Participants who have anyone in their household who have worked in advertising or public relations; marketing or market research; federal government; or journalism, media, or the press will not be able to participate in the study.

Participants will be randomly assigned across treatment and control groups so that each group has the same proportion of people in two education classes (i.e., college degree v. no college degree) and four age groups (i.e., 18-34, 35-44, 45-64, 65 and over).

We need a sample size of approximately 775 participants per condition to have 80% power to detect a difference between a treatment condition and the control. (See Figure 1.) With 1,000 respondents per group, we will be able to detect a difference of at least 0.035 between a treatment and control group with 80% power.

Use of Incentive: Panels incentivize their panel members to complete studies via a points-based incentive system. Panel members redeem points earned over time for a variety of prizes or cash incentives. The average range for the points-based incentive value is $0.75-$5. All incentives are distributed upon completion via virtual currency in the form of a correlated point value.

Below is a list of materials to be used in the current study:

1. Attachment 1: 2020 Census Quantitative Copy Testing Instrument

Length of Interview: The questionnaire will take 15 minutes.

For each treatment, we will need 1,000 respondents per group. The burden hours will be 2,500 hours (1000 respondents x 10 groups x 15 minutes per questionnaire).

The contact persons for questions regarding data collection and the design of this research are listed below:

Gina K. Walejko, Ph.D.

Decennial Statistical Studies Division

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-1643

[email protected]

Page 2 of 2