OMB Memo

University of Virginia-Census Persuasion Survey

Submitted Under Generic Clearance for Internet Nonprobability Panel Pretesting

Request: The Census Bureau plans to conduct additional research under the generic clearance for Internet Nonprobability Panel Pretesting (OMB #0607-0978). We plan to administer an online survey experiment aimed at understanding the effectiveness of different persuasion techniques on eliciting response to the decennial census. As part of an ongoing partnership with researchers at the University of Virginia (UVA), this instrument has been through pilot testing with 170 respondents.

Purpose: The purpose of conducting this survey is to understand the effectiveness of different persuasion techniques on eliciting response to the decennial census. The current thinking in the persuasion literature provides evidence that there are six archetypes of persuasion: reciprocity, consistency, social proof, liking, authority, and scarcity (Cialdini 2016). Examples of each of these archetypes will be used in experimental panels to test their effectiveness.

The full questionnaire is attached (See Attachment I: Persuasion Survey Experiment Questionnaire). Participants are asked to answer a series of questions meant to measure their perceptions of risk and trust in the decennial census and the Census Bureau in general. Then they are asked to read an invitation to complete the decennial census. The invitation contains the experimental messages that we are testing. We will randomly assign respondents to read one of 48 invitations. The invitations vary by the following characteristics:

• Letter or email invitation: The mock invitations we ask respondents to read will look like either a letter or an email. This manipulation is to test whether the mode of invitation matters for the effectiveness of the persuasion message. See Attachment II: Experimental Invitations.

• Mode of response request: The invitations invite participants to respond to the census by clicking a link in a future email, by following a personalized link in a letter, or by returning a paper questionnaire. This manipulation is to test whether the offered mode of response has an effect on intent to respond and the effectiveness of the persuasion messages.

• Persuasion messages: There are six archetypes of persuasion: reciprocity, consistency, social proof, liking, authority, and scarcity. We are testing two different authority messages and will also include a control that has no message in this experiment. See Attachment II: Experimental Invitations.

The fully crossed design is 2 x 3 x 8 = 48. After reading one of the messages, participants will respond to questions about the message they read and their intent to respond to the census based on that message. We also ask basic demographic questions and present two short personality scales.

Population of Interest: Residents of the United States and its territories.

Timeline: We intend to conduct this survey with a non-probability sample drawn from the Census Bureau’s Contact Frame in December 2019. The survey will be open for a period of two weeks after the first email invitation is sent.

Sample: In December 2019, Census Bureau staff will sample 72,000 MAFIDs from the Census Bureau’s Contact Frame. The Contact Frame includes email addresses and phone numbers purchased from third party vendors and matched with addresses from the Master Address File. Based on past use of this frame, we expect a 4% response rate. Using an alpha level α = .05 and assuming a small effect size our goal is 2,880 complete responses.

Recruitment: Respondents will be invited to respond to the online survey by means of a series of emails containing a link to the survey (see Attachment III: Persuasion Survey Experiment Email Invitations). For this study, each email address in the sample will receive a maximum of three notification emails:

1. An initial email on a Monday,

2. A reminder email on the following Thursday (if they have not yet clicked on the link to the survey), and

3. A final reminder email on the following Monday with the survey closing the following Friday.

Each MAF-ID from the Contact Frame has up to three email addresses associated with an address. We will email all three addresses associated with a particular MAFID.

Survey Administration: The questionnaire will be administered online only using the survey platform Qualtrics. Respondents will receive an invitation with a link to the survey which will then take them to the Qualtrics instrument.

Questionnaire: The questionnaire, programmed in the online survey platform Qualtrics, begins with a consent form that is required by UVA’s Institutional Review Board (IRB). Demographic screeners that ask about age and US residency follow the consent screens. Next, we ask a series of perceptual questions about the decennial census, opinions on privacy and confidentiality, and perceptions of the Census Bureau more generally. An instruction screen informs respondents that this invitation is not a real invitation to complete the decennial census and that they will not receive a follow-up survey as the invitation suggests. Each participant is then shown one of the 48 invitations and then asked a series of follow-up questions about the message. Finally, participants are asked to answer a series of demographics questions as well as two short personality measures: the 4-item short Affinity for Technology Interaction (ATI-S) scale (Franke, Attig, & Wessel 2019) and the extra short 15-item Big Five Inventory-2 (BFI-2-XS) scale (Soto & John 2017). We expect these personality measures to correlate with perceptions and intent to respond based on the privacy and confidentiality messages.

Informed Consent: In the survey invitation materials, we will inform participants that their response is voluntary and that the information they provide is confidential and will be accessed only by employees involved in the research project. Informed consent is required by UVA’s IRB and is included at the beginning of the instrument.

Incentive: Participants will not receive any payment for their participation in the survey.

Length of Interview: We estimate that the survey will take an average of 15 minutes for each complete response. We expect 2880 complete responses for a total of about 720 hours. Each email will take about 1 minute to read. Each person could get up to three emails and we will send emails to 72,000 email addresses for a total burden of 3,600 hours. The total estimated burden of this research is 4,320 hours.

Table 1. Total Estimated Burden

Category of Respondent	No. of Respondents	Participation Time	Burden
Reading email invitations	72,000	3 minutes	3,600 hours
Survey	2,880	15 minutes	720 hours
Totals			4,320 hours

The following documents are included as attachments:

Attachment I: Persuasion Survey Experiment Questionnaire

Attachment II: Experimental Invitations

Attachment III: Persuasion Survey Experiment Email Invitations

The contact person for questions regarding data collection and the design of this research is listed below:

Aleia Fobia

Center for Behavioral Science Methods

U.S. Census Bureau

Room 5K024E

Washington, D.C. 20233

(301) 763-4075

[email protected]

References

Cialdini, R. (2016). Pre-suasion: A revolutionary way to influence and persuade. Simon and Schuster.

Franke, T., Attig, C., & Wessel, D. (2019). A personal resource for technology interaction: development and validation of the affinity for technology interaction (ATI) scale. International Journal of Human–Computer Interaction, 35(6), 456-467.

Soto, C.J., & John, O.P. (2017). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of personality and social psychology, 113 1, 117-143 .

Page 3 of 3