OMB Memo

Measuring Respondent Privacy Concerns Cognitive Interviews

Submitted Under Generic Clearance for Questionnaire Pretesting

Request: The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB #0607-0725). We will be conducting a second round of cognitive interviews to evaluate a revised self-administered questionnaire intended to measure respondents’ privacy concerns for the 2020 Census. This first round package was accepted on 8/17/17.

Purpose: The purpose of this cognitive research is to (1) evaluate the effectiveness of a revised draft questionnaire for measuring respondent privacy risk tolerance in the context of the 2020 Census and (2) identify potential improvements to the draft questionnaire items.

The full questionnaire is attached (see Attachment I: Respondent Privacy and Confidentiality Concerns Questionnaire). As in the first round, we intend to administer an abbreviated decennial census questionnaire to respondents, followed by a series of privacy risk perception questions developed by our research team. The privacy risk questions focus on the specific types of data collected in the decennial questionnaire.

In the first round of testing, we found that respondents often forgot the context of the decennial census questions by the time they answered the privacy risk questions. As a result, they often answered the latter questions while thinking about other (and more sensitive) types of information than that collected by the decennial census. Respondents also generally had no knowledge of the specific type of disclosure that is the focus of some of the privacy risk questions (re-identification), and were typically unable to report accurately a level of perceived likelihood or personal impact of such a disclosure. In addition, our initial questionnaire also contained questions about other types of disclosure risks which were much more salient to respondents (data breach and hacking), so their responses to questions about re-identification were conflated with the other, more familiar concepts. To address these issues, our revised questionnaire will rearrange the item order to first ask about privacy concerns for the specific information collected in the decennial questionnaire before asking about types of disclosure.

Once respondents complete the entire set of questions (decennial questionnaire and experimental privacy items), we will probe them about their responses to and interpretation of the privacy risk perception questions. By combining the responses across interviews and making iterative changes to the questionnaire as necessitated by respondent feedback, we can evaluate the strengths of the draft questionnaire and identify any potential problems in terms of respondent comprehension or interpretation. The results of the cognitive testing will be used to design a field test to quantitatively measure individuals’ privacy risk tolerance with the goal of informing the privacy loss budgets allowed in mathematical privacy models for decennial data releases.

Population of Interest: Residents of the United States and its territories.

Timeline: From July to September 2018, staff from the Center for Survey Measurement will conduct an iterative series of cognitive interviews with up to 20 respondents. As needed throughout the interview period, CSM staff will update the draft questionnaire and interview materials to address feedback from respondents.

Sample: We will conduct the interviews with a total of up to 20 respondents using printed copies of the draft questionnaire (to be updated iteratively based on respondent feedback during the course of the interview period).

Our recruiting efforts will aim for diversity on characteristics believed to be related to privacy attitudes and concerns:

 

• Geography

• Age

• Education and use of federal statistics

• Device ownership and level of internet use

Recruitment: Respondents will be recruited through fliers posted at local community organizations; advertisements on Craigslist.com; and broadcast messages distributed through the Census Bureau’s daily online newsletter. All recruiting materials are attached (see Attachment II: Recruitment Advertisements).

Location of Interviews: Interviews will be conducted at the Census Bureau’s Response Research Laboratory and at locations convenient to interviewees. The interviews will be conducted in-person in the local metropolitan area (DC, Maryland, Virginia).

Protocol: We will conduct our cognitive interviews using printed versions of the draft questionnaire. We intend to administer an abbreviated decennial census questionnaire to respondents, followed by a series of privacy risk perception questions developed by our research team. We will ask respondents to think aloud about their reaction to the privacy risk perception questions. Once respondents complete the entire questionnaire, we will probe them about their responses to and interpretation of the privacy risk perception questions. General probes will be asked at the end during respondent debriefing (see Attachment III: Privacy and Confidentiality Concerns Questionnaire Cognitive Interview Protocol).

Informed Consent: We will inform participants that their response is voluntary and that the information they provide is confidential and will be accessed only by employees involved in the research project. The consent form will also indicate that the respondent agrees that the interview can be audio-taped to facilitate analysis of the results. Participants who do not consent to be audio-taped will still be allowed to participate.

Incentive: Participants will receive $40 for their participation in this research. A copy of the consent form is attached (see Attachment IV: Consent Form).

Length of Interview: We estimate that each of the 20 interviews will take approximately one hour for a total of 20 hours. The pre-approved generic screening questionnaire will take approximately ten minutes per person, and the additional screening questions specific to this research will take two minutes per person (see Attachment V: Recruitment Eligibility Screeners).

We estimate that we will screen three people for each successful recruit for each of the 20 interviews. Therefore, for the first round of interviewing, we estimate a total of 60 people screened for a total of 2 hours (60 people at 2 minutes each). Thus, the total estimated burden of this research is 22 hours.

Table 1. Total Estimated Burden

Category of Respondent	No. of Respondents	Participation Time	Burden
Screening	60	2 minutes	2 hours
Cognitive Interviews	20	60 minutes	20 hours
Totals			22 hours

The following documents are included as attachments:

Attachment I - Respondent Privacy and Confidentiality Concerns Questionnaire

Attachment II - Recruitment Advertisements

Attachment III - Privacy and Confidentiality Concerns Questionnaire Cognitive Interview Protocol

Attachment IV - Consent Form

Attachment V - Recruitment Eligibility Screeners

The contact person for questions regarding data collection and the design of this research is listed below:

Casey Eggleston

Center for Survey Measurement

U.S. Census Bureau

Room 5K020H

Washington, D.C. 20233

(301) 763-6144

[email protected]

Page 3 of 3