Cognitive Protocol

Cognitive Testing of 2020 P&C Evaluation

RESPONDENT ID #: _________________________ DATE: ____ / ____ / _____

INTERVIEWER’S NAME: _________________________________________________

START TIME: ______: ______ AM / PM

Introduction

Hello, my name is ________________, and I work for the U.S. Census Bureau. Thank you for agreeing to help us today. Let me start by telling you a little about what we will be doing today.

The Census Bureau counts the population in the United States every ten years. The next census is in 2020. After the 2020 Census, the Census Bureau will conduct a follow-up survey to learn more about people’s experiences with the census. Before we conduct this survey, it’s important to try out questions with the help of people such as yourself. It is important that the questions make sense, are easy to answer, and that everyone understands the questions the same way.

If you agree to take part in this study, I will ask you to answer the questions as you would if you were at home. I am interested in your answers, but I am also interested in the process you go through in your mind when you answer the questions. I will ask you some questions about your answers, or about the questions themselves. There are no right or wrong answers. Our purpose is not to compile information about you. Instead, your interview, along with those of others, will show us how to improve these questions for this 2020 Census follow-up survey.

Your participation in this interview is voluntary, and you may decline to answer any question at any time.

[IF OBSERVER PRESENT]: I also want to let you know that this interview is being observed by [DESCRIBE OBSERVER’S ROLE]. They are observing the interview because they are also interested in how the questions work for people like you.

Do you have any questions about what we’ll be doing?

Informed Consent

• [HAND RESPONDENT COPY OF THE CONSENT FORM.]

Before we start, I would like you to read over the document in front of you. This document explains a little bit about this interview and provides information about your rights as a respondent, such as that your participation is optional, all information you provide is confidential, and we won’t use your name in any report. It also asks for your permission to have this session audio (if applicable: and video) recorded so that I can concentrate on what you are saying rather than having to take extensive notes. Please ask me any questions you have about this document. Once you have finished reading the document, please sign and date it.

• [RESPONDENT READS FORM THEN SIGNS AND DATES FORM]

Thank you for signing the consent form. I will also sign this form to show that I agree to everything in the form.

• [INTERVIEWER SIGNS AND DATES FORM]

• [IF RESPONDENT PROVIDES CONSENT TO HAVE THE SESSION RECORDED]: I will now turn on the recorder.

• [TURN ON RECORDER]

Census Check

CONFIRM THAT RESPONDENT HAS COMPLETED CENSUS:

• If yes: Continue to think aloud.

• If no: Skip to Appendix.

Think Aloud

You answered the 2020 Census [before coming in today/just now]. For this next part of the interview, I will be asking you to answer some more questions, but I’d also like you to think aloud as you answer the questions. Tell me everything you are thinking about as you go about answering each question. I may remind you to think aloud as we go through the survey questions.

Let’s begin with a practice question. Remember to try to think aloud as you answer.

How many windows are there in the place where you live? _____________

If needed: Try to visualize the place where you live, and think about how many windows there are in that place. As you count up the windows, tell me what you are seeing and thinking about.

• How did you come up with that answer?

• Tell me more about that. Why did you say [ANSWER]?

• I noticed that you hesitated. Tell me what you were thinking.

Encourage respondents to think aloud as needed throughout interview:

• Remember to tell me what you what are thinking about as you answer these questions.

• How did you come up with your answer?

Okay, let’s move on.

Introduction and CAPI Administration

Now I would like you to answer some questions for a follow-up to the 2020 Census. The survey is called the 2020 Census Experience Follow-up Survey.

Probe:

• What do you think about the name of the survey?

• What kinds of questions do you think you will be asked on the survey?

Thank you. I will now read the questions to you and ask you to answer them. Please answer these questions as you would if I were an interviewer who had come to your home. Remember to think aloud as you answer.

• ADMINISTER ALL QUESTIONS TO RESPONDENT FROM QUESTIONNAIRE HANDOUT AND RECORD ANSWERS ON PAPER.

• YOU WILL NEED TO FILL RESPONDENT’S NAME into the first question, AND KNOW WHETHER THERE IS MORE THAN ONE PERSON IN THE HOUSEHOLD.

• IN THE PROTOCOL, NOTE ANY PROBLEMS YOU HAVE ADMINISTERING THE QUESTIONS TO THE RESPONDENT, SUCH AS DIFFICULTY READING THE RESPONSE OPTIONS.

• PROBING IS DONE RETROSPECTIVELY.

Retrospective Probing

Thank you for answering these questions. Now I would like to go back and talk a little bit more about how you came up with your answers so I can see how we can improve these questions.

Section 1: Census Experience

1. If respondent selected “extremely sensitive” or “very sensitive” to any of Q5-18:

   1. How did you come up with your answer to this question? Repeat for each question. If more than 3, choose 3 to ask about.

2. Q6: What does the word “household” mean to you in this question?

3. Q8: What does the phrase “household members” mean to you in this question?

4. Q17:

   1. In your own words, what is this question asking?

   2. What does the word “householder” mean to you in this question?

Section 2: Privacy and Confidentiality

1. Q25: Next, I’d like to ask you to put a few questions in your own words. Starting first with this question, in your own words, what does this sentence mean?

2. Q26: How about this one? [If needed: In your own words, what does this sentence mean?]

3. Q27: How about this one? [If needed: In your own words, what does this sentence mean?]

4. Q28: How about this one? [If needed: In your own words, what does this sentence mean?]

5. Q29:

   1. Note whether respondents understood that “businesses and private industry” were part of the same response option.

   2. How did you come up with your answer to this question?

   3. If needed: What kinds of risks were you thinking about?

6. Q30:

   1. If option other than “never” selected: Tell me more about that.

      1. If needed: What question was it?

7. Q31:

   1. If “yes” selected: Tell me more about that.

   2. If no selected: Can you think of any examples of questions people might give wrong answers to because of privacy concerns?

8. Q34:

   1. What, if any, steps have you taken to protect the privacy of your personal information online?

9. Q36: How did you come up with your answer to this question?

   1. If no: Can you think of any concerns people might have about responding to the 2020 Census online?

10. Q38: I’d like to ask you to put a few more questions in your own words. In your own words, what is this question asking?

11. Q39: In your own words, what is this question asking?

12. Q40:

    1. In your own words, what is this question asking?

    2. What, if any, difference do you see between these two questions [point to Q39 and Q40]?

13. Q41:

    1. If yes: Tell me more about how you came up with your answer to this question.

14. Q43: How did you come up with your answer?

15. Q46:

    1. How did you come up with your answer?

    2. What does this statement mean to you [point to first response option]?

    3. What about this statement [point to second response option]?

16. Q47:

    1. If yes: In your own words, what does “differential privacy” mean?

    2. If no/DK/REF: If you had to guess, what do you think “differential privacy” might mean?

Section 3: Administrative Records

1. Q49:

   1. What does the word “vacant” mean to you in this question?

      1. If needed: Do you think this question is asking whether you happened to be away from your home on a particular date, or whether no one was living in your home at all on a particular date?

Section 4: General Attitudes

1. Q59: Can you think of any other examples of federal statistical agencies?

   1. If respondent only mentions Census Bureau: Can you think of any other examples?

Section 5: Demographics

No probes.

Debriefing Questions

Thank you. I have just a few more questions before we wrap up.

1. Were there any questions I asked you today that stood out to you for any reason?

   1. If needed: Which questions? Why did those questions stand out to you?

2. If needed: Of the questions I asked you today, which question(s) are the most important in capturing how you felt about your overall experience with the census? (MAKE SURE RESPONDENT IS NOT THINKING OF CENSUS ITSELF)

3. Were there any questions you think some people might find difficult to answer?

4. Was there anything else that we haven’t talked about that you would like to mention?

Great, thank you very much. Those are all of the questions that I had for you today. Thank you very much for your participation.

[TURN OFF THE RECORDER.]

Here is an envelope containing your $40, and a voucher form that verifies I gave you the money. Please complete the information in the highlighted areas and sign and date.

[HAND THE CASH INCENTIVE TO THE PARTICIPANT AND COLLECT VOUCHER FORM.]

Appendix

First, I’m going to ask you to answer questions from the 2020 Census. I will read the questions to you and ask you to answer them. Please answer these questions as you would if I were an interviewer who had come to your home.

• SEE CENSUS QUESTIONNAIRE HANDOUT

• ADMINISTER ALL 2020 CENSUS QUESTIONS TO RESPONDENT AND RECORD ANSWERS ON PAPER.

Thank you. GO BACK TO THINK ALOUD SECTION.

8