2016 Census Letter Cognitive Testing

Package Testing

Protocol for 2016 Census Test letters and postcards

Introduction:

Hello, my name is _______________. I work for the Census Bureau. Thanks for agreeing to help us today. Let me start by telling you a little about what we will be doing today. Every ten years the United States conducts a census, or count, of everyone who is living in the United States at that time. In order to improve this Census for 2020 we are conducting a Census test. Today we are going to look at some of the letters and postcards that you would get in the mail before and during the 2016 Census test. There are four things to look at. I’ll explain each one to you as we go along. I’m going to ask you to read the letters and postcards, and then ask you some questions about them: what they mean to you, and what your reactions to them are.

There are no right or wrong answers. We are interested in what people thinks about these letters and postcards.

Permission to Tape-Record:

So I don’t have to rely on my memory later on, I’d like to tape record this interview. That way, I can focus today on what you’re saying rather than having to concentrate on taking notes. Is that ok with you? [Hand respondent consent form.] Please read this over and sign it. I want you to know that your responses will be kept strictly confidential and will only be seen by Census employees. Your participation in this study is completely voluntary and you can decline to answer any particular question.

Think-out-loud Instructions:

As I said before, we are interested in your thoughts on these letters and postcards. What I would like you to do is to read each one as I give it to you. But so I know what you are thinking and how you are reacting to it, as you read it, I’d like you to tell me everything that’s going through your mind. Instead of thinking to yourself, I’d like you to think out loud. From time to time, I may ask you what you are thinking to remind you to do this.

I find it helps people to get the idea of thinking aloud while they are answering a question if they do a practice question first. Let’s begin with a practice question. Remember to think aloud as you answer.

How many windows are there in the house or apartment where you live?

_____________ WINDOWS

[IF NEEDED:] Try to visualize the place where you live, and think about how many windows there are in that place. As you count up the windows, tell me what you are seeing and thinking about.

PROBES:

• How did you come up with that answer?

• Tell me more about that. Why did you say [ANSWER]?

• I noticed that you hesitated. Tell me what you were thinking.

Great! We are ready to get started, do you have any questions before we begin?

TURN ON TAPE RECORDER

General Probes:

What are you looking at?

What are you thinking?

INTIAL LETTER:

SET-UP. The first piece of mail you would receive is this letter I am going to show you now. It doesn’t come with a form. Please look at this letter, and remember to think aloud as you go through it.

Standard Probes: (when R indicates he/she is done reading.)

1. BY OBSERVATION: Does R turn the letter over and find text on back? Read it?

2. How much of the letter did you read? Where did you stop?

3. What do you think the main point of the letter was?

4. Was there any part of the letter you spent more time on than other parts?

5. What do you think you would do after receiving this letter?

6. Do you think you have to do the census, based on reading the letter?

7. Did the letter mention any reasons for doing the census? What were they? Did you feel the reasons were valid? [If they say more than one reason then ask “which reason would you say is the most important”]

8. What was your reaction, in general to paragraph 2?

9. What did you think about paragraph 4? [Interviewer point to paragraph that says census is required by law/ for Reingold version ask about the line “If you don’t respond, we will need to send a Census Bureau interviewer to your home to collect your answers in person”]

10. Did you notice anything about whether you could respond in Spanish if you need to?

REMINDER POSTCARD/LETTER:

SET-UP: After about a week, you would receive this Postcard/ letter [Fill in as appropriate]. Please look over it and remember to think aloud as you go through it.

1. How much of the letter/postcard did you read? Where did you stop? Was there any part of the letter/postcard you spent more time on than other parts?

2. What do you think the main point of the letter/postcard was?

3. What do you think you would do after receiving this postcard?

4. Did the letter/postcard mention any reasons for doing the census? What were they?

5. Did you notice any new reasons to do the census that weren’t in the other letter? What were they?

SECOND REMINDER POSTCARD:

SET-UP: A little while later, you would receive this postcard in the mail, if you had not already responded online by a certain date. Please look at the card and remember to think aloud as you go through it.

1. How much of the postcard did you read? Where did you stop?

2. Was there any part of the postcard you spent more time on than other parts?

3. What do you think the main point of the postcard was?

4. What do you think you would do after receiving this letter?

5. What was the second paragraph trying to tell you?

6. What was the third paragraph trying to tell you?

QUESTIONNAIRE MAILING COVER LETTER:

SET-UP: Shortly after you get the postcard, if you haven’t completed your online survey, we will send you a package with a questionnaire. This is the letter that would come with your questionnaire. Please look at the letter and remember to think aloud as you go through it.

1. How much of the letter did you read? Where did you stop?

2. Was there any part of the letter you spent more time on than other parts?

3. What do you think the main point of the letter was?

4. What do you think you would do after receiving this letter with a census form?

5. Did you notice any new reasons to do the Census that you hadn’t seen yet?

6. What is the main point of the first paragraph? The second paragraph? Do you think these paragraphs would encourage you to fill out your census form?

DEBRIEFING:

I just have a few more questions for you.

1. In general, would you say you were likely or not very likely to read these materials if they came in the mail to you?

2. Let’s say that you completed your census online, and then got the postcard reminding you to respond to the census. What would your reaction be?

3. If you respond online but we don’t receive it right away for some reason, you could still get the paper questionnaire. What would your reaction be to that?

4. Do you think that the language of the letters and postcards is too difficult? Too easy? About just right? Are there any specific words and phrases you think that some people might not be familiar with?

5. Did the messages in the letters bring up any issues in your mind that some people might find sensitive or uncomfortable?

6. Was there anything else that we haven’t talked about that you would like to mention?

Thank you for participating in this study. Your feedback has been very helpful.

TURN OFF TAPE RECORDER AND GIVE INCENTIVE. HAVE R SIGN THE PAYMENT VOUCHER.

Here is your money for helping me today. I need you to sign this payment voucher so the government knows that I gave you the money.