Various Pretesting Activities (see attached list)

STANDARD PROBE:

1. What are you thinking?

PART 1: PRESENTATION OF MAILINGS AND OBSERVATION

I. Census Pre-notice

SET-UP: Around March 7^th, [show date on calendar] you would receive this letter in the mail. Please treat it as you would if you were getting it at your house, and remember to think out loud as you do. [Hand respondent mailing.]

OBSERVE:

1. To what extent does R look at envelope?

   • didn’t look

   • glance

   • scan

   • read

   • study

2. Does R seem to notice any physical features of the envelope?

• logo

• other

3. Does R say anything about the envelope? If so, what?

4. To what extent does R seem to look at/read the letter text?

How much of it?

1. didn’t look

2. glance

3. scan

4. read

5. study

5. Does R say anything about the text? What?

6. Does R ask any questions? What?

7. List any facial expressions or body language related to this task.

Anything else?

STANDARD PROBE, if needed

II. Census Initial Questionnaire Package

SET-UP: Around March 16^th, [show date on calendar] you would receive this in the mail. Please treat it as you would if you were getting it at your house, and remember to think out loud as you do. [Hand respondent mailing]

INTERVIEWER NOTE: If R asks about whether or not to complete form, ask them just to take a look through it.

OBSERVE:

1. To what extent does R look at envelope?

• didn’t look

• glance

• scan

• read

• study

2. Does R seem to notice any physical features of the envelope?

• logo

• mandatory

• thickness

• other

3. Does R say anything about the envelope? If so, what?

4. To what extent does R look at/read the letter text?

How much of it?

a. didn’t look

b. glance

c. scan

d. read

e. study

5. Does R seem to notice any physical features of the letter?

• color

• bold print

• logo

• other

6. Does R say anything about the text? If so, what?

7. Does R ask any questions? What?

8. Does R compare this letter to the pre-notice letter?

9. Does R ask if he/she should fill out questionnaire?

10. Does R look through the questionnaire?

Ignore it?

11. If R completes looks through, does R show any reaction (annoyance,

impatience, acceptance…)? How so?

12. List any facial expressions or body language related to this task

Anything else?

STANDARD PROBE, if needed

III. ACS Pre-notice envelope and letter

SET-UP: Around March 18^th, [show date on calendar] you would receive this letter in the mail. Please treat it as you would if you were getting it at your house, and remember to think out loud as you do. [Hand respondent mailing.]

OBSERVE:

1. To what extent does R look at envelope?

• didn’t look

• glance

• scan

• read

• study

2. Does R seem to notice any physical features of the envelope?

• color

• ACS name

• census logo at bottom

• other

3. Does R say anything about the envelope? If so, what?

4. Does R seem to compare ACS envelope with Census envelope?

5. To what extent does R look at/read text of letter?

How much of it?

a. didn’t look

b. glance

c. scan

d. read

e. study

6. In particular, does R seem to look at/read second paragraph?

7. Does R seem to notice any physical features of the letter?

• bold print

• other

8. Does R say anything about the text, including:

• “ACS is separate from Census”

• “required by U.S. law”

• “some households, including yours, will receive both”

• “must do both”

9. Does R ask any questions? What?

10. Does R compare this ACS pre-notice letter with any census letter?

11. Does R place the ACS pre-notice letter back in its envelope?

12. List any facial expressions or body language related to this task.

Anything else?

STANDARD PROBE, if needed

IV. ACS Initial Questionnaire Packet

SET-UP: On March 22^nd, [show date on calendar] you would receive this packet in the mail. Please treat it as you would if you were getting it at your house, and remember to think out loud as you do. [Hand respondent mailing.]

INTERVIEWER NOTE: If R asks about whether or not to complete form, ask them just to take a look through it.

OBSERVE:

1. To what extent does R look at envelope?

• didn’t look

• glance

• scan

• read

• study

2. Does R seem to notice any physical features of the envelope?

• Color

• ACS name

• mandatory

• thickness

• other

3. Does R say anything about the envelope? If so, what?

4. Does R seem to compare ACS envelope with Census envelope?

5. How does R take materials out of the envelope? Does R look at the letter before going to the questionnaire?

6. To what extent does R look at/read the cover letter?

How much of it?

a. didn’t look

b. glance

c. scan

d read

e. study

7. Does R seem to notice any physical features of the letter?

• bold print

• other

8. Does R say anything about the text? If so, what?

• “ACS is separate from Census”

• “required by U.S. law”

• “some households, including yours, will receive both”

• “must do both”

• other

9. Does R ask any questions? What?

10. Does R seem to compare this ACS cover letter or mailing package with any

other census or ACS letter or package?

11. Does R ask if he/she should fill out questionnaire?

12. Does R look through the questionnaire?

Ignore it?

Compare it to Census questionnaire?

13. If R looks through questionnaire, does R show any reaction (annoyance,

impatience, acceptance….) How so?

14. List any facial expressions or body language related to this task.

15. Does anything happen with the FAQs or ACS brochures?

Anything else?

V. Field Flyer Portion

1. Before we move on to the next part of this interview, I’d like to ask you if you have any questions or comments about the materials you just saw.

2. The last mailing you received was an American Community Survey questionnaire. If you received this at your home, how likely would you be to complete it? (Very likely, somewhat likely, somewhat unlikely, very unlikely.)

2a (If likely) Okay, I’d like you to imagine the following situation. You’ve just told me that you would be likely to do this if you received it in the mail. Imagine that you had every intention of completing this, but a family emergency came up and this form just slipped your mind. Now imagine that it’s six weeks later, and there’s a knock on your door. You answer the door, and a Census Bureau interviewer is there. The interviewer asks you if you’d be willing to be interviewed to complete the American Community Survey.

2b. (If not likely) Okay, I’d like you to imagine the following situation. You’ve just told me that you wouldn’t be likely to complete the American Community Survey if you received it in the mail at your home. Imagine now that you are at your home and it is six weeks later. There’s a knock on your door. You answer the door, and a Census Bureau interviewer is there. The interviewer asks you if you’d be willing to be interviewed to complete the American Community Survey.

3. (If R asked questions during interview) Now I’d like you to think about some of the questions you had when you saw the items. I seem to recall you mentioning ____. Is that right? (Skip to 5)

4. (If R did NOT ask questions during interview) Now I’d like you to think about any questions that came to your mind as you were looking at the items. Was there anything at all you wondered about or were unsure about?

5. Now I’m going to ask you to look at a flyer. As you look at it, I’d like you to think out loud and let me know what’s going through your mind.

Does R say anything? What?

6. Now I’d like to ask you a few questions about this flyer. Did this flyer answer your questions about the American Community Survey and Census fully, partially, or not at all?

If partially or not at all: What additional information would you like to see there?

7a. Now I’d like you to look at the (other) frequently asked questions on this side. Please take a look at the first one, “What is the difference between the American Community Survey and the Census?” Do you think the answer to this question is clear or not?

Yes

No

If no, what additional information should be there?

7b. Now please take a look at the next one, “Didn’t I already answer the Census?” Do you think the answer to this question is clear or not?

Yes

No

If no, what additional information should be there?

7c. Now please take a look at the third one, “Do I have to answer both the American Community Survey and the Census?” Do you think the answer to this question is clear or not?

Yes

No

If no, what additional information should be there?

7d. Now please take a look at the next one, “Why was I chosen for the American Community Survey?” Do you think the answer to this question is clear or not?

Yes

No

If no, what additional information should be there?

7e. Now please take a look at the last one, “Why can’t you use my Census answers for the American Community Survey?” Do you think the answer to this question is clear or not?

Yes

No

If no, what additional information should be there?

8. To what extent would the information on this flyer affect your decision about whether to go ahead with the ACS interview? To a large extent, some extent, wouldn’t affect decision.

9. After reading this flyer, would you be more likely or less likely to do the interview, or would it not make any difference?

10. Did you take any time to look at the pictures on the bottom of the page?

11. Does anything come to your mind as you look at each of these

pictures?

12. Do you think these pictures have anything to do with the American

Community Survey, or are they just there to make this flyer more

appealing?

If linked to ACS: What do you think they have to do with the ACS?

If just appealing: What pictures do you think we could use to help

people think of the ACS?

13. Do you like these pictures or not?

14. Do you think in people in general will like these pictures or not?

15. Do you think these pictures would make you more likely to agree to

the interview, less likely to agree, or would they have no effect on

agreeing to the interview?

More likely

Less likely

No effect

16. Now I’d like you to turn the flyer over to the other side. On the other

side of the flyer, do you think having copies of the Census form and the

ACS form is helpful, somewhat helpful, or not helpful?

If helpful: How so?

If somewhat helpful: How so? Could it be better some how?

If not helpful: Why not? What would you rather see there, if anything?

17. Now I’d like you to think about the flyer in general. Do you have any

comments on the format of this flyer? The colors? The fonts?

18. Are there any other questions you think should be answered on this

flyer?

19. If an interviewer used this flyer in an interview with you, would

you want to keep the flyer for any reason, or would you to give it back to

the interviewer? Why?

20. The interviewers will be carrying these flyers around to help answer people’s questions. If an interviewer came to your door, would you rather have the interviewer hold it in front of you and point out things for you to read, or would rather hold it and look at it yourself?

21. Do you have any suggestions for improving this flyer?

Thank you very much for answering our questions. Is there anything else you’d like to tell me or ask me?

11