Interview Protocol

Attachment D:

OY4 ACS Respondent Burden Testing

Cognitive Interview Protocol

Introduction

Hello, my name is ______________ and I work for Westat, a survey research company based in Rockville, Maryland. Thank you for taking the time to participate in this research study.

Westat is working with the Census Bureau on a study to find out what people think about the mail that they receive.

There are no right or wrong answers to the questions I’ll ask today. Our purpose is not to compile information about you. Instead, your interview along with those of others will allow us to give feedback to the Census Bureau about how people handle the mail in different types of households.

The information collected in this interview is covered under OMB # 0607-0725.

Informed Consent

Before we get started, there are a few things I should mention. This is a research project, and your participation is voluntary. If you prefer not to answer any questions just say so, and we can go on to the next one. It’s also okay if you change your mind after starting and would rather not participate.

All your answers, everything you say, will be kept confidential. We will not use your name in any reports. We may use quotes in our report that you provide during the interview, but never in a way that would identify you. The interview will take about one hour and you will receive $40. We also would like to audio record our conversation. This helps me so I can listen to what you are saying and won’t have to take a lot of detailed notes while you are talking; it will also help when we write up a summary of this interview. Only project staff will have access to the recording and other project materials, and those materials will be stored according to federal requirements for protecting the identity of individual respondents.

HAND CONSENT FORMS TO PARTICIPANT, This form contains all of the things I just told you about your rights in this interview. Please read it over and sign both copies if you are willing to take part in the study.

HAVE R SIGN TWO CONSENT FORMS, KEEP ONE AND GIVE ONE TO PARTICIPANT.

TURN ON RECORDER. The date and time is ____________. Now that the recorder is running, let me ask again, is it okay with you if we record this interview?

Protocol

1. MAIL HANDLING

Research Questions:

How and how often is mail handled in participants’ households?

1. First I’d like to ask some questions about mail in your household. Can you paint a picture for me of how your household deals with mail? I’m interested in as much detail as you can give me.

IF NEEDED, FOLLOW UP PROBES:

1a. Who in your household retrieves the mail?

1b. How often is mail retrieved? (e.g. daily, every few days, once a week, etc.)

1c. What do you/does this person do with the mail?

1c1. (IF SORTS MAIL) How do you/does this person sort the mail? (e.g. bills, personal correspondence, junk mail, other)? What happens once the mail is sorted?

1d. How do you/does this person decide what to open?

1e. What do you/does this person do with mail you don’t open?

1f. What kinds of things do you/does this person pay attention to on the envelopes? What makes you/this person pay attention to these things?

ALL CONTINUE TO SECTION II (RECALL)

2. RECALL

Research Questions:

What do participants remember about the ACS and the 2020 Census mailings and questionnaires?

What were participants’ general reactions to the mailings?

How did participants feel when they received the ACS and the 2020 Census mailings?

What thought processes did participants go through when they received the ACS and 2020 Census mailings?

What were participants’ perceptions of the 2020 ACS FAQs? (Did they notice them? Did the FAQs answer any questions participants had? Did the FAQs help differentiate the ACS and 2020 Census?)

What action, if any, did participants take to get additional information about the ACS or 2020 Census?

2. Now I’d like to ask, do you recall receiving any mail from the Census Bureau in the past few months?

___ YES (CONTINUE)

___ NO (GO TO SECTION III – RECOGNITION)

2a. IF YES (RECALL), Please tell me in as much detail as possible what you recall about the mail you received from the Census Bureau.

IF PARTICIPANT HAS MAILING MATERIALS ON HAND, ASK THAT THEY BE SET ASIDE FOR LATER DISCUSSION AND, IF POSSIBLE, MOVE THEM TO INTERVIEWER SIDE OF THE TABLE.

FOLLOW UP PROBES, IF NEEDED:

2b. Do you remember how many mailings you received? IF YES, How many?

2c. Do you remember roughly when you got these mailings?

2d. What did it/the mailings look like?

2e. (IF RECALLS MORE THAN ONE) How were the mailings different from each other, if at all?

2f. What did you think was the purpose of the mailing/these mailings?

2g. What, if anything, stood out to you about the mailing(s)?

2h. What did you think it was/they were trying to tell you?

2i. Who did you think it was/they were for?

2j. What other reactions or feelings, if any, did you have when you received the mailing/these mailings?

2k. What do you recall doing with these mailings when you received them?

2l. Did you notice any frequently asked questions in the mailing(s)? IF YES, What do you remember about them? IF NEEDED, Did you use them to answer any questions you had? IF YES, What were your questions? Did the information in the frequently asked questions section answer your specific questions?

2m. What actions, if any, did you take to find out more about what you saw in the mailing/mailings? IF NEEDED, Where did you go for information? Did you get the help you needed?

ALL SKIP TO SECTION IV (DIFFERENTIATION)

3. RECOGNITION

Research Questions:

What do participants remember about the ACS’s and the 2020 Census’ mailings and questionnaire?

What were participants’ general reactions to the mailings?

How did participants feel when they received the ACS and the 2020 Census mailings?

What thought processes did participants go through when they received the ACS and 2020 Census mailings?

What were participants’ perceptions of the 2020 ACS FAQs? (Did they notice them? Did the FAQs answer any questions participants had? Did the FAQs help differentiate the ACS and 2020 Census?)

What action, if any, did participants take to get additional information about the ACS or 2020 Census?

INTERVIEWER INSTRUCTION: LAY MATERIALS FROM FIRST 3 MAILINGS (ENVELOPES AND CONTENTS) IN FRONT OF PARTICIPANT FOR VISUAL RECOGNITION, NOT FOR IN-DEPTH READING/REVIEW.

3. Now I’d like to show you some mailings that you may or may not have received recently. Without spending a lot of time reading these, do any of them look familiar to you?

___ YES (CONTINUE)

___ NO (GO TO SECTION VI – EXPOSURE)

FOLLOW UP PROBES:

3a. Without looking too closely at what’s in front of you, can you point to the ones you remember receiving?

MARK EACH MAILING THE PARTICIPANT POINTS TO:

___ FIRST MAILING

___ SECOND MAILING

___ THIRD MAILING

3b. Now that you’re looking at them, what do you remember about them?

FOLLOW UP PROBES, IF NEEDED:

3c. What did you think was the purpose of the mailing/these mailings?

3d. What did you think it was/they were trying to tell you?

3e. Who did you think it was/they were for?

3f. What other reactions or feelings, if any, did you have when you received the mailing/these mailings?

3g. What do you recall doing with these mailings when you received them?

3h. Did you notice any frequently asked questions in the mailing(s)? IF YES, What do you remember about them? IF NEEDED, Did you use them to answer any questions you had? IF YES, What were your questions? Did the information in the frequently asked questions section answer your specific questions?

3i. What actions, if any, did you take to find out more about what you saw in the mailing/mailings? IF NEEDED, Where did you go for information? Did you get the help you needed?

3j. Do you remember recently receiving any other mailings from the Census Bureau in addition to these?

ALL CONTINUE TO SECTION IV (DIFFERENTIATION)

4. DIFFERENTIATION

Research Questions:

Did participants notice that they were asked to complete two different questionnaires?

How did participants know that they were being asked to complete two different questionnaires?

What did participants think were the differences between the ACS and the 2020 Census questionnaires?

What aspects of the mailings, if any, helped participants understand that the ACS is different from the 2020 Census?

Did the FAQs help differentiate the ACS and 2020 Census questionnaires?

4. IF NOT ALREADY MENTIONED BY PARTICIPANT, When you received mail from the Census Bureau, did you receive mail for one survey or for two different surveys?

___ ONE SURVEY (GO TO SECTION V - RESPONSE DECISION)

___ TWO DIFFERENT SURVEYS (CONTINUE)

FOLLOW UP PROBES:

4a. How did you know that you were being asked to complete two different surveys?

4b. In your own words, what are the two different surveys? IF NEEDED, What makes them different from each other?

4c. Why do you think the Census Bureau is doing both surveys?

4d. Do you recall anything in the mailings that helped explain that these were two different surveys? IF YES, What do you recall?

4e. IF NOT ALREADY MENTIONED AND PARTICIPANT NOTICED FAQs IN 2L or 3H, Do you remember if anything in the frequently asked questions helped you understand that these were two different surveys? IF YES, Please say more.

ALL CONTINUE TO SECTION V (RESPONSE DECISION)

5. RESPONSE DECISION

Research Questions:

How did participants decide to answer either or both questionnaires?

Were participants aware that their response was mandatory (to either or both)?

What aspects of the mailings influenced participants’ decision to participate (to either or both)?

How did participants feel when they received the ACS and the 2020 Census questionnaires?

What thought processes did participants go through when they received the ACS and 2020 Census questionnaires?

What action, if any, did participants take to get additional information about the ACS or 2020 Census questionnaires?

5a. DID PARTICIPANT ALREADY INDICATE IF THEY RESPONDED TO THE ACS?

___ YES, ALREADY MENTIONED THIS (GO TO 5C)

___ NO, DID NOT MENTION THIS YET (GO TO 5B)

5b. Do you recall whether you responded to any Census Bureau surveys in the past few months?

___ RECALLS BEHAVIOR (CONTINUE TO 5C)

___ DOES NOT RECALL BEHAVIOR (GO TO SECTION VI, EXPOSURE)

5c. We asked you to come to today’s interview because you were recently asked to participate in two Census Bureau surveys – the 2020 Census and the American Community Survey. Help me understand your thought process of how you decided whether or not to respond to these survey requests. I’m interested in everything you can recall about your decision-making process.

FOLLOW UP PROBES, IF NEEDED:

5d. What did you ultimately decide to do about the survey requests?

5e. What factors went into your decision?

5f. Was anyone else in your household involved in making this decision? IF YES, How so?

5g. Was there anything in the mailings you received from the Census Bureau that helped you decide whether or not to respond to one survey or the other? IF YES, What aspects of the mailings helped you make the decision?

5h. Did you think you were required or not required to respond to either or both surveys? IF REQUIRED, How did that impact your decision about whether to respond?

5i. Did you turn to any sources of information to help you make the decision of whether or not to participate in either of the two surveys? (IF NEEDED: for example, a website, a family member or friend, social media, etc.) IF YES, Where did you turn for information? How, if at all, did that information influence your decision about responding to the surveys?

5j. What other reactions or feelings, if any, did you have when deciding whether to participate in the surveys?

ALL CONTINUE TO SECTION VI (EXPOSURE)

VI. EXPOSURE

RESEARCH QUESTIONS:

What do participants remember about the ACS’s and the 2020 Census’ mailings and questionnaire?

What were participants’ general reactions to the mailings?

How did participants feel when they received the ACS and the 2020 Census?

What thought processes did participants go through when they received the ACS and 2020 Census?

6a. Now let’s take a moment to a look at some of the language in the materials that were sent by the Census Bureau for the American Community Survey, or ACS. This first mailing I’m going to hand you is the initial invitation you should have received to participate in the ACS. Please read this as you would at home if you received it in the mail, opened it, and decided to read it.

INTERVIEWER INSTRUCTION: HAND FIRST ACS MAILING TO PARTICIPANT AND PROVIDE 2-3 MINUTES FOR REVIEW.

6a1. POINT RESPONDENT TO 2^nd PARAGRAPH AND ASK THEM TO READ IT. Do you recall if you noticed this information when you got the letter? What made this information stand out? IF NOTICED: Do you remember if this statement impacted your decision of whether or not to respond to the ACS? If so, how did it impact your decision?

6a2. POINT RESPONDENT TO FIRST TWO POINTS OF FAQ ON BACK OF LETTER AND ASK THEM TO READ IT. Do you recall if you noticed this information when you got the letter? What made this information stand out? IF NOTICED: Do you remember if this information impacted your decision of whether or not to respond to the ACS? If so, how did it impact your decision?

6a3. Now that you’re seeing this language (again), how helpful is this information to distinguish the ACS from the Census? What makes it helpful/not helpful?

6a4. Do you have any other feedback for me on how well this letter explains that the ACS and the 2020 Census are two different surveys and that you are required to answer both of them?

6b. PRESSURE SEAL MAILER: Now let’s look at the second mailing that some households received if they did not respond to the mailing you just looked at. Again, please read this as you would at home if you received it in the mail, opened it, and decided to read it.

INTERVIEWER INSTRUCTION: HAND PRESSURE SEAL MAILING TO PARTICIPANT AND PROVIDE 2-3 MINUTES FOR REVIEW.

6b1. POINT RESPONDENT TO 2^nd PARAGRAPH AND ASK THEM TO READ IT.

6b2. IF NUMBER OF ACS MAILINGS >=2 ON PARTICIPANT DATA SHEET: Do you recall if you noticed this information when you got the letter? What made this information stand out? IF NOTICED: Do you remember if this statement impacted your decision of whether or not to respond to the ACS? If so, how did it impact your decision?

6b3. ALL: Now that you’re seeing this language (again), how helpful is this information to distinguish the ACS from the Census? What makes it helpful/not helpful?

6b4. ALL: Do you have any other feedback for me on well this letter explains that the ACS and the 2020 Census are two different surveys and that you are required to answer both of them?

6c. THIRD MAILING: Now let’s look at the third mailing that some households received if they did not respond to the mailing you just looked at. Again, please read this as you would at home if you received it in the mail, opened it, and decided to read it.

INTERVIEWER INSTRUCTION: HAND THIRD MAILING TO PARTICIPANT AND PROVIDE 2-3 MINUTES FOR REVIEW.

6c1. POINT RESPONDENT TO 1^st SENTENCE AFTER BULLET POINTS AND ASK THEM TO READ IT.

6c2. IF NUMBER OF ACS MAILINGS >=3 ON PARTICIPANT DATA SHEET: Do you recall if you noticed this information when you got the letter? What made this information stand out? IF NOTICED: Do you remember if this statement impacted your decision of whether or not to respond to the ACS? If so, how did it impact your decision?

6c3. ALL: Now that you’re seeing this language (again), how helpful is this information to distinguish the ACS from the Census? What makes it helpful/not helpful?

6c4. ALL: Do you have any other feedback for me on how the Census Bureau well this letter explains that the ACS and the 2020 Census are two different surveys and that you are required to answer both of them?

7. GENERAL ENGAGEMENT

Research Questions:

What are participants’ general engagement with civic activities or community involvement (e.g. voting, Parent Teacher Association, Homeowner Association, volunteer work, other political activities)?

7. Finally, we are interested in understanding how people are involved in the government and in their communities.

7a. Are you involved in any community organizations or civic activities in this area? IF YES: What types of organizations or activities?

7b. How often do you vote in elections? What types of elections do you usually vote in, if any?

7c. Aside from voting, how active would you say you are politically?

7d. In general, how do you feel about responding to different types of survey requests, like phone surveys, customer satisfaction surveys, or other types of survey requests you receive?

7e. How do you generally feel about survey requests that come from a federal government agency?

7f. How much trust do you have in federal statistics in the United States, like the population count, the unemployment rate, and the crime rate? IF NEEDED: Would you say that you tend to trust federal statistics or tend not to trust them?

7g. What about other kinds of statistics? How much trust do you have in statistics put out by other organizations, like universities or the media?

7h. How about statistics you see posted on social media, like Facebook or Twitter?

7i. How much do you trust federal statistical agencies like the Census Bureau to keep information about you confidential?

Closing and Incentive

MARK HERE IF PARTICIPANT BROUGHT OUT THEIR OWN COPIES OF ACS OR CENSUS MAILING MATERIALS DURING THE INTERVIEW.

___ ACS MATERIALS (LIST THEM)

___ CENSUS MATERIALS (LIST THEM)

MARK HERE IF PARTICIPANT SEEMED TO HAVE PREPARED FOR THE INTERVIEW BY ALREADY HAVING THEIR ACS MAILINGS HANDY AND PROVIDE SUPPORT FOR YOUR IMPRESSION IN THE INTERVIEW SUMMARY.

___ YES

___ NO

IF OBSERVERS ARE PRESENT, CHECK TO SEE IF THEY HAVE FURTHER QUESTIONS.

Those are all the questions I have for you. Is there anything we haven't discussed that you would like to mention?

DISCUSS ANY PARTICIPANT COMMENTS.

Thank you for your time.

STOP TAPE RECORDER.
GIVE INCENTIVE AND HAVE PARTICIPANT SIGN RECEIPT.

OY4 ACS Respondent Burden Testing – Cognitive Interview Protocol 15