Various Demographic Pretesting Activities

Be Counted – Automated Protocol

INTERVIEW ID: ____________________

INTERVIEW DATE: ____/____/____ START TIME: ___________ AM / PM

Hello, my name is ___________. I work for RTI/RSS a research company that has been hired by the Census Bureau to conduct research on the census forms. Thanks for agreeing to help us today. Let me start by telling you a little about what we will be doing. Every ten years the United States conducts a census, or count, of everyone who is living in the United States. The form we’re working on today is the kind you might access online, using a computer, tablet, or smartphone if you thought you had not been counted in the census.

Permission to Record:

Because it would be hard to keep track of everything you say today, I would like to record this session. [HAND RESPONDENT THE CONSENT FORM IF YOU HAVE NOT ALREDY.] Please read over this consent form and let me know if you have any questions. When you are ready, please sign the form. I want you to know that your responses will be kept strictly confidential and will only be seen by members of the project team and Census employees. Your participation in this study is completely voluntary and you can decline to answer any particular question. At the end of the study, we will delete any record of your responses from our machines. Your name will never be used in any reports.

Think-out-loud Instructions:

What I’ll ask you to do today is to fill the questionnaire out, just as you would if you accessed it online. But, so I can keep track of what you are doing, I’d like you to describe what you are doing and thinking as you go along. For example, I may ask you a question that seems relatively simple such as “How many cars do you have?” Instead of just providing a number, tell me what you were thinking when you answered it. For example, my response would be “Well I have an SUV that I drive every day and I also have an old truck. But the truck does not currently run. I assume that you mean working cars, so…one. I just have one car.” This kind of response will help me understand how you understood and interpreted the question so that we can see what each question means to you. I may remind you to think aloud as we go along. After some of the questions, I will ask you some additional questions about your answers to the Census. Some of these questions may sound a little strange. For example, I might ask you what a certain word means to you or how you came up with your answer. Again, the reason for this is to see how people understand each question. Even though some of these questions may sound like I am giving you a test, there are no right or wrong answers. We are interested in how these questions work for you.

Do you have any questions before we begin?

IF CONSENT WAS GIVEN, START TAPE RECORDER AND DEVICE RECORDER.

Logon TO APP (PASSWORD: CSM922).

[GIVE DEVICE TO R AND SAY: So that I can follow along more easily, please read the survey questions aloud and tell me your answers.]

[administer scripted probes after respondent has answered all questions in the section; note usability issues as they occur. Administer debriefing probes after scripted probes.]

Roster Questions (1-4)
The Census Bureau needs to count everyone where they were living or staying on Census Day, which was October 1. It should take about 10 minutes. What is your full name? ______________________________________ [First Middle Last] Were you already counted in the 2012 Census? Yes No Don’t Know	Q1 USABILITY: NOTE WHETHER RESPONDENT READS INTRODUCTION FIRST TIME THROUGH AND HOW NAME IS LISTED. What are your thoughts on this screen? What is it telling you or asking you to do? What does the phrase “living or staying” mean to you as it is used in the introduction?
1a. What was the physical address where you were living or sleeping on October 1, 2012? ADDRESS FIELDS	Q1a USABILITY: NOTE COMPLETENESS OF ADDRESS. IF PARTIAL, PROBE ABOUT THE “If you do not know…” TEXT AT BOTTOM. What does “living or sleeping” mean to you as it is used in this question? How does that compare to “living or staying?” Please tell me about the address you provided for Q1a. What kind of place is it? How long were you there?
1b. On October 1, 2012, were you staying on the street, in a shelter, or in another location for persons experiencing homelessness? Yes No	Q1b USABILITY: What do you think they mean by “experiencing homelessness” in this question? IF YES: Why did you choose that answer? VERIFY WHETHER THIS IS THE ADDRESS IN Q1A.
1c. Was there anyone else living or staying at 1234 Home Address on October 1, 2012? Yes (If not homeless, go to Q2; else go to 1d) No (If not homeless, go to Q3; else go to Q7) 1d. IF YES: Are you completing this form for… All of them – Go to Q2 Some of them – Go to Q2 Just yourself – Go to Q7	Q1c USABILITY: Why did you say [YES/NO]? IF YES: Why did you decide to complete it for all/some/just yourself? IF SOME OR JUST SELF: How are the other people living or staying at this address related to you? Are they friends, relatives, or what?
2. What are the full names of the other people who were living or sleeping at 1234 Home Address on October 1, 2012? [First Middle Last] ___________________________________________ ___________________________________________ ___________________________________________	Q2 USABILITY:
Question 2 Continued. Anyone else? Yes No [If homeless – go to Q7]	NOTE WHETHER FULL NAME IS PROVIDED AND IF IT IS ENTERED CORRECTLY (E.G. ONE NAME PER LINE). How did you determine who was living or sleeping here? [Was there anyone you were unsure about including? IF YES: Can you tell me more about that?]
3. Were there any additional people staying here on October 1, 2012 who you did not include yet? For example: Babies? Foster children? Any other relatives? Roommates or people not related to you? Yes – Go to 3F No – Go to 4	Q3 USABILITY: In your own words, what is this question asking? [NOTE WHETHER THEY UNDERSTAND THAT THIS QUESTION IS ASKING ABOUT ADDITIONAL PEOPLE.]
3F. What are the full names of the babies, foster children, other relatives or people not related to you staying with you? [First Middle Last] ___________________________________________ ___________________________________________ ___________________________________________ Anyone else? Yes No	Q3F USABILITY: Why did you not initially include [NAME] as living or staying at <ADDRESS>? FOR ANY PEOPLE ADDED, PROBE TO UNDERSTAND THEIR RELATIONSHIP TO THE PARTICIPANT AND HOW FREQUENTLY THEY LIVE/STAY THERE. NOTE WHETHER FULL NAME IS PROVIDED AND IF IT IS ENTERED CORRECTLY (E.G. ONE NAME PER LINE).
4. Was there anyone else staying at 1244 Home Address on October 1, 2012 who had no permanent place to live? Yes – Go to 4F No – Go to 4a	Q4 USABILITY: What does “no permanent place to live” mean as it is used in this question? What were you thinking of?
4F. What are full the names of the people who had no permanent place to live? [First Middle Last] ___________________________________________ ___________________________________________ ___________________________________________ Anyone else? Yes No	Q4F USABILITY: FOR ANY PEOPLE ADDED, PROBE TO UNDERSTAND THEIR RELATIONSHIP TO THE PARTICIPANT AND HOW FREQUENTLY THEY LIVE/STAY THERE. NOTE WHETHER FULL NAME IS PROVIDED AND IF IT IS ENTERED CORRECTLY (E.G. ONE NAME PER LINE).
4a. Was there anyone else living or staying at 1234 Home Address on October 1st who is no longer living there? Yes No	Q4a USABILITY: In your own words, what is this question asking?
4F. What are the full names of the people who are no longer staying at 1234 Home Address but were there on October 1, 2012? [First Middle Last] ___________________________________________ ___________________________________________ ___________________________________________	Q4F USABILITY: FOR ANY PEOPLE ADDED, PROBE FOR DETAILS. NOTE WHETHER THEY LIST ADDITIONAL PEOPLE VS SOMEONE THEY ALREADY LISTED. NOTE WHETHER FULL NAME IS PROVIDED AND IF IT IS ENTERED CORRECTLY (E.G. ONE NAME PER LINE).

Roster Questions (1-4) Debriefing

I would like to stop here for a moment to ask you a few follow-up questions about the items you just answered.

28. Is there anyone you listed or thought about listing for any question, but you were unsure whether he or she should be included on the census or not? [Tell me more about that?]

29. Was there anyone you included as living or staying here, who was actually away on October 1. For example away on a business trip, for vacation, or for another reason? [How did you know to include this person?]

Now I’d like you to do something a little different. I have some brief descriptions of different households. After I read each description, I’d like you to answer a few questions.

30. Jane’s brother visited her for 7 days from September 28 to October 5. When Jane answers the census survey, should Jane include her brother or not? [Why?] [SHOW SCREENSHOTS FOR Q2, Q2a, Q2F.]

31. Sam’s friend moved to the area on September 15. The friend is crashing on Sam’s couch until he can find his own apartment. When Sam answers the census survey, should Sam include his friend or not? [Why?] [SHOW SCREENSHOTS FOR Q2, Q2a, Q2F.]

Those are all the follow-up questions I have for this part. Let’s continue with the rest of the interview.

Demographic Questions (5-9):
Administer demographic questions. Note any usability or cognitive issues below. administer spontaneous probes as necessary if R has difficulty with any question.
Question 10-10c Series
The next few questions are about the people who live in your household. This is who we have listed at 1234 Home Address: Person 1 Person 2 Person 3 Person 4 Person 5 Person 6	Q10 USABILITY: WAS IT CLEAR TO R WHAT R WAS SUPPOSED TO DO ON THIS SCREEN? IF SOMETHING WAS INCORRECT, WHAT DID R DO? WAS R ABLE TO FIX IT SUCCESSFULLY? DESCRIBE.
10M Did you or any of these people move into 1234 Home Address after September 1, 2012? Yes – Go to 10L No – Go to 10 10L. Who moved in after September 1, 2012? Person 1 Person 2 Person 3 Person 4 Person 5 Person 6	Q10.0 USABILITY: IF YES, PROBE FOR DETAILS. WHEN, WHERE FROM, ETC. IS THE ADDRESS PROVIDED THEIR PREVIOUS ADDRESS OR NEW ADDRESS? NOTE COMPLETENESS OF ADDRESS.
10. Do you or any of these people sometimes live or stay somewhere else with a parent, grandparent, friend, or other person? Yes – Go to 10L No – Go to 10a 10L. Who stayed with a parent, grandparent, or other person? Person 1 Person 2 Person 3 Person 4 Person 5 Person 6	Q10 USABILITY: What does the phrase “sometimes live or stay somewhere else” mean in this question? What time period were you thinking about when you answered this question? FOR EACH YES: Can you tell me more about that? [Who did NAME stay with? When was NAME there last? How frequently does NAME stay there?]
10a. Do you or any of these people sometimes live or stay at another home, like a seasonal or second residence? Yes – Go to 10L No – Go to 10b 10L. Who stayed at a seasonal or second residence? Person 1 Person 2 Person 3 Person 4 Person 5 Person 6	Q10a USABILITY: What does “at another home, like a seasonal or second home” mean to you in this question? What were you thinking of? FOR EACH YES: Can you tell me more about that? [What type of places is it? Who owns it? When was NAME there last? How frequently does NAME stay there?]
10b Do you or any of these people sometimes live or stay at another address for college, for the military, or for a job? Yes – Go to 10L No – Go to 10c 10L. Who stayed at another address for college, for the military or for a job? Person 1 Person 2 Person 3 Person 4 Person 5 Person 6	Q10b USABILITY: What does it mean to live or stay at another address for college, military or a job? [What sort of situations were you thinking of?] FOR EACH YES: Can you tell me more about that? [Where was name staying? A hotel, a house, etc.? How long/often is name there?]
10c On October 1, 2012, did you/any of these people stay at a jail or prison, nursing home, group home, shelter, or other facility? Yes – Go to 10L No – Go to 10d 10L. Who stayed at a facility on October 1, 2012? Person 1 Person 2 Person 3 Person 4 Person 5 Person 6	Q10c USABILITY: IF YES: Can you tell me more about that? [How long did name stay there?]

Questions 10-10c Debriefing

43. IF OTHERS: When you answered these questions about living or staying someplace else, were you thinking about everyone in the household [SUCH AS PEOPLE ADDED AFTER THE INITIAL ROSTER] or just some people [SUCH AS PEOPLE INCLUDED IN INTIAL ROSTER]?

44. IF OTHERS IN HHD: How confident are you in the answers you provided for the other people in the household?

45. Are there any other places that you (or the other people you mentioned) stay that we did not ask about or that you were not sure counted?

46. IF ANY RESPONSES TO THE CENSUS QUESTIONS ARE INCONSISTENT WITH COMMENTS PROVIDED, PROBE FOR MORE INFORMATION. FOR EXAMPLE IF THE R SAYS THAT HE/SHE SOMETIMES STAYED SOMEWHERE, BUT THEN DID NOT MARK ANY YES TO ANY OF THE Q10 SERIES QUESTIONS.

Enumeration Questions (Person ___)
10F You reported that PERSON X stayed someplace else [Fill]. What is the address of the other place PERSON X stayed? Enter a new address ADDRESS FIELDS Any other address? Yes No	Q10F USABILITY: NOTE HOW MANY NEW ADDRESSES WERE PROVIDED AND COMPLETENESS OF THE ADDRESSES BY PLACE. [IF SEASONAL:] Is [ADDRESS] anyone’s usual residence? [If YES: Whose usual residence?] [IF SEASONAL:] IF NEEDED: What does “usual residence” mean to you in this question? [IF SEASONAL:] How did you determine who usually lives at that residence?
11. Where does PERSON X live or stay most of the time? Here, at this house or residence Address 2 Address 3 Address 4 In above places equally Some other place	Q11 USABILITY (FOR ALL PERSONS): What time period were you thinking of on Q11? What does the phrase “most of the time” mean to you as it’s used in this question? IF ADD’L ADDRESSES PROVIDED: How did you determine where you live or stay most of the time? If I had only asked “where did you live most of the time” or “where did you stay most of the time”, would that have changed your answer? How so? If I had asked, “Around October 1, 2012, where did you live and sleep most of the time?” would that have changed your answer? How so?
11a. On October 1, 2012, where was PERSON X staying? Here, at this house or residence Address 2 Address 3 Address 4 Some other place	Q11a USABILITY (FOR ALL PERSONS): What does “staying” mean as it is used in this question? IF ANSWER IS DIFFERENT THAN WHAT YOU WOULD EXPECT: How did you come up with your answer to this question?
Enumeration Questions (Person ___)
10F You reported that PERSON X stayed someplace else [Fill]. What is the address of the other place PERSON X stayed? Address 1 Enter a new address ADDRESS FIELDS	Q10F USABILITY: P47. NOTE HOW MANY ADDRESSES WERE PROVIDED AND COMPLETENESS OF THE ADDRESSES.
11. Where does Person X live or stay most of the time? Here, at this house or residence Address 2 Address 3 Address 4 In above places equally Some other place	Q11 USABILITY (FOR ALL PERSONS): P53. IF ADD’L ADDRESSES PROVIDED: How did you determine where Person 2 lives and sleeps most of the time? P55. If I had asked, “Around October 1, 2012, where did you live and sleep most of the time?” would that have changed your answer? How so?
11a. On October 1, 2012, where was Person X staying? Here, at this house or residence Address 2 Address 3 Address 4 Some other place	Q11a USABILITY (FOR ALL PERSONS): P57. IF ANSWER IS DIFFERENT THAN WHAT YOU WOULD EXPECT: How did you come up with your answer to this question?
Enumeration Questions (Person ___)
10F You reported that PERSON X stayed someplace else [Fill]. What is the address of the other place PERSON X stayed? Address 1 Enter a new address ADDRESS FIELDS	Q10d USABILITY: P47. NOTE HOW MANY ADDRESSES WERE PROVIDED AND COMPLETENESS OF THE ADDRESSES.
11. Where does Person X live or stay most of the time? Here, at this house or residence Address 2 Address 3 Address 4 In above places equally Some other place	Q11 USABILITY (FOR ALL PERSONS): P53. IF ADD’L ADDRESSES PROVIDED: How did you determine where Person 3 lives and sleeps most of the time? P55. If I had asked, “Around October 1, 2012, where did you live and sleep most of the time?” would that have changed your answer? How so?
11a. On October 1, 2012, where was Person X staying? Here, at this house or residence Address 2 Address 3 Address 4 Some other place	Q11a USABILITY (FOR ALL PERSONS): P57. IF ANSWER IS DIFFERENT THAN WHAT YOU WOULD EXPECT: How did you come up with your answer to this question?
Enumeration Questions (Person ___)
10F You reported that PERSON X stayed someplace else [Fill]. What is the address of the other place PERSON X stayed? Address 1 Enter a new address ADDRESS FIELDS	Q10d USABILITY: P47. NOTE HOW MANY ADDRESSES WERE PROVIDED AND COMPLETENESS OF THE ADDRESSES.
11. Where does Person X live or stay most of the time? Here, at this house or residence Address 2 Address 3 Address 4 In above places equally Some other place	Q11 USABILITY (FOR ALL PERSONS): P53. IF ADD’L ADDRESSES PROVIDED: How did you determine where Person 4 lives and sleeps most of the time? P55. If I had asked, “Around October 1, 2012, where did you live and sleep most of the time?” would that have changed your answer? How so?
11a. On October 1, 2012, where was Person X staying? Here, at this house or residence Address 2 Address 3 Address 4 Some other place	Q11a USABILITY (FOR ALL PERSONS): P57. IF ANSWER IS DIFFERENT THAN WHAT YOU WOULD EXPECT: How did you come up with your answer to this question?
Enumeration Questions (Person ___)
10F You reported that PERSON X stayed someplace else [Fill]. What is the address of the other place PERSON X stayed? Address 1 Enter a new address ADDRESS FIELDS	Q10d USABILITY: P47. NOTE HOW MANY ADDRESSES WERE PROVIDED AND COMPLETENESS OF THE ADDRESSES.
11. Where does Person X live or stay most of the time? Here, at this house or residence Address 2 Address 3 Address 4 In above places equally Some other place	Q11 USABILITY (FOR ALL PERSONS): P53. IF ADD’L ADDRESSES PROVIDED: How did you determine where Person 5 lives and sleeps most of the time? P55. If I had asked, “Around October 1, 2012, where did you live and sleep most of the time?” would that have changed your answer? How so?
11a. On October 1, 2012, where was Person X staying? Here, at this house or residence Address 2 Address 3 Address 4 Some other place	Q11a USABILITY (FOR ALL PERSONS): P57. IF ANSWER IS DIFFERENT THAN WHAT YOU WOULD EXPECT: How did you come up with your answer to this question?
Enumeration Questions (Person ___)
10F You reported that PERSON X stayed someplace else [Fill]. What is the address of the other place PERSON X stayed? Address 1 Enter a new address ADDRESS FIELDS	Q10d USABILITY: P47. NOTE HOW MANY ADDRESSES WERE PROVIDED AND COMPLETENESS OF THE ADDRESSES.
11. Where does Person X live or stay most of the time? Here, at this house or residence Address 2 Address 3 Address 4 In above places equally Some other place	Q11 USABILITY (FOR ALL PERSONS): P53. IF ADD’L ADDRESSES PROVIDED: How did you determine where Person 6 lives and sleeps most of the time? P55. If I had asked, “Around October 1, 2012, where did you live and sleep most of the time?” would that have changed your answer? How so?
11a. On October 1, 2012, where was Person X staying? Here, at this house or residence Address 2 Address 3 Address 4 Some other place	Q11a USABILITY (FOR ALL PERSONS): P57. IF ANSWER IS DIFFERENT THAN WHAT YOU WOULD EXPECT: How did you come up with your answer to this question?

Enumeration Questions Debriefing

58. HHD: How easy or difficult was it for you to type the addresses on the device?

59. How sure or unsure are you that you provided the correct address(es)? IF UNSURE: If this were the real census, what would you do? [Would you look them up, call someone, make your best guess?]

60. IF OTHERS IN HHD: How sure or unsure are you about where the other people in your household were staying on and around October 1?

61. IF ANY RESPONSES TO THE CENSUS QUESTIONS ARE INCONSISTENT OR UNEXPECTED, PROBE FOR MORE INFORMATION. FOR EXAMPLE IF THE R SAYS THAT HE WAS AT THEIR USUAL RESIDENCE AROUND OCTOBER 1, BUT HAD TOLD YOU PREVIOUSLY THAT HE WAS TRAVELING THEN.

Questions 12 - 14
12. You listed the following people at 1234 Home Address: Person 1 Person 2 Person 3 Person 4 Person 5 Person 6 Was there anyone else living or staying at 1234 Home Address on October 1st who you did not complete the form for? Yes – Go to Q13 No – End interview	Q12 USABILITY: IF SOMETHING WAS INCORRECT, WHAT DID R DO? WAS R ABLE TO FIX IT SUCCESSFULLY? DESCRIBE. In your own words, what is this question (below the list of names) asking?
13. Were they already counted in the 2012 Census? Yes – End interview No – Go to Q14 Don’t Know – Go to Q14	Q13 USABILITY: IF YES: How do you know they were counted?
[IF NOT HOMELESS] 14. In case we need to contact them, can you provide a name and phone number where we can reach them? [NAME AND PHONE FIELDS] Anyone else? Yes No	Q14 USABILITY: What are your thoughts about this question?

FINAL DEBRIEFING

Thanks, we’re almost finished. I have a few more questions for you.

[SCROLL BACK TO THE “This is who we have listed at [address]” QUESTION SO THAT YOU CAN SEE THE NAMES PROVIDED OR ELSE REFER TO YOUR NOTES.]

67. I see that you [HHD: typed/NRFU: provided] ________________ as your name. Would you ever use a different version of your name on a form like this?

1. IF YES: How else might you list your name? NOTE DIFFERENCES WITHOUT USING PII.

2. IF YES: How do you decide which way to write it on different forms?

1. [IF NEEDED:] Is __________ your full name or another form of your name such as a nickname? NOTE DIFFERENCES WITHOUT USING PII.

2. IF MIDDLE NAME BLANK OR JUST INITIAL: Did you see that you were supposed to provide your [full] middle name? Do you have a middle name? How did you decide to [leave it blank/just provide an initial]?

71. Do any of the other people you listed sometimes use a different version of their name on forms like this? NOTE DIFFERENCES WITHOUT USING PII. How did you decide how to list their name?

72. Do you have any other opinions about the format of the name question on this form? SHOW SCREENSHOTS IF NEEDED.

73. Was there anything in these questions that you found sensitive or uncomfortable to answer?

74. Was there any information for yourself or others that you would be hesitant to provide on the census form?

75. Aside from the things we’ve talked about, were there any questions or instructions that you find confusing or difficult to answer?

76. Did you find parts of the interview repetitive? If so, what parts?

77. If you were going to answer your census questionnaire on your own, without an interviewer, how would you prefer to do it: CHECK ALL THAT APPLY

____ On a smartphone? (What kind:____________________________________________)

____ On a tablet? (What kind:____________________________________________)

____ On a laptop or desktop?

____ On a paper form?

FOR EACH ANSWER CHOSEN, ASK: Why?

78. If you don’t answer the census on your own, an interviewer might be sent to your home to get your census information. How would you feel about the interviewer using….. CHECK ALL THAT APPLY.

____ a smartphone to do the interview?

____ a tablet, like an iPad?

____ a laptop computer?

____ a paper form?

Which do you prefer and why?

79. Do you own or have access to…

____ A smartphone? (What kind:____________________________________________)

____ A tablet? (What kind:____________________________________________)

____ A laptop or desktop?

HAND R THE PRINTOUT WITH THIS QUESTION: In order to count people in the right place in the census, the Census Bureau must collect detailed geographic information. This information can be time-consuming for respondents to provide and it can be expensive for the census to process. To assist in our efforts to increase the accuracy of the census and save a lot of taxpayer’s money, we could use cell phone or tablet technology, such as GPS, to determine your location more easily and accurately. If during the completion of your 2020 Census questionnaire we asked to use this technology to obtain geographic information directly from your mobile device, would you allow it?

80. Tell me why/why not? [IF NECESSARY: What additional information would you want to know?]

81. What does geographic information mean to you in this question?

End Time:________

Be Counted Automated Protocol 9