Protocol

SmaRT Testing of Decennial Mail Materials – Instrument Testing

RESPONDENT ID #: _________________________ DATE: ____ / ____ / _____

INTERVIEWER’S NAME: _________________________________________________

START TIME: ______: ______ AM/PM VERSION of Vacancy Question 1AB or 2AB: ________

Session start

For the remainder of the session you’ll answer the 2023 Census Test survey that you read about in the letters. Then we will have you answer a few questions about your experience with the survey. I want you to respond to it naturally but just like you have been doing so far, remember to think aloud as you do it. If you fall silent, I may remind you to think aloud by saying “keep talking” or “what are you thinking?”

[HAVE RESPONDENT COMPLETE THE ONLINE QUALTRICS DECENNIAL QUESTIONNAIRE;

URL for Qualtrics Decennial Questionnaire:

Make note if they had any difficulty with the address screen.

Make note of if the address standardizer found their address and how they answered the question.

Make note of how they answered the whole/partial question

Make note of which vacancy question they got– 1A/B or 2A/B, and how they answered

Now I’m going to have you answer some satisfaction questions about the survey you just completed.

URL for Satisfaction/Debriefing/Quizlette/:

[HAVE RESPONDENT COMPLETE THE SATISFACTION QUESTIONS IN QUALTRICS]

Debriefing

[HAVE RESPONDENT REVIEW THE SCREENS OF NOTE IN QUALTRICS]

Now I just have a few more questions for you about the survey.

[Show Address Standardizer] Take a look at this screen. When you did it, it displayed the machine’s understanding of your address. I think you said it was [accurate/inaccurate]. Can you tell me more about that?

[If count discrepancy screen is triggered] Take a look at this screen. Any thoughts when you saw this?

[Show respondent the whole/partial question screen] Take a look at this screen. I believe you answered ______. Can you tell me more about that? [If needed] Who were you including when you answered that question? Did you think of including or excluding anyone [else]? What did you think about that screen?

[Show count discrepancy and whole/partial together] Looking at these together, do you have any additional thoughts?

[Show respondent the vacancy screen they saw] Take a look at this screen. You answered______. Tell me about that.

What does the word vacant mean to you in this question?

[If they saw version 1A - ] Now take a look at all the response options. Can you tell me in your own words what each of them means to you?

Now here is another way of asking that question. Take a look at this version. How would you answer this question?

[If they saw version 1B --we want to probe about meaning of the response options in Version 1A] Now take a look at the response options in this other version. Can you tell me in your own words what each of them means to you?

[Show respondent paper questionnaire – Vacancy path] I’d like to direct you back to the paper questionnaire. Looking at page 8 of the paper questionnaire. The question is similar to the ones we just looked at. But tell me what you are thinking about as you look it over.

So how would you answer this question?

What do you think about the location of this question in the paper questionnaire?

Is there a better place for it?

Talk me through the different pages you think you would read over if you were answering the survey on paper.

[If necessary - and they answered no and they didn’t say more/give context for the no] Did you answer “no” because you didn't know the street address, or because you don’t know the address of the vacant unit/house?

Another way to ask this question would be: “Do you know any houses, apartments, or mobile homes that were vacant on November 1, 2023? This could be homes inside or outside your neighborhood.”

Yes

No

What are your thoughts?

Which do you prefer?

Do you have any feedback about the layout or design of this page?

Quizlette

We’re almost done. Now we would like you to complete a few more short questions before we wrap up.

[HAVE THE RESPONDENT COMPLETE THE QUIZLETTE QUESTIONS IN QUALTRICS]

Quizlette URL:

Online, Paper, Or in-Person

If you were filling out this survey at home, would you prefer to complete it online, with a paper form, or talking to someone over the phone?

[IF ONLINE:] Would you prefer to do this on a computer, on your phone, on a tablet, or a different device? (If different device, please specify)

Is there anything else that you’d like to share with us about the questions you have answered for us today?