SmaRT Testing of Decennial Instrument Protocol March 2024

SmaRT Testing of Decennial Mail Materials – Instrument Testing

RESPONDENT ID #: _________________________ DATE: ____ / ____ / _____

INTERVIEWER’S NAME: _________________________________________________

START TIME: ______: ______ AM/PM VERSION of Vacancy Question 1AB or 2AB: ________

Session start

For the remainder of the session you’ll answer the 2024 Census Test survey that you read about in the letters. Then we will have you answer a few questions about your experience with the survey. I want you to respond to it naturally but just like you have been doing so far, remember to think aloud as you do it. If you fall silent, I may remind you to think aloud by saying “keep talking” or “what are you thinking?”

[HAVE RESPONDENT COMPLETE THE ONLINE QUALTRICS DECENNIAL QUESTIONNAIRE;

URL for Qualtrics Decennial Questionnaire:

Make note if they had any difficulty with the address screen.

Make note of if the address standardizer found their address [if all capital letters in address that is returned to them] and how they answered the question.

Make note of how they answered the whole/partial question

Make note of which vacancy question they got– 1A/B or 2A/B, and how they answered

Make note of comments on the Targeted UC questions

Now after you hit submit, you’d receive an email confirmation. I’m going to show you what it would say and let me know if you have any feedback. [Send link in chat Insert URL here].

[THIS LINK CONTAINS THE EMAIL CONFIRMATION, THEN SATISFACTION, THEN DEBRIEFING]

Thank you. You can go to the next page. And I’m now going to have you answer some satisfaction questions about the survey you just completed.

Debriefing

Now I just have a few more questions for you about the survey. Please go to the next page.

[Show Address Standardizer] Take a look at this screen. When you did it, it displayed the machine’s understanding of your address. I think you said it was [accurate/inaccurate]. Can you tell me more about that?

[Targeted_UC screen] Take a look at this screen.

Any thoughts when you saw this?

Was there anyone you thought about including but didn’t?

[If count discrepancy screen is triggered] Take a look at this screen. Any thoughts when you saw this?

[Show respondent the whole/partial question screen] Take a look at this screen. I believe you answered ______. Can you tell me more about that? [If needed] Who were you including when you answered that question? Did you think of including or excluding anyone [else]? What did you think about that screen?

[Show respondent the vacancy screen they saw] Take a look at this screen. You answered______. Tell me about that.

What does the word vacant mean to you in this question?

[Show respondent the vacancy screen they didn’t see] Now here is another way of asking that question. Take a look at this version. How would you answer this question?

Between these two different vacancy questions, which do you prefer, if either?

Do you have any feedback about the layout or design of this page?

Knowledge Check

We’re almost done. Now we would like you to complete a few more short questions before we wrap up.

[HAVE THE RESPONDENT COMPLETE THE KNOWLEDGE CHECK QUESTIONS IN QUALTRICS]

Quizlette URL: https://research.rm.census.gov/jfe/form/SV_1FGXDvyS5TVXuvQ

Online, Paper, Or in-Person

If you were filling out this survey at home, would you prefer to complete it online, with a paper form, or talking to someone over the phone?

[IF ONLINE:] Would you prefer to do this on a computer, on your phone, on a tablet, or a different device? (If different device, please specify)

Is there anything else that you’d like to share with us about the questions you have answered for us today?