Usability Debreifing Slides

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

[PROBES FOR SPANISH ONLY – INTERVIEWER: PLEASE POINT OUT TO THE SPECIFIC SENTENCES WHERE THESE CONCEPTS APPEAR IN THE INSTRUCTIONS]

 

- What are you thinking when we say "We intend to keep your answers anonymous "?

- What are you thinking when we say "You can choose not to answer any question "?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

[PROBES AFTER Q1 & Q2]

-  How did you come up with your answers for question 1 and question 2?

-  What are you thinking when we say "impacts you and your community"?

 

[PROBES FOR SPANISH ONLY]

Can you tell me more about what “how familiar” means to you both in question 1 and question 2?

<number>

<number>

 

Do  you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[PROBE AFTER Q3 & Q4]

Let us reread question 3 and question 4.  Are these questions asking you the same thing or something different?  Can you tell me more about that?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[PROBE FOR Q5 & Q6]

-  How did you come up with your answers for question 5 and question 6?

-  What does “census form” mean to you in question 6?

[PROBE FOR Q7]

Can you tell me more about how did you come up with an answer for question 7?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

[PROBE FOR  Q10]

How did you come up with your answer to question 10?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

[PROBES FOR Q12]

 

How did you come up with your answer to question 12?

What does "comfortable" mean to you in question 12?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

[INTRODUCTION FOR INTERVIEWER   FOR Q13-Q22: THE FOLLOWING QUESTIONS ARE RELATED TO CENSUS KNOWLEDGE FACTS. PLEASE TAKE NOTES ON QUESTIONS THAT CAUSE HESITATION OR PERCEIVED DIFFICULTY AND THEN USE THOSE NOTES TO PROBE RETROSPECTIVELY AFTER RESPONDENT FINISHED ANSWERING Q13-Q22.]

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[PROBES FOR Q13-Q22. ONLY FOR THOSE QUESTIONS THAT CAUSED HESITATION AND/OR PERCEIVED DIFFICULTY. PROBE ONLY ON A MAXIMUM OF THREE ITEMS UNDER THESE CRITERIA]  

How easy (or not) was to come up with an answer for… [FOR TIME CONSTRAINS ASK ONLY ON A MAXIMUM OF THREE ITEMS]

[ONLY FOR THOSE THAT SELECTED "DON'T KNOW" AS AN ANSWER ON Q13-Q22 ]  How do you feel about not having the "Don't know"option [POINT TO THE DON’T KNOW ANSWER. FOR TIME CONSTRAINTS, ASK ONLY ON A MAXIMUM OF THREE ITEMS?

 

[PROBES FOR SPANISH ONLY]

Can you tell me more about what “break the law” means to you?

[PROBES FOR Q23-Q24.]

 Can you tell me more about how easy (or not) was to come up with an answer for Q23 and Q24?

 

[PROBE FOR Q23. INTERVIEWER SHOW THE PIECE OF PAPER WITH THE ALTERNATIVE WORDING FOR Q23]: We are thinking on alternatives to ask question 23. Can you tell me in your own words what do you think about this version? Which version do you prefer between the two? For what reasons?

 

As far as you know, if you DO NOT fill out a Census form what will happen?

 

You will get a phone call to complete the form

You will be fined

A Census worker will come to your door to complete the form in person

You will receive an email link to complete the form

Nothing will happen

 

[PROBE FOR Q24. INTERVIEWER SHOW THE PIECE OF PAPER WITH THE ALTERNATIVE WORDING FOR Q24]: We are thinking on alternatives to ask question 23. Can you tell me in your own words what do you think about this version? Which version do you prefer between the two? For what reasons?

 

 

How often does the U.S. Constitution require a census to be conducted?

Every 10 years

Every 5 years

The Constitution does not require a census at all

Don’t know

 

[PROBE ONLY AFTER Q24]

 

Before you answered Q13-Q24. There was an introduction to the section that read as follows: “The following questions are about the Census in general. Please answer to the best of your knowledge. There is no need to look up this information.” What does this instruction mean to you in your own words? We want to make sure people know they should not look up information when they answer these questions. Is this how you understood the instruction? [IF NO] How can it be improved?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[PROBE AFTER Q25-Q33.]

 

Can you think on more public services that we did not include in these questions (Q25-Q33)?

 

[PROBES FOR SPANISH ONLY]

 In your own words, what does “Job training programs” in question 30 mean to you?

<number>

<number>

 

Do  you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[INSTRUCTIONS FOR INTERVIEWER: AFTER READING THE INTRODUCTORY STATEMENT FOR Q34-Q36, ASK THE RESPONDENT TO STOP READING THE QUESTIONNAIRE AND PROBE]

 

Before you answer the next survey questions, I want to know what uses for the Census you think the survey will be asking you about?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

[PROBE AFTER Q35.]

In your own words, can you tell me what "changes in your community" mean to you in question 35?

 

[PROBE AFTER Q36.]

What do “civil rights laws” mean to you in question 36?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[PROBE AFTER ASKING  Q37-Q40.]

 

[PROBE FOR Q37.]

In your own words, can you tell me what Q37 is asking? [IF NECESSARY] What does "making sure your voice is heard" mean?

[PROBE FOR Q38.]

What about Q38, can you tell me in your own words what Q38 is asking? [IF NECESSARY]  What does "proud of your cultural heritage" mean?

 

[PROBE FOR Q39.]

In your own words, can you tell me what Q39 is asking? [IF NECESSARY]  What does "a better future for your community" mean?

[PROBE FOR Q40.]

In your own words, can you tell me what Q40 is asking? [IF NECESSARY]  What does "civic responsibility" mean?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[PROBE AFTER Q41]

Can you think on more options that were not included in this list?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[PROBE AFTER  Q42]

What does “your community” mean to you in this question?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[PROBE AFTER ASKING  Q46-Q49.]

 

[PROBE FOR Q46]

In question 46, what does the phrase “not keep answers confidential” mean to you?”

[PROBE FOR Q47]

When  question 47 asked about information being miused, what type of information were you thinking about?

[PROBE FOR 48]

What does “share answers with other government agencies” mean to you in question 48?

 

[PROBE FOR Q49]

What does “used against you” mean to you in question 49?

 

[PROBES FOR SPANISH ONLY]

Can you tell me more about what “misused” means to you in question 47?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[PROBE FOR Q52]

How did you come up for answer to  question 48?

[IF HARM]: How do you think answering your Census form could harm you?

[IF BENEFIT]: How do you think answering your Census form could benefit you?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[PROBE AFTER Q54]

Are there any activities you think that are missing from the list?  Which ones?

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.  

[PROBES FOR SPANISH ONLY]

Can you tell me more about what “local broadcast radio” means to you in option 2?

 

[INSTRUCTION FOR INTERVIEWER: PAY ATTENTION TO ANY PERCEIVED DIFFICULTY AND OR HESITATION ON THE DEMOGRAPHICS AND PROBE WHEN NECESSARY]

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

[IF R SAW THIS SCREEN]

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

[IF R SAW THIS SCREEN]

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

NO THANK YOU MESSAGE OR YOU HAVE Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.NO COMPLETED THE SURVEY, ETC.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>

 

Do you have any comments about this screen? [IF R HAD PROBLEMS] Tell me more about your experience with this screen.

<number>

<number>