Various Pretesting Activities (see attached list)

Protocol for CPEX-OMB Alternative Race Question

(FORM X17/B4d)

Section I. Cognitive Probes

RESIDENCE RULES AND QUESTION 1:

WHILE R IS READING, INTRODUCE THINK ALOUD PROBES

• (AS NEEDED) What are you thinking? / What are you looking at?

• (AFTER R ANSWERS Q.1) How did you arrive at this answer?

• (ABOUT THE RULES AND Q.1) Can you tell me what this part is asking you to do?

QUESTION 2:

• (AS NEEDED) What are you thinking? / What are you looking at?

AFTER R ANSWERS Q2:

• How did you choose your answer?

• What does “people staying here temporarily” means to you in this question?

QUESTION 3:

• (AS NEEDED) What are you thinking? / What are you looking at?

• Can you tell me what “occupied without payment of rent” means to you in this question?

QUESTION 4:

• No probes

QUESTION 5:

Remind R to write his/her own name in this space.

• (AS NEEDED) What are you thinking? / What are you looking at?

• Do you think you would have written your name here if I hadn't asked you to? Why or why not?

QUESTION 6:

• No probes

QUESTION 7:

• (AS NEEDED) What are you thinking? / What are you looking at?

• What information is this question asking for?

QUESTION 8 AND 9:

• ALL PROBING WILL TAKE PLACE AFTER R HAS COMPLETED BOTH QUESTIONS. (UNLESS R STARTS TO COMPLAIN OR HAS NOTICEABLE DIFFICULTY, IN WHICH CASE, ASK ABOUT THE PROBLEM.)

OBSERVATION NOTES: RECORD ANY NOTABLE BEHAVIORS BELOW AND BE SURE TO INCLUDE THEM IN YOUR SUMMARY. INCLUDE:

- EVIDENCE THAT R READ Q8 OR Q9 PARTIALLY

- EVIDENCE THAT A QUESTION OR PART OF A QUESTION IS RE-READ?

- EVIDENCE THAT R SKIPS THE QUESTION, PARTS OF A QUESTION OR READS ONLY ANSWER CATEGORIES?

- DOES R BOTH MARK THE BOX AND PROVIDE A WRITE IN, IF APPROPRIATE?

- DOES R FILL BOTH QUESTIONS IN ORDER OR OUT OF ORDER?

- DOES R GO BACK AND CHANGE ANY ANSWER AT ANY POINT?

- DOES R APPEAR TO BE SEARCHING FOR ANYTHING OR HAVE DIFFICULTY IN LOCATING A RESPONSE?

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

INTERVIEWER: ASK R TO STOP AFTER COMPLETING Q8 AND Q9 FOR PERSON 1, SAY OR PARAPHRASE: “I see you have finished answering for yourself. Before you go on, I’d like to ask you some questions. I'm going to ask you about Question 8 first.”

PROBES FOR PERSON 1 (RESPONDENT’S INFORMATION):

Q8. Hispanic origin

INTERVIEWER: If R had a notable problem with Q8, probe that now. For example: “I noticed that you spent a long time looking for an answer (thinking about what to write). I know it might be a little hard to recall, but can you tell me what was going on for you then?”

Standard Probes:

• How did you choose your answer(s) for Question 8?

• [AS APPROPRIATE]:

  • Why did you write in _____?

  • I notice you didn't write anything in down here. Can you tell me why?

• What information did you think this question was asking for?

Q9. Race

INTERVIEWER: If R had a notable problem with Q9, probe that now. For example: “I noticed that you spent a long time looking for an answer (thinking about what to write). I know it might be a little hard to recall, but can you tell me what was going on for you then?”

• How did you choose your answer(s) for Question 9?

• [AS APPROPRIATE]:

  • Why did you write in _____?

  • I notice you didn't write anything in down here. Can you tell me why?

  • (IF R MARKED MORE THAN ONE: How did you decide to mark more than one answer?)

• What information did you think this question was asking for?

• If you can remember, did you read the whole question? Where did you stop? What parts did you see?

• Did you notice the heading above the Asian and Native Hawaiian and Other Pacific Islander categories? What did you think of that? Are these headings helpful or not helpful?

• Did you notice these examples after ‘Other Asian’? (If no, direct R to read them now.) Was there something you expected to see, or would have been helpful to you, in the examples that wasn’t there? What do you think of the order or the examples?

• Did you notice these examples after ‘Other Pacific Islander’? (If no, direct R to read them now.) Was there something you expected to see, or would have been helpful to you, in the examples that wasn’t there? What do you think of the order or the examples?

• How about “Some other race”, who do you think this applies to?

NOTE TO INTERVIEWER: We are saving probing on instructions for subsequent Persons. If this is a one-person household, however, ask about the “mark one or more” instructions and the instruction prior to Question 8 NOW.

PROBES FOR PERSON 2:

INTERVIEWER: ASK R TO COMPLETE THE FORM FOR PERSON 2: “Thanks for your answers. Please continue filling your form for the next person in your household, and remember to tell me what you are thinking as you come up with your answers.”

INTERVIEWER: DO NOT PROBE ANY OF THE OTHER DEMOGRAPHIC QUESTIONS PRIOR TO HISPANIC ORIGIN AND RACE. NOTE ANY OBSERVED DIFFICULTIES BELOW AND BE SURE TO INCLUDE THEM IN YOUR SUMMARY.

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

STOP R AFTER INFORMATION FOR PERSON 2 IS COMPLETE AND PROBE: “I have a few more questions to ask you about Questions 5 and 6, before you go on to the next person.”

Q5. Hispanic origin for Person 2

• Probe any difficulty noted, for example: “I noticed that you spent more time on question 5 this time. Can you tell me what you were thinking?”

• How did you choose your answer(s) for Q5?

Q6. Race for Person 2

• Probe any difficulty noted with this question.

• How did you choose your answer(s) for Q6?

• [AS APPROPRIATE]:

• Why did you write in _____?

• I notice you didn't write anything in down here. Can you tell me why?

• (IF R MARKED MORE THAN ONE: How did you decide to mark more than one answer?)

• Did you see the part of the instruction about “mark x one or more boxes”?

  • If yes: what was that instruction telling you? Who do you think this applies to?

  • If no: how many boxes did you think you could mark? If you had noticed it, would it have changed your answer?

• Do you recall seeing the instruction “➜ NOTE: Please answer BOTH Question 8 about Hispanic origin and Question 9 about race.”

  • (GET AN ANSWER; IF NO, POINT TO IT) What was this instruction asking you to do?

• Did you see the instruction “ For this census, Hispanic origins are not races.”

  • (GET AN ANSWER; IF NO, POINT TO IT) What does that mean to you in this question?

INTERVIEWER, SAY/PARAPHRASE: “Thanks for your answers. Please continue filling your form for the next person in your household, and remember to tell me what you are thinking as you come up with your answers.”

PROBES FOR PERSONS 3 – 6:

NOTE: We are primarily interested in how responses for children are handled in these persons. We are interested in how respondents answer for multiracial children and U.S.-born children of immigrant parents.

Q5. Hispanic origin/Q6. Race for Persons 3 – 6

• AS NECESSARY: Why did you choose that answer (those answers) for Person (fill in)?

• IF APPROPRIATE: I noticed that you answered differently for yourself and for your child. Can you tell me why you decided to do that?

Section II. Alternative Forms

INTERVIEWER, SAY/PARAPHRASE: “Thank you, we're done with this part of the interview. Now, I'd like to show you two more forms to compare with the one you just filled out.

SHOW FORM X14 / B4a: Please take a look at this form. You don't have to fill it out, but please read it over. [Give a few moments for R to read the form].

What are some of the most noticeable differences between X14 and X17 (the form you saw first)? Anything else?

How would you have answered this form?

Did you notice that Q9 is different in X14?

What information do you think Q9 in X14 is asking for?

Is it the same or different from X17?

If you had to pick one wording, which do you prefer X14 or X17? Why?

Did you notice the differences in the Black/African American category?

Which do you prefer? Why?

NOW SHOW FORM X16 / B4c

And now take a look at form X16 [give a few moments for R to read the form].

What are some of the most noticeable differences between X16 and X17 (the form you saw first)? Anything else?

How would you have answered this form?

Did you notice the differences in the Black/African American category?

Which do you prefer? Why?

Did you notice that X16 does not have the heading above Asian and NHPI?

Do you prefer the form with the headings above Asian and NHPI or the one without the headings? Why?

Did you notice that the examples for Other Asian are listed differently? Which do you prefer? Why?

RANKING

Please rank these forms in the order of your preference. Why did you put them in that order?

Section III. Debriefing

Just a few final questions to wrap up –

• Overall, would you say the survey questions were easy or difficult to respond to? Why?

• Do you think there are questions some people would find difficult? Sensitive?

• Do you have anything else you would like to tell us that you haven’t had a chance to

6

CPEX-OMB FORM X17/B4d