AIAN Tribal Enrollment Question Research
Cognitive Interview Protocol
Protocol 3
PARTICIPANT ID #: _________________________ DATE: ____ / ____ /2016
COGNITIVE INTERVIEWER NAME: _________________________
START TIME: ______: ______ AM/PM
|
|
|||
|
(Cognitive Interviewer: PLACE THE CONSENT FORM IN FRONT OF PARTICIPANT, THEN Read)
|
|
|
|
A. Hello, I’m [NAME OF COGNITIVE INTERVIEWER]. I work for the
Census Bureau. Thanks for agreeing to help me today.
Before we start, I would like you to read over the document in front of you. This document explains a little bit about this interview and provides information about your rights as a participant. It also asks for your permission to have this session audio recorded. Please ask me any questions you have about this document. When you finish reading the document, please sign it if you agree to the terms.
|
PARTICIPANT READS AND SIGNS FORM
|
B. IF PARTICIPANT PROVIDES CONSENT TO HAVE THE SESSION AUDIO-TAPED: I will now turn on the audio recorder to ask for your consent again.
TEST AUDIO RECORDER AND TURN IT ON AGAIN AFTER
TEST Do you consent to being recorded for this interview?
|
|
SECTION II. COGNITIVE INTERVIEW |
Introduction
(Cognitive Interviewer: Read/Paraphrase)
“Let me begin by telling you more about what we’ll be doing today. The United States Census Bureau counts how many people live in the U.S., and also conducts different kinds of surveys.
Today, with your help, we will test some new questions that could be used in the 2020 Census.
I'll give you the form to fill out. I'd like you to fill it out as you would if you received it in the mail, at home, but with one big difference:
What I would like you to do today, is tell me what you are thinking and feeling as you come up with your answers and fill out your Census form. Instead of thinking to yourself, I’d like you to think out loud, and I will remind you to do this from time to time. I will ask you some questions about your answers, and I’ll also ask you about the meaning of some words and what those words mean to you. The questions I will ask are only to test the quality of our questions and not yourself. Please remember that I am really interested in how these questions work for you, so there are no right or wrong answers.
Do you have any questions for me before we begin?”
A. FORM EQ3
Here is the first form [HAND FORM EQ3 TO R]. This form asks questions about you, and you will be Person 1. [INTERVIEWER: POINT OUT THE TERM “PERSON 1” IN QUESTION 5.] Please start at Question 1 and then stop at Question 10. I will give you a few minutes to fill it out. Please remember to tell me what you are thinking as you come up with your answers.
INTERVIEWER: WE ARE PROBING ON QUESTIONS 1, 3, and 8 TO TRAIN R TO THINK ALOUD AND RESPOND TO PROBES. THESE QUESTIONS ARE NOT A RESEARCH FOCUS. ALL PROBES SHOULD BE ASKED AFTER R PROVIDES A WRITTEN RESPONSE, EXCEPT FOR THINK ALOUD PROBES, WHICH ARE USED ANY TIME THAT R FALLS SILENT OR HAS A VISIBLE REACTION.
TRAINING PROBES
WHILE R IS READING, INTRODUCE THINK ALOUD PROBES
QUESTION 1: How many people were living or staying in this house, apartment, or mobile home on April 1, 2016?
(AS NEEDED) What are you thinking?
How did you come up with your answer?
QUESTION 3: Is this a house, apartment, or mobile home?
(AS NEEDED) What are you thinking?
In your own words, what information is this question asking for?
QUESTION 8: Which categories describe Person 1?
(AS NEEDED) What are you thinking?
In your own words, what information is this question asking for?
QUESTIONS 9a, 9b, and 10:
OBSERVATION NOTES: RECORD ANY BEHAVIORS BELOW AND BE SURE TO INCLUDE THEM IN YOUR SUMMARY. INCLUDE:
EVIDENCE THAT R READ THE QUESTION (NOT AT ALL/PARTIALLY/FULLY)
EVIDENCE THAT A QUESTION OR PART OF A QUESTION IS RE-READ
EVIDENCE THAT R SKIPS PARTS OF A QUESTION OR READS ONLY ANSWER CATEGORIES
DOES R BOTH MARK A BOX AND PROVIDE A WRITE IN, IF APPROPRIATE?
ARE ALL WRITE-IN ANSWERS APPROPRIATE?
DOES R FILL BOTH QUESTIONS IN ORDER?
DOES R GO BACK AND CHANGE ANY ANSWER AT ANY POINT?
DOES R APPEAR TO BE SEARCHING FOR ANYTHING OR HAVE DIFFICULTY IN LOCATING A
RESPONSE?
DOES R SHOW ANY HESITATION AND/OR DIFFICULTY IN COMING UP WITH AN ANSWER?
DOES R SEEM TO HAVE ANY DIFFICULTY UNDERSTANDING THE QUESTION?
IS THERE A LITERACY AND/OR A LANGUAGE BARRIER?
DOES R ASK FOR CLARIFICATION?
ANYTHING ELSE?
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
SAY OR PARAPHRASE: “I see you have finished answering the questions. I'm going to ask you about Questions 9a, 9b, and 10.”
QUESTION 9a:
PROBES FOR INTERVIEWER: If R had any problem with Question 9a, probe that now.
For example:
“I noticed that you spent a long time looking for an answer (thinking about what to write). I know it might be a little hard to recall, but can you tell me what was going on for you then?”
Standard Probes:
How did you choose your answer for Question 9a?
[IF THE RESPONDENT REPORTS A TRIBE, VILLAGE, OR CORPORATION IN QUESTION 8, AND CHECKS “NO, NOT ENROLLED” FOR QUESTION 9a] What is the relationship you have with your tribe (village, or corporation)?
[AS APPROPRIATE]:
(IF R MARKED MORE THAN ONE:) How did you decide to mark more than one answer?
(IF R CHANGED THEIR ANSWER TO QUESTION 8 AFTER LOOKING AT OR ANSWERING QUESTION 9a): For what reasons did you decide to change your answer to Question 8? Tell me more about that.
What information did you think this question was asking for?
If you can remember, did you read the whole question? Where did you stop? What parts of the question did you see?
QUESTION 9b:
PROBES FOR INTERVIEWER: If R had any problem with Question 9b, probe that now.
For example:
“I noticed that you spent a long time looking for an answer (thinking about what to write). I know it might be a little hard to recall, but can you tell me what was going on for you then?”
Standard Probes:
How did you choose your answer for Question 9b?
[AS APPROPRIATE]:
For what reasons did you write in _____?
[IF RESPONDENT REPORTS BEING ENROLLED IN QUESTION 9a, AND DOES NOT WRITE IN A TRIBE/VILLAGE IN THE WRITE-IN SPACE] I notice you didn't write anything in down here [in the write-in line]. Can you tell me for what reasons?
(IF R WROTE IN MORE THAN ONE ANSWER): How did you decide to write in more than one answer?
(IF R CHANGED THEIR ANSWER TO QUESTION 8 AFTER LOOKING AT OR ANSWERING QUESTION 9b): For what reasons did you decide to change your answer to Question 8? Tell me more about that.
What information did you think this question was asking for?
If you can remember, did you read the whole question? Where did you stop? What parts of the question did you see?
QUESTION 10:
PROBES FOR INTERVIEWER: If R had any problem with Question 10, probe that now.
For example:
“I noticed that you spent a long time looking for an answer (thinking about what to write). I know it might be a little hard to recall, but can you tell me what was going on for you then?”
Standard Probes:
How did you choose your answer for Question 10?
[IF THE RESPONDENT REPORTS A REGIONAL OR VILLAGE CORPORATION IN QUESTION 8, AND CHECKS “NO, NOT A SHAREHOLDER” FOR QUESTION 10] What is the relationship you have with your corporation? Tell me more about that.
[AS APPROPRIATE]:
For what reasons did you write in _____?
(IF R MARKED MORE THAN ONE:) How did you decide to mark more than one answer?
(IF R WROTE IN MORE THAN ONE ANSWER): How did you decide to write in more than one answer?
(IF R CHANGED THEIR ANSWER TO QUESTION 8 AFTER LOOKING AT OR ANSWERING QUESTION 10): For what reasons did you decide to change your answer to Question 8? Tell me more about that.
What information did you think this question was asking for?
If you can remember, did you read the whole question? Where did you stop? What parts of the question did you see?
B. FORM EQ1
Thanks for your answers to the first form. Now I will ask you to complete a second form. Here is the form [HAND FORM EQ1 TO R]. Like the last form, this one asks questions about you, and you will be Person 1. [INTERVIEWER IF NECESSARY: POINT OUT QUESTION 5.] But this time, please start at Question 5 and stop at Question 9. I will give you a few minutes to fill it out, and remember to tell me what you are thinking as you come up with your answers.
QUESTION 9:
OBSERVATION NOTES: RECORD ANY BEHAVIORS BELOW AND BE SURE TO DESCRIBE THEM IN YOUR SUMMARY. INCLUDE:
EVIDENCE THAT R READ THE QUESTION (NOT AT ALL/PARTIALLY/FULLY)
EVIDENCE THAT A QUESTION OR PART OF A QUESTION IS RE-READ
EVIDENCE THAT R SKIPS PARTS OF A QUESTION OR READS ONLY ANSWER CATEGORIES
DOES R BOTH MARK A BOX AND PROVIDE A WRITE IN, IF APPROPRIATE?
DO ALL WRITE-IN ANSWERS APPEAR APPROPRIATE?
DOES R FILL BOTH QUESTIONS IN ORDER?
DOES R GO BACK AND CHANGE ANY ANSWER AT ANY POINT?
DOES R APPEAR TO BE SEARCHING FOR ANYTHING OR HAVE DIFFICULTY IN LOCATING A
RESPONSE?
DOES R SHOW ANY HESITATION AND/OR DIFFICULTY IN COMING UP WITH AN ANSWER?
DOES R SEEM TO HAVE ANY DIFFICULTY UNDERSTANDING THE QUESTION?
IS THERE A LITERACY AND/OR A LANGUAGE BARRIER?
DOES R ASK FOR CLARIFICATION?
ANYTHING ELSE?
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
PARAPHRASE: “I see you finished answering the questions. I'm going to ask you about Question 9.”
QUESTION 9:
PROBES FOR INTERVIEWER: If R had any minor or major problem with Question 9, probe that now. For example:
“I noticed that you spent a long time looking for an answer (thinking about what to write).
I know it might be a little hard to recall, but can you tell me what was going on for you then?”
Standard Probes:
How did you choose your answer for Question 9?
[IF THE RESPONDENT REPORTS A TRIBE, VILLAGE, OR CORPORATION IN QUESTION 8, AND CHECKS “NO, NOT ENROLLED” FOR QUESTION 9] What is the relationship you have with your tribe (village, corporation, or what you wrote on the AIAN line in Question 8)?
[AS APPROPRIATE]:
For what reasons did you write in _____?
[IF RESPONDENT REPORTS A TRIBE, VILLAGE, OR CORPORATION IN QUESTION 8, AND DOES NOT WRITE ANYTHING IN THE WRITE-IN SPACE IN QUESTION 9] I noticed you didn't write anything in down here [in the write-in line]. Can you tell me for what reasons?
(IF R MARKED MORE THAN ONE CHECKBOX): For what reasons did you decide to mark more than one answer?
(IF R WROTE-IN MORE THAN ONE ANSWER): For what reasons did you decide to write in more than one answer?
(IF R CHANGED THEIR ANSWER TO QUESTION 8 AFTER LOOKING AT OR ANSWERING QUESTION 9): For what reasons did you decide to change your answer to Question 8? Tell me more about that.
What information did you think this question was asking for?
How much of the question did you read? Where did you stop? What parts of the question did you see?
C. FORM EQ2
Thanks for your answers to this form. Now I will ask you to complete a third form. Here is the form [HAND FORM EQ1 TO R]. Like the last form, this form asks questions about you, and you will be Person 1. Please start at Question 5 and end at Question 10 [INTERVIEWER IF NECESSARY: POINT OUT QUESTION 5.] I will give you a few minutes to fill it out, and remember to tell me what you are thinking as you come up with your answers.
QUESTIONS 9 and 10:
OBSERVATION NOTES: RECORD ANY BEHAVIORS BELOW AND BE SURE TO INCLUDE THEM IN YOUR SUMMARY. INCLUDE:
EVIDENCE THAT R READ THE QUESTION (NOT AT ALL/PARTIALLY/FULLY)
EVIDENCE THAT A QUESTION OR PART OF A QUESTION IS RE-READ
EVIDENCE THAT R SKIPS PARTS OF A QUESTION OR READS ONLY ANSWER CATEGORIES
DOES R BOTH MARK A BOX AND PROVIDE A WRITE IN, IF APPROPRIATE?
DO ALL WRITE-IN ANSWERS APPEAR APPROPRIATE?
DOES R FILL BOTH QUESTIONS IN ORDER?
DOES R GO BACK AND CHANGE ANY ANSWER AT ANY POINT?
DOES R APPEAR TO BE SEARCHING FOR ANYTHING OR HAVE DIFFICULTY IN LOCATING ARESPONSE?
DOES R SHOW ANY HESITATION AND/OR DIFFICULTY IN COMING UP WITH AN ANSWER?
DOES R SEEM TO HAVE ANY DIFFICULTY UNDERSTANDING THE QUESTION?
IS THERE A LITERACY AND/OR A LANGUAGE BARRIER?
DOES R ASK FOR CLARIFICATION?
ANYTHING ELSE?
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
SAY OR PARAPHRASE: “I see you have finished answering the questions. I'm first going to ask you about Question 9, and then I’ll ask you about Question 10.”
QUESTION 9:
PROBES FOR INTERVIEWER: If R had any problem with Questions 9, probe that now. For example:
“I noticed that you spent a long time looking for an answer (thinking about what to write). Can you tell me what was going on for you then?”
Standard Probes:
How did you choose your answer for Question 9?
[IF THE RESPONDENT REPORTS A TRIBE OR ALASKA NATIVE ENTITY IN QUESTION 8, AND CHECKS “NO, NOT ENROLLED” FOR QUESTION 9] What is the relationship you have with your tribe (council, association, or community)?
[AS APPROPRIATE]:
For what reasons did you write in _____?
[IF RESPONDENT REPORTS A TRIBE/ALASKA NATIVE ENTITY IN QUESTION 8, AND DOES NOT WRITE IN A TRIBE/ALASKA NATIVE ENTITY IN THE WRITE-IN SPACE IN QUESTION 9] I noticed you didn't write anything in down here [in the write-in line]. Can you tell me for what reasons?
(IF R MARKED MORE THAN ONE CHECKBOX): For what reasons did you decide mark more than one answer?
(IF R WROTE IN MORE THAN ONE ANSWER): For what reasons did you decide to write in more than one answer?
(IF R CHANGED THEIR ANSWER TO QUESTION 8 AFTER LOOKING AT OR ANSWERING QUESTION 9): For what reasons did you decide to change your answer to Question 8? Tell me more about that.
What information did you think this question was asking for?
If you can remember, did you read the whole question? Where did you stop? What parts of the question did you see?
QUESTION 10:
PROBES FOR INTERVIEWER: If R had any problem with Question 10, probe that now. For example:
“I noticed that you spent a long time looking for an answer (thinking about what to write). Can you tell me what was going on for you then?”
Standard Probes:
How did you choose your answer for Question 10?
[IF THE RESPONDENT REPORTS A REGIONAL OR VILLAGE CORPORATION IN QUESTION 8, AND CHECKS “NO, NOT A SHAREHOLDER” FOR QUESTION 10] What is the relationship you have with your corporation? Tell me more about that.
[AS APPROPRIATE]:
For what reasons did you write in _____?
[IF RESPONDENT REPORTS A CORPORATION IN QUESTION 8, AND DOES NOT WRITE IN AN ALASKA CORPORATION IN THE WRITE-IN SPACE] I notice you didn't write anything in down here [in the write-in line]. Can you tell me for what reasons?
(IF R MARKED MORE THAN ONE CHECKBOX): How did you decide to mark more than one answer?
(IF R WROTE IN MORE THAN ONE ANSWER): How did you decide to write in more than one answer?
(IF R CHANGED THEIR ANSWER TO QUESTION 8 AFTER LOOKING AT OR ANSWERING QUESTION 10): For what reasons did you decide to change your answer to Question 8? Tell me more about that.
What information did you think this question was asking for?
If you can remember, did you read the whole question? Where did you stop? What parts of the question did you see?
D. PROBES DISCUSSED FOR ALL FORMS
INTERVIEWER, SAY/PARAPHRASE: “Thanks for your answers to the third form. Now I will ask you questions about the forms you completed [BE PREPARED TO SHOW R ANY OR ALL OF THE FORMS].”
FORM EQ1
[POINT TO THE TERM “ENROLLED” IN THE QUESTION STEM] What does the word “ENROLLED” mean to you in this question?
[POINT TO THE PHRASE “AMERICAN INDIAN TRIBE” IN THE QUESTION STEM] What does “AMERICAN INDIAN TRIBE” mean to you in this question?
[POINT TO THE PHRASE “ALASKA NATIVE TRIBE, VILLAGE, OR CORPORATION” IN THE QUESTION STEM]
What does “ALASKA NATIVE TRIBE” mean to you in this question?
What about “ALASKA NATIVE VILLAGE?”
What about “ALASKA NATIVE CORPORATION?
[IF NOT ALREADY ANSWERED] Do “TRIBE,” “VILLAGE,” AND “CORPORATION” mean the same thing to you in this question, or do they have different meanings? Do they have anything in common? Tell me more about that.
[IF NOT MENTIONED BY THE RESPONDENT] Did you think you could write down more than one tribe, village, or corporation?
[POINT TO THE (S) IN THE INSTRUCTION “PRINT ENROLLED TRIBE(S), VILLAGE(S), OR CORPORATION(S) BELOW”] Did you see the “(s)” in this instruction? What does this “(s)” mean to you in this instruction?
Can someone be enrolled in more than one tribe?
What about being enrolled in more than one village?
What about being enrolled in more than one corporation?
FORM EQ2
[POINT TO THE PHRASE “ALASKA NATIVE COUNCIL, ASSOCIATION, OR COMMUNITY” IN THE QUESTION 9 STEM]
What does “ALASKA NATIVE COUNCIL” mean to you in this question?
What about “ALASKA NATIVE ASSOCIATION?”
What about “ALASKA NATIVE COMMUNITY?
Do “COUNCIL,” “ASSOCIATION,” AND “COMMUNITY” mean the same thing to you in this question, or do they have different meanings? Tell me more about your answer.
Would having the phrase “federally recognized” in front of “Alaska Native council, association, or community” make it more clear that “council, association, or community” are meant to be about Alaska Native tribes or villages?
[POINT TO THE PHRASE “REGIONAL OR VILLAGE CORPORATION” IN THE QUESTION 10 STEM]
What does “REGIONAL OR VILLAGE CORPORATION” mean to you in this question?
[IF NOT MENTIONED BY THE RESPONDENT] Do “REGIONAL CORPORATION” and “VILLAGE CORPORATION” mean the same thing to you in this question or do they mean something different? Tell me more about your answer.
Show the respondent the words “COUNCIL,” “ASSOCIATION,” “COMMUNITY,” “TRIBE,” “VILLAGE,” AND “CORPORATION.”
Do any of these words have the same meaning? [IF YES] Which words?
Were there any other types of groups you expected to see listed with these words, but did not see?
FORM EQ3
[POINT TO THE PHRASE “ALASKA NATIVE CLAIMS SETTLEMENT ACT (ANCSA) CORPORATION SHAREHOLDER” IN THE QUESTION 10 STEM]
Have you ever heard anything about the Alaska Native Claims Settlement Act (ANCSA)?
What does “ALASKA NATIVE CLAIMS SETTLEMENT ACT (ANCSA CORPORATION” mean to you in this question?
What does “SHAREHOLDER” mean to you in this question?
Do you have to be enrolled in an ANCSA corporation to be a shareholder in the corporation?
If a person inherits shares from a different Alaska Native corporation, does that person has to enroll in that corporation before receiving those shares? What about for people who are gifted shares?
Can a person can be a shareholder of more than one regional corporation? What about more than one village corporation?
What is the difference between being enrolled in an ANCSA corporation and being a shareholder in an ANCSA corporation.
Do you use the word “enrolled” to describe being a shareholder in an ANCSA corporation? What about anyone you know?
Are there Alaska Native corporations that are not recognized by ANCSA? If yes, which ones?
E. FORM COMPARISION
SHOW ALL FORMS TO THE RESPONDENT AT THE SAME TIME.
Please take a look at all three forms. [Give a few moments for R to read the forms].
What are some of the most noticeable differences among these forms? Anything else?
[FOR INTERVIEWS WITH IMCSA RESPONDENTS, SKIP TO QUESTION 5]
Point to the phrase “ALASKA NATIVE TRIBE, VILLAGE,” on Form EQ1. Then point to the phrase “ALASKA NATIVE council, association, OR community” Form EQ2.
Do you prefer one phrase more than the other? For what reason(s)? Did these phrases help you decide how to answer the question? For what reason(s)?
Point to the phrase “A SHAREHOLDER OF ANY ALASKA NATIVE REGIONAL OR VILLAGE CORPORATION,” on Form EQ2. Then point to the phrase “a shareholder of an alaska native claims settlement act (ANCSA) corporation” on form EQ3.
Do these two phrases refer to the same kinds of groups, or do they refer to different groups? Tell me more about that.
Do you prefer one phrase more than the other? For what reason(s)?
Did these phrases help you give an answer to the question? For what reason(s)?
Is there a better way to ask about membership in an Alaska Native Corporation?
Point to the QUESTION 9 STEM on FORM EQ1 and THE QUESTION 9a and QUESTION 9b STEMS ON FORM EQ3.
Do you prefer to write the name of your tribe or village after the same question that asks you whether you are enrolled (as in in Form EQ1), or do you prefer to write your tribe or village in a separate question (as in Form EQ3)? For what reason(s)?
Which form do you think is best to capture how you self-identify? For what reason(s)?
Which form do you think is best for how to count your own group in the Census? For what reason(s)?
After the interview:
Now, I am going to ask you some questions about your answers and about the enrollment questions themselves. I am really interested in how these questions work for you, so there are no wrong answers.
Overall, would you say the questions about enrollment were easy or difficult to respond to? For what reason(s)?
How clear do you think these questions were? Tell me more about that.
How easy or difficult would these enrollment questions be if you were answering them for the other people you live with in your household?
SECTION III. DEBRIEFING QUESTIONS |
Do you have any suggestions on improving Form EQ1? What about EQ2? What about EQ3?
How helpful would it be to have a checkbox that allows people to say they’re eligible to be enrolled, but not enrolled? For what reason(s)?
Are you eligible for enrollment, but not currently enrolled, with any tribe, village, or corporation? [If R included this tribe in any response] For what reasons did you include this tribe in your response? [If R excluded this tribe from all responses] For what reasons did you exclude this tribe in your response?
Would it help to add a checkbox that allows people to answer that they don’t know or are not sure whether they are enrolled?
Is there a better way to ask whether someone is enrolled in a tribe, village, or corporation than the questions we’ve talked about?
Have you answered a question about enrollment in a tribe, village, or corporation on past surveys? (If yes), How did you answer? For what reasons? What was the survey?
Just a few final questions to wrap up –
Overall, do you think some people might find any of these questions sensitive? Which one(s)?
Do you think the Census Bureau should ask one or more enrollment questions in the next census? For what reasons?
Do you have anything else you would like to tell us that you haven’t had a chance to mention yet?
I want to thank you very much for your participation. I will now give you a $40 prepaid gift card and I will ask you to sign a receipt form as proof you received the money.
TURN OFF THE RECORDER. HAND THE MONEY INCENTIVE TO THE PARTICIPANT.
END TIME: ______: ______ AM/PM
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Rafi M. Goldberg |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |