AIAN TEQ Cog Research REV

Generic Clearance Request

The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). The purpose of this research is to conduct cognitive interviews with respondents from the American Indian and Alaska Native (AIAN) population on the topic of a separate and specific tribal enrollment question. The research is sponsored by the Census Bureau’s Population Division (POP) and Decennial Census Management Division (DCMD).

In response to extensive discussion regarding the importance of tribal enrollment data during the 2007 American Indian and Alaska Native Tribal Consultations and a formal request from the U.S. Department of Housing and Urban Development (HUD), the Census Bureau committed to testing the feasibility of a tribal enrollment question.  The Census Bureau is currently exploring the feasibility of collecting data on tribal enrollment in a census environment through a combination of qualitative and quantitative testing.

Potentially feasible tribal enrollment questions will be developed in part through qualitative testing. The Census Bureau’s tribal enrollment cognitive testing is solely linked to the upcoming 2017 Census Test, and is not part of the current research that is being done to improve the design of questions that collect data on race/ethnicity (e.g., 2015 National Content Test), as the tribal enrollment question collects information that is based on a concept separate and distinct from race and ethnicity.

The purpose of the qualitative research we are undertaking this year on tribal enrollment is to determine which two versions of tribal enrollment questions will be quantitatively tested, using a field test in 2017 (i.e., the 2017 Census Test). The results from the 2017 Census Test research, along with continued stakeholder engagement with tribal leaders, organizations such as the National Congress of American Indians, and advisors such as the National Advisory Committee on Racial, Ethnic, and Other Populations, will inform whether or not the Census Bureau proposes to include an item on tribal enrollment in the American Community Survey or the decennial census.

As previously mentioned, the present cognitive interview research is one phase of qualitative research on the development of a separate and specific tribal enrollment question. The preceding phase consisted of 11 focus groups the Census Bureau conducted with AIAN participants in January and February of 2016. Results from the focus groups informed the development of three tribal enrollment question formats.

The first experimental format is a streamlined, one-question format that asks for American Indian tribe or Alaska Native tribe, village, and corporation enrollment in a single question (see Enclosure 1). The second experimental format is a two-question format that asks for enrollment in an American Indian tribe or Alaska Native council, association, or community in its first question, and then asks about Alaska Native regional or village corporation shareholder status in its second question (see Enclosure 2). The third experimental format is a three-question format that asks for AIAN tribe and village enrollment status in its first question, the name of the enrolled tribe or village in its second question, and Alaska Native corporation shareholder status (without mentioning “enrollment”) in its third question (see Enclosure 3).

These formats were revised after incorporating feedback from various AIAN tribal leaders and the Census Bureau’s National Advisory Committee on Race, Ethnic, and Other Populations (NAC). The NAC raised several points in their feedback to the enrollment question formats and the recruitment plan. In response, we incorporated a comment to remove the term “federally recognized” from both questions in the second question format because the term “federally recognized” could be redundant, potentially confuse to respondents, or lead to erroneous exclusion of enrolled AIAN citizens. We also incorporated another comment to develop a strategy to recruit Alaska native respondents. We tentatively incorporated a comment to replace one of the six interview sites with a second reservation, and we are currently investigating the feasibility of conducting interviews at a second reservation.

In addition, there were several NAC comments we either partially incorporated or decided not to incorporate. We partially incorporated a comment to not use the term “enrolled” to describe shareholder status in a corporation by not using the term “enrolled” in the second question format, and keeping the term “enrolled” in the first question format. The term “enrolled” was kept in the first question format to maintain its simplicity, and because recent Census Bureau focus group research with Alaska Native participants found that some participants may understand the term “enrolled” as a proxy for shareholder status. We did not incorporate a comment to avoid using the term “tribe” to describe Alaska Native tribal entities because some Alaska Natives use the term “tribe” to describe their tribal entities, and thus we plan to cognitively test the term “tribe” with other similar terms (e.g., village, association, etc.) to determine which terms respondents prefer.

We did not incorporate a comment to increase recruitment beyond 63 interviews because 63 interviews are methodologically sufficient to cognitively test three question formats. In terms of sample size, 10 to 20 respondents per format is the standard range for a cognitive test. In this case, the research team is interviewing 21 respondents per format. In addition, all 63 respondents will see all three forms as a result of the form comparison at the end of the protocol. Thus all respondents will give opinions about the three formats. For analysis, the research team will conduct traditional cognitive interview analysis that mostly comprises a content analysis of their answers to our scripted questions and spontaneous probing. The team will also analyze interviewer notes on how respondents navigated the forms to assess form navigation, and have respondents compare all three formats to see which ones they prefer. Using these analysis, sampling, and protocol plans, the research team is confident that most if not all of the relevant issues will be discovered with the 63 respondents.

Finally, the NAC’s primary recommendation was to not cognitively test the three enrollment question formats, out of concern that the tribal enrollment question would not be feasible in a census environment. We did not incorporate this recommendation so that the Census Bureau can: (a) address the previously mentioned views expressed by delegates of the 2007 AIAN Tribal Consultations and HUD, and (b) keep its commitment to explore the feasibility of a tribal enrollment question in research that ends with the 2017 Census Test.

The goal of this research is to further refine tribal enrollment question design by investigating how AIAN respondents react to elements of each of the three experimental question formats, including the question stem and response options. The research will also assess format navigation and respondents’ preferences between the three formats.

From July – August 2016, staff from the Center for Survey Measurement (CSM) will conduct 63 cognitive interviews with respondents from the following five subgroups:

(1) American Indians in federally-recognized tribes (n = 15);

(2) American Indians in state-recognized tribes (n = 15);

(3) American Indians in tribes not recognized by either the federal government nor any state government (n = 15);

(4) Alaska Natives with tribe, village, corporation, or similar identities (n = 12); and

(5) Indigenous Mexican, Central, and South American groups (n = 6)

For the first three subgroups, we plan to conduct interviews by tribe recognition status because respondents from tribes of different recognition statuses will likely have different perspectives on a tribal enrollment question administered by the federal government. For the fourth subgroup, interviews will be conducted with Alaska Native people because of their unique and complex relationships to tribes, villages, corporations, and similar entities in Alaska. For the last subgroup, interviews will be conducted with respondents from Indigenous Mexican, Central, and South American groups because they are also defined as AIAN, according to the OMB definition of the AIAN race category.

To achieve geographic diversity in tribe heritage and cognitive interview location, the cognitive interviews will be conducted in several regions identified by the Bureau of Indian Affairs, as shown in Table 1 below.

Table 1. Cognitive Interview Sites

Number	City	BIA Region
1	Fairbanks, AK	Alaska
2	Rapid City, SD	Great Plains
3	Detroit, MI	Midwest
4	Phoenix, AZ	Western
5	Kansas City, KS	Southern Plains
6	Window Rock, AZ	Navajo

Regarding respondent recruitment, we will use a recruitment contractor who has the expertise to recruit specifically from the AIAN population. This contractor will coordinate with contacts from their AIAN networks to screen and recruit AIAN respondents. The contractor will use the attached recruitment flyers, tribal publication ads, and Craigslist ads to recruit respondents (see Enclosure 4). The Census Bureau will provide the recruitment contractor staff with the attached screener questionnaires to screen potential respondents during recruitment. One screener questionnaire will be used for Alaska Natives, and another will be used for American Indians and Indigenous Mexican, Central, and South American Indians (see Enclosure 5 and Enclosure 6). Census Bureau researchers will select the final respondents, for each cognitive interview.

For all cognitive interviews, we plan to recruit respondents who are not employed by the federal government. They will be screened on and vary by:

• race (alone or in combination)

• tribe(s)

• tribal enrollment status

• whether living on or off an American Indian reservation

• level of tribal engagement

• education level

• age

• gender

Attached to this letter are mock-ups of the three experimental enrollment question formats previously described, as well as three versions of the cognitive interview protocol (see Enclosure 7, Enclosure 8, and Enclosure 9).

In each cognitive interview, respondents will complete all three experimental enrollment question formats. Interviewers will probe respondents on their experiences of completing each form, as well as their interpretations of each question stem and write-in line instructions. In addition, interviewers will probe respondents on how they navigated the formats, and their preference between the three formats. Presentation order effects will be reduced by systematically varying the order in which the forms are presented to the respondent across the three protocols. All interviews with Indigenous Mexican, Central, and South Americans will be conducted in Spanish. Thus, the test protocols and questionnaires for these respondents will be translated into Spanish.

All cognitive interviews will be audio-recorded to facilitate a summary of the results. The respondents will be asked to sign consent forms (see Enclosure 10) in English or Spanish and give permission to be audio-recorded. All respondents will be informed that their responses are voluntary and that the information they provide is confidential. After the cognitive interview concludes, respondents will receive a $40 incentive to offset the cost of participation, such as transportation or childcare costs.

For this project, we expect to screen up to 315 people to arrive at 63 respondents to invite to the cognitive interviews. Screening takes approximately five minutes per person. Thus, the maximum burden is 90 hours total: 63 hours for the cognitive interviews (one hour × 63 respondents), and 27 hours (five minutes × 315 people) for recruiting.

Below is a list of materials to be used in this study.

1. Mock-up of enrollment question format 1 (Enclosure 1)

2. Mock-up of enrollment question format 2 (Enclosure 2)

3. Mock-up of enrollment question format 3 (Enclosure 3)

4. Draft recruitment flyers and tribal publication ads (Enclosure 4)

5. Draft AIAN recruitment screener for Alaska Natives (Enclosure 5)

6. Draft AIAN recruitment screener for American Indians and Indigenous Mexican, Central, and South Americans (Enclosure 6)

7. Draft AIAN enrollment question cognitive interview protocol 1 (Enclosure 7)

8. Draft AIAN enrollment question cognitive interview protocol 2 (Enclosure 8)

9. Draft AIAN enrollment question cognitive interview protocol 3 (Enclosure 9)

10. Draft consent form for cognitive interview participation (Enclosure 10)

The contact person for questions regarding data collection and study design is:

Rodney L. Terry

Center for Survey Measurement

U.S. Census Bureau

Room 5K022E

Washington, D.C. 20233

(301) 763-5479

[email protected]

5