ASSESSMENT OF NATIVE AMERICAN, ALASKA NATIVE, AND NATIVE HAWAIIAN HOUSING NEEDS:
SUPPORTING STATEMENT FOR REQUEST FOR CLEARANCE: PART B
Revised November 7, 2013 (originally approved 10/12/2012 with amendment 5/21/2013; OMB Control Number 2528-0288)
TABLE OF CONTENTS: PART B
COLLECTION OF INFORMATION EMPLOYING
STATISTICAL METHODS
Assessment of Native American, Alaska Native, and Native Hawaiian Housing Needs: Revised for Household Survey of Native Hawaiians
B1. Respondent universe, sample selection and expected response rates |
3
|
1a. Respondent universe |
3
|
1b. Sample selection |
4
|
1c. Expected response rates |
8
|
B2. Procedures for the collection of information |
9
|
B3. Methods to maximize response rates and to deal with issues of non-response
|
21 |
B4. Pre-testing procedures and methods |
38
|
B5. Individuals or contractors responsible for statistical aspects of the design |
|
|
|
|
|
Appendix E. Memorandum of Understanding (MOU)
Appendix F. Native Hawaiian Household Survey Instrument
Appendix G. 30 Day Federal Register Notice
PART B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
Three data collection efforts employing statistical methods are being conducted. The respondent universe for each is as follows:
A major in-person household survey designed to provide national estimates of housing needs and conditions for Native Americans living in tribal areas. Household interviews will be conducted in 40 sampled AIAN tribal areas (target to complete observations and interviews in 1,280 housing units). According to the 2010 Census, there are 617 tribal areas and 967,135 individuals who list Native American only as their race.
A survey of Tribal Housing Offices and Tribally Designated Housing Entities (called the Tribe/TDHE survey). These are the organizations that administer the Indian Housing Block Grant program under NAHASDA. We will survey a target of 104 of these organizations in the national sample of tribal areas.
More in-depth in-person interviews with Tribes/TDHEs, tribal leaders, housing officials, and community leaders in 24 of the 40 tribal areas selected for the household survey.
There are four other proposed primary data collections that do not employ probability sampling and therefore were not originally included in Part B of our request. The methodology for the following primary data collection activities is discussed at the end of section B.3:
A web-based survey of tribal housing offices/TDHEs
Site visits to five urban areas with concentrations of Native American populations and telephone interviews with staff at Urban Indian Community Centers and other informed individuals in other urban areas.
Telephone interviews with lenders that originate home loans in Indian Country.
Telephone interviews concerning the assessment of native Hawaiian housing needs.
Table B-1 provides a crosswalk tying each of the seven primary data collections to research purposes and questions. Population growth since the 1996 study is an important issue for this study because this will provide a comparison to the last assessment, which predates the implementation of NAHASDA. This issue will be addressed through analysis of secondary (primarily Census) data, but we will also ask Tribal Housing Offices and TDHEs about their perceptions regarding population growth and changes in their tribal areas. Responses to these questions will provide important context for understanding how population growth affects housing needs and conditions and how tribal housing offices and NHASDA have responded to population growth in tribal areas where such growth has occurred, although such responses cannot be used to measure effects or implications.
Socioeconomic conditions are an important factor in housing needs and conditions and in the capacity of a tribe to address housing needs. Gaming is a significant economic development activity for some tribes, and one that has grown considerably since the 1996 study. Relevant study questions include: “How do housing and socioeconomic conditions vary by the presence of gaming?”; “How important is this industry compared to other economic activities?”; and “How has this changed over time?” We will use secondary data to identify tribes with gaming operations, employment rates by industry, and local unemployment rates. Perceptions pertaining to the effects of gaming will also be addressed in the interviews with tribal housing officials and TDHEs, although such responses cannot be used to measure effects or implications. Gaming operations are pertinent to changes in tribal economic, and therefore, housing conditions and may affect how tribal housing authorities/TDHEs prioritize their activities and use of NAHSDA funds and availability of other resources. Based on recent consultations with tribal leaders, we will be mindful of the fact that the main concern is economic development, of which gaming in only one components as tribes have many and varied economic development initiatives.
This revised information collection request augments the household survey to include Native Hawaiians. The Native Hawaiian survey is designed to provide representative estimates of housing needs and conditions of Native Hawaiians eligible for the homestead benefit (that is, people who are at least 50% blood quantum Native Hawaiian). The respondent universe is comprised of Native Hawaiians on the waiting list for a residential allotment in the Home Lands communities in Hawaii. Applicants on the Wait List are deemed ineligible for the sample if they live on the mainland, have invalid addresses, are deceased, or are applicants who have not yet provided full documentation indicating Native Hawaiian status. Only applicants for residential homesteads are eligible. The Applicant Wait List has 14,350 eligible applicants. Household interviews will be conducted with 500 sampled households.
Sites that will be the subjects of the in-person household survey, the survey of Tribal Housing Offices and Tribally Designated Housing Entities, and in-person interviews with Tribes/TDHEs, tribal leaders, housing officials, and community leaders have been selected via an integrated probability sample design to produce reliable national estimates. In brief, the design calls for proportional stratification by region and size (AIAN population). Within each region, tribal areas were selected with probability proportionate to size. We selected a sample of 120 tribal areas in this manner. The sample was selected from the universe of AIAN tribal areas as identified and defined by the U.S. Bureau of the Census as of 2010, consistent with applicable law. There is not a one-to-one correlation between tribal areas as defined by the Census and NAHASDA grantees because a few tribes choose not to participate or do not qualify. Also some TDHEs serve as “umbrella” organizations, and represent more than one tribe. However, we sampled based on Census tribal areas because this is the most clearly defined frame and the primary purpose of this study is to assess housing needs and conditions of Native Americans living in tribal areas.
Of the 120 tribal areas selected, 40 (with 20 back-ups) will be targeted for administration of the household survey. Then, 24 of the 40 will be selected for visits by the teams that will conduct the in-person interviews with officials (purposive sample, designed to assure that an appropriate variety of conditions will be observed). All 120 will be contacted for the administration of the Tribe/TDHE survey by telephone, with the expectation that 104 will result in complete interviews.
While we will fix the number of sites for the household survey at 40, the survey subcontractor’s1 extensive data collection experience in Indian Country suggests that we need to be concerned about levels of non- participation at the tribal level—regardless of the efforts we intend to make to encourage participation. Such instances can arise from limited tribal capacity to host researchers, inability to identify tribal partners to help collect data, tribal events that are incompatible with the data-collection schedule, or considerable difficulties gaining approval from tribal IRBs. Through our experience on other projects in tribal areas, we are very aware that there are wide disparities in the qualifications and capabilities of tribal administrative personnel. Also, administrative personnel at smaller tribes often have multiple responsibilities and, thus, a limited amount of time to respond to information requests.
For this reason, we oversampled tribal areas. Our approach to oversampling balances the need to maintain control over sample size with the importance of avoiding convenience sampling—which we believe undermines the credibility of survey results. This is critical because, left uncontrolled, the sample size (number of interviews) could swing by a sizeable number, depending on the number of reluctant tribal sites that cannot be encouraged to participate—representing a large risk to the government investment in this survey.
Accordingly, in addition to the basic 40 site sample, we also selected a reserve sample of 20 sites that can replace non-participating sites from the original group. The 40 sites and 20 alternate sites will be contacted at the start of the survey effort and informed of their status. As described below (section B.2), pre-data collection planning will be initiated with both groups; however we will be very clear that intensive efforts will be made to obtain the participation of the 40 sites before replacing any of them with a reserve site. (This is a very important point because selecting the first 40 that agree to participate among the 60, without regard to sample or reserve status, would amount to convenience sampling).
Within each region, we implicitly stratified by size of tribal area, excluding tribal areas with fewer than 150 AIAN persons. We excluded the very small sites because they account for such a small share of the overall AIAN population living on tribal land; we want to ensure that our 40-site clusters provide as much coverage and diversity as possible; and we want to restrict ourselves to sites that can yield the desired number of interviews. Based on data from the 2010 Census, this sampling strategy covers over 98 percent of the AIAN population in tribal areas. In Alaska, the strategy covers 91 percent of the population because of the preponderance of small tribal villages in Alaska; however, this should not have a substantial negative effect on our estimates as the sampling procedure is intended to obtain national estimates rather than estimates for Alaska Natives alone.
Size was determined using Census 2010 counts of American Indians and Alaska Natives. The tribal areas sampled for the survey are spread throughout the distribution of size for each region, with larger areas having a higher probability of inclusion into the sample. The resulting national estimates will reflect the housing conditions of the average person or household rather than the average tribal area.
For the household survey, we will sample 1,600 households/addresses and attempt to complete 1,280 interviews across the 40 tribes (yielding an average of 32 AIAN respondents in each tribal area). Our sample is designed to produce national estimates of AIAN households residing in tribal areas.
One complication is that the four largest tribal areas (Navajo, Cherokee, Creek, and Lumbee) are quite large: based on the 2010 census, they accounted for 40 percent of the total AIAN persons in tribal areas with over 150 AIAN persons. Allocation of the sample strictly in proportion to population would lead to interviewing 40 percent of the sample in these four areas. We believe such heavy interviewing in these four areas is neither feasible nor a good use of resources. Instead, we propose to interview 2.25 times as many persons in three of these large areas (Cherokee, Creek, and Lumbee) as are interviewed in other areas and then weight any national estimates to account for this disproportionately low sampling rate. This weighting will reduce the precision of any national estimates, but this is unavoidable as it would be inappropriate to exclude the largest tribes from the survey. Further increasing the number of interviews in these large areas would improve the precision of our national estimates.
This is a particular concern with respect to the Navajo tribal area, which is by far the largest tribal area, accounting for 17.5 percent of the AIAN population in the tribal areas covered by our study. Therefore, we have arrived at a strategy that is conditional on Navajo participation. To gain participation of the Navajo Nation, we will begin our outreach efforts early and comply with their required IRB approval process. If Navajo Nation agrees to participate, we will double the number of interviews conducted there relative to other certainty areas, slightly reducing the number of interviews in each of the non-certainty sites. If surveys cannot be conducted there, we would add an additional non-certainty site – and increase the number of interviews in each of the non-certainty sites. The results would then reflect AIAN housing in non-Navajo tribal areas.
Assuming Navajo participation, we plan to complete 125 interviews in the Navajo tribal area; 61 interviews in the other three certainty areas; and 27 interviews in the 36 non-certainty sites. The Navajo area would account for 9.7 percent of the sample as compared with its 17.5 percent population share. The three remaining certainty sites would have 4.7 percent of the sample as compared with their 6.7, 6.7 and 9.4 percent shares of the population. The non-certainty areas would account for 76 percent of the sample as compared with its 60 percent share of the population.2
The Hawaiian Homelands Applicant Wait List has both the current residence island as well as the island for which the applicant is waiting for a homestead. The sample will be selected by the residence island (i.e., the island where the applicant is currently living). Due to survey cost limitations the sample will be limited to a subset of islands. However, the sample will be stratified (through sorted systematic sampling) by the wait list island so that all five wait list islands are represented.
The target sample size is 500 total completed interviews. In order to reach that goal, the researchers need to target 625 respondents that have not opted out of the survey when initially contacted by DHHL.3 Table 1 shows the distribution of the 14,350 eligible applicants by their residence island as well as their wait list island. Table 1 shows that there is no Lanai Wait List, but 11 of the eligible applicants (0.08 percent) live on Lanai. The survey will exclude Lanai residents. Also, 84.1 percent of the applicants are on the wait list for their residence island.
Table 1. Distribution of the Eligible Applicants by Residence and Wait List Island
Residence Island |
Wait List Island |
TOTAL |
|||||
Hawaii |
Maui |
Molokai |
Lanai |
Oahu |
Kauai |
||
Hawaii |
2,702 |
90 |
18 |
0 |
149 |
27 |
2,986 |
Maui |
42 |
1,618 |
23 |
0 |
47 |
9 |
1,739 |
Molokai |
9 |
32 |
349 |
0 |
13 |
6 |
409 |
Lanai |
2 |
5 |
0 |
0 |
4 |
0 |
11 |
Oahu |
798 |
630 |
102 |
0 |
6,660 |
233 |
8,423 |
Kauai |
11 |
5 |
5 |
0 |
25 |
736 |
782 |
TOTAL |
3,564 |
2,380 |
497 |
0 |
6,898 |
1,011 |
14,350 |
Anticipated response rates for the three data collection efforts associated with the nationally representative sample of tribal areas are as follows: 80 percent for the household survey; 87 percent for the Tribe/TDHE survey, and 90 percent for the site visits.
We anticipate that the greatest challenge will be obtaining approval from each tribe to conduct the study. The 1996 survey attempted to include 36 tribal sites, with a sample size of 20 households per site. However, only 24 sites completed the household survey. In the 24 sites that did complete the household survey, individual household participation was high, indicating that once a tribe agreed to be part of the study and engaged with researchers to promote the study, individual survey response rates were high. Consequently, we plan an intensive outreach effort to achieve a high participation rate in the household survey. The sample design, along with an increase of sample sites with interviews and an increase in sample size to 1,280 households, will yield reasonably reliable national measures of housing need and conditions such as overcrowding, homelessness, adequacy of infrastructure (e.g., sanitation, electricity), types of financing available and used, and rent burden/cost—as well as information on factors likely to be associated with variations in these measures.
For the Tribe/TDHE survey, we expect high participation among tribes that have agreed to participate in the study. This is an opportunity for NAHASDA grantees to share their successes as well as their challenges and offer recommendations for improving operations under NAHASDA. We expect the survey to yield reliable estimates of housing conditions and needs, key challenges, and sources of funding leveraged.
The expected 90 percent site visit data collection is based upon past experience and our plan to implement an integrated approach to the collection of data for the 40 tribes selected for the household survey and the 24 of those tribes that also have a site visit involving in-person interviews. This will provide ample time for establishing a rapport with the site and identifying a time that is convenient for the tribe and identified respondents.
We assume an 80 percent response rate for the Native Hawaiian household survey. We anticipate that the greatest challenge will be obtaining release forms from the selected waiting list sample, followed by cooperation of all of the Native Hawaiian households sampled. Consequently, we are working very closely with DHHL to assure a coordinated and culturally appropriate outreach effort to achieve a high participation rate in the household survey. The memorandum signed by HUD and DHHL (Appendix E) outlines responsibilities of both parties and specifically mentions outreach activities.
This section describes procedures for: 1) the household survey; 2) the Tribe/TDHE survey; and 3) the site visits.
For the household survey we have detailed procedures for: a) creating the sample frame; b) site replacement; and c) survey administration.
Having selected tribal areas for the household survey, we must create the household sample frame for each tribal area. Our experience with data collection in Indian Country suggests that obtaining lists of AIAN households and their addresses on tribal lands will be challenging. We plan to use the following sources/methods for constructing the list of households/addresses that will be the universe from which the sample to be targeted for interviews will be selected:
United States Postal Service (USPS) address lists. NORC (the survey subcontractor) maintains a database containing USPS address lists for the entire country. Only city-style addresses are suitable for a sampling frame, however, so NORC’s database cannot serve as the sampling frame for all tribal areas. In particular, post office box addresses and other rural route addresses do not provide a physical location for data collection. Where analysis suggests our database of USPS addresses will account for at least 80 percent of all households in a tribal area, this is our preferred method of data collection since the coverage of the tribal area is high and the cost is minimized. However, we anticipate that many households in Indian Country may not have a registered address and might use post office boxes that would not be usable for sampling.
Tribal Membership Lists. A second option is to use tribal membership lists for sampling rather than a strict address-based sample. Many tribes maintain a regularly updated list of all households that are members of their tribe and reside within tribal area boundaries. Where such a list is maintained to high standards, is computer-based, and contains addresses, it can be the basis for sample selection. The list may be provided through highly secure means to the designated members of the research team or the tribe may draw the sample following procedures established by the research team, as described below:
Tribal Membership Lists Used by the Research Team. If a tribal list is available, is deemed to be of high-quality to ensure AI/AN coverage, and the tribe is willing to share a list, without names, with the study team, we will use this list as the sampling frame and proceed with random sampling of households. Lists provided by tribes will be treated as private data used only for research puposes and only by the researchers.
Tribal Membership Lists with Tribal Sampling. In some cases, tribal lists may be maintained to high standards and be good source for sampling, but the tribe may not wish to share the list with outside researchers. In these cases, we will work with the tribe to identify a skilled and trusted tribal member who can draw the sample for us following procedures specified by the project team.
Listing. For areas where USPS coverage is insufficient; tribal lists are outdated, incomplete, or otherwise inadequate for sampling; or tribes do not wish to provide the list or draw the sample for the research team, a listing process will be undertaken.
Full Listing. A full listing process involves field researchers visiting the sites ahead of time and, with a survey methodologist, constructing a list of every housing unit in selected census blocks. This requires an additional visit to the tribal area as well as the time spent listing the housing units, so it is the most costly option, and will be undertaken in a limited number of sites (up to eight) which are not amenable to other alternatives.
List-and-Go. The list-and-go method is a listing alternative where a trained field Interviewer, supervised by a field manager, screens for interviewing eligibility at every kth housing unit listed, where k is determined in advance by the research team. List-and-go-can be accomplished on the same trip as household interviewing.
One disadvantage of the USPS address lists, full listing, and list-and-go is that non-AIAN households who live in tribal areas are likely to be selected as a part of the sample. Where these approaches are used, it will be necessary to oversample and then ask identifying questions at the beginning of all interview visits so non-AIAN households can be screened-out expeditiously. An advantage of the tribal membership list approach is that it eliminates the need for any screening of selected households.
Now that the sample of sites has been selected, 40 sites plus 20 alternates, we are undertaking a pre-outreach reconnaissance process. We are assembling readily available information about each of the 60 sites. For example, we are obtaining information from various sources on the quality of each tribe’s membership list and on whether a USPS list is available (and if so, calculating how big a share listed addresses represent of total households). We are also finding out which of these tribes has an IRB process and about other characteristics that could influence how we might best approach them during outreach.
We will use this information to make a tentative selection of which of the alternatives described above appears to be the best for creating the household sampling frame at each of the 40 sites in the base sample.
If a USPS list is available for the tribal area and provides an acceptable level of coverage, then this list will be used for the sample frame and sampling will be conducted by the survey subcontractor.
If a tribal list is available, then we will ask if the tribe is willing to share a de-identified list with the study team. If yes, we must also ask if the tribal IRB or regulatory body approves of sharing this list. If this list is deemed to be of high-quality to ensure AIAN coverage, then we will use this list as the sampling frame and proceed with random sampling of households.
If the tribal list is of high quality but the tribe does not wish to share it with the research team, we will work with the tribe to identify a skilled and trusted tribal member who can draw the sample for us following procedures specified by the project team.
In cases where USPS lists are not adequate and a tribe elects to not share a membership list, and the tribe is unwilling or unable to work with the research team to conduct the sampling using the tribal list, then we will ask if the tribe will allow the site to be listed. If yes, then we will proceed with either full listing or list-and-go, as described above. If not, then we will renegotiate with the tribe to seek cooperation. If this effort is unsuccessful and there is no other alternative to developing the sampling frame, then we will have to replace the site with one of the replacement sites within the designated region.
As outreach proceeds, it will be possible to find out more specific and complete information on the circumstances relating to approaches to develop the sampling frame. For example, we may find that the tribe’s membership list is actually in either better or worse form than our initial information led us to believe. This may cause us to select a different approach than the one initially chosen.
Where the full listing approach appears to be the only workable approach for a particular tribal area, we will request formal permission from tribal officials during outreach to conduct that approach. The work will involve in-person enumerations with a two-person team—the Field Manager and a tribal driver—who will conduct a thorough sweep of the tribal areas to account for the locations and addresses of all dwellings. We anticipate that some tribes will provide maps of the tribal lands or Native Villages to use as a guide. During the pretest, we will explore the use of alternative methodologies, including geographic information tools such as Google maps, to verify the address lists developed for the sampling frame.
Compiling the information to develop the sampling frame for full listing will involve a three step-process. First, each large tribal area will be partitioned into tracts (based on postal address files) or “segments” of housing units (based on Census block-level data). For all but the 4 largest tribal areas with more than 2000 housing units, we will select five sub-areas of at least 300 housing units to allow for approximately 30 interviews per tribal area. For the second to fourth largest tribal areas, we will select 10 sub-areas for sampling/listing. For Navajo Nation we will select 15 to 20 sub-areas. Next, the listing sheets and maps will be generated. This activity involves preparing the listing sheets (i.e., making changes to the listing sheet generation program, generating listing sheets, and conducting quality control), editing the listing sheets and conducting quality control both before and after data entry, making the listing maps, and printing and binding of listing sheets and maps. After the listing of segments is complete, data entry of the listing sheets will occur.
Specialized software will be used to allocate the proposed sample of 1,600 potential respondents across the segments within the 40 tribes. Systematic sampling will be used to select household units within a segment. These activities will be conducted by a NORC Senior Statistician. Where tribal membership lists are used and where feasible, sampling will occur by a designated tribal representative with oversight by NORC to ensure quality control.
The information below describes the approach to the sampling frame for the Native Hawaiian household survey.
Since funds allocated to the Hawaii Household Survey preclude conducting household interviews on five islands, trade-offs must be made. Several options were developed that considered the coverage of the sample (number of islands), the mode of data collection (in-person or telephone), and the ability to provide island-specific estimates. If the sample were to include all islands except Lanai, and the interviews were distributed proportionately across islands, the distribution of interviews across the islands would be as shown in Table 2.
Table 2. Sample Sizes for Proportional Selection Using Five Island Sample
Residence Island |
Eligible Applicants |
Percentage of Applicants |
Proposed Proportional Interviews |
Hawaii |
2,986 |
20.81% |
104 |
Maui |
1,739 |
12.12% |
61 |
Molokai |
409 |
2.85% |
14 |
Lanai |
11 |
0.08% |
0 |
Oahu |
8,423 |
58.70% |
294 |
Kauai |
782 |
5.45% |
27 |
TOTAL |
14,350 |
100.00% |
500 |
This allocation is optimal for statewide wait list estimates, but is not sufficient to provide island-specific estimates. Furthermore, in order to conduct household interviews with residents of five islands, interviews for the smaller islands would be conducted by telephone, excluding enumerator observations of the respondent’s current housing conditions. After reviewing various options with DHHL, the three-island option described below was selected.
The sample will be selected from eligible applicants on only three islands (excluding Lanai). The three islands will be Oahu, Hawaii, and Molokai. This decision is based on discussions with DHHL. DHHL agrees that in-person interviews that include household enumeration will be most informative. DHHL recommended Molokai as the third island for the following reasons: it more rural and more remote (thus adding diversity to the sample); DHHL is a large landholder on this island, and it has one of the oldest homesteads. Under this scenario (shown in Table 3), Molokai receives the full set of interviews allocated to Maui, Molokai, and Kauai, as noted in Table 2 (61 + 27 + 14 = 102). By sorting the eligible applicants living on each resident island by the wait list island, the sample can include respondents from all five wait lists. In fact, the eligible applicants from the different wait lists can be sampled at different rates so that every wait list has an appropriate number of interviews. This approach reasonably represents everyone on the waiting list only to the extent that housing conditions on the selected small island are relatively similar to those on the two islands not selected. This approach does allows for estimates for the selected smaller island and the two larger islands. However, it will not capture the diversity or unique characteristics of each of the smaller islands.
Table 3. Proposed Sample Sizes for Three Island Sample
Residence Island |
Eligible Applicants |
|
|
|
Percentage to be in Sample |
Proposed Interviews |
|||
Oahu |
8,423 |
100.00% |
294 |
|
Hawaii |
2,986 |
100.00% |
104 |
|
Maui |
1,739 |
59.35% |
102
|
|
Kauai |
782 |
26.69% |
||
Molokai |
409 |
13.96% |
||
Lanai |
11 |
0.00% |
0 |
|
TOTAL |
14,350 |
|
500 |
|
Pilot Testing the Sampling Approach
Our approach to sampling will vary by site and is intended to ensure that (a) overall coverage will be greater than 85 percent; and (b) no method will be used that is estimated to yield a coverage rate of below 70 percent in any site. In-person listing will achieve a very high coverage rate, but is the most expensive option. If listing is required in more sites than our budget can sustain, we will replace one or more of the 40 selected tribal areas with one of our twenty alternates. With these guidelines, we are confident that overall coverage should be at least 85 percent. If overall coverage is below 85 percent, we will conduct a demographic analysis of non-coverage at the end of the project. In that analysis, we would compare the characteristics of households (and, to the extent available, housing) in tribal areas for which membership lists were used with the 2010 Census data for the same sites. The non-coverage analysis findings would be presented along with the primary findings of the study.
Prior to obtaining full OMB approval, we propose conducting a pilot study in 5-8 of the tribal areas for which postal lists are not available. The pilot sites would all have membership lists with reputations of decent but probably varying quality and include several tribes with which UI and NORC do not have previous relationships. For the pilot sites, we would gather sufficient information to support a good estimate of the tribal membership list coverage of AIAN population and to evaluate the feasibility of using the tribal membership lists for sampling given tribal rules as to how the lists need to be handled. Our discussions with the pilot sites will also enable us to learn about local IRB processes and pave the way for gaining their approval of this work.
Concurrently, we would: (1) make telephone/email inquiries to learn about circumstances in other tribal areas where we suspect that tribal membership lists will be the best approach (we will learn some of the basics about the lists, but not nearly as much as we will learn in the pilot sites); (2) firm up the numbers on sites where we have USPS lists and do some additional fact finding about sites where we suspect NORC on-site listing will be necessary; and (3) NORC would complete its ongoing research on the feasibility of sampling based on satellite photographs in rural areas.
In late October 2012, we will submit our full plan for implementation including our proposed approach to sample frame development and estimated coverage rates site by site. The plan will contain descriptions of all we learned in the pilot phase and how that supports the estimates we have made. The plan will include a statement of our expected overall coverage, and a description of how we would respond (e.g., choose alternative sites) if we run up against unexpected problems in implementation.
As described above, several options will be considered for creating the sampling frame and a detailed decision tree will be followed in order to minimize the need for replacement. As outreach proceeds, we may find new information suggesting that conducting the survey in a particular site is likely to be infeasible, or at least very risky. If so, it may be necessary to replace some of the initial 40 sites with alternates from the list of 20. Replacement will be done sparingly and will occur only after other options have proven infeasible. The decision to replace a site will be approved by NORC, UI, and HUD.
The plan for implementing the sample selection process in each specific site will have been refined based on the experience of the pilot, and then worked out with the tribe during the outreach process, as described above. When the team arrives at the site, the first step will be to visit the appropriate tribal office and confirm these arrangements.
With smaller tribal areas (fewer than 2,000 housing units), the team should be able to directly implement the chosen approach immediately after the visits. For larger tribal areas, however, the geographies will be divided into at least five “segments” for sampling purposes. This will save substantial costs in the listing process and it will save travel costs where the USPS list approach is used. It is recognized that it will be difficult to link tribal membership lists back to Census-defined geographical units, so segmenting will seldom be implemented where the tribal membership list approach has been selected.
In segmenting, for all but the 4 largest tribal areas with more than 2000 housing units, we will select five sub-areas of at least 300 housing units. For the second to fourth largest tribal areas, we will select 10 sub-areas for sampling/listing. For the Navajo Nation we will select 15 to 20 segments. We will consider several approaches for defining sub-areas, depending upon the type of site and the source of address information, considering both size of the geographic area and density of AIAN population. In some areas, we may consider PPS sampling of sub-areas rather than random sampling.
To select the segments for any tribal area, we first need to build a sampling frame of segments. Block level housing unit counts will be obtained from Census 2010 data. Specifically, we will use the counts from Summary File 1. To determine which blocks to include in the tribal areas, GIS layers will be used to include a tribal lands layer and a census block layer. Spatial intersection will be determined using our Geographic Information System (GIS) database. The resulting block key is composed of county_fips (i.e., zip codes), tribal tract, and block.
To build segments, we will use an algorithm to group blocks into sizes of approximately 300 housing units. The algorithm will group blocks sequentially based on the order of blocks within tracts. This will facilitate the grouping of adjacent blocks when possible. Prior to selection, segments from each tribe will be sorted to ensure that those areas selected are geographically dispersed and we may oversample segments that have high proportions of AIAN residents.
Implementation of sample selection for the Household Survey will vary depending on whether the sample is drawn by the research team (research team provides list of cases to field interviewer), the tribe (sample shared with research team to develop the list of cases for the field interviews), or through the list-and-go procedure (field Interviewer will select a random sample of households using a pre-defined sampling procedure [i.e., every kth household] developed by the research team.
Survey Procedures
As described in detail in Section B-3, extensive outreach activities are planned before the survey data collection begins. Prior to any household contact, we will contact chapter houses and/or tribal leaders, even after tribal IRB approval has been obtained, to let them know that we are about to begin data collection. Although these dates will have been discussed with tribal leaders and agreed to in advance, this confirmation is important to maintain transparency and trust. As described in Section B.3, we anticipate that one or more in-person visits will be required to some tribes, to communicate study plans, address concerns, and share information about data collection procedures.
Launching the survey at each site will involve a number of sequential tasks. First, the lists of selected households will be generated, which will be used for mailing the advance package to the household, producing case face sheets, and for interviewer assignments. Then, the following procedures will implemented for: contacting sampled households, key respondent rules, completed interview definitions, obtaining informed consent, and conducting the interviews.
Implementing the Native Hawaiian sampling process and launching the survey will involve additional steps. First, DHHL will provide the research team with an initial de-identified list with unique IDs of the families on the waiting list (approximately 14,000). From this list, the research team will draw an initial probabilistic sample of 2,400 and return the unique identifiers to DHHL. For this sample, DHHL will send out an initial mailing and will request permission to release contact information (telephone number and address). DHHL will provide NORC with a de-identified file of all who have signed the release form. This form will contain only telephone numbers and addresses and will be sent to NORC via a secure FTP server. NORC will draw the survey sample from this file. In this way, DHHL will have a record of all families that have provided release forms, but will not know which of the families that signed release forms have been selected for the sample.
HUD leadership will provide a letter explaining the purpose of the study, which will be included in the initial mailing from DHHL. The content of the letter from DHHL will emphasize the importance of the study, the study’s sponsorship by HUD, confidentiality of the data, and that participation allows each family the opportunity to “tell their story” about their housing conditions and experiences. Finally, the letter will include a toll-free 800 line for respondents to call for more information or with questions and to indicate that a $25 incentive will be provided for participation. A self-addressed stamped envelope (provided by NORC) and letter with a signature line will be provided for the applicant to return to DHHL. NORC will prepare the advance mailing materials and provide them in bulk to DHHL. NORC will also work with DHHL to customize a brochure about the survey that can be used for outreach. Then DHHL will paste address labels and mass-mail the letters. This will preserve confidentiality and guard against contact information being provided prior to release.
After allowing sufficient time for receipt of the materials, DHHL will provide the contact information for those households that have signed to schedule an appointment to conduct the in-person interview and the enumerator observation. The next step would be informed consent for an interview, and refusals could also occur at this point.
Contacting procedures. Each household selected will receive an advance package about 10 days before the field period start. The Field Interviewer will mail or hand deliver (for those with post office boxes) advance letters to all sampled households and will—assuming this information is contained in the sample file—send special advance letters to multi-unit buildings that may require additional effort to gain access. Some households have post office boxes rather than door-to-door delivery or may not have read the letter. For these cases, the interviewers will also have a supply of advance packages to give out at the door.
The advance materials emphasize the importance of the study, the study’s sponsorship by HUD and non-affiliation with the TDHE, confidentiality of the data, and that participation in the study provides for each family the opportunity to “tell their story” about their housing needs and experiences. Advance materials will include a toll-free 800 line for respondents to call for more information or with questions.
After allowing sufficient time for receipt of the materials, the Field Interviewer will telephone or visit the household to schedule an appointment to conduct the in-person interview and the enumerator observation.
Key Respondent Rules. The key respondent for the household survey will be the owner or renter of the house/apartment or his or her spouse/partner. If the owner/renter is unable to participate in an interview then we will request to speak with the person who is most knowledgeable about family housing (e.g., adult child of an elder). For alternate respondents we will collect information on his or her relationship to the owner/renter, a first name and contact number, and the reason that the owner/renter cannot participate. This protocol will be observed using the Household Screener module of the household survey.
For the Native Hawaiian survey, the key respondent will be the contact listed on the DHHL waiting list, e.g., the individual that is eligible and had applied for housing on the Home Lands.
Informed Consent Procedures. The Head of Household or alternate respondent will be asked to consent to participation in the Household Survey. As with the information materials, the content of the consent form will be tailored to different tribal research conditions. The consent form will explain the sample selection procedures; the purpose of the study; the study’s sponsorship and non-affiliation with the local TDHE or other tribal entities; the data collection procedures and types of questions to be asked; the approximate length of the survey; any discomforts, risks, or benefits associated with participation; and data confidentiality protections. The Field Interviewer will be responsible for reading the full consent form to the respondent, assuring understanding, and obtaining his/her signed approval before proceeding with the interview.
Conducting the Interview and the Enumerator Observation. The Household Survey will be conducted in-person with the identified Head of Household. In addition to gaining cooperation and obtaining informed consent, key elements of administering the Household Survey include:
Obtaining a complete roster of persons living in the household at the time of the interview
Administering the modules of the paper and pencil interview (PAPI) instrument in accordance with the procedures covered in the Field Interviewer Training, including resolving inconsistencies detected during the interview and probing to elicit more detailed responses (as appropriate)
Using the interview aides, including “show cards” listing respondent choices for specific questions in the instrument.
Upon completion of the interview in the home, the Field Interviewer will conduct the observation of housing conditions in accordance with the procedures covered in the training and tribal approvals. A completed interview will comprise responses to all modules and the enumerator observation of housing conditions.
The TDHE sample is comprised of a total of three mutually exclusive samples, each drawn with probability proportionate to size. The first consists of the tribal housing offices and/or TDHEs in the sample of 40 tribal areas selected for the household survey. The second sample consists of the TDHEs in the 20 tribal areas identified as reserve sites for the household survey. The remainder of the TDHEs to fill out the sample of 120 tribal areas will not be included in the household survey. Each of these groups will require a different approach and schedule.
The TDHEs that are tied to the household survey sample will most likely already be on our radar because of the outreach effort with the tribe for the household survey. Tribal outreach for this group will include the TDHE sample as well as the household sample. As permission is gained from each reservation, the TDHE interview will automatically be assigned to the field manager managing the household sample for the site. The TDHEs in the reserve sample will also have received some outreach related to the household survey.
For the remaining sample those tribal areas not tied to the household survey or the reserve sample for the household survey – a separate approach and schedule will be used. As permission is gained to interview the TDHE, the case to be interviewed will be assigned to a field manager based on geography and time available for this task.
Site Visit Plans: Conducting On-Site In-Person Interviews
This section describes our plan for conducting in-person interviews in 24 sites, including site selection, respondents, site visit scheduling and on-site activities.
Site Selection. No survey, or combination of surveys, can fully capture the range of tribal circumstances, priorities, relationships, and approaches with respect to addressing housing needs and conditions. The site visit interviews provide an opportunity to present a richer and more nuanced description of these issues in 24 tribal areas. To capture the greatest diversity within project resources, we will select a purposive sample of 24 sites, based on consideration of the following factors:
Size
Region
NAHSDA grantee administration (e.g., tribal housing office, TDHE for a single tribe; multi-tribe/consortia TDHE)
Need component of the IHBG formula
The sites will be selected from the 40 sites that are participating in the household survey. Initial outreach for the site visits will be conducted in tandem with the outreach for the household survey, so that if a site is replaced for the household survey, it will no longer be considered for the in-person interviews.
Respondents. Respondents will vary in each site depending upon the tribal organization and administration of the NAHASDA and other tribal housing activities. For purposes of developing discussion guides and planning our visits, we group respondents into the following general categories: Tribal Housing Office /TDHE Director, Tribal Leader, Tribal Housing Official or Other Housing Official, and Community Leader.
Scheduling Site Visits. Site visits will be planned over the same data collection period as the household survey. As part of the outreach process, those sites that have been selected for in-person interviews as well as the household survey will receive information about the on-site interviews, including the respondents we will want to interview, the estimated time needed for each interview, and the discussion topics.
For each site, we will assign a member of the project team who will be a site visitor to act as the liaison with the tribe. This person will conduct preliminary calls with the tribe to identify appropriate respondents and suggest possible site visit dates that are convenient for the tribe. Follow-up calls will be made with each respondent to confirm the best dates and schedule interview dates and times. In addition to arranging the logistical details of the visit, these calls serve two other important functions. First, they provide an opportunity to answer questions that tribal respondents may have prior to our visit, and second, they provide a chance for the study team to learn more about the tribal organization. This allows us to tailor discussion guides accordingly for each site. A confirmation of the visit and the schedule of interviews will be provided to the site in advance.
Methods to maximize response rates include: intensive outreach; use of tribal interviewers; extensive training for interviewers, including scheduling procedures and field supervisor coaching; specific approached to address barriers to cooperation; incentives for household survey respondents; and follow-up of non-response.
The purpose of the outreach effort is to build relationships and effective communication with the sampled households early in the study. This is crucial to gaining cooperation for the household survey so that the project team conducts the survey successfully and in a manner that is culturally appropriate and complies with human service protections as specified in IRB requirements.
Highest priority for our outreach efforts are with the Navajo Nation, the Cherokee Nation, the Muskogee (Creek) Nation, and the Lumbee Tribe of North Carolina. We can say with certainty that these tribes will be included in the sample, based on 2010 Census tabulations.
Outreach and engagement focuses on obtaining firm commitments to cooperate with the household survey and the 24 site visits for the case studies. To do this, we will enlist the assistance of key stakeholders (i.e., “Champions”) who can be instrumental to this effort by facilitating outreach to tribes and encouraging partnerships with the tribes to promote participation. The strategy will be jointly implemented by HUD, the Urban Institute, and its subcontractors. Key steps in the process are:
Research the IRB requirements for each tribe and review this information prior to contacting the Champions, including developing a timeline for needed actions.
HUD Assistant Secretary sends a letter to a person (or persons) designated to be a key “champion” for each of the four tribes (likely to be the ONAP Area Administrators) who will promote the benefits of participation in the study and solicit cooperation.
After the HUD letter goes out, the Urban Institute PI or Project Manager and the NORC Task Leader will call the Champions to discuss the best strategy for working with each of these tribes and jointly develop a more detailed strategy. This will specify the sequence of actions (phone calls, presentations, etc.) and specify who will be involved in each action and the target date (actions will run from the first call through negotiations with IRBs). Champions will be asked to offer advice and assistance to ensure strong tribal participation in the study.
We will reach out to each tribe selected for the household survey. For the subset of 24 tribes that are selected for the on-site visits by the research team, we will refine the outreach strategy to incorporate site visit planning and protocols as needed into the approach below. The following steps will occur:
Research the tribal history and tribal leadership for each reservation or native village selected.
Inform appropriate ONAP area offices of the tribes and Native Alaskan villages that have been selected for the study sample.
Through communications with Area Administrators and Area staff persons, obtain the list of those individuals currently serving as the tribal Chairpersons/Governors, etc., and Executive Directors of housing entities for the selected tribes/villages.
Prepare a formal letter of notification of a tribe or village’s selection for the sample from HUD. These letters can be sent to the elected chief political official of the tribe/village, as well as to the Executive Director of the housing entity, or they can be included in the initial material sent to tribes. This will include notification that the tribe has been selected to one of three conditions: (1) household survey and tribal/TDHE survey; (2) household survey, tribal/TDHE telephone survey, and on-site visits with in-person interviews, or (3) tribal/TDHE telephone survey only.
Through email or phone, confirm a tribal contact that should be sent the project information and accompanying materials.
Provide informational material to a tribally-designated contact for dissemination.
Conduct an overview presentation by Webinar for tribal leaders either by phone or in person (see item 10) that addresses:
The study and its importance/benefit to tribe
An overview of topics addressed and types of questions asked in the Household Survey, tribal/TDHE telephone survey, and in-person interviews during the on-site visit
Tribal approval to conduct the study and agreement to assist in and facilitate sample selection where appropriate
Hiring tribal members to conduct the household survey
A description of NORC's role in the project for the household survey (i.e., impartial data collector), along with NORC's Pledge of Confidentiality and Ethics Standards
Overview of field work methods for the household survey (sampling, listing, recruiting and hiring, interviewing)
Community presentations
If time allows, address key study tasks (as identified in Item 8 below)
(If needed) conduct a follow-up call at a pre-arranged time with each tribe to address key study tasks, obtain agreement, and develop an Action Plan to execute these processes: research approvals, sampling activities, preparation for fieldwork, appointment of a tribal liaison (if desired) to facilitate contact with the study team, and discussion of tribal-specific concerns.
Obtain site agreements. We will draft a memorandum that details the tribal-specific protocol for field interviewers to observe and provide tribal leaders with a copy for their reference.
Conduct on-site visits. In some cases it will be necessary to go on site to meet with tribal officials and engage in face-to-face discussion on the topics noted above. We may also be asked to make a presentation in person before a Tribal Council, the tribal IRB, a research review committee, or the community. We intend to do so in a manner most conducive to the concerns of each tribe. We will produce a set of PowerPoint slides or talking points that conveys key information about the study and the study team, and will tailor these materials, as needed. During the visits, we will also share information about the types of questions asked during the household interview. We will circulate copies of the Household Survey instrument, if requested, but will ask that they be returned at the conclusion of the meeting.
The Institutional Review Board (IRB) process is being handled on a case-by-case basis for each tribe in the sample. Some tribes do not have a research review processes, some have their own research review procedures that are less formal than Institutional Review Boards, some have their own Institutional Review Boards, and some tribes participate in IRB or research review processes through tribal consortia. As described above, we are obtaining the information necessary to follow the appropriate process for each tribe in the sample. This information will help to determine how we might best approach tribes during outreach. As some tribes will take longer to obtain tribal IRB approval or to gain cooperation, this will allow for staggered data collection and for the Field Manager to work with Field Interviewers over different time periods.
The outreach strategy for the Tribal/TDHE survey is just as important as the strategy for the household survey. The same project staff will conduct outreach for the sites with no household component. For tribes participating in the household survey component, the trained field manager who is also supervising the household efforts will be responsible for the tribal/TDHE survey outreach.
Contact will be made very early in the outreach efforts for the sites selected for both the household and TDHE interviews. The TDHE respondents will be given the full set of introductory materials presented to the tribal authorities and informed of the step by step process that led to the site approval. As a courtesy, if the site requires a personal visit before approval is given for the household data collection, the TDHE will also be visited in person.
Outreach for those sites not involved with the household survey will require a different approach for gaining cooperation. The first task will be to identify the TDHE, followed by an information gathering telephone call. Once the right person is identified, specially designed outreach materials will be emailed or sent. The materials will include an advance letter, project fact sheet and materials describing the Urban Institute and NORC. The TDHE will be then called back to have any of their questions answered, gain cooperation and set up an appointment to complete the interview. If it is deemed that an in-person visit is needed in order to gain cooperation, and a NORC field manager is within 150 miles radius, a visit will be arranged as the budget permits.
As previously noted, DHHL is a partner in this survey process, and the MOU signed by HUD and DHHL specifies that DHHL will conduct informational outreach activities to inform the public about the study (community disclosure), including informing the community through newsletters. The research team will provide a one page overview of the project and a brochure that can be used by DHHL in initial outreach as well as other materials (such as brief articles for newsletters or Frequently Asked Questions) as requested by DHHL. DHHL’s involvement will assure that the timing of outreach activities coordinates with key relevant newsletter publication dates and community gatherings, and takes advantage of web sites and media outlets that are most likely to reach the intended respondents.
Econometrica will provide substantial outreach support to NORC in working with DHHL, Native Hawaiian organizations and community stakeholders, including meetings with and or presentations for community groups and housing agencies.
To demonstrate respect for tribal sovereignty in the research process, build tribal capacity for data collection, and ensure cultural competency in conducting the household survey, the contractor will make every effort to recruit tribal interviewers within the designated tribal area. Suggestions for recruitment and approval for any hiring activities will be sought from each tribe.
Staffing of field interviewers who are right for the task at hand is essential to the success of the data collection. The contractor will do everything possible to hire two interviewers at each of the 40 reservations (one will serve as the alternate). In the event that they are unable to recruit a qualified field interviewer from the sampled tribe, contractor staff experienced that are experienced AI/AN interviewers will be used instead. Tribal interviewers will be screened, hired, and trained by the survey subcontractor.
The contractor will make every effort to recruit Hawaiian interviewers to administer the household surveys. Suggestions for recruitment and approval for any hiring activities will be sought from DHHL staff. Staffing of field interviewers who are right for the task at hand is essential to the success of the data collection. NORC plans to hire 12 Hawaiian interviewers for the 500 household surveys. In the event that they are unable to recruit a qualified field interviewer, NORC staff that are experienced with interviewing Native Hawaiians will be used instead. Interviewers will be screened, hired, and trained by NORC.
Training of field interviewers will include home study and in-person project training at a centralized location in Hawaii. Support of data collection activities and direct oversight of field interviewers will be provided by one field manager and one regional manager. The field manager will make monthly visits to oversee data collection and monitor interviewer performance. Based on NORC experience conducting in-person surveys of Native Hawaiians, it is assumed that respondents of the Household Survey may require greater efforts to locate and gain cooperation (i.e., living in remote locations, a population known to reject outsiders). It is estimated that about 10 percent of the interviews would occur by phone so that geographically dispersed respondents could be contacted.
As tribal populations are considered hard-to-enumerate, we anticipate challenges in data collection for the household survey. While cooperation, planning, and training are practical efforts that reduce the likelihood of data collection barriers, we recognize that there will be unanticipated challenges to address. We have identified seven categories of barriers to achieving the target number of interviews at each site.
Sampling. Problems could arise both during the pre-interview phase or while the interviewing is underway. In places where listing has not occurred, the field interviewers may find houses are vacant or have been deemed uninhabitable. In this event, we will work with tribal personnel from the housing office, tribal enrollment office, or natural resources/mapping office who are knowledgeable about the area.
Tribal cooperation or approval rescinded: It has to be recognized that a tribe may change its mind about participating in the survey project mid-stream. In such cases we (1) gather details on the nature of the tribe’s concerns, (2) renegotiate with the Tribal Council (through the Principal Investigator), and (3) address concerns as directed by the Tribal Council. In the event that a resolution cannot be reached, then we will replace the site with the alternate site.
Field Interviewer issues: Even with extensive recruiting, comprehensive training, and quality field management, sometimes interviewers do not work out on a project. To address this we have a number of do’s and don’ts in place as well as a range of back-up options. We will hire an alternate field interviewer at the beginning of the project, rehire and train as time and budget allow, provide extra oversight and nurturing by the Field Manager, retrain the original interviewer as needed, provide flexibility during a family crisis, send in travelers from other areas or research team interviewers who are experienced working on reservation). We avoid in most cases hiring multiple family members and ‘best friends.’ We have found that that if one does not work out, then the other one often ends up quitting as well, and thus we risk losing the whole team.
Gaining Cooperation. Approximately one week after mailing or delivery of the advance letter, the Field Interviewer will attempt telephone and in-person contact. The study protocol mandates that the initial contact is by telephone, unless information from the tribe indicated that an in-person visit was preferable or if the family did not have a phone. The telephone contact will be used as a means to schedule an interview or an in-person meeting to address any concerns that the respondent has. Field interviewers are issued photo ID badges, a project authorization letter, a confidentiality agreement, and a copy of the study’s Certificate of Confidentiality. These tools establish the study’s validity and the representative’s authority.
Respondent Issues: Field staff will be encouraged to visit each household within the first two weeks of data collection so that any concerns about the cases can be defined very early in the field period. They will be trained to be prepared to address a variety of issues that arise from non-cooperative respondents. This includes respondent refusal, gatekeeper refusal, unavailable for field period, illness, language, and avoidance, to name a few. Our approach to offset non-cooperation will be to offer refusal conversion tools (e.g., letters, calls, a visit), use a different interviewer, employ a telephone conversion team, discuss concerns with the Field Manager on weekly reporting call, and contact households early so that cases in which there is illness or a family crisis can be scheduled at a more optimal time.
Community events: We are mindful that data collection needs to be scheduled around community events and religious ceremonies. It is equally important to recognize that a tragedy, death or crisis in the family or in the community may affect data collection efforts. In such cases we will confer with the tribe about how to proceed in a respectful and responsive manner (e.g., suspending data collection for a limited period of time).
Weather and natural disasters: Weather problems should never be a surprise in field data collection efforts. Natural disasters, to include fires, earthquakes and the like, should always be considered possible. It is our practice to plan for bad weather that may occur during any season, offer weather-related solutions (such as renting a four wheel drive vehicle) to enable field interviewers to travel and get the interviews done, and create a schedule and work diligently to complete interviews before seasonal bad weather sets in. Although it is our aim to keep on schedule, in the event a natural disaster does occur, we must, in deference to the population, stop work for a reasonable amount of time.
Field Interviewers will record each attempt to contact a household. Interviewers will vary their contact attempts to the selected households across the most probable times of contact. Persistent non-contact households will be discussed with field managers; the resulting discussion will generate a new approach. Similarly, the interactions for resistant cases will be discussed and a strategy prepared. Copies of the instrument will be mailed to the field interviewers for in-person administration.
The Field Management staff will meet with the NORC team and produce a list of why respondents may refuse. Refusal letters will be developed to answer these issues. Refusal letters will be available to order from NORC by the Field Interviewer. Possible topics for refusal letters include: (1) too busy, (2) mistrust of the government, (3) confidentiality issues and (4) other topics.
At the close of both parts of the interview, the respondent will receive an incentive valued at $20. The Field Interviewer will obtain a signed receipt for this transaction. Field interviewers will receive an advance from the project to cover these incentive fees. Incentives have been shown to be effective in increasing overall response rates in all modes of surveys4 and may also reduce nonresponse bias. During the ONAP outreach sessions, tribal representatives suggested that Wal-Mart gift cards or gas vouchers would be suitable incentives in lieu of cash. This will be re-confirmed with each tribe prior to conducting household surveys for specific communities.
Below we describe the methodology for the following primary data collection activities:
A web-based survey of tribal housing offices/TDHEs
Site visits to five urban areas with concentrations of Native American populations and telephone interviews with staff at Urban Indian Community Centers and other informed individuals in other urban areas.
Telephone interviews with lenders that originate home loans in Indian Country.
Telephone interviews concerning the assessment of native Hawaiian housing needs.
Web-based survey of tribal housing offices/TDHEs
Resource limitations and the need to maintain a nationally representative sample of tribal housing offices/TDHEs limits the number of tribes that can be included in the telephone survey of tribal housing offices/TDHEs. In outreach sessions sponsored by ONAP that included discussion of this study, tribes expressed an interest in expanding participation in the survey, so that all tribes could be included. This survey is important in order to engage the tribal community and to respond to their request to have the opportunity to provide this information even if they were not selected for sample. In response to tribal interest, HUD has adapted the tribal/TDHE telephone survey for the web. The web-based survey will be administered by HUD staff. This survey will be conducted prior to the survey of a national sample of TDHEs and will serve as field test and outreach activity. Information from this survey (such as lack of variation in responses to particular questions, item nonresponse) will inform subsequent collections in this research project.
Respondent universe and identification of potential participants. The respondent universe for the web survey of TDHEs will not overlap with the TDHE telephone survey sample or the in-person interview subsample, but will be completed by tribes that were not part of the nationally representative sample. A primary purpose of this survey is to inform the telephone survey of the sampled tribal housing authorities/TDHEs and other data collection components of this study. Since inclusion in the sample of participants for the telephone survey is PPS, small tribes will have a higher probability of being eligible for the web-based survey, and this may add valuable information. In presenting findings, we do not intend to incorporate these responses into the response for the telephone survey sample. We understand the potential for selection bias in the web survey, and we will carefully consider how we report the results of the web survey. This will depend largely on the response rate achieved in the web survey. If all tribes respond, we will have achieved a census and will report accordingly, but we recognize that this is unlikely. We will emphasize to tribes that the reports of these results will be most useful and informative if participation is high.
The urban study component of the overall project will involve collecting data for 5 case study sites and 25 other sites that staff will not visit. We will draw on both primary and secondary data sources for both sets of sites; data collection for the case study sites will be more extensive.
Primary data collection efforts will focus on issues such as factors that affect residential location decisions among AIAN households living in urban areas and the challenges people face in finding and affording decent housing. Data sources will include telephone interviews with Indian Community Center (ICC) staff in select areas and HUD staff in select regional offices and at headquarters for all 30 sites. For case study sites, we will conduct in-person interviews with key stakeholders who are members of, or otherwise involved with, the AIAN community and hold small group discussions with adult AIAN community members. Key stakeholders will include: ICC staff, staff from other organizations familiar with AIAN housing conditions and needs such as service organizations and agencies, local public housing authority staff, and leaders within the AIAN community.
We will draw secondary demographic, socioeconomic, and housing data from the Census and the ACS. These data, drawn at the MSA level, will support site selection, site descriptions, and analyses of population changes since the 1996 study.
Respondent universe and identification of sites and potential participants. Using 2010 Census and ACS data, we will sort MSAs in the continental US and Alaska to identify those with the highest concentrations of AIAN populations.5 After reviewing AIAN population sizes, we will select a minimum population cutoff to capture a higher percentage of the urban AIAN population than was captured in the 1996 study (61 percent).6 We will also identify sites that have an Indian Community Center (ICC). If the list of potential sites includes more than 30 locations, the list will serve as the sample from which we will select sites for the telephone and urban case studies. Since we have not completed the analysis of Census data, we cannot estimate the respondent universe. Sample selection is purposive, so probability of selection cannot be calculated. Our purposive samples will not be used to estimate population differences between population subgroups. Furthermore, the method of selection means that we will not make inference from the sample to the population.
The first set of selection criteria will include AIAN population growth rate and proximity to tribal areas. In addition to our interest in areas with relatively large AIAN populations, we are interested in areas with growing AIAN populations that might become significant AIAN population centers in the future. We will identify those MSAs with a relatively faster growth rate for the AIAN population since 1990. Because the AIAN population is highly mobile and connected to tribal area communities, we want to include some MSAs that are proximate to these areas in order to better understand the dynamics of housing and socioeconomic patterns related to moves to and from urban areas. Additional selection criteria will include geographic region, metro size as indicated by the total population, and local economic strength as indicated by select housing market and employment activity indicators.
Ultimately, we will select study sites to achieve as diverse a set of sites as possible within the limitations posed by the small sample. From the sampling frame, we will select a total of 30 sites plus identify an additional 10 sites, if possible, to serve as replacements should the need arise. From these 30 sites we will select five for case studies that will best maximize site diversity based on the factors identified above.
Once sites are selected, we will need to secure participation agreements from ICC staff before finalizing sites for data collection. UI will contact the leader of the ICC in each of the five case study sites to explain the study and request participation as a case study site. If an ICC refuses to participate, we will replace the site with one of the sites identified for the telephone interviews, and replace the telephone interview site with a site from the replacement list. Only after we have five sites secured for the case studies will we begin the telephone interview component with the remaining 25 sites. Should we need additional replacement sites, we will select other sites based on the criteria discussed above.
Rationale for the number to be selected. The number of sites selected for telephone interviews (25) and case studies (5) was determined based on: sample size and experience of the 1996 study; understanding of the types and number of relevant stakeholders addressing housing issues of Native Americans in urban areas; and budget constraints. The population minimum for selection of MSAs in the 1996 study was set at 10,000, which led to the selection of 15 MSAs that accounted for 61 percent of the total metropolitan AIAN population. Our goal is to increase that percentage, and we determined that, within our budget, we could include up to 30 MSAs. In making this determination, we reviewed the prior study and conducted additional background research on the number of ICCs as well as the extent to which public housing developments serve Native Americans residing in urban areas. The purpose of the urban component of the data collection is to describe the housing conditions and needs of Indians residing in the selected metro areas, rather than to draw any national inferences.
Outreach, Field Agenda, and Data Collection Procedures. We plan to conduct outreach activities, which will take place over approximately 2 months’ time. We will conduct telephone interviews over the course of 2 months after confirming sites for the case study visits. We will begin the telephone interviews first and visits to the 5 case study sites will begin shortly after researchers commence the telephone interviews. Two-person research teams will carry out on-site data collection activities (interviews and discussion groups) during 3-day site visits.
The first phase of outreach will focus on securing agreement from ICCs to participate in the case study component of the project. Researchers will send a letter by postal service or email to the director of each local ICC to explain the purpose of the study and notify staff that a researcher from UI will contact them to discuss participation. If the organization is willing to participate, we will identify dates for a site visit. During the follow-up phone call, UI will solicit names of other relevant agencies and organizations that provide housing support used by local AIAN households. Stakeholders will include, in addition to ICC staff, staff of the local public housing authority, staff of other organizations identified by ICCs as being familiar with the housing conditions of AIAN households, and persons identified as leaders among the local AIAN community. Once dates are set for the visit, researchers will contact stakeholders with a letter similar to the one sent to ICC directors and then follow up by telephone to schedule interviews.
Researchers will work with ICC staff to recruit adult participants (age 18 and older) for the small group discussion in each site. UI will provide guidance for recruitment to ensure participants are diverse in housing tenure (renters, owners, homeless), age, tribal membership or affiliation, and time in the community, and are willing to discuss issues related to housing quality, access to housing, housing challenges, and factors that affect their decisions to remain in or leave urban areas. ICC staff will be asked to recruit 15 adult heads-of-households to ensure 10 to 12 actual participants. Participants will be offered an incentive of $20, either in cash or a gift card.
Outreach for the telephone-interview element of the urban study will begin similarly to that for the case study component. Once case study sites are confirmed, researchers will send a letter by postal service or email to the directors of the remaining 25 local ICCs to explain the purpose of the study and notify staff that a researcher from UI will contact them to discuss participation and, if the organization is willing to participate, identify potential dates for a telephone interview. During the follow-up phone call, UI will confirm with the director which staff members would be most appropriate to interview and ask ICC staff to identify any other entities that provide housing services to the local AIAN population in order to pursue one additional interview in each site, if appropriate. UI will contact the prospective interviewee by letter (postal or email), followed by a telephone call to schedule an interview. We will remain open to the possibility of interviewing PHA staff if an ICC suggests we talk to them or ONAP indicates that PHAs now serve more AIAN households than in the past. In such cases, the PHA interview would serve as the one additional interview.
Researchers will conduct telephone interviews with staff members of HUD headquarters and regional field offices that serve the selected metropolitan areas as part of the outreach activities. Initial contact with the offices will take place via email with telephone follow-up. Information collected during the interviews will be used to help prepare for the primary data collection activities.
Although there have been a number of changes since 1996 to increase the volume of mortgage lending on Indian Land, there are still challenges. In addition, the mortgage crisis complicated lending in any market, including Indian land. We will conduct telephone interviews with lenders that originate home loans in Indian Country to determine answers to the following research questions:
What are the major challenges associated with mortgage lending on Indian Land?
What are some of the strategies used by lenders to overcome these challenges?
How well are mortgage lending programs, such as HUD’s Section 184 Program working to increase the availability of mortgages on Indian Land?
Under what circumstances are lenders willing to originate mortgages on Indian Land?
How has mortgage lending on Indian Land been affected by the mortgage crisis?
To ensure that we receive information from lenders that originate a relatively large number of loans on Indian land, we will select a purposive sample of 30-35 lenders, including: approved Section 184 Program lenders (primary focus); tribe-owned credit unions; Native American CDFIs; and lenders using funds financed through state housing finance agency (HFA) initiatives.
Selecting lenders to be Interviewed. In principle, we could select lenders that originate mortgages on Indian Land with HMDA data. However, as documented by Manchester (2001) and Listokin, et al. (2006), HMDA does not provide good coverage of lending volume in all Tribal areas. As an alternative we detail below a method to select Section 184 Program lenders using the list of approved lenders under the Section 184 Program as a sampling frame7 and, for the other three categories (tribal credit unions, Native Community Development Financial Intermediaries – CDFIs - and state HFAs ), lenders identified through key informant interviews with people who are knowledgeable participants and observers of the Native American lending market.
The Section 184 Program list of approved lenders provides an excellent sampling frame because, by submitting an application to become an approved lender, the company has demonstrated an interest in making loans on Indian land,8 as compared to lenders who, for whatever reason are not interested in originating such mortgages. We can use data from HUD that measures Section 184 Program origination volume by lender, Tribal Area and whether a loan is for a property located on Tribal Land to identify “high-volume” lenders that originate (1) a relatively large number of loans when compared to the overall Section 184 program, to a lender’s total originations (that we can collect from HMDA if the lender reports such information) or to a lender’s total assets (as reported in June Call Reports); (2) and “low-volume” lenders, that, based on these measures, are approved Section 184 Program lenders but do not make many loans, particularly on properties that are located on Indian Land.
Using this method, we propose to interview 20 lenders that are high-volume Section 184 Program originators, and 5 low-volume lenders. This will allow us to identify the strategies used by lenders to make loans on properties located on Indian land, and remaining challenges to originate such loans, even for approved Section 184 Program lenders.
In addition to the 25 surveys of Section 184 Program approved lenders (20 high-volume and 5 low-volume), we propose to identify 3 tribe-owned credit unions, 3 Native CDFIs and 4 lenders using state HFA initiatives by first speaking to representatives of knowledgeable entities, including the CDFI Fund, the Native American Indian Housing Council, the National Credit Union Association, HUD senior Section 184 Program staff and other HUD staff (these discussions have already started), other Treasury officials knowledgeable about Native American CDFIs and senior state HFA officials in states with larger concentrations of Native American areas. In these conversations, we will ask key informants to identify lenders that originate relatively large numbers of mortgage loans on Indian Lands.
Respondent universe and selection of potential participants. In summary, the sample of lenders to be interviewed will be selected purposively. In the survey of lenders, we plan to select for interviews the 5 smallest and 20 largest of the 162 approved Section 184 lenders. Although the 25 lenders represent 25/162 of all lenders, we would not conclude that the lenders had a 25/162 probability of inclusion. Instead, all of the 20 largest lenders have a 100 percent chance of inclusion as do the 5 smallest lenders. The remaining 137 lenders have no chance of selection. There is no weighting of the sample using the inverse probability of selection that will lead to a credible statistical estimate for the entire population. The interviews will therefore not be used to make statistical estimates for the entire population of lenders. Instead, the interviews are intended to provide insight into the range of attitudes and behaviors of largest and smallest lenders based on the interviews. This sample will allow us to describe the distributions of attitudes and behaviors among the largest and smallest lenders, but it will not allow us to characterize the frequency with which such attitudes or behaviors occur among all lenders.
The lenders’ survey will also include interviews with CDFIs, tribe-owned credit unions, and lenders using state HFA initiatives obtained through word-of-mouth recommendations. Although we know the number interviewed and the number in the population, we cannot use that information to obtain population estimates or statistical properties of the information learned from the interviews. The reason is that given the non-random selection we have no way to generalize from those chosen for interview to the entire population. Care will be needed to understand how the selection process affects the information and attitudes that we would get from these interviews.
Rationale for the number of respondents to be selected. The rationale for the number of interviews planned was based on budget. We determined that the budget could support 35 interviews. Since Section 184 is the predominant product used by lenders to originate mortgages on tribal land, we decided to conduct the bulk of the interviews (25) with Section 184 lenders and interview a small number of other lenders originating mortgages on tribal trust land to obtain information on other products.
The purpose of the lender survey is purely descriptive, to update and better understand the status of home mortgage lending on Indian land. The interviews will not be used to make statistical estimates for the entire population of lenders.
Outreach and Interview Procedures. After we receive OMB approval for the survey instrument we will initiate contact, first through an email that will discuss the purpose of our study and summarize the purpose of the lender survey. We will follow-up this email with a telephone call. The initial contact will be with a person within each selected lender organization that is responsible for managing marketing and underwriting of loans originated on Indian land. For example, Wells Fargo Home Mortgage has a Native American Lending manager based in its Sioux Falls, SD office. We will identify a similar position in each lender organization selected for a survey, and, through a phone call with this person, determine the most appropriate staff members to respond to the survey. The interviewers assigned to this data collection component, two experienced researchers with experience in housing research and conducting lender interviews, will be responsible for contacting a lender, scheduling times to administer the survey and conducting the survey with the lender’s representative(s). The results of each survey will be recorded on an instrument (see the instrument package requesting OMB clearance) completed by the senior team member administering the survey.
The only direct housing funding for native Hawaiians is under the Native Hawaiian Housing Block Grant, which is administered by the Department of Hawaiian Home Lands, and they will be the focus of our data collection. Primary data collection for the Hawaii study component will consist of three semi-structured discussion guides that will be administered primarily by telephone with an option for the respondent to send a completed questionnaire to a designated e-mail address. The questions will be identical for both the telephone and e-mail versions of the questionnaire. Respondent populations for these interviews will be:
Presidents/Executive Directors of Hawaiian Homestead Associations These non-profit organizations represent the interests of the homestead households on nearly all of the state recognized home lands. These organizations promote community development efforts on their home lands as well as representing the interests of homestead households to DHHL and other stakeholders. The Presidents/Executive Directors are elected by residents. There are approximately 50 Homestead Associations and the study team intends to administer the survey instrument to the entire universe of Presidents/Executive Directors.
Managers and administrators within DDHL. This discussion guide will be administered not only to senior DHHL management but also managers in the functional divisions within the agency as well as mangers with responsibility from administering DHHL programs within various sub-regions. The study team anticipates 15 to 30 respondents for this survey.
Representatives of stakeholder organizations. These are organizations that have had extensive experience either studying the NH population or providing services to that population. The respondent population will include representatives of the aforementioned Office of Hawaiian Affairs (OHA) and research centers at the University of Hawaii. In addition, the study team will seek out other respondents including social service agencies (both state-wide and local), churches serving the NH population, community development organizations that have extensive experience working with the NH population, mortgage and financial organizations with experience working with the NH population, and foundation representatives. It is anticipated that data from these surveys will provide high quality data on the entire NH population and not just the population resident on home lands. The study projects a respondent pool of 30 to 50 individuals for this survey instrument.
Outreach and Selection of Respondents. The selection of the three respondent populations mentioned above was determined primarily by the resources currently available for the Hawaii study component. Identification of respondent populations was based on two criteria, namely:
The only direct housing funding for native Hawaiians is under the NHHBG, which is administered by the state Department of Hawaiian Home Lands (DHHL). Accordingly, the Homestead Association Directors and DHHL staff will be key respondents; and,
Respondents with detailed understanding of the housing needs of the overall NH population in the state. This led to the selection of the stakeholder respondent pool. Lacking the resources necessary to conduct any on-site data collection, this group provides the best substitute for obtaining information housing conditions and needs from the perspective of residents.
The project team has prior contacts with several of the Homestead Association Directors, senior DHHL staff, and senior staff of the state Office of Hawaiian Affairs. We will finalize the list of stakeholder respondents by building on these contacts and linkages. Association Directors, DHHL staff, and OHA staff will be asked to supply lists of stakeholder organizations meeting study criteria. Organizations appearing on all three or at least two lists will be considered as the primary candidates for inclusion in this respondent pool. The maximum number of organizations to be included in the pool will be 50. If the initial list of recommendations does not achieve the desired number of potential respondents, members of the project team will contact a number of the recommended organizations and obtain additional recommendations for the respondent pool.
Respondent universe and identification of potential participants. There are three groups of potential respondents for the telephone interviews pertaining to housing needs of native Hawaiians: 1) Interviews will be conducted with all Presidents/Executive Directors of Hawaiian Homestead Associations, so the probability of inclusion for this respondent group is 100%. 2) We also plan to include all senior managers and managers of functional divisions within DHL, as well as managers in the six DHHL sub-regions, so probability of inclusion for this respondent group is also 100%. 3) Potential respondents for the stakeholder interview will be developed from lists and recommendations provided by Homestead Association Directors, DHHL staff, and OHA staff. We do not know the total number of potential respondents and , since the selection of respondents is not random, we have no way to generalize from those chosen for interviews to the entire population. Care will be needed to understand how the selection process affects the information and attitudes that we would get from these interviews. The purpose of all of these interviews is to gain a through description of the housing needs and conditions of native Hawaiians and the programs and systems that address these needs.
Rationale for the number of respondents to be selected. The selection of the three respondent groups and the number of interviews was determined primarily by the resources available for this component of the study. Since the only direct housing funding for native Hawaiians is under the Native Hawaiian Housing Block grant, which is administered by DHHL, this group and the Homestead Association Directors, who receive NHHBG funding, were considered key respondents and we plan to contact the entire group of potential respondents. The extensive telephone data collection from stakeholders is in lieu of site visits and will be used to provide a detailed understanding of the housing needs of the native Hawaiian population in the state. The findings will be purely descriptive and will be not be used to draw statistical inferences to all native Hawaiians or all stakeholders.
The pretest procedures were designed to determine if the questionnaire, procedures, and documents work as planned in a field setting. The project team was especially interested in ensuring that time estimates accurately reflected the timing of each survey (see updated burden estimates in Supporting Statement Part A). We began outreach for pretest participants in August 2011, relying primarily on previous contacts with tribal areas established by members of the research team. Pretests took place from late August to mid-September. The number of pretests completed for each of the primary data collection instruments is summarized below:
|
Pretest Interview |
Interviews completed |
HH Survey |
6 Households (3 tribal areas) |
|
TDHE Survey |
2 TDHE Administrators (2 tribal areas) |
|
Lender Interviews |
1 lender pretest 1 expert review |
|
Urban Interviews |
2 Indian Community Center Directors |
|
Hawaii Interviews |
2 Associate Directors 2 Department of Hawaiian Home Lands Staff 2 Office of Hawaiian Affairs |
|
On-Site Interview Guide |
1 Tribal Leader |
|
On-Site Interview Guide |
2 Tribal Housing Officials/TDHEs (2 tribal areas) |
|
On-Site Interview Guide |
1 Other Community Leader |
|
The household survey was pretested in-person in three tribal areas with a total of six respondents. The TDHE/tribal officials survey was pretested by telephone in two tribal areas with a total of two respondents. Interview guides for lenders, Indian Community Center Directors, Directors of Hawaiian Home Land Associations, Department of Hawaiian Home Lands staff, and representative of other Hawaiian stakeholder groups were pretested over the telephone. Due to time and budget constraints, the on-site interview guides for tribal leaders, tribal housing officials, TDHEs, and other community leaders were also tested over the telephone. We did not pretest the web-based survey since HUD staff will be administering it [Note: HUD plans to conduct pretests of the web-based survey]. Changes made to the tribal housing officials/TDHE telephone survey instrument as a result of the pretest will be incorporated into the web-based survey instrument. Burden estimates and procedures for the urban case study group discussions are based on prior experience with similar data collection instruments and respondents.
Pretest interviewers underwent training prior to conducting the tests. For the Household Survey, field interviewers were from the communities in which households were interviewed and all interviewers except one were of Native American or Native Alaskan heritage. These field interviewers were trained as a group (all sites together on the phone). All other surveys and interviews were conducted by research team members. Training was done over the phone before pretesting began in late August.
In addition to reviewing the questionnaire and conducting a mock interview, interviewers were also trained in what to look out for during a pretest. These items included seeing if the questionnaire flowed smoothly, whether the respondents have difficulty answering any of the questions or sections, and suggestions for improving the process of gaining cooperation, and for conducting the interview.
Pretests were followed by a debriefing interview during which interviewees gave feedback about the instrument. Topics included an in-depth review of the questionnaire and all materials used to accomplish each task. Pretest interviewers documented responses and used them to write a report covering any suggested improvements and revisions to the instruments. Pretesters used their own observations about the interviewing process as well as feedback from participants in order to revise each of the survey instruments.
Overall, survey instruments were found to be well designed, easily understood, and comprehensive of the research topics. Revisions were minimal and, for most instruments, the burden on each respondent, as measured in the amount of time taken to complete the survey accurately, was similar to that estimated by the instrument authors. We increased the estimate time for the household survey and the tribal housing official/TDHE survey from 45 to 50 minutes and we eliminated a number of questions from the Urban Indian Community Center (ICC) interview guides and the other interview guides to reduce interview length. Other revisions typically involved rewording introductions and questions in order to make them clearer for the respondent. In some cases, repetitious questions were removed. The most notable challenge across all pretests was the difficulty reaching prospective interviewees and scheduling interviews. This suggests that reaching participation goals for the actual surveys will require significant efforts. Prior to data collection, the team plans to put considerable effort into developing complete lists of interviewees and accurate contact information. Revised instruments (shown with changes tracked) are included in Appendix A. In the following sections, we provide additional detail on the pretest activities and findings for each data collection component.
With respect to the interview process, household survey pretest interviewers reported that it is important to take the time to build trust and confidence in the beginning, even before the interview starts, so that the interview will go more smoothly. Overall, the reaction of the respondents to the interview process and questionnaire indicated nothing out of the ordinary. A couple of respondents, however, reacted to questions about whether the respondent lived with his/her spouse, the household composition, and income, as if they were “personal.” In the main fielding, the interviewers will receive more training on how to deal with these concerns, i.e., by explaining the purpose of these questions, and reiterating the project’s commitment to confidentiality. Additional language will be added to the training materials and questionnaire about how to handle questions perceived as sensitive by the respondent.
With respect to length, including the time spent administering the informed consent and conducting the enumerator observation, the interview time was closer to one hour than to 45 minutes. One interview took less than an hour to complete and three interviews took in excess of an hour to complete. Interviews with multi-generational households took longer to complete. One respondent stated that the interview was a bit long. The other five respondents were fully engaged to the end and thought it was an interesting experience. Two respondents mentioned that their voices, in general, are not being heard and that the household survey is a way for people to be heard. Overall, the respondents thought it was an interesting, worthwhile experience. Consequently, our suggested revisions are aimed at reducing survey administration time, particularly reducing the complexity of the household composition section. For example, we will develop a showcard with the categories of possible household occupants and some of the instructions. We are also looking into developing a single chart that can be coded to enter and reconcile the number of people in the household. With these changes we estimate the household interview can be reduced to 50 minutes.
With respect to content, we have made revisions to simplify the household composition section. Questions in the sections on Housing Unit Characteristics and Conditions, Needed Services and Amenities, and Attitudes Toward Tribally-Assisted Housing were clarified and response options were added to a number of questions.
NORC conducted one round of pretesting for the Native Hawaiian Household Survey component of the Assessment of Native American, Alaska Native and Native Hawaiian Housing Needs utilizing the paper and pencil instrument. The purpose of the pretesting, conducted June 25, 2013 through July 2, 2013, was to refine the questionnaire instrument and estimate duration of the interview. To ensure the questionnaire is appropriate for in-person interviewing, seven interviews were completed.
The pretest sample consisted of respondents that met the following criteria: Native Hawaiian living in Hawaii; not on the DHHL waiting list unless waiting for pastoral land; and not working in DHHL or a housing office. The pretest respondent contact information was provided by DHHL. As a token of appreciation for their participation, all pretest respondents received $25.
The seven interviews completed during the pretest were conducted by telephone. The pretest data collection was carried out by one NORC Field Manager and one NORC Field Interviewer who was of Native Hawaiian heritage.
Median administration time for the questionnaire
The median administration time for the seven interviews completed during this pretest was about 75 minutes. The administration times include the time spent on the start of the interview and administering the informed consent, but exclude conducting the enumerator observation. Since the pretest interviews were conducted over the telephone, showcards were not used in the administration of these interviews. The interviewers wanted to make sure the respondents heard each of the categories. It is estimated that not being able to use showcards and having to read all response options for all questions aloud added about ten minutes to the administration time of the interview, beyond what would likely be required for in-person interviews where showcards will be used. For any telephone interviews conducted during the main data collection, showcards could be mailed in advance or sent via email.
Another suggestion for reducing the interview time is training the interviewers to ‘keep on track.’ During the debriefing, the interviewers indicated that pretest respondents were very aware of their role in this pretest effort and often asked why certain questions were being asked, and also were very forthcoming with suggestions. While, this is less likely to occur in the main data collection, we will remind interviewers of the importance of minimizing respondent burden by focusing on the survey questions and avoiding other conversations. Training will include suggestions about how to balance maintaining a rapport with the respondent while discouraging questions and discussion beyond the survey requirement. This could save from 10-15 minutes of interview time without hurrying the respondent.
In addition, using shorter/simpler words as suggested below for certain questions should take off another 3-4 minutes of administration time. These suggestions are incorporated in the attached survey instruments (Appendix F) and summarized below by survey section.
With these changes and the above suggestions, we estimate the median administration time of the instrument for an in-person interview using showcards to be about 60 minutes. Interviewer experience with Native Hawaiians, as well as discussions with DHHL staff, indicate that this length of time for a survey is realistic for cultural reasons and the need for a slower pace.
Summary
of field interviewer comments and suggestions
The changes describes below have been addressed in the attached survey instrument (Appendix F).
In the interest of reducing the survey administration time, the first two questions (S1 and S2) about the respondent’s Federal Census response have been deleted. DHHL advised that respondents are unlikely to recall this information accurately, and responses to these questionnaires are not essential to the survey objectives.
Several edits have been made to Section A, Household Composition, primarily changes to skip patterns, survey format, and wording clarifications:
A1 – To begin, can you tell me how long you have lived here? – For ease of administration, move the skip (IF MORE THAN 5 YEARS SKIP TO A3) up to appear right after the response blank for number of years.
A2 – ROSTER OF RESIDENCES FROM PAST FIVE YEARS – Remove column for Line # since it is never referred to later in the questionnaire. Add more room for Street/City. Also indicate in training that we do not need exact details for this list, and that it is really for the R’s benefit as a mind jogger so the R can keep the places straight and in order.
A4 – HOUSEHOLD ROSTER – Clarify for the EMPLOYED column that the response type sought is Yes/No. Also, for the HOW LONG LIVING IN HOUSEHOLD column, clarify if it means how long the person has been in the house or how long the person has been living with the others as a household.
A6 – Correct the skip logic to A9, not A8.
In Section B, Housing Unit Characteristics and Conditions, several questions needed clarification. Comments addressing verbatim questions from the American Housing Survey (AHS) are so noted. Since the AHS questions are standard questions, the suggestions may have to be covered in training.
(AHS) B7 – Does your house/apartment have some type of cooking stove, or a range with an oven—one that is in working order? – Be sure to point out in training that this question is asking about an oven and the next question (B8) is asking about burners – and they could be on the same appliance.
B15 – How do you obtain electricity? – Add “Propane Gas” and “Solar Panels” as response options. Many Hawaiian households have propane and solar panel heating. These households are not on the grid.
(AHS) B24 – What is the main source of heat for this unit? – Add skip logic to skip out of the rest of the heating questions (SKIP to B29) if the respondent reports not having any heating equipment.
(AHS) B30 – Was that because the MAIN heating equipment broke down? – Add the option “DON’T HAVE HEATING EQUIPMENT” to the response options and have that option skip to the air conditioning subsection (SKIP to B34).
B35 – Does a heat pump/heating equipment provide air conditioning for this home? – Most of the pretest respondents did not know what a ‘heat pump’ was. Include a definition for this term.
(AHS) B42 – Does most of the water for your house/apartment come from a water system, either public or private, from an individual well, or from some other source? – The word “private” in the first response option confused some of the pretest respondents. Change the wording of the response option “Rain barrel” with a clarifier—change it to “Water catchment, i.e. rain barrel.” Also add the response option “Swimming pool.”
(AHS) B67 – Have you ever seen signs of mice, rats, insects, termites or other pests INSIDE your house/apartment? – The interviewers and respondents stated that in a tropical climate there are always some kinds of the pests referenced in the question around. The question wording should have the word “unwanted” added for clarification; for example, “Is there a problem with unwanted . . .”
Several wording changes/clarifications were suggested for Section D, Community Preferences and Section E, Culturally Responsive Housing:
D1f – (I am going to list some reasons people may have for living in a particular community. How important are these reasons to you?) Family and friends living on homestead and it is a life style that would be appealing – This question item seemed too wordy. Perhaps remove the words “and it is a lifestyle that.”
D1j – Access to other government subsidies/assistance – Include an example in the question of what is meant by this.
D1k – Beneficial assistance from DHHL – Include clarification or an example of what is meant by this.
D1l – Ideal location to raise a family – Change the word “ideal” to “good.”
D1m – Ideal location for retirement – Change the word “ideal” to “good.”
D2 – What features are you looking for in a new community? – Change the response option “Improved transportation options” to “Access to public transportation.” Add a response option “Opportunity to grow things.”
E1 – Thinking about this house/apartment, can you tell me if each of the following features of your home supports Hawaiian cultural values and practices? I am going to read each feature aloud to you. – We might want to add to the list of features: Land to grow Hawaiian plants, fruits, and flowers to preserve the culture.
E1d – Number and type of rooms, for example, for family gatherings; quilting; weaving; native crafts; sewing; tools; activities; sharing music; hula; worship; ceremonies; places for children to study; rooms for extended family, etc. – There was general agreement that this question item was much too wordy. Some thought the categories were more relevant for a community center, since these features were unrealistic for a house but desirable for a community. Some suggested moving some of these items to Section F.
Finally, in Section G, Housing Tenure and Preferences, the questions, skip patterns, and in this section need to be looked at carefully and made a bit easier to follow. For example,
G8 – Would you prefer to rent your home rather than owning? – Clarification is needed on whether the “I would prefer to own” response option should skip to G14 (the question right after the “FOR RENTERS ONLY” subsection instead of G10 as indicated. Clarification is also needed on whether the “Don’t know” or “Refused” response to this question should then skip to G14 instead of G10.
Such changes will improve data quality by helping respondents better understand what is being asked of them and improving the flow of the interview.
The pretest data collection was carried out by an experienced NORC Regional Field Manager. The same person interviewed both pre-test respondents. She thought the questionnaire was in excellent shape and will be easily understood by the respondents and interviewers. There were a few questions that need clarification.
Each survey took 60 minutes, longer that our goal of a 45-miute survey. There were recommendations about providing tools to help respondents jog their memories. In the actual survey administration, respondents will also receive the questions in advance. These steps will shorten the survey time, so we have revised our estimate to 50 minutes.
Both respondents indicated they saw only minor problems with the questionnaire. The respondents indicated that the questionnaire covered all the necessary points related to the TDHE and will be well received. Based on the pretest, we will develop a glossary of commonly-used terms and provide technical definitions that use clear, simple language to aid respondent comprehension, as well as examples. Such changes will improve data quality by helping respondents better understand what is being asked of them. As each TDHE is unique, we anticipate that technical terms may differ across entities.
Due to time and budget constraints, the on-site guides were pretested over the telephone with two tribal housing officials/TDHEs, one tribal leader, and one other community leader. For the tribal housing officials/TDHE guide, the first respondent was a TDHE Assistant Director, and the second interview was completed jointly by the Executive Director of the Tribal Housing Department and the Chief Administrative Officer for the Tribal Council. There were no difficulties in contacting these individuals or obtaining cooperation. Both interviews took 50 minutes. All respondents felt that the interview length was appropriate: it was long enough to incorporate a wide range of questions, but not so long that it was overly burdensome or inconvenient. During interviews, the order and flow of the questions was easy for the respondent to follow. Respondents suggested that having more time to prepare for the interview would be helpful (our outreach and site visit scheduling will allow for this). Based on respondent feedback, we made minor revisions to the guides to address issues of concern and eliminate redundancy.
The tribal leader interviewed for the pretest was identified for us by the TDHE Director. This individual was familiar with the study and willing to assist us with finding respondents for the various interview guides. We then contacted the tribal leader, who responded to our request promptly and an interview was scheduled. The interview lasted 45 minutes and went smoothly. The respondent was not in his office, so was not able to answer a few questions that require data. He suggested sending such questions ahead of time to allow for preparation. The order and flow of questions was easy for the respondent to follow and the questions were easily understood. He indicated that the questions were not leading or biased, and that all expected topics were covered during the interview. A small number of changes were made to in order to reduce redundancy and improve the flow of the guide.
We conducted one pretest interview of the guide to be used with a person in the tribal community who is knowledgeable about the area’s housing conditions and other service needs. The pretest interview was conducted with a leader of tribal programs for elders. The respondent was identified for us by the TDHE Director. The interview took approximately 35 minutes to complete, but the respondent had limited information about NAHASDA . If the respondent is more knowledgeable about NAHASDA, the interview could take slightly longer. Modifications to the guide were made to reduce wordiness, clarify wording of some questions, and reduce redundancy.
We were not able to recruit anyone to pretest the guide for a tribal housing official or other housing official. This guide is intended for respondents in tribal housing offices or other offices in the tribal government that are responsible for housing in some way, but are not the Tribally Designated Housing Entity for NAHASDA. Not all tribes will have such an office or person operating within the tribal government, which is likely why we were unable to identify a pretest participant. However, the questions in this guide are similar to those in the TDHE and Community Leader onsite guides, so we are able to extrapolate what we have learned from those pretest interviews to improve this instrument.
Two pretest interviews were completed with staff of Indian Community Centers. To complete the pretests, we searched for community organizations located in prospective pretest sites on the internet. Once an organization was identified, we made telephone contact using a number found on each organization’s website.
Rather than speaking first with the Executive Director, we asked the person answering the telephone to connect us with a staff member who worked with housing services or was otherwise familiar with housing issues. In some cases, this approach was successful for pretest purposes. However, during the actual data collection phase, we will first speak with the ED and gain the ED’s agreement to participate and then ask to speak with the appropriate staff member. The staff members with whom we conducted the pretests were amenable to participating.
The contact process was challenging. Among the other nine organizations contacted, one had a disconnected number, and three organizations only had a general voicemail service; we left messages but no one returned the calls. Two organizations declined to participate, citing insufficient staff and time. We identified appropriate staff at two other organizations, but were unable to make contact with them to schedule an interview. Based on this experience, we plan to obtain updated ICC contact information and allow sufficient time for scheduling interviews. We believe that our outreach process will successfully address these challenges.
The average length of the two pretest interviews was 50 minutes. One interview took 45 minutes, but the organization no longer had any housing-specific services to discuss and the person interviewed had limited familiarity with housing services offered by other organizations in the community. The other interview lasted 55 minutes, but might have extended slightly longer had the person not had limited time available. Respondents felt the guide was too long and that some information, such as the number of homeless persons, could be gathered from secondary data sources rather than by asking respondents. We have deleted a number of questions to shorten the guide, simplified wording of other questions, and improved the flow of the instrument. These changes should reduce the length of the interview considerably, now estimated at 35 minutes.
One pretest interview of the lender survey instrument was conducted with a bank representative. In order to get additional feedback, the instrument was also reviewed by Patricia Nie of Wells Fargo (a member of the Expert Panel for this project) and Thomas Wright of HUD’s Office of Native American Programs, which administers the Section 184 program.
Regarding potential changes to the methods: researchers selected and contacted seven lenders to participate in the pre-test. Of the three lenders who responded, one lender’s representative said that her company no longer originated Section 184 program loans, and so did not want to participate. The second lender reviewed the draft discussion guide, which she found on HUD’s website, and after reviewing the questions, said that because her company stopped originating Section 184 loans in 2008, she did not have current information about the program and so did not want to participate. A lower refusal rate is expected when contacting sampled lenders in the study because selection of lenders will be based on their Section 184 volume since 2008. Researchers will be able to identify lenders who make Section 184 loans in Indian land and, in addition, lenders who originate Section 184 loans but not on Indian land.
For the lender that participated in the pretest interview, the instrument took 60 minutes to complete. The respondent was able to understand all of the questions and had no suggested topics to add to the survey. At the suggestion of the other reviewers, we added questions specifically about the effect of Bureau of Indian Affairs’ policies and practices on the ability to originate mortgages on Indian land, since this greatly influences lending on Indian land. A number of questions were eliminated to reduce the length of the interview to 45 minutes.
Pretest interviews were completed with two Homestead Association Directors, one Department of Hawaiian Home Lands (DHHL) staff person, and one stakeholder. The pretest process indicated that there will be a need for considerable planning for outreach and development of respondent contact lists during actual data collection. Suggestions for pretest interviewees were provided by the project team’s established contacts in Hawaii, but it was still very difficult to reach prospective interviewees and schedule interviews. We will address this challenge by developing lists of interviewees and assuring the we have accurate contact information, and through outreach to let key informants know about the schedule, purposes, and objectives of the study.
The interviews for both the Association Director and stakeholder questionnaires took 45 minutes, while the interview for the DHHL staff lasted 25 minutes. Respondents did not suggest any changes regarding the terminology used in the three questionnaires, the flow of the questions, or the relevancy of the questions.
Respondents recommended that our interview guides take into account the fact that the quality of housing can vary by home land. For the stakeholder guide, respondents suggested asking questions regarding the “demand side” of housing choice as well as the affordability and availability of the supply. For the Association Directors questionnaire, respondents suggested asking questions concerning the amount of home maintenance services the Associations are providing or should provide. They indicated that this is important because respondents indicated that homestead residents have little economic incentive to maintain their homes, and the poor quality of home maintenance is an important issue on the home lands. Native Home Lands Association Questionnaire and the Stakeholder Organization Questionnaire were modified accordingly. No changes were made to the DHHLS staff interview guide.
The agency responsible for receiving and approving contract deliverables is:
Office of Policy Development and Research (PD&R)
U.S. Department of Housing and Urban Development
451 Seventh Street, SW
Washington, DC 20401
Person Responsible:
Paul Joice
202-402-4608
The organization responsible for administering the household survey and the Tribal TDHE telephone survey is:
NORC at the University of Chicago
4350 East West Highway
Bethesda, MD 20814
Persons Responsible:
Dr. Carol Hafford, Senior Research Scientist 301-634-9491
|
Steven Pedlow, Senior Statistician 312-759-4084
|
The organization responsible for statistical design of data that will be collected is:
The Urban Institute
2100 M Street, NW
Washington, DC 20037
Persons Responsible:
Dr. Doug Wissoker, Senior Statistician 202- 261-5622 |
Mr. Rob Santos, Senior Institute Methodologist 202-261-5904 |
The organizations responsible for analyzing all data to be collected are:
The Urban Institute
2100 M Street, NW
Washington, DC 20037
Persons Responsible:
Mr. Tom Kingsley, Principal Investigator 202- 261-5585 |
Ms. Nancy Pindus, Project Manager 202-1261-5523 |
and Econometrica, Inc.
7475 Wisconsin Ave., Suite 1000
Bethesda, MD 20814
Person Responsible:
Mr. Charles Hanson
240-333-4807
1 NORC at the University of Chicago
2 The analyses based on the household survey will be weighted to offset the under-representation of the population in the largest tribes in our sample. As noted in the text, the largest four tribes represent roughly 40 percent of the population, but only 24 percent of the sample. To adjust for this, we will develop a weight based on population and sample shares that ensures that the households from each of the certainty areas are given their full population share. In addition, we anticipate some weighting to adjust for non-response, especially that resulting when sampled sites are replaced.
3 DHHL cannot share contact information of applicants without their permission and requires that individuals “opt-in” meaning that they affirmatively agree to allow DHHL release their contact information for sampling purposes. The Hawaii Attorney General is reviewing language for a release form to be included in a mailing to applicants
4 Singer, Eleanor (2002). “The Use of Incentives to Reduce Nonresponse in Household Surveys.” In Survey Nonresponse, eds. Robert M. Groves, Dan A. Dillmon, John L. Eltinge, and Roderick J.A. Little. p. 163-77.
5 The urban study will not include Native Hawaiians living in Hawaii or in the continental US because of the separate, Native Hawaiian component of the study.
6 The population minimum for that study was set at 10,000, which led to the selection of 15 MSAs that accounted for 61 percent of the total metropolitan AIAN population.
7 The list of currently approved lenders is available at http://portal.hud.gov/hudportal/HUD?src=/program_offices/public_indian_housing/ih/homeownership/184/lender_list. Last accessed, March 31, 2011.
8 To be approved as a Section 184 Program lender, an organization must submit the following to ONAP's Office of Loan Guarantee (OLG) for review: (1) evidence of federal approval, supervision, regulation or insurance. For example, all HUD/FHA approved lenders must submit the HUD/FHA lender identification number for each approved branch office that will originate Section 184 loans; (2) address, telephone number and name of the branch manager for each branch office that will originate Section 184 loans. HUD/FHA approved lenders must provide the HUD/FHA lending area for each branch office; (3) if a HUD/FHA correspondent/sponsor relationship exists, provide a copy of the HUD/FHA approval for the relationship; including the sponsor's lender identification number and (4) the name and location of the entity that will service the Section 184 loans must be submitted for review. See: http://home.pon.net/kashayapomo/hayu/_6949258/184loanguarantee.html. Last accessed March 31, 2011.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Table of Contents |
| Author | npindus |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |