Housing Search Process for Racial/Ethnic Minorities
Task Order GSA Contract No. C-NYC-00936-004
UI No: 08577-004-00
Paperwork Reduction Act (PRA) Submission for Housing Search Process for Racial and Ethnic Minorities Large Scale Cognitive Testing and Qualitative Interviews
Part B: Statistical Methods
July 22, 2015
Prepared for:
Judson James, PhD
Government Technical Representative
U.S. Department of Housing and Urban Development
Office of Policy Development and Research
451 Seventh St, SW
Washington, DC 20410
Prepared by:
Maria Krysan
University of Illinois Chicago
Molly M. Scott
Josh Leopold
Martha Galvez
Jessica Luna
The Urban Institute
Urban Institute
2100 M St NW
Washington, DC 20037
B1. Respondent Universe, Sample Selection, and Expected Response Rates 2
B1.3. Expected Response Rates 4
B2. Procedures for the Collection of Information 5
B3. Methods to Maximize Response Rates and to Deal with Issues of Non-response 6
B4. Pre-testing of Procedures and Methods 7
B5. Individuals or Contractors Responsible for Statistical Aspects of the Design 7
The universe for this data collection is renters who have either moved in the last 2 months or are currently moving over a limited 4-month field, from approximately September 2015 to December 2015, in the Washington D.C. metropolitan area.
The research team selected the area for this exploratory work (cognitive testing and in-depth qualitative interviews) for several reasons. First, DC is a diverse metropolitan area both in terms of racial/ethnic composition and geographic features, and has a concentration of renters (35%) similar to the nation as a whole (32%). Represented in the region are quite diverse sub-markets, some with very high foreclosure rates (e. g. Prince George’s County) and others (e.g. Arlington, D.C.) where the housing market has bounced back rather quickly. This permits us to examine patterns for the processes in different types of areas. In addition, there are efficiencies for the cognitive testing recruitment as well as the fielding of the in-depth interviews in the metropolitan area in which the project manager, the research analysts, and assistants are geographically located.
The data gathered through the cognitive tests are quantitative in nature in order to facilitate description and simple analyses of a relatively large number of tests. However, these methods are not designed to produce valid and reliable results than can be generalized to the universe of the study for multiple reasons:
This work is exploratory and meant to lay the groundwork for future HUD data collection and study, by providing insights into potential racial/ethnic differences in housing search as well as into the methods HUD should consider for detecting these differences in the general population.
There are no existing sampling frames that would allow the research team to identify the universe of renters who have either moved in the last 2 months or are currently moving over a limited 4-month field period.
Using random digit dialing to find these individuals among the general population would be prohibitively expensive given the resources allotted to this project.
Consequently, the research team plans to use a convenience sample for the cognitive tests with stratified sampling targets in place to ensure that we recruit a diverse group of respondents-- stratified by mover or searcher status, race/ethnicity, and income-- to help us accomplish the goals of this study.
The research design process-- which included extensive literature review, preliminary analyses of existing quantitative and qualitative data, and an expert panel--directly informed the stratification targets for the Housing Search Study’s cognitive tests illustrated in Exhibit 1.
Exhibit 1. Respondent Sampling Targets at time 1
|
|
Recent Movers |
Current Searchers |
||||
|
Total |
White |
African American |
Latino |
White |
African American |
Latino |
Low Income |
233 |
58 |
58 |
58 |
19 |
19 |
19 |
Medium Income |
233 |
58 |
58 |
58 |
19 |
19 |
19 |
High Income |
233 |
58 |
58 |
58 |
19 |
19 |
19 |
Total |
700 |
175 |
175 |
175 |
58 |
58 |
58 |
Note: Numbers are rounded to the ones place to sum to a total number of 700 respondents.
First, because of the difficulty that the research team anticipates recruiting participants for the cognitive tests who are actively engaged in housing search-- particularly during our expected data collection period from September to December-- we have chosen larger sampling targets for recent movers. In total, we aim to recruit 525 recent movers and 175 current searchers.
Second, within the DC metro, we found that both Latinos and Asians accounted for relatively small proportions of the renter population and that trying to include both at significant numbers would not be viable given our limited resources. In light of these data and insights from the expert panel on the likely diversity of Asian respondents, the research team decided to limit the cognitive tests to whites, African Americans, and Latinos, and dedicate the resources saved to double-down on recruitment of Latinos.
Third, information gathered during the research design process indicated strongly that differences in income may drive racial/ethnic differences in housing search behavior. In order to be able to compare cognitive tests for households of similar income, the research team chose to further stratify sampling targets using inflation-adjusted tertiles generated from the overall distribution of household income among DC renters. The research team intends to execute this strategy by asking each respondent whether his/her household income is higher or lower than two household income cut-off points, $25,000 and $65,000, which respectively, correspond roughly to the 33rd and 67th percentile of renter household incomes in the Washington, D.C. metro area based on 2008-2012 ACS data. 1
Within these basic cells, we do want to see a good amount of variation among other dimensions like age, household composition, Housing Choice Voucher status so that we will be able to explore their interplay with race/ethnicity to the extent possible, particularly among the larger sample of recent movers. However, because of the uncertainty of recruitment as well as the lack of clear information about which other factors might warrant additional stratification, the research team has opted to not construct additional strata for this exploratory work.
Nevertheless, as the data from the cognitive tests start to come in, the research team will closely monitor the other characteristics of the respondents appearing in each of our sampling target cells to ensure that participants of different races within income strata do not fall into single, limited profiles. That is, if we see that the all the low-income African American respondents we are getting are college-aged singles and all the white low-income respondents are single-mother households with children, we will cap those numbers, screen them out, and re-double recruitment efforts to try to locate low-income participants who will help balance out the sample. Similarly, the research team will monitor the recruitment methods through which participants arrive—particularly those associated with specific ways of searching for housing—to make sure that no one method is over-represented in our sample. This close monitoring is especially important for current searcher cells which are much smaller than those of recent movers.
The objective of the in-depth interviews is to better explain how and why racial/ethnic differences in the housing search process surface, but because this work is so exploratory, we do not yet know where these differences are likely to lie. For this reasons, we will examine early data on things like the length of housing search, satisfaction and ease of search, and the differences between initial search criteria and housing outcomes from the cognitive tests of both recent movers and current searchers completed by the end of the first month of data collection. These early analyses will inform purposive sampling to recruit and select a total of 48 respondents (16 White, 16 African American, and 16 Latino) for the in-depth interviews.
For example, if the preliminary data suggest differences by group in search context or outcomes (i.e., number of abandoned searches, the need for urgent moves, or dissatisfaction with move outcomes) or strategies used to find housing (use of social networks, use or avoidance of certain information sources), we can explore whether it is feasible to recruit in-depth interview respondents based on those reported experiences. Or, if it appears that recent mover and current searcher responses vary across groups in unexpected ways, we may be able to explore these emerging patterns.
In the event that preliminary analysis is inconclusive we will select households based on income and presence of children, using the preliminary data to refine appropriate income ranges and household characteristics (i.e., householder or children’s ages) for IDI selection. We anticipate selecting 8 low-income households (4 with children, 4 without children) and 8 higher-income households (4 with children, 4 without) for each race/ethnic group.
Because we are not using survey statistical methods, we do not have traditional estimated response rates but rather the sampling targets explained above.
However, current searchers will have to be interviewed at three different time points over the span of no more than 28 days and we do expect a certain amount of attrition at time 2 and time 3. No study to date has attempted to gather information about housing search from people who are actively engaged in housing search-- a process that has the potential to be disruptive and stressful-- at the time of data collection. Further, most studies follow-up with respondents over long time periods (i.e. 6 months or a year) instead of within 10-14 days. For this reason, there is no clear academic scholarship on this topic to guide our assumptions.
Because the follow-up time between interviews is fairly short— DIR will allow 2 weekends to pass between time 1 and time 2, and then between time 2 and time 3— we expect response rates among current searchers at time 2 to be fairly high, about 80% or 140 of the original 175 participants. At time 3, we expect the same 20% attrition due to non-response, and possibly another 10% due to housing searches that ended by the time 2 test, leaving us with an estimated 98 respondents at T3.
Most potential respondents interested in participating in the cognitive tests will call a toll-free phone number to communicate directly with a DIR interviewer. If the research team has difficulty meeting its recruitment targets, DIR will also call potential respondents referred by family and friends who have already participated in the HSS.
The interviewer will administer active consent procedures and assure privacy as described in detail in Section A10. The interviewer will then screen the respondent to determine eligibility, including renter status, status as a mover in the last 2 months or a current searcher, racial/ethnic identification, income, household size and composition, and recruitment source. If the potential respondent meets eligibility criteria and the sampling targets have not yet been reached for his/her corresponding cell in Exhibit 2, then the interviewer will administer the modules of the Housing Search cognitive testing study as appropriate for recent movers and current searchers. These phone tests are intended to last an average of 30 minutes.
At time 2—after two weekends have passed since the initial call, DIR interviewers will call current searchers who have consented to a follow-up call directly and administer the modules appropriate to their housing search. If a respondent has completed his/her housing search, this time 2 call will be the last cognitive test. However, if he/she is still looking at time 3, DIR interviewers will call him/her back after two more weekends have passed for a third interview. Interviewers will repeat the procedures as described in more detail above.
For the In-Depth Interviews, UI staff will call and/or email potential respondents only if they have given their consent for their contact information to be shared with UI for this purpose. Once contact is made, UI staff will set up a time to conduct the interview in person at the location of the respondent’s choosing (i.e. her home, place of work, a public library, etc.). At the scheduled time and place, UI staff will administer all consent and privacy procedures as described in Section A10. Once active consent has been obtained, UI staff will administer a semi-structured interview protocol designed to better understand the narrative around individuals’ housing search experiences and outcomes. These interviews will last no more than 1 hour and will be recorded for subsequent transcription and coding.
This study does not require estimates of response rates since it does not employ traditional survey methods. However, the research team is deploying varied intensive recruitment efforts to maximize our ability to meet our sampling targets. Our varied methods, designed to reach out to diverse populations, include:
• Online presence: UI will host a simple web page for the study that explains its purpose, HUD funding of this work, human subjects protections, incentives, the organizations implementing the study, and instructions for participation.
• Snowball sampling: All potential respondents who call in to the HSS-- both those who are screened out and those who actually participate will be encouraged to refer friends, family, or acquaintances who might be recent movers or current searchers by voluntarily sharing their contact information or alternatively sharing the study call-in number with them. HSS respondents will also receive flyers in the mail along with their incentive payment to distribute as they choose.
Methods targeted specifically to current searchers and recent movers will be used to try to minimize the cost of screening out inappropriate respondents.
• Social Media: UI staff will utilize the prominent social media platforms Facebook and Twitter to reach out to potential respondents. UI staff plan to use Facebook’s targeted advertising service – which allows targeting by geographic area, demographic profile, and residential profiles – to direct users to the study’s website. UI staff also plan to create and manage a Twitter profile for the study and will rely on relevant hashtags and direct messaging to reach out to users in the DC metropolitan area who have indicated that they are looking to move or have recently moved.
• Postcard Mailing: UI staff will create a postcard to be mailed to targeted addresses within the DC metropolitan area. The mailing list for the postcards will be complied using a web extraction program developed by UI staff to collect addresses from listing of single family homes for rent that are posed to the classified advertisement website Craigslist.
Partnerships with property management companies: UI have established relationships with a variety of different rental properties across DC, and will plan to distribute flyers to these buildings and at open houses for prospective tenants
Methods specifically targeted to Latinos, the smallest racial/ethnic group in the sample
• Community partnerships: UI staff will work to establish partnerships with a variety of different private and nonprofit organizations working closely with Latino constituents, including housing counseling agencies, community organizations, and businesses to help promote the study among their constituents through flyers, emails, newsletters, posters, Facebook postings, and personal interactions with clients. The specific strategies employed will vary by partner. UI staff will may also reach out to specialized groups online through open email mailing lists (i.e. mailing lists for Latino professionals, parents of young children), message boards, and targeted web ads.
Latino Media: UI promote the study on local Latino radio stations and in newspapers with high Hispanic readership.
General recruitment. As data collection progresses and data on participants accumulate, the UI team may adjust its recruitment strategy to focus on particular groups (race/ethnicity, household composition, household income, etc.) In addition, if general response rates are low, UI employ broad recruitment strategies, with higher costs, including:
Craigs list
Bus ads
Mass media; newspaper
Appendix A contains the language UI expects to use to promote the study on its webpage and in promotional materials.
Recent movers who participate in the housing search cognitive tests will receive a one-time $30 incentive for their participation. Current searchers will receive $30 for their initial interview, but will be promised an additional $30 for their participation at time 2. At time 3, current searchers who still had active searches at time 2 will receive an additional $30 incentive. Participants in the in-depth interviews will receive $50, since the time allotted will be double that of the cognitive tests.
Because this work is exploratory, the cognitive tests in and of themselves are tests of the viability of questions and methods for asking about housing search. That said, the research team is taking various steps to make sure the cognitive tests align with the estimates of respondent burden outlined in Part A and yield the most valuable information for this project.
During the research design process, DIR—the research partner implementing the cognitive tests—timed the modules for the recent movers and current searchers. This information, coupled with input from the GTR and the expert panel allowed the research team to cut down the draft instruments to an estimated 30-minutes per test for the recent movers and time 1 of the current searchers and 20 minutes for waves 2 and 3 of current searchers. Once OMB approves a final version of the testing instruments, DIR will program them and pilot them with research team staff to formally time each module again, ensure that skip patterns work properly, and that the flow from individual question to individual question maximizes the efficiency of the tests.
The agency responsible for receiving and approving contract deliverables is:
Office of Policy Development and Research, Program Evaluation Division
U.S. Department of Housing and Urban Development
451 Seventh St. SW, Room 8120
Washington, DC 20410
Person Responsible:
Judson James
Government Technical Representative & Social Science Analyst, Program Evaluation Division
The organization responsible for administering the Housing Search cognitive tests is:
Decision Information Resources (DIR), Inc.
2600 SW Freeway, Suite 900
Houston, Texas 77098
Person Responsible:
Pamela Wells
Vice-president, Director of Survey Operations
(832) 485-3720
The organization responsible for statistical design of data to be collected is:
University of Illinois at Chicago
1007 W Harrison St
Chicago, IL 60607
Person Responsible:
Dr. Maria Krysan
Professor of Sociology
(312) 996-5575
The organization responsible for analyzing all data to be collected is:
The Urban Institute
2100 M Street, NW
Washington, DC 20037
Person Responsible:
Molly M. Scott
Senior Research Associate
(202) 261-5888
Cognitive Testing Recruitment Language
Are you thinking about looking for new rental housing? Are you currently searching? Or did you move in the last 2 months?
Do you have 30 minutes to spare?
At the Urban Institute, a nonprofit research organization, we are currently conducting new research on how renters in the DC Metro area look for housing. We are interested in talking to people who are currently searching for rental housing or who recently searched for rental housing in the District of Columbia, Prince George’s County, Montgomery County, Arlington County, or Fairfax County.
We need your help!
Participation may include:
A 30-minute phone interview and the possibility of up to 2 additional 20-minute phone calls.
What do I receive?
Up to $90, if interviewed three times.
To learn more about participating in this study, please call:
(XXX) XXXX-XXXX
Your participation is voluntary. All of your responses will be kept private. We will never release your name or any other individually identifying information.
This project is funded by the U.S. Department of Housing and Urban Development.
1 Exact values are $25,956 for the 33rd percentile and $68,873 for the 67th percentile. We rounded down for ease of use during recruitment and because these figures include higher-income outlying counties that will not be included in this study.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Housing Search Process for Racial/Ethnic Minorities |
| Author | Luna, Jessica |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |