Supporting Statement for Paperwork Reduction Act Submission–Part B
Understanding Rapid Re-housing Study
OMB Number 2528-New
November 30, 2017 - Revised
Prepared for:
Mindy Ault
Social Science Analyst
U.S. Department of Housing and Urban Development
Office of Policy Development and Research
451 7th St. SW, Room 8120
Washington, DC 20410
B.1 Potential Response Universe 2
Justification of level of accuracy 4
Unusual problems requiring specialized sampling procedures 4
Any use of periodic (less frequent than annual) data collection cycles to reduce burden 4
B.3 Maximizing Response Rates 4
B.4 Tests of Procedures or Methods 7
B.5 Statistical Consultation and Information Collection Agents 7
For the Understanding Rapid Re-housing Study web-based census, the universe of Continuums of Care (CoCs) and rapid re-housing (RRH) programs will be invited to participate.
After reviewing data collected from the web census of all RRH programs, the research team will purposively select 20 rapid re-housing programs to take part in in-depth telephone interviews. Topics of interest that will help inform the selction of these 20 programs include: use of a progressive engagement service model; location in a city with right-to-shelter laws; and receipt of VA Self-Sufficency for Homeless Families (SSVF) funding. Per HUD’s request, one of these programs will be located in the Boston area (a high cost, low vacancy rental market with right to shelter laws) and one will be located in North Carolina (a lower cost, high vacancy rental market).
The research team will subsequently recruit households for the RRH participant interviews and ethnographic work from RRH programs located in the Boston area and North Carolina.
For the CoC section of the web census, the primary respondent from each CoC will be the Collaborative Applicant. For the RRH program section of the web census, the primary respondent from each RRH program will be the Executive Director. In some instances, the research team anticipates that multiple respondents from RRH programs may be necessary to complete the census. Particularly in RRH programs, it is likely that the RRH program Executive Director will complete the census.
The primary respondent to the in-depth RRH program interview will be the Executive Director of each rapid re-housing program.
For the RRH participant interviews, the research team will recruit 28 households participating in RRH programs in the Boston area and North Carolina. Of these households, six households will be in the process of entering RRH, 16 will be currently enrolled in RRH, and six will have exited the RRH program in the last 6 months.
For the ethnographic research, the research team will ask all 16 households who participated in the currently enrolled in RRH program group to participate. This will include two follow-up interviews, quarterly check-in meetings, and site observations.
The Understanding Rapid Re-housing Study does not require a sampling plan, as the universe of CoCs and RRH programs will be invited to participate in the web census. The census invitation will be sent to the CoC leads for all 400 CoC using contact information provided by HUD. The Collaborative Applicants in each CoC will then in turn provide appropriate contact information for the estimated 2,400 RRH programs across the country. Using this contact information, the research team will administer the RRH component of the web census.
The sample for the qualitative component of the study will be respondents from 28 households: all 28 will participate in a one-time interview, and 16 of these respondents (all current RRH participants) will be invited to participate in the applied ethnographic component of the study. The objective of this research is to understand and identify themes in households’ experiences during and after RRH assistance. We will not be drawing statistical inferences from this sample.
The research team anticipates that the sample sizes of 28 one-time interviews and 16 households in the ethnographic panel will be adequate to answer the study’s research questions to reach thematic saturation. Thematic saturation means that no additional findings (themes) could be generated through additional data collection. In practice, thematic saturation is a concept that can best be assessed during and after data collection. An emerging body of research is examining when thematic saturation has been reached using various qualitative methods. An important caveat is that different research questions require different levels of thematic saturation (i.e., high-level findings or uncovering a full range of nuance). Suggested ranges for interviews are 5 to 11 for high-level findings and 16 to 26 interviews for reaching more detailed findings.1
For the one-time RRH participant interviews, the research team will recruit a mix of individuals and families in approximate proportion to their representation among all RRH participants in the existing RRH programs identified in Boston and North Carolina. These research sites represent different housing market conditions—a high-cost, low-vacancy housing market in Boston and a market in North Carolina similar to the national median vacancy rate and monthly rent. Additionally, these sites are near to the research team’s Site Directors for data collection, to enable them to collect data efficiently, including when opportunities for ethnographic data collection arise on short notice or when participants’ schedules change.
The research team’s recruitment strategy for the 28 one-time interviews will vary depending on respondents’ status in RRH program participation. The research team will identify one or more RRH providers to work with closely to identify potential respondents. In consultation with program managers, case workers, and emergency shelter staff, the research team will create recruitment processes that maximize the study’s reach to prospective participants while minimizing disruption to providers’ work. All of the programs’ current and former participants (who exited in the previous year) will be considered eligible and will be invited to participate in the study.
For households in emergency shelter, the study team will coordinate with shelter staff to incorporate information about and eligibility for the study into intake processes, including administering informed consent for interested participants, until the team enrolls three households per site.
For households who are currently receiving RRH, the research team will use an approach coordinated between case workers, who have established rapport with clients and research staff. The research team will ask case workers to present information about the study to current RRH clients during their next contact with them. This will occur either during their next regularly scheduled appointment, if it is within a month, or by contacting recipients specifically to inform them of the study opportunity. Case workers will be trained to use a standard description and have study materials prepared by the study team to share with prospective participants. Those clients who express interest in the study will complete informed consent with case workers and then be contacted by the research team Site Director to schedule a first meeting. Recruitment methods may differ slightly between the two sites in order to accommodate the differences in modalities that might be effective at each location.
All former RRH clients will be identified through a roster from the program of households who have been served over the past two years. RRH programs are directed to maintain current contact information for exited clients; however, the quality of these data is sometimes poor. This project needs to recruit only three exited households per site for one-time interviews. Because the research team will select programs with moderate to high client volumes and require a small sample size for interviews, we are confident that we will be able to use client rosters to identify former clients.
The households who are currently receiving rapid re-housing will be invited to continue to participate in the study as part of the ethnographic panel, with a goal of having at least five households per site remain in the ethnographic component of the study for the duration of the follow-up period.
The study will report on the census results using descriptive analytic techniques. Because this is a census and we anticipate a high response rate, the resulting estimates should have a high level of accuracy. However, we will conduct analysis for non-response bias to decide whether to weight the data to be representative of the universe of RRH programs.
The study will also report on the program and participant interviews using descriptive analytic techniques. The research team will not use the ethnographic research to generalize the information collected to the larger population or to draw statistical inferences.
There are no unusual problems associated with this sample. The study seeks responses from the entire population of CoCs and RRH programs rather than sampling from that population. In addition, the study team will ask rapid re-housing program staff to help recruit households to participate in the qualitative data collection (e.g., one-time participant interviews and ethnographic research).
Not applicable to this study.
HUD will introduce the study and the web census through an email message to the CoC Collaborative Applicants. HUD maintains and frequently updates a list of CoC Collaborative Applicant email addresses. Later on the same day that HUD sends out its message to all CoC Collaborative Applicants, the research team will send out a second email invitation with a link to the web census. This will include the URL for accessing the web census, the log in information, and a deadline date for completing the census. If a respondent prefers to complete the census over the phone, a toll-free number will be provided that he/she can call.
We anticipate that it will be possible to achieve a 100-percent response rate for the Survey of CoC Contacts to provide accurate contact information to administer the Survey of Rapid Re-housing Programs.
The research team anticipates that outreach and administration of the CoC portion of the web census will begin in January 2018 and will take approximately six weeks. During the initial outreach, the research team will provide each CoC with a description of the study and a list of RRH programs in its jurisdiction. Included in this initial email will be a letter of encouragement from HUD. Two weeks after the initial email, the research team will then follow up by phone with CoCs that have not yet completed the web census. The research team expects that after one month, approximately 60 and 80 percent of CoC Collaborative Applicants will complete their section of the web census. During the final two-week period, the research team will ask HUD to provide a signed letter on HUD letterhead that the study team can send to encourage non-responsive CoCs to complete the survey. If some CoCs still do not respond, the research team will attempt to identify contact information for RRH programs through other means (e.g., inquiry with the VA SSVF Program Office or HUD SNAPS Office).
Using the information collected from each CoC, the research team will then contact each RRH program and request their participation in the web census. The research team will again provide a description of the study, the survey’s URL, a user login and password required to access the web survey, and an invitation to log onto the URL to complete the survey by a deadline date. The research team also will provide a toll-free number the respondent can call if he or she prefers to complete the survey by phone; a trained interviewer will then administer the survey.
A web census offers respondents the convenience of completing the survey in stages and on their own schedule, but sometimes respondents miss or accidentally delete the email invitation. To ensure the research team achieves the targeted response rate of 80 percent, it will employ a strategic follow-up approach:
After two weeks, the research team will send an email reminder to all RRH programs that have not completed the survey, asking them to complete the survey as soon as possible;
If there is still no response, at the end of week 4, the research team will re-confirm the RRH’s contact information with the local CoC;
Between weeks 4 and 6, the research team will request that HUD communicate directly with RRH programs that have not yet responded, to encourage their participation;
Between weeks 8 and 10, the research team will conduct telephone follow-up calls with non-responders. Calls will be prioritized based on a few factors including the size of RRH program (per the HIC), tightness of its housing market, and size of its non-chronic homeless population (per the most recent PIT count available).
Using these techniques, along with encouraging emails from HUD, we expect to achieve an overall response rate of at least 80 percent for the Survey of RRH Programs. Previous experience indicates that this response rate is achievable. For example, Abt Associates recently administered a survey to public housing agencies to gather information on practices used to serve households who are homeless, and achieved a response rate of 80 percent.
Follow-up telephone calls will be completed by a survey professional that is specially trained to then call, conveying the importance of participation and “converting” those who initially are reluctant to participate. Thus, while the census will be conducted online, the research team will use one to three telephone reminders, depending on the CoC, and if needed will offer telephone administration. The research team estimates a 25 percent return rate in the first month of deployment.
It is especially important to obtain a high response rate to the web census from RRH programs operating in CoCs with the largest homeless populations. The research team will designate these programs as “high priority,” as well as programs that operate in high-cost, low vacancy rental markets. For these high-priority programs the research team will take extra steps to encourage them to respond to the web survey. In addition to the four email reminders, the team will make at least two telephone calls to the high-priority RRH programs that do not respond, including offering to complete the survey with them over the telephone. Professional interviewers with refusal conversion expertise will make these follow-up attempts with non-responsive RRH programs in the high-priority group.
The research team will keep track of the number of completed surveys. Every week, the team will check to see which RRH programs responded. The team will then produce tracking sheets identifying which programs have not. This information will enable the study team to gauge how close they are to achieving the target response rates and to identify programs that require telephone follow-up.
The ethnographic panel includes frequent and prolonged contact and multiple data collection methods that require active participation by RRH participants. Households receiving RRH assistance are highly mobile and, by definition, going through a crisis, which present challenges that make it difficult to remain in contact and committed to sustained participation in a research study. For these reasons, the research team anticipates attrition in the ethnographic sample despite several strategies planned to encourage retention.
Strategies to limit attrition will include up-to-weekly contact with participants through multiple modes (telephone, texting, email, and in-person). Many homeless or deeply poor households struggle to maintain a consistent phone number, making follow-up a significant challenge. The research team will be providing RRH participants with base model cell phones to be used for regular communication with the research team. The research team will also provide minutes and texting to each participant’s phone sufficient to cover contact with them (but provided in small increments so as not to overspend allotted credits too early in the study process).
The research team aims to have a final sample of five (out of eight) households per site complete participation through the full follow-up period, noting that there is still valuable information to be learned those households that may drop out earlier.
HUD personnel and staff from Contractor Abt Associates reviewed draft versions of the web-based census instrument. Their comments are reflected in the version of the Rapid Re-housing CoC and Program Survey included in this package.
During the review period of the request for OMB clearance, the study team plans to complete a pretest with four CoCs and four rapid re-housing programs.
HUD’s Office of Policy Development and Research will work with the contractor, Abt Associates, to conduct the proposed data collection. Mindy Ault, a Social Science Analyst in HUD’s Office of Policy Development and Research, Program Evaluation Division, serves as Government Technical Representative (GTR). Within Abt Associates, Michelle Wood and Brooke Spellman (the study’s co-Principal Investigators) and Dr. Jill Khadduri (the study’s Project Quality Advisor) contributed to the review of the CoC and RRH census survey instruments, as well as consultant Dennis Culhane.
1 Guest, Greg, Arwen Bunce, and Laura Johnson. 2006. How Many Interviews are Enough?: An Experiment with Data Saturation and Variability. Field Methods 18(1): 59-82. Namey, Emily, Greg Guest, Kevin McKenna, and Mario Chen. 2016. Evaluating Bang for the Buck: A Cost-Effectiveness Comparison Between Individual Interviews and Focus Groups Based on Thematic Saturation Levels. American Journal of Evaluation 37(3) 425-440.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Abt Single-Sided Body Template |
| Author | Lauren Dunton |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |