Impact of Housing and Services Inteventions for Homeless Families

The Impact of
Housing and
Services
Interventions
on Homeless
Families
Baseline Survey Pre-Test Report

Contract No. C-CHI-00943
Task Order: CHI-T0001

September 29, 2009

Prepared for
Paul Dornan
U.S. Department of HUD
Office of Policy Development
and Research
451 Seventh Street SW Room 8140
Washington, DC 20410

Prepared by
Debi McInnis
Michelle Wood
Abt Associates Inc.
55 Wheeler Street
Cambridge, MA 02138

Contents

Introduction .................................................................................................................................... 1
Pretest Methodology ....................................................................................................................... 1
Pretest Results ................................................................................................................................. 2
Length of Interview......................................................................................................................... 2
Detailed Comments/Recommendations .......................................................................................... 3
General Comments.................................................................................................................. 3
Section A: Pre-Shelter Housing ............................................................................................... 3
Section B: Housing Barriers .................................................................................................... 4
Section C: Housing History..................................................................................................... 4
Section D: Employment .......................................................................................................... 5
Section E: Family Composition ............................................................................................... 5
Section F: Income and Income Sources ................................................................................... 5
Section G: Adult Health .......................................................................................................... 5
Section H: Other Health (primarily Adult mental health) ......................................................... 5
Section I: Family Head Substance Use .................................................................................... 6
Section J: Social History ......................................................................................................... 6
Section K: Types of Homeless Intervention Requirements....................................................... 6
Section L: Demographics ........................................................................................................ 6
Section M: Contact Information .............................................................................................. 6
Debriefing Questions .............................................................................................................. 6

Abt Associates Inc.

Contents

i

Introduction
In August 2009, Abt Associates and AbtSRBI completed a pretest of the Baseline Survey for the
study of the Impact of Housing Services Interventions on Homeless Families. The purpose of the
pretest was to confirm the length of time needed to complete the interview and to assess the question
wording and flow. We also wanted to gather feedback about the clarity of the questions and any
concerns about specific topics addressed in the questionnaire from homeless families in similar
residential situations as respondents who will complete the interview for the study. The results of the
pretest are summarized in this document.

Pretest Methodology
Heading Home, Inc., a homeless assistance provider in Cambridge, Massachusetts assisted Abt
Associates with the survey pretest. Heading Home operates emergency shelters, transitional housing,
and permanent supportive housing programs for individuals and families throughout the Boston
metropolitan area. Heading Home recruited pretest respondents from two of their programs—the
Boston Family Shelter that provides emergency shelter, and from Homeless to Housing, a permanent
supportive housing program for families that provides scattered-site permanent housing and services
to homeless families. We completed pretest interviews with a total of nine families across the two
programs, five from Boston Family Shelter and four from Homeless to Housing. All of the
respondents interviewed were female. Two field interviewers conducted the pretests, one male and
one female. From our assessment of the pretest interviews, it appears that the gender of the
interviewer did not affect the respondent’s willingness to participate in the interview pretest.
We attempted to identify pretest respondents who are similar to future study participants. In two
respects this pretest sample differs somewhat from the study sample who will be recruited during the
evaluation. While all of the pretest respondents had been homeless, for convenience in scheduling
and completing the pretest, some of the respondents were residing in a permanent supportive housing
program. Participants eligible for PSH will not be included in the study, and the point of intake for
the study will be the emergency shelter. However, we believe that by virtue of having experienced
homelessness, these respondents provided a reasonable test of the baseline instrument question
wording, question flow, and interview length, as well as valuable feedback on the topics addressed in
the interview. In addition, for the study families will be defined as at least one parent and at least one
child age 15 or under.1 All of the pretest respondents except one (a pregnant woman with no other
children) met this criterion. The interviewer did not realize that this pretest respondent did not have
other children until midway through the interview. Since the interview was already in progress when
the interviewer discovered this, the interview was completed.
The Heading Home program staff assisted the Abt interviewers to recruit the pre-test participants.
The interviews were conducted one after the other over the course of two days, providing a test of the
procedures that may be used during the study when one or more interviews will be conducted in a
participating shelter in a given day. The method of recruitment varied somewhat across the two
1

The age restriction will be applied to ensure that there will be at least one child under the age of 18 at the
time of the follow up survey to serve as the focal child for parent interviews.

Abt Associates Inc.

Baseline Survey—Pretest Report

1

shelters. Boston Family posted a sign-up sheet inviting shelter residents to participate in the pretest
interview. The sign up sheet explained the requirements for participating, indicated that up to five
families were needed and that selection would be on a first-come first-served basis. Pre test
respondents in the Homeless to Housing were recommended to the Abt interviewers by their case
managers.

Pretest Results
Overall, interviewers reported that that pretest interviewing went smoothly. They reported that the
flow of the instrument worked very well, and most respondents did not have difficulties responding to
the questionnaire.

Length of Interview
The table below summarizes the average length of interview by section and overall for the nine
pretest respondents. The average length of time needed to complete the interview was 40 minutes,
which is consistent with the estimated length used to budget for the baseline survey and to calculate
burden estimates. The longest interview took approximately 53 minutes, and the shortest just 33
minutes. All of the time estimates include a standard estimate of 5 minutes for Section M, Contact
Information. Since the baseline instrument developed for this study uses a standard set of secondary
contact questions that have been used in many other studies, we have reliable estimates of the time
needed to complete these questions. In addition, contact information is not needed for the pretest
respondents since they will not be enrolled in the study. As a result, we did not administer the contact
information questions to the pretest respondents. The time estimates shown below also include a
short debriefing module we administered to all of the pretest respondents to solicit feedback on the
interview questions.

Section
Section A: Pre-Shelter Housing
Section B: Housing Barriers
Section C: Housing History
Section D: Employment
Section E: Family Composition
Section F: Income and Income Sources
Section G: Adult Health
Section H: Other Health
Section I: Family Head Substance Use
Section J: Social History
Section K: THI
Section L: Demographics
Debriefing Section
Section M: Contact Information
(estimated)
Total Average Length

Abt Associates Inc.

Average Time to
Administer
3
4
4
3
4
4
2
5
2
2
2
2
2

Minimum
Administration
1
2
3
1
2
2
1
3
2
1
.5
1
1

Maximum
Administration
5
5
7
9
8
5
2
6
3
3
4
2
2

5

5

5

40

33

53

Baseline Survey—Pretest Report

2

Detailed Comments/Recommendations
The interviewers reported that the interview instrument worked well. Interviewers did not encounter
serious problems with the questionnaire, perhaps in part because many of the questions have been
used in other studies to collect information from people who are homeless. Based on the results of
the pretest we do not recommend substantial revisions to the baseline instrument. We list below the
comments that interviewers provided and suggestions for improving the instructions provided to
interviewers and interview probes, and in some cases minor changes to the wording of interview
questions.
General Comments



Pretest respondents did not appear to have any problems shifting between the various
scales and different timing reference points used throughout the questionnaire. For
example, Question B1 uses a three-point response scale while H2 uses a five-point scale.
Question D1 asks about employment in past week; F2 asks about income in previous
calendar year; G1 asks about health status in the past month; H2 asks about the previous
two weeks. The interviewers believe that the probes and introductions to each question
provided sufficient explanation about the scale and timeframe referenced in each question
to avoid confusion among respondents.



The recall effort required to calculate total time homeless in Question C6 was challenging
for several of the pretest respondents. Interviewers provided assistance in calculating the
total amount of time homeless for this question.



For the most part, pretest respondents did not object to the length of the interview, nor did
they exhibit fatigue. The exception was the respondent whose interview took
approximately 53 minutes, who commented that the interview took a long time. The
number of children in the family appears to be the most important factor that influences
the length of the interview.



The pretest respondents did not appear to be concerned about responding to the sensitive
questions regarding alcohol and drug use included in Section I of the questionnaire.
Some of the respondents laughed when asked the drug and alcohol questions, and one
said she was surprised at being asked the questions, but was not offended by them. Most
simply indicated that these questions did not apply to their situation, but they were
willing to answer the questions.

Section A: Pre-Shelter Housing



Question A1 asks the respondent about her living situation prior to coming to this shelter.
In the study, respondents will be interviewed after seven days in emergency shelter, and
the emergency shelter is expected to be the first point of contact with the homeless
assistance system (unless referral to emergency shelter occurs through a centralized
intake system). In contrast, all of the pretest respondents had resided first in a hotel prior
to entering the shelter or permanent supportive housing program and were a bit confused
as to how to respond to this question, with some answering the hotel and others referring

Abt Associates Inc.

Baseline Survey—Pretest Report

3

back to their housing in their own place or with others prior to the shelter hotel. The
intent of the question is to collect the type of housing situation the respondent had prior to
entry into the emergency shelter. The interviewers recommend adding interviewer
instructions about the purpose of the question and instructions that respondents should
report on the housing situation prior to entering the emergency shelter.
Section B: Housing Barriers



Two of the pretest respondents answered yes for many of the questions in B1. Midway
through the series, they stopped to clarify whether we were asking about barriers in
general or if we were asking specifically about barriers they had experienced themselves.
When the interviewer clarified that the questions refer to specific barriers that were
relevant to the family, they asked to begin again to change some of their responses. The
interviewers thus recommend emphasizing the qualifier “for you and your family” that
appears at the end of the question, and they recommend that interviewers be instructed to
repeat the qualifier for the first three items to remind the respondent about the focus of
the question.

Section C: Housing History



Question C1 posed some problems for both the respondents and the interviewers and will
require additional review and careful training for interviewers. Although the question
asks “Just before you came to this shelter, this time, how long had you been
homeless?” all of the pretest respondents answered ‘this is my first time being homeless,’
and were confused about how to provide a length of time homeless prior to shelter entry.
When interviewers probed, one respondent said she had been homeless for one week
prior to entering the shelter. Since most of the pretest respondents spent time in a hotel
prior to being referred to the program where the interview took place, it appears they did
not count their time in the hotel as part of their current spell of homelessness. The
interviewers recommend careful training for the interviewers on this question and
additional probes to explain the question to respondents.



Several of the pretest respondents had difficulty calculating the total time they had been
homeless in their lifetime (Question C6). We will provide interviewers with a calendar
for prior years to help guide respondents’ recall.



When asked for their reactions to the questions about being homeless, one respondent
indicated that she was confused by them, primarily because she had spent a great deal of
time staying with friends and family and it was challenging to calculate the total length of
time she had been in that situation. On the other hand, another respondent indicated that
these questions were not difficult to answer, and she thought the questionnaire should
have included more questions about her experiences with homelessness. On the whole,
we do not recommend revising this section of the interview, except to add the detailed
instructions for interviewers described above.

Abt Associates Inc.

Baseline Survey—Pretest Report

4

Section D: Employment



Most pretest respondents did not have a job, and this section was completed quickly. A
few respondents who were working asked for assistance to calculate earnings and income
information.

Section E: Family Composition



The interviewers did not encounter problems administering this section of the interview.
The pretest respondents were able to respond without confusion or problems. One
interviewer noted that the questions as structured do not capture situations where children
spend some time each week in different situations. For example, a couple of the pretest
respondents said that their child(ren) stayed with grandparents on the weekends, but with
the custodial parent in the shelter or program during the week, to give the child(ren) some
time away from the shelter environment. The questions as structured do not capture this
type of special circumstance (the respondent would report that the child(ren) in these
cases reside with the parent in the shelter). This can be addressed by allowing
interviewers to provide supplemental comments about the special circumstances of
respondents that are not captured in the structured questionnaire, using an interviewer
comments form.

Section F: Income and Income Sources



Some respondents requested help to calculate income information.



Some respondents were confused as to whether the SCHIP insurance program was the
same as Mass Health or Medicaid. When the survey is administered in the study, we will
provide interviewers with local names of all public insurance programs, the TANF
program, and other assistance programs to ensure they are equipped to answer such
questions from respondents.

Section G: Adult Health



Respondents asked if Hepatitis C should be counted in G3f. We will clarify the
interviewer instructions to indicate that yes, it should be counted in G3f (problems with
my liver).



Some of the respondents commented that they wished the survey asked about some of the
health care challenges they face while homeless, such as illnesses and medication costs.

Section H: Other Health (primarily Adult mental health)



This section was one of the longer sections to administer. Some respondents seemed to
answer without problems; others appeared to default to the “not at all” issues responses.
To address this we will instruct interviewers to repeat the response categories for the first
three items to remind respondents about the possible responses.

Abt Associates Inc.

Baseline Survey—Pretest Report

5

Section I: Family Head Substance Use



I5 may need some modification. One respondent answered the question no, but then
qualified it with a yes, but the yes was because of the partner’s drinking, not the
respondent’s. We have reworded and clarified I5 to read “During the past year have you
lost friends or boy/girlfriends because of your drinking?” This question will also include
probes and instructions to interviewers to clarify that we are referring to the respondent’s
own drinking.

Section J: Social History



No problems were reported in this section.

Section K: Types of Homeless Intervention Requirements



No problems were reported in this section.

Section L: Demographics



No problems were reported in this section.

Section M: Contact Information



No problems were reported in this section.

Debriefing Questions



No problems were reported in this section.

Abt Associates Inc.

Baseline Survey—Pretest Report

6