VHPD_OMB_Package_Part_A 8-22-2012

This supporting statement provides detailed information on proposed data collection activities associated with the Veterans Homelessness Prevention Demonstration (VHPD) Evaluation administered by the U.S. Department of Housing and Urban Development (HUD).

A1. Circumstances that make the collection of information necessary

The purpose of this study is to conduct the congressionally mandated evaluation component of the Veterans Homelessness Prevention Demonstration (VHPD). The FY 2009 budget for the U.S. Department of Housing and Urban Development (H.R. 1105, became Public Law 111-8, Omnibus Appropriations Act) included a $10 million set-aside “to conduct a demonstration program on the prevention of homelessness among the Nation’s veterans.” The law allows for up to $750,000 of the funding to be used for an evaluation of the demonstration. Senate Report No. 110-418 elaborated that the demonstration was to “test the effectiveness of strategies to prevent veterans from becoming homeless.” The Senate Report also directs the VA to dedicate at least $5 million for caseworkers for the demonstration program. Combined, these directives became the Veterans Homelessness Prevention Demonstration (VHPD), which were enacted in the 2009 Omnibus Appropriations Act.

The law specifies that the HUD Secretary work with the VA and DOL to select both urban and rural sites for the demonstration. As directed by PL 111-8, the VA, in collaboration with HUD and DOL, selected five military bases and their surrounding communities to participate in the demonstration.

The VA, in collaboration with HUD and DOL, selected five military bases and their surrounding communities to participate in the demonstration. Camp Pendleton in San Diego, CA; Fort Hood in Killeen, TX; Fort Drum in Watertown, NY; Joint Base Lewis-McChord in Tacoma, WA; and MacDill Air Force Base in Tampa, FL. HUD demonstration funds were allocated directly to the largest Continuum of Care (CoC) in each geographic area covered by the VHPD programs: the City of San Diego; Austin/Travis County; Utica/Rome/Oneida County; Tacoma/Lakewood/Pierce County; and Tampa/Hillsborough County. HUD awarded each grantee $2 million for a period of 3 years starting in February 2011; grants went to homeless assistance programs in designated CoC or to the CoC itself, to deliver housing and supportive services in collaboration with VA medical centers and DOL One-Stop workforce development centers. As specified by the Act, HUD selected these sites based on the following factors (1) communities with high rates of veterans experiencing homelessness; (2) grantees experienced in coordinating with VA and DOL to help veterans access mainstream services, including education and job training programs; (3) communities located close to military installations where service members are transitioning from military service to civilian life for up to three sites; and (4) rural areas.

Fort Drum was selected because it most closely met the definition of a rural community, it had limited access to VA facilities and a high rate of Operation Enduring Freedom (OEF)/Operation Iraqi Freedom (OIF) military members with families and limited supports. The other 4 sites were selected based on the following criteria: (1) the number of homeless veterans reported for that geographic area through HUD’s Continuum of Care (CoC); (2) the number of unique returning OEF/OIF veterans who accessed health care through VA between FY2002 and the second quarter of FY 2009; (3) the number of homeless veterans reported through VA’s Community Homelessness Assessment, Local Education and Networking Group (CHALENG) and the Northeast Program Evaluation Center (NEPEC); the range and diversity of military represented by the selected sites (e.g., all branches of the U.S. military, the National Guard and Reserves); access and availability to VA health care; overall geographic distribution; capacity of the community to carry out the demonstration project.

VHPD sites are required to spend 65 percent of their grant on housing assistance, but beyond that requirement, they have discretion to develop program activities that reflect local need. VHPD grantees and their subgrantees provide a range of financial, case management, and housing location services to homeless households and those at risk of homelessness. VHPD provides short- or medium-term housing assistance (3 to 18 months), including security deposits, rent, rental arrearages (up to six months back rent), moving cost assistance, and utilities, as well as case management and referrals to community-based services and supports. Service providers may also use VHPD funds for childcare, credit repair, and transportation expenses.

Grantees have discretion in targeting veterans most in need of homelessness prevention and rapid re-housing. Eligibility criteria set by HUD include veteran status and that the household is at or below 50 percent of AMI. The target groups are veterans and veterans with families at risk of homelessness or experiencing short-term homelessness (fewer than 90 days). Veterans from all periods of service (Vietnam, Persian Gulf, etc.) are eligible, but HUD is encouraging focused outreach to OEF/OIF veterans. National Guard members and those who served in the reserves and saw active duty for two years or more are also eligible for VHPD financial assistance and services.

To track and monitor outcomes, the sites are required to participate in HMIS and, either already or in the near future, HOMES, a VA database that includes all veterans who receive homeless services from the VA. VHPD programs are already serving clients, with some beginning enrollment as early as March or April 2011.

As noted earlier, in the legislation creating and funding the VHPD, Congress included an evaluation requirement (Public Law 111-8, Omnibus Appropriations Act). This study will provide detailed information as to how the five VHPD sites have implemented the program with special attention to the structure and effectiveness of local level partnerships between the CoC, the VA, and the DOL as well as measuring the effectiveness of the prevention assistance offered through VHPD.

A2. How, by whom, and for what purpose the information is to be used

To understand the effectiveness of VHPD, the study will survey and conduct focus groups with veterans who enroll in VHPD. Further, the study will collect administrative data on two comparison groups: (1) veterans similar to those who are eligible for the program, but do not receive services because they enrolled in a Veterans Affairs Medical Center (VAMC) before the program was available; and (2) similar non-veterans who received services from the Homelessness Prevention and Rapid Rehousing Program (HPRP) during the same time frame as VHPD. These respondents are described in more detail below.

A2.1 Project Overview

HUD contracted with Silber & Associates along with its subcontractor the Urban Institute to conduct the Veterans Homelessness Prevention Demonstration Study. VHPD is HUD’s first prevention program to target at-risk and homeless veterans. This process and outcomes evaluation, will describe program models at each of the five VHPD sites, evaluate VHPD’s efficacy in preventing homelessness among veterans, and provide policymakers with greatly needed knowledge on how to design effective prevention programs. Three questions will guide the research: (1) what are effective ways to identify, reach, and assist veterans who are at risk for homelessness or are experiencing short term homelessness; (2) are the services provided through VHPD effective?; and (3) what are the barriers to providing services? To address all of the research questions and to satisfy Office of Management and Budget’s (OMB) requirements that federal agencies undertake independent evaluations of program performance, new data (for which this OMB clearance is requested) must be gathered. Existing administrative data sources alone are not adequate to answer the research questions required for the evaluation mandated by Congress. To answer the questions, the research team will collect the following data:

Moderate Focus Groups with Program Participants. We will conduct focus groups with a sample of veteran VHPD participants, gaining their perceptions of how well the assessment process identified their needs and how well the network of VHPD agencies was able to get them services that met their needs and helped them move forward. Asking these questions in a focus group setting, where the responses can be open-ended will help uncover issues that may not be captured in the survey. The research team drafted a moderator’s guide for the focus groups (Appendix A). Information obtained through focus groups will be used with findings from the Interim Report to enhance the Process Evaluation summary in the Final Report.
Baseline and Follow-Up Survey Interviews. S&A will conduct baseline and follow-up telephone interviews with VHPD participants in the sample. We anticipate a response rate of upwards of 80 percent of each group. The research team developed survey instruments for these interviews (Appendix B) that covers the major areas on which VHPD interventions are expected to have an impact, or that are important baseline characteristics for understanding veterans’ situation at intake and how it might affect their experience with and outcomes from VHPD.
Administrative Data from HMIS and HOMES. The research team will collect administrative data to construct the comparison groups (detailed below) and to examine outcomes of interest—most notably shelter entry.

Sample Overview¹
Ideally this evaluation would include experimental methods, but this was not possible both because of the costs associated with such a study and the fact that it was expected that sites would be implementing the program before the research study was launched, making random assignment to a treatment and control impractical. Further, this program is relatively new, no standardized interventions exist, the program sites vary considerably in their service configuration and local circumstances, and the interventions change in greater or lesser ways over time as programs gain experience with their clientele and with what seems to work. Together, these factors made using random assignment impractical.

One of the biggest challenges to understanding program effects in non- experimental designs such as the VHPD evaluation is selection bias. To understand the true impact of the VHPD on program participants, it is critical to create a counterfactual that answers the question: all things equal, what would have happened absent the VHPD intervention? This requires selecting samples of one or more groups that did not receive VHPD but that look similar to program participants who did receive services.

The research team will collect data for three sample groups:

Group 1. 500 VHPD participants enrolling between July 2012 and June 2013, for baseline and follow-up interviews (up to 1,000 interviews total);

Group 2. Comparison group of approximately 300 to 500 veterans who would have qualified for VHPD services but did not receive them, enrolling in VA services before June 1, 2011 (administrative data only); and

Group 3. Comparison group of approximately 300 to 500 non-veterans who received services from the Homelessness Prevention and Rapid Re-Housing Program (HPRP) (administrative data only).

Simple selection procedures will not result in groups that are comparable, even if matched on initial characteristics. To compensate for the consequences of serious differences in groups at baseline and to produce unbiased estimates of program effects, we will use propensity score matching, using baseline characteristics available for each group that are associated with risk of homelessness. We will use these characteristics in logistical regression analyses to predict the probability of group membership (VHPD vs. Group 2, and VHPD vs. Group 3). We will use the resulting scores to weight members of the comparison groups so the overall group characteristics look similar to those of VHPD participants. Probable matching variables are OEF/OIF or not (for VHPD vs. Group 2) and families/singles, male/female head of household, VHPD site, and possibly one other characteristic for both comparison groups.² We are confident that we have a very strong research design. However, we understand that there may be differences between the comparison groups that we cannot control and that might not be compensated for through propensity scoring methods. We will note this limitation of the evidence in all reports of study results.

With data from the groups described above, the research team can examine how well VHPD prevents homelessness among veterans, and, by comparing outcomes for a similar group of non-veterans, we can highlight how well VHPD participants compare to other at-risk populations. The first comparison group (veterans who did not receive VHPD services) offers the opportunity to assess the impact of the VHPD services on veterans. The second comparison group (non-veterans receiving prevention services) offers the opportunity to compare veterans’ outcomes to outcomes of non-veterans who receive similar services. This comparison will provide an important outcomes benchmark and help to shed light on veteran-specific barriers to prevention. Since we know very little about how homelessness outcomes among veterans compares to non-veterans, this comparison (Group 1 to Group 3) is particularly important---and since we are using administrative data, this comparison can be made for relatively low cost. However, the evidence produced by this research will be qualified because we cannot control other factors that may distinguish the comparison groups from the VHPD client group. This limitation will be clearly stated in all reports on this study.

A2.2 Purpose of the Data Collection

The purpose of the data collection is to fulfill HUD’s statutory requirement outlined in Public Law 111-8, Omnibus Appropriations Act.

This PRA submission requests approval for a baseline and follow-up survey with VHPD clients and several focus groups with VHPD clients. The data collected will provide evidence for answering both process evaluation and outcome evaluation research questions. Further research questions will be answered through site visits that are excluded from this request and therefore are not discussed here. (Information about the site visits is available in Appendices C and E.)

Process Evaluation Research Questions

Specifically, the focus groups and baseline and follow-up survey will contribute to answering the process questions outlined in the table below.

Research Question

Method

Who is served through the program? What are their needs? Is it possible to identify specific constellations of needs that characterize subgroups of veterans, in particular: younger veterans, OEF and OIF veterans, National Guard, women, and young families?

Survey

Focus Groups

What barriers limit effective services or stability, from the veteran’s perspective?

Focus Groups

Are any barriers unique to preventing homelessness among veterans in general or specific subgroups of veterans, in particular younger veterans, OEF and OIF veterans, National Guard, women, and young families?

Focus Groups

Outcomes Evaluation Research Questions

The data collected through the baseline and follow-up surveys and focus groups, supplemented by administrative data, will address whether or not VHPD prevents homelessness among veterans. The key issue is whether veteran households that participate in VHPD are able to avoid homelessness, as measured by (1) the housing status they report in follow-up telephone interviews and (2) whether they have become homeless as indicated by shelter or other homeless service use recorded in a Homeless Management Information System (HMIS) within the VHPD catchment area, or use of a VA homeless service recorded in HOMES. Other outcomes of interest, measured by the Group 1 personal interviews, are housing stability (getting into and/or remaining in an appropriate housing unit and being current on rent, utilities, etc.) and improvements in income levels, employment status, and income sources. Specifically, the outcomes study will answer the following key questions outlined in the table below.

Research Questions

Methods

What happens to program participants after receiving VHPD program benefits? Do program participants avoid homelessness? Do they experience housing stability? Do program participants have better housing stability than similar veterans who do not participate in VHPD? How do VHPD participant outcomes compare to those who receive prevention services through HPRP?

Survey
Administrative Data
Focus Groups

Do program participants report any increases in non-housing-related benefits after program participation, particularly in the areas of health, employment/earnings, or receipt of VA pensions/benefits and other mainstream benefits?

Survey
Administrative Data
Focus Groups

A2.3 Who Will Use the Information
Policymakers at HUD will use this information to understand the impact of VHPD on homelessness among veterans. In addition, the study findings will inform the development of future prevention programming for veterans at-risk, particularly those returning from Iraq and Afghanistan. The study will also contribute to the research literature on homelessness prevention.

A2.4 Instrument Item-by-Item Justification

Focus Group Protocols/Moderator Guides

The study uses focus groups to collect information on the experiences and perspectives of VHPD clients. This information is critical for understanding the best ways to identify and reach out to veterans at risk of homelessness and the veterans’ perspectives on barriers to providing prevention services. Exhibit 1 outlines the topic areas covered by the focus group protocol and a clear justification for each area’s inclusion in our instruments. The proposed focus group protocol is included in Appendix A. Note that since separate focus groups may be conducted with veterans with families (i.e., those with minor children in the household), women, OEF/OIF veterans, and single adults, several probes specific to particular subgroups are included in the protocol. The justification follows the generic protocol but highlights opportunities where we may target questions or probes to specific subgroup specific populations.

EXHIBIT 1. Item-by-Item Justification of Focus Group Protocol – VHPD Participants
Question(s) / Topic(s)	Content and justification for inclusion
Opening	The opening question asks participants to introduce themselves with their name, where they served and how long they’ve been home since their last tour of duty. This question is meant to be one that is easily answerable and helps participants feel more comfortable in sharing their opinions with the rest of the group. If we are able to hold a focus group of only OEF/OIF veterans we may also ask how they believe their experiences are different than veterans of the Vietnam War, many of whom became homeless a few decades after service. OEF/OIF veterans are believed to have special needs as compared to veterans of older conflicts and as VHPD emphasizes serving OEF/OIF veterans this question has particular relevance in the service needs of this group.
Housing Instability and Pathways to VHPD	One of the key issues we want to discuss in the focus groups is the paths to participants’ homelessness or at-risk status and how they were able to find about VHPD. We will ask participants about the challenges they faced in transitioning to civilian life, how their housing struggles started, what led them to seek assistance in general and from VHPD specifically. If we are able to hold subpopulation specific focus groups (for single adults, veterans with families, women, and OEF/OIF veterans), we will probe for subpopulation-specific factors that might have also played a role. For example, we will probe for challenges specific to having children to provide for or challenges specific to women. As part of the process evaluation of this study, these responses will allow us to identify successful outreach strategies.
VHPD Experience and Services	Other key issues for the focus groups are how participants moved through VHPD and the types of assistance they received and their impressions of what aspects of VHPD or types of assistance were most helpful to them. To that end, we will ask participants to talk about their overall impressions of VHPD, what services they were able to access, whether they believed those services were helpful, and how VHPD helped them access additional services through VA and DOL. Another key issue for focus group discussion is how VHPD assistance has changed the participants’ situation (in terms of housing security, job skills and employment, mental and physical health, access to benefits, financial stability, and education level). To this end, we ask participants to discuss how they think their situation has changed since starting VHPD and what aspects of VHPD they think were most helpful for them in making that progress. As part of the process evaluation component of this study, these responses will be integral to our understanding of which services are most effective and what services are not helpful to veterans.
Prospects for the Future	The last issue we will address in the focus groups is the participants’ prospects for the future: more specifically, what services they think will be useful going forward and how they think VHPD will affect their housing stability in the long-term. Answers to these questions will also inform our analysis of which services offered by VHPD are most important for improving the housing stability of veterans.

Baseline Survey

Exhibit 2 shows the types of information we will collect through the baseline telephone survey and the justification for including each question/topic. A copy of the survey instrument is included as Appendix B. Note that all question numbers within the survey instrument, with the exception of the first three questions, are preceded with a letter prefix. The letter prefix relates to the topic area to which the question relates.

EXHIBIT 2. Item-by-Item Justification of Baseline Survey Instrument – VHPD Participants
Question(s) / Topic(s)	Content and justification for inclusion
Questions 1-4	The first question asks the respondent to provide the names of the local VHPD participant organizations (grantee, VA medical center, and worksource center). These names will be used for the remainder of the baseline survey and also in the follow-up survey so that the questions are easier for the respondent to answer. Questions 2 and 3 ask for the respondent’s address and the address where we should send his or her $30 incentive. Questions 4 and 4a asks when the respondent was discharged from the military, Reserves, or National Guard. This question is asked in the opening section, because the information obtained from it will be used throughout the remainder of the survey. Throughout the rest of the baseline survey, the respondent will be to reflect over one of two timeframes: (1) the last 12 months, which will be asked of veterans who were discharged from the military more than 12 months ago, or (2) since the respondent left the military, which will be asked of veterans who were discharged from the military less than 12 months ago.
SECTION A: Housing at VHPD Program Entry and Housing History A1-A10	The first series of questions (A1 – A6a6) asked in this section of the survey targets the respondent’s housing history, determining the respondent’s current housing situation – whether they were in their own home, living in someone else’s home, homeless, or institutionalized. We also ask how long they have lived at that place and for their assessment of the housing quality of that place. We also ask for the total number of places they lived over the past 12 months or since they were discharged from the military and the types of places – the respondent’s own place, someone else’s place, a homeless shelter, someplace not meant for habitation, or in an institutional setting – they lived over the past 12 months or since they were discharged from the military.. Taken together, the respondent’s answers to these questions give us a picture of the respondent’s current housing situation and how stable it has been during the time leading up to VHPD assistance so that we can better understand the housing circumstances that lead veterans to seek VHPD assistance. The final series of questions in this section (A7-A10) asks specifically about the respondent’s history of homelessness, including the number of incidences of homelessness over the life course, total time spent homeless over the past 12 months, and the respondent’s age when he or she was first homeless. These questions allow us to better assess whether the veteran’s homelessness or risk of homelessness has been a recurring problem and whether the veteran experienced homelessness only after being discharged from the military. Also, experiencing recent homelessness is a risk factor for subsequent homelessness, so it is important that we capture the respondent’s history of homelessness.
SECTION B: Household Composition B1-B2	The series of questions asked in this section of the survey captures the composition of the respondent’s current household. We ask specifically for the relationship of the current household members to the respondent and the total number of adults and children. If children are present, we ask for their age range, which is important since having children under the age of five has been identified as a risk factor for homelessness. The size of the household also allows us to assess overcrowding – another risk factor for homelessness.
SECTION C: Education and Training C1-C6	This section asks what level of education and any vocational training the respondent had when he or she began applying for VHPD, as well as the training or schooling the respondent is currently in and whether the programs associated with VHPD (grantee, VA, DOL) helped them access that schooling or training program. This allows the researchers to determine the education level of VHPD participants and the types of education/schooling VHPD participants are able to access through the program as well as the role of the specific VHPD programs in helping participants access these opportunities. Item D5 in the baseline survey asks about the role of each VHPD partner in turn in accessing this education/training. The survey will be automatically pre-populated with the actual name of the VHPD program the respondent participated in and so appears in the current survey draft as “PN-HL”, “PN-VA”,”PN-DOL”. This section also asks whether the respondent is receiving benefits from the Post 9/11 GI Bill to help pay for school costs. Anecdotally, VHPD program staff indicate that many of the veterans they serve are accessing this benefit; however, education is not captured in HMIS, so it is important to capture and quantify this in the evaluation.
SECTION D: Income and Employment D1-D17	Loss of income, recent unemployment, and housing cost burden are all risk factors for homelessness. The series of questions posed in this section identify the overall level of the respondent’s household’s income and the household’s sources of income, as well as any benefits the household received in the past 30 days, including education benefits. We also ask about their work history, whether the respondent is working, looking for work, reasons why they were not able to work or look for work, and whether the VHPD DOL partner has helped them find work. Item D6 in the baseline survey asks about the role of each VHPD partner in turn in accessing benefits. Item D15a asks about the role of each VHPD program in getting the respondent’s current job. The survey will be automatically pre-populated with the actual name of the VHPD program the respondent participated in and so appears in the current survey draft as “PN-HL”, “PN-VA”,”PN-DOL”. Since having combat related injuries also increase the risk of homelessness for veterans, we also ask whether the respondent has a disability that impedes their ability to work and if this disability is related to their military service.
SECTION E: Housing Costs E1-E7	Housing cost burden is a key factor in increasing a household’s risk of homelessness. This section asks questions to determine how much respondents and their families are paying for housing each month, if the respondents receive rental assistance from government or other programs, and whether there was ever a time over the past 12 months that the respondent was unable to pay rent or utilities and how often that has happened. In conjunction with the income information from Section E, these questions will enable us to estimate the degree to which VHPD households are cost burdened and thus at greater risk of homelessness.
SECTION F: Family Health and Wellbeing F1-F4	Having health problems, including Traumatic Brain Injury increase the risk of homelessness. Traumatic Brain Injury is more commonly experienced by veterans. In order to assess the presence of these risk factors in VHPD participants, we ask for the respondent’s self-assessment of their health, whether they experience a range of mental health issues, , and traumatic brain injury. We also ask which of the organizations involved in VHPD have helped them get help with any of their health issues Item F3 in the baseline survey asks whether each of the VHPD partners helped with any of the respondent’s health problems. Lastly, we ask whether the respondent has health insurance, and if so, whether or not that health insurance is provided by the VA to determine the respondent’s access to affordable health care.
SECTION G: Veteran Status/Military Experience G1-G8	As VHPD is targeted to veterans with an emphasis on OEF/OIF veterans, and that certain aspects of military service increase the risk of homelessness (e.g., multiple deployments, combat related injuries, transition to civilian employment), we need to capture the respondent’s veteran status and military experience. We ask whether they were active duty military or in the reserves or national guard, in what conflicts they served, exposure to combat and friendly/unfriendly fire, how long they served, month and year of discharge, type of discharge, and the greatest challenges that the respondent faced in returning to civilian life. A critical question in this section is whether the respondent thinks their military service increased their risk of becoming homeless and if so how.
SECTION H: Demographics H1-H5	This section asks for the respondent’s race and ethnicity, gender, date of birth, and marital status. These demographic factors will be used to draw the comparison group samples of non-VHPD veterans and non-veteran HPRP participants that resemble our sample of VHPD participants as closely as possible.
SECTION I: Contact Information I1-I21	This section asks for contact information for three people who will know how to reach the respondent in case we cannot get in touch with them for the follow-up survey. This information is necessary so that we can achieve the highest response rate possible for the follow-up survey and minimize attrition.

Follow-Up Survey

Exhibit 3 shows the types of information we will collect through the follow-up telephone survey and the justification for including each question/topic. A copy of the survey instrument is included as Appendix B. Note that all question numbers within the survey instrument are preceded by a letter prefix. The letter prefix relates to the topic area to which the question relates.

EXHIBIT 3. Item-by-Item Justification of Follow-Up Survey Instrument – VHPD Participants
Question(s) / Topic(s)	Content and justification for inclusion
SECTION A: Housing Now and Since Baseline / Services received from VHPD and Perspectives on Impact of Services A1-A11	Since the baseline interview captured the respondent’s housing situation at that time and housing history prior to VHPD, the follow-up survey seeks to capture any changes in residence since the baseline. We ask what the respondent’s housing situation currently is – whether the respondent is living in the same place as they were at the time of the baseline interview, and if not whether the respondent is in his or her own home, living in someone else’s home, homeless, or institutionalized – as well as how long they’ve been in that place and an assessment of the housing quality and unit size. We also ask for the number of places they have lived since the baseline interview and in what type of places the respondent lived – his or her own place, someone else’s place, a homeless shelter, someplace not meant for habitation, or in an institutional setting To assess the impact of VHPD assistance, we also ask what types of housing assistance the respondent has received through VHPD (i.e. paying off rent arrears, paying a deposit for a different apartment) and if the services they received offered the respondent sufficient help. In order to determine the role of supports from social networks, questions A11 and A11a ask whether the respondent received help in his or her housing crisis from family, friends, or other social networks and, if so, whether that assistance was more, less, or equally as useful as the assistance provided by VHPD.
SECTION B: Household Composition B1-B5	The series of questions asked in this section of the survey targets the change in the composition of the respondent’s household since the baseline survey. We ask specifically for the relationship of the household members to the respondent and the total number of adults and children present in the household. If there are children present in the household, we ask how many are under the age of 5, between 6 and 17, and 18 and older. We ask this for multiple reasons. First, having young children is a risk factor for homelessness. Second, we need to discern whether the respondent has school age children to determine whether questions on children’s schooling should be asked. For those respondents that have school age children, we ask whether their children were attending school regularly or if they have trouble attending school regularly because of their housing situation or other reasons prior to VHPD enrollment and since the time they enrolled in VHPD to capture the impact of housing situation on children’s school outcomes.
SECTION C: Housing Barriers C1-C2	This section identifies which factors contributed to the respondent’s difficulties in finding and maintaining housing since the baseline survey to determine what struggles are faced by VHPD participants. Specifically, we ask respondents if respondents had trouble keeping their housing or had to move since the baseline interview. We ask respondents to identify factors that make it difficult for people to keep housing or find new housing (i.e. not having enough income, owing too much back rent or utilities, not being employed, and having trouble with drugs at alcohol).
SECTION D: Education and Training D1-D8	This section seeks to determine what educational and vocational training opportunities the respondent has been able to access since the baseline survey and the role of the VHPD partner organizations in helping VHPD participants access these opportunities. Items D4 and D7 ask specifically if each of the VHPD partner organizations were helpful in accessing these opportunities. We ask specifically what levels of education or types of training they were able to achieve and whether any of the VHPD partner organizations (grantee, VA, or DOL) helped the respondent access those opportunities. In order to determine the role of supports from social networks, questions D4e and D4f ask whether the respondent received help getting education or training from family, friends, or other social networks and, if so, whether that assistance was more, less, or equally as useful as the assistance provided by VHPD. We also ask whether the respondent is using the Post 9/11 GI Bill to help pay for school costs.
SECTION E: Income and Employment E1-E19	Loss of income, recent unemployment, and housing cost burden are all risk factors for homelessness. The series of questions posed in this section identify the respondent’s household’s sources of income (if any) in the past 30 days, the amount of income received by the household, as well as any benefits the household has received in the past 30 days. We also ask about their work history since the baseline. Item E4 asks specifically about types of assistance that VHPD partners provided, and E5 asks the respondent for his or her assessment of the impact of this assistance. Item E14 specifically asks about types of assistance that may have helped the respondent find work that VHPD partners provided, and E15 asks the respondent for his or her assessment of the impact of this assistance. E18 asks if the VHPD helped the respondent get work (if the respondent has a current job.) Since having combat related injuries also increases the risk of homelessness for veterans, we also ask whether the respondent has a disability that impedes their ability to work and if this disability is related to their military service. In order to determine the role of supports from social networks, questions E15a, E15b, E19, and E19a ask whether the respondent received help finding work from family, friends, or other social networks and, if so, whether that assistance was more, less, or equally as useful as the assistance provided by VHPD.
SECTION F: Housing Costs F1-F7	Housing cost burden is a key factor in increasing a household’s risk of homelessness. This section asks questions to determine how much respondent’s household is paying for housing each month, whether the household receives assistance from rental assistance programs from the government or another program and whether there was ever a time over the since the baseline that the respondent was unable to pay rent or utilities and how often that has happened. In conjunction with the income information from Section E, these questions will enable us to estimate the degree to which VHPD households are cost burdened and thus at greater risk of homelessness. By having these measures at baseline and follow-up we can determine the change in housing cost burden over time and the impact of VHPD on the household’s housing cost burden.
SECTION G: Family Health and Wellbeing H1-H7	Having health problems, including Traumatic Brain Injury, increase the risk of homelessness. Further, Traumatic Brain Injury is more commonly experienced by veterans. In order to assess the presence of these risk factors in VHPD participants, we ask for the respondent’s self-assessment of their health, whether they experience a range of mental health issues and traumatic brain injury. We also ask whether the organizations involved in VHPD have helped them get help with any of their health issues and the health of children in the household. Item G4 asks specifically about the role of VHPD partners in helping with health issues. Lastly, we ask whether the respondent has health insurance, and, if so, whether insurance is provided by the VA or military. By having this information at baseline and follow-up we can determine the change in these factors over time and the effect of VHPD on the respondent’s health and the wellbeing of children in the household and the extent to which the respondent believes the services he or she was able to access through VHPD were helpful.
SECTION H: Demographics and Closing I1 – I3 and closing statement	This section asks whether the respondent’s marital status has changed since the baseline interview and if so we ask for his or her current marital status, so that we have a current record of the demographic composition of the sample. Verify that the address we have on file is the correct address to send their $30 incentive for participating in the follow-up survey. If not, get the correct address. The closing statement thanks respondents for their time and asks if they have any questions prior to ending the interview.

A3. Use of automated, electronic, mechanical or other technological collection techniques to reduce burden

The study will deploy a telephone survey for the baseline and follow-up survey. The research team believes that this mode of data collection is the easiest way for the respondents to answer the survey questions. A web survey would not be appropriate for this study because many of the participants do not have access to the internet. In addition, web surveys typically have significantly lower response rates compared to telephone surveys. The baseline and follow-up interviews are each expected to last about 25-30 minutes. Silber & Associates' interviewers will conduct telephone interviews using computer-assisted telephone interviewing technology (CATI). The CATI questionnaire has built-in logic that automatically customizes the interview based on the respondent's answers to previous questions, streamlining the interviewing process. Using CATI technology will help ensure that the survey flows quickly and makes answering questions easier for the respondent.

A4. Efforts to identify duplication

During the process of designing the survey instrument, the research team carefully reviewed the data HUD currently collects through local Homeless Management Information Systems (HMIS), VA HOMES data, and quarterly reports and made sure that none collect the kinds of data this survey will provide. Since VHPD is a new program, there have been no national studies that examine its efficacy. An extensive review of the literature by UI revealed no other studies collecting the same information evaluating VHPD or any other systematic study of homelessness prevention for veterans on a national scale, so the study will produce the first qualitative and quantitative data on how communities are implementing VHPD and how effective the program is in helping veterans maintain stable housing.

A5. Methods to minimize the burden on small businesses or other small entities

Not applicable. No small businesses or other small entities will serve as respondents for this study.

A6. Consequences if data are not collected

As noted earlier, the Omnibus Appropriations Act of 2009 (PL 111-8) included a $10 million set-aside “to conduct a demonstration program on the prevention of homelessness among the Nation’s veterans.” This included up to $750,000 for an “evaluation of this demonstration.” The Act (PL 111-8) provides no further direction. Senate Report 110-418 elaborates that the evaluation, for which The Committee includes $750,000, should examine “the most effective ways to identify, reach, and assist veterans who are at-risk of homelessness or are experiencing short-term homelessness; the extent to which services and activities meet the needs of veterans experiencing a housing crisis and contribute to their longer-term economic stability; and the identification of any barriers that limit prevention activities.”

Importantly, this will be the first comprehensive study of VHPD or any other homelessness prevention strategy targeted to veterans. The information collected through the study is critical to understanding the effectiveness of homelessness prevention programs for veterans. From this study, policymakers will learn about different approaches to prevention and about the unique needs of at-risk veterans. This information is particularly relevant as many veterans return from serving in Iraq and Afghanistan. Furthermore, information collected through the study will help the administration get closer to achieving its goal of ending homelessness among veterans. Without this study, there will be no evidence upon which to base future policy decisions related to efforts to prevent homelessness among veterans.

A7. Special Circumstances

The proposed data collection activities are consistent with the guidelines set forth in 5 CFR 1320.6 (Controlling Paperwork Burden on the Public—General Information Collection Guidelines). There are no special circumstances that require deviation from these guidelines.

A8. Federal Register Notices

In accordance with 5 CFR 1320.8(d) a Notice was published in the Federal Register on November 14, 2011 (pages 70470 and 70471) announcing HUD’s intention to request OMB review of this data collection effort and soliciting public comments. No comments were received.

A9. Remuneration to respondents

Veterans who participate in the study will receive incentives as a token of appreciation for participating in the study. In addition, monetary incentives are a powerful tool for maintaining low attrition rates in longitudinal studies. Monetary incentives will help ensure a high response rate among veterans participating in the study. Focus group participants will receive a $50 incentive. Participants in the outcomes evaluation will receive a $30 incentive for each survey interview they complete. Past studies conducted by Silber & Associates and the Urban Institute with similar populations have found that this level of remuneration is a necessary incentive to achieve adequate response rates. Other studies involving a homeless population have used a similar incentive (e.g., “Ending Homeless in the City of Red Deer” ($25 honorarium); “Measuring Pain in the Context of Homelessness” ($25); “Homeless Perspectives on the Public Library” ($10); and “Homeless Employment Access Niagara Region 2006” ($20)). Slightly higher incentive payments are useful in studies with long, complex surveys and that require longitudinal follow-up. For example, respondents in HUD’s random assignment study of interventions for homeless families will receive $50 for completing the follow-up survey. For the VHPD evaluation Silber & Associates similarly chose a slightly higher incentive payment because of the length and complexity of the interview and the critical importance for the study design of being able to follow-up with respondents who filled out the baseline survey. Moreover, because the questions collect information more so than opinion, the demand on the respondent is greater.

A10. Assurances of confidentiality

The information requested under this collection is protected and held confidential in accordance with 5 U.S.C. 552a (Privacy Act of 1974) and OMB Circular No. A-130. This data collection is also protected under the System of Records Notice (SORN), which HUD published in the Federal Register on May 2, 2012 (FR-5613-N-02). Detailed procedures used to obtain informed consent are discussed below. (Note: HUD’s authority to conduct research and program evaluations is the Housing and Urban Development Act of 1970, Section 502 (Pub. L. 91-609; 84 Stat. 1784; 12 U.S.C. 1701z-1 et seq.) )

As previously indicated, the survey data collection will be conducted by Silber & Associates. The Urban Institute maintains an Institutional Review Board (IRB) to ensure that research practices and procedures effectively protect the rights and welfare of human subjects, consistent with the requirements set forth in Title 45, Part 46 of the Code of Federal Regulations (45 CFR 46). The Urban Institute’s policy is that all research involving human subjects must adhere to the following principles, among others:

Risks to human subjects from research must be reasonable in relation to anticipated benefits, and must be minimized to the extent possible;
Human subjects must be fully and accurately informed of the nature of the research in which they will be involved, whether their participation is mandatory or voluntary, any consequences of non-participation, any risks associated with their participation, and how the research will be used;
Adequate provision must be made to protect the privacy of human subjects and to maintain the confidentiality of data that are collected, where promised and as appropriate.

In accordance with these policies, we will maintain the following procedures. First, before they agree to participate all research subjects will be given a clear overview of the study and its goals, the data security plan, the staff confidentiality agreement, and our methods for safeguarding anonymity in our reports and publications. A study informed consent form (Appendix F) outlining this information will be administered by program staff when the participant enrolls in the VHPD program. In addition, we will stress the voluntary nature of their participation and make clear to all parties that there are no negative consequences for their person or household should they choose not to participate. For the focus groups, we will provide an oral overview of all of the above points as an introduction to the focus group as well as providing a written consent form that explains the above points. We will obtain written consent from all focus group participants. For the telephone survey, Silber & Associates interviewers will provide an oral overview of the same points as an introduction to the survey and will obtain oral consent from all participants.

Second, we will take care to safeguard the information gathered from participants in this research effort. The data gathered from the telephone survey will be analyzed and discussed exclusively in aggregate; no published reports using the survey data will single out a particular respondent. Similarly, everything focus group participants share during our site visits will be treated as confidential – that is, no comments will be attributed to them as individuals. Because we expect to produce case study and cross-site analyses of themes, however, it is likely that the comments may be associated with a particular site. In these cases, we will take special care with particularly sensitive information to ensure that it cannot be traced back to a particular respondent.

The data security plan submitted with our Privacy Impact Assessment, which was approved by the Privacy Office along with the SORN for this information collection, details technical and administrative procedures for safeguarding the confidentiality of personally identifiable information. Upon completion of the study, PII will be destroyed in according with applicable regulations. The data set delivered to HUD will be de-identified.

A11. Questions of a sensitive nature

Since veterans must be at or below 50 percent AMI to be eligible for VHPD and homeless or at risk, participants in the focus groups and telephone surveys will be part of a vulnerable population. The telephone survey questions pertain entirely to their housing history, housing costs, barriers to finding and maintaining housing, income and employment, military experience, health, wellbeing, as well as demographic and contact information. These questions are not sensitive in nature. The focus group questions pertain to participants’ housing struggles, services received through VHPD and what aspects of their VHPD program were most helpful for them. These questions are not sensitive in nature. This can also be verified by reviewing the focus group guides provided. Because of the vulnerable nature of this population, we will explain the purposes of the study as an introduction to the focus group and survey interviews, stress that participation is voluntary, that participants have the right to refuse to answer any question without consequence, and that their identity will be kept confidential, with answers only reported in the aggregate (as discussed in Section A10). This can be verified by reviewing the data collection instruments provided.

A12. Estimates of the burden of the collection of information

A12.1. Estimate of respondent burden hours

Respondents to the baseline survey will total 500 VHPD participants, and, with attrition, we estimate 400 respondents for the follow up survey. The average response time for each of the telephone surveys is 30 minutes. This will result in an estimated response burden of 450 hours (see Exhibit 4). The instruments will be pretested with no more than nine respondents to make sure the wording is clear, and to confirm the length of the survey.

During the second round of site visits, we will conduct two focus groups at each of the five sites for a total of 10 focus groups. Each focus group will include 8 participants, which amounts to 80 participants across all 10 focus groups. Each focus group will last no longer than 2 hours. We estimate the total reporting burden for this stage of data collection to be 160 hours (5 sites x 2 focus groups x 8 participants x 2 hours).

Exhibit 4. Telephone Surveys and Focus Group Respondent Burden Estimates
Description	No. of Respondents	Estimated Hours per Response	Total Burden Hours
Telephone Surveys
Baseline Survey	500	.5	250
Follow Up Survey	400	.5	200
Survey Total			450
Site Visits
Focus Groups	80	2	160
Surveys + Site Visits
Total			610

A13. Total annual cost burden to respondent or record keepers

There are no capital/start-up or ongoing operation/maintenance costs associated with this data collection.

A14. Estimate of annual cost to the government

The total cost to the government for this study, including but not limited to the data collection activities described in this submission, is $740,256 over a 36-month period. Included are costs associated with background research, evaluation design, development of data collection instruments, data collection activities, remuneration to study participants, analysis, and reporting. The cost to the government for the survey and focus group data collection is $352,058. These costs include background research, evaluation design, development of data collection instruments, data collection activities, remuneration to study participants, analysis, and reporting.

A15. Reasons for any program changes or adjustments

This submission is a new request for approval; there is no change in the burden.

A16. Plans for tabulation, analysis, and publication

A16.1 Plans for tabulation

The focus groups will occur between the weeks of October 22, 2012 and January 18, 2013. Baseline telephone surveys will begin during the week of September 3, 2012 and end during the week of June 28, 2013. The follow-up surveys would begin six months after the first Group 1 members we recruit stop receiving assistance through VHPD (approximately June 1, 2013) and continue until all Group 1 members have received a follow-up interview (approximately June 1, 2014).

A16.2 Plans for analysis

Focus Group Analytic Strategy

The focus groups will focus on three key topic areas that correspond with elements of the above framework: (1) housing instability and pathways to VHPD, (2) VHPD experience and services, (3) prospects for the future. We will prepare a Grantee Memo for each site describing our findings, organized according to the implementation and process components listed above. We will either prepare one memo for each site or one integrated memo comparing findings across all five sites, according to the preferences of HUD.

To analyze the information gained during the focus groups we will store notes in the master Process Evaluation folder for each site, which also contains the VHPD proposal, other VHPD program documents, screening, assessment, and case management tools, and reconnaissance and site visit findings. We created a process evaluation file with sections corresponding to the following implementation and process components, which will guide the analysis of findings from the key informant interviews and focus groups:

Program Participants

Pathways to enrollment in VHPD—identification and outreach, recruitment
Screening and eligibility determination; how the “but for” requirement is implemented;^³
Processes of assistance—assessment, case plan development and support for implementation, primary and secondary goals, follow-up, reassessments;
Types and levels of assistance—months of rental assistance, types and length of supportive services;
Data entry and tracking.

Service Agencies and Systems

Program structure, partners, relationships with other aspects of the homeless and other assistance systems; how this particular structure and participants were selected for each VHPD community;
Place of VA, workforce development, and the primary housing/service partner in VHPD in the community, historical relationships that may have eased or complicated implementation of VHPD, perceived value to the VA and homeless systems of new relationships developed and/or new systems brought into interaction;
Interactions and approaches to integrating housing and services receipt across VHPD partners (and others, if relevant);
System changes already accomplished, plans and implications for future joint work.

Outcomes—Analytic Strategy

Our goal is to interview 500 VHPD participants in Group 1 (100 per site), plus get information on the use of shelters and other homeless services from HMIS and/or HOMES⁴ for this group. In addition, we will have HMIS and/or HOMES data on post-VHPD rental assistance use of emergency shelters and other homeless services if they are reported in local HMISs, and VA homeless services if they are reported in HOMES. With these data we can do:

Pre-post comparisons of VHPD participants (follow-up vs. baseline) on any variable or scale included in both surveys, and
Treatment (Group 1) to comparison (Groups 2 and 3) analyses of shelter/homeless service use at follow-up for Group 1 and after an equivalent time lapse for Groups 2 and 3.

We have sufficient sample size for pre-post analyses for Group 1/VHPD participants (500/~400), after accounting for likely attrition—that is, sample members who do not complete the follow-up survey. We also have sufficient power to detect significant differences between Group 1 to either Group 2 or Group 3 (500/500). As the evaluation proceeds, it will be very important to be aware of the consequences of possibly reduced sample sizes (for example, if OMB approval takes more time than expected or other delays are encountered). For this reason, we examine the implications of these possible changes through power calculations of alternative sample sizes.

In Exhibits 6 and 7, we show the power calculations for two hypothetical measures that may be included in the outcomes examined for VHPD impacts: (1) a 5-point well-being scale comparing the baseline survey with the follow-up survey for Group 1, and (2) the percentage of Group 1 respondents who remain housed at follow-up to Group 2 or Group 3 members at an equivalent time period post-program enrollment. The data for the latter comparison will come from administrative data, HOMES for Group 1 vs. Group 2 and HMIS for Group 1 vs. Group 3. A power of 80 percent to detect a difference at the 5% level is generally considered adequate for such comparisons. As the Exhibits show, for the first measure the power is adequate to detect the level of differences observed in the VHPD study with samples of only 400, which we expect to get with a follow-up survey response rate of 80 percent. The power is also adequate to detect a percentage point difference as small as 6 percent in housing retention rates (an effect size that seems plausible with the intervention).

Thus the group sizes we propose should give us ample power to detect differences that will make a difference to policy. Such power calculations vary from measure to measure, but these examples show that the scale measures (often scales from 1 to 5) are likely to require larger samples to measure the impact of the intervention. In addition, even if impacts can be detected with the larger sample sizes for each group, the planned group sizes may not allow for sub-group analyses for many measures.

Exhibit 6: Power to Detect Difference for a Hypothetical Measure on a 5-Point Scale—Pre-post comparisons for respondents to the baseline and follow-up surveys (VHPD Clients)

Measure: Well-Being Scale-2/10 of a point difference Baseline Mean—3.5; Follow-up Mean—3.7	Sample Size Baseline Follow-up		Power to Detect the Difference at 5% Level
	500	400	90.9%
	400	320	84.7%
Assumptions: Standard Deviation=1; One-tailed test.

Exhibit 7: Power to Detect Difference for a Hypothetical Percentage Measure—Comparing Treatment Group (VHPD Clients) to Comparison Groups (Veterans not in VHPD & non-Veterans in HPRP)

Measure: % of respondents who remain housed Veterans Mean—85%; Comparison Mean—75% Veterans Mean—85%; Comparison Mean—79%	Sample Size VHPD Comp.		Power to Detect the Difference at 5% Level
	500	500	99%
	500	500	80%
Assumption: One-tailed test.

Quantitative Analysis. The quantitative analysis will describe the VHPD participants and provide the results of t tests that examine pre- and post- intervention differences on measures of housing stability for VHPD participants. In addition, we will identify predictors of homelessness among VHPD participants. To do this, we will build an econometric model that includes risk factors identified through the literature (e.g., multiple deployments, combat injuries, trouble transitioning to civilian employment, young children, pregnancy, residential history, etc.) and regress them, using ordinary least squares (OLS) regression on the number of days spent in homeless shelter. To understand the impact of VHPD on homelessness and housing stability we will compare outcomes between Groups 1, 2, and 3. We will assess the impact of VHPD by using ordinary least squares (OLS) regression to assess the effects of independent variables on continuous dependent variables (e.g., days spent homeless as measured by shelter utilization) to examine differences in homeless rates within and across the various groups matched using propensity scoring techniques described above. A list of independent variables of interest is provided in Exhibit 8. For the impact analysis, we plan to rely heavily on administrative data, but will supplement with survey data where appropriate.

EXHIBIT 8. Quantitative Analysis Comparisons and Variables
Group	Independent Variables	Dependent Variables	Data Sources
Group 1 compared to Group 2	Demographics, military background, living situation, employment, Income and clinical status.	Days spent in VA homeless programs: HUD‐VASH, Residential Programs, HCHV Case Management, HCRV, VJO	HOMES
Group 1 compared to Group 3	Demographics, veteran status, housing status, income sources, physical and mental health, substance abuse, domestic violence, and VHPD and HPRP service provision.	Days spent in emergency shelter	HMIS

A16.3 Plans for Publication

The research team will analyze, integrate, and summarize data from the process and outcomes evaluations in interim and final reports. The interim report with focus on the results of the process study and will describe the VHPD program model and how VHPD is being implemented at the five demonstration sites. The final report will include a section summarizing the major findings of the Process Evaluation, and then devote itself largely to the Outcome Evaluation Results.

A16.4 Time Schedule

Recruitment for Baseline Surveys 9/1/12-8/31/13
Focus Groups 10/22/2012-1/18/13
Interim Report 11/26/2012
Follow-Up Telephone Surveys 7/1/13-7/1/14
Final Report 10/29/14

A17. Approval to not display the OMB expiration date

Not Applicable. Silber & Associates and Urban Institute will display the expiration date for OMB approval of the information collected on all instruments and correspondence with prospective respondents.

A18. Exception to the certification statement

This submission, describing data collection, requests no exceptions to the Certificate for Paperwork Reduction Act (5 CFR 1320.9).

1 We describe the sampling plan in detail in Part B: Statistical Methods.

2 Burt used a similar statistical procedure in her forthcoming article (2012), “Impact of Housing and Work Supports on Outcomes for Chronically Homeless Adults With Mental Illness: LA’s HOPE,” in Psychiatric Services in Advance. Also see Pirog MA, Buffardi AL, Chrisinger CK, et al (2008), Are the alternatives to random assignment nearly as good? Statistical corrections to nonrandomized evaluations. Journal of Policy Analysis and Management 28:169–172 and Rubin, DB: Using propensity scores to help design observational studies: application to the tobacco litigation. Health Services and Outcome Research Methodology 2:168–188, 2001

3In HPRP terminology, eligibility is ultimately determined by the “but for” requirement—screening and eligibility determination staff must make the judgment that the household would become homeless but for this assistance. As the VHPD grant announcement also includes the “but for” language, the VHPD programs will be finding themselves up against the same difficulties that HPRP communities encountered in judging household eligibility for the program—and may come up with just as many variations on how to do it, what to count, and what to discount as they make this determination.

4 HOMES (VA Homeless Operations Management Evaluation System), is a new database that tracks service utilization and outcomes for VA-funded homeless services; it became operational in April 2011. HOMES will include data management information from VHA Service Support Center (VSSC) & HUD‘s Homeless Management Information System (HMIS). HUD’s HMIS system includes name, social security number, date of birth, race, ethnicity, gender, veteran status, disabling condition, residence prior to program entry, zip code for last permanent address, housing status, program entry date, program exit date, personal identification number and household identification number, income and sources, non-cash benefits, physical disability, developmental disability, chronic health condition, HIV/AIDS, mental health, substance abuse, domestic abuse, destination, date of contact, date of engagement, financial assistance provided, housing relocation and stabilization services provided.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2021-01-31