(Final) YHDP OMB SS-A 4-23-18

Supporting Statement for Paperwork Reduction Act Submission

Evaluation of the U.S. Department of Housing and Urban Development

Youth Homelessness Demonstration Project

OMB # 2528-XXXX

1. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary.

In early 2017, the U.S. Department of Housing and Urban Development’s (HUD’s) Youth Homelessness Demonstration Program (YHDP) began supporting 10 communities in developing and executing a coordinated community approach to preventing and ending youth (U.S. Department of Housing and Urban Development, 2017). YHDP funds communities to work with youth advisory boards, child welfare agencies, and other community partners to create comprehensive community plans to end youth homelessness. The grants fund a variety of housing options, including rapid re-housing, permanent supportive housing, and transitional housing, as well as innovative programs such as host homes. YHDP also supports youth-focused performance measurement and coordinated entry systems. In order to obtain a clear picture of YHDP grant activities, a longitudinal, multi-level evaluation will measure activities and performance of grantees essential to building and sustaining effective community change. This submission is a request for approval of data collection instruments and burden for the evaluation of the YHDP.

1. Background

The YHDP sites represent 10 diverse communities, including six urban and four rural areas; a range of geographies; and a span of population sizes. They were selected for YHDP funding in part because they appeared to be poised to make significant progress towards establishing a system level approach to addressing youth homelessness, with the anticipation that the grant would be sufficient to bring their efforts to fruition (U.S. Department of Housing and Urban Development, 2017).

Each site has an active Continuum of Care (CoC) within which a site-specific YHDP is being implemented. The CoC model for homelessness service delivery involves a system that guides and tracks clients over time through a comprehensive array of health and support services spanning all levels and intensity of care. It is an organizational model that involves collaboration across agencies that serve homeless and at-risk populations to provide an array of effective, community-based, culturally- and linguistically-appropriate services and supports for families and individuals (U.S. Department of Housing and Urban Development, 2012). Services provided can include housing, behavioral health care, food, transportation, independent living skill training, and employment support. Development of the CoC model began in the 1990s in response to the need to improve the organization, coordination, and delivery of homeless services and improve outcomes for homeless populations.

b. The Need for Evaluation

On December 18, 2015, the Consolidated Appropriations Act, 2016 (Public Law 114-113) made funding available to implement the YHDP, provide technical assistance (TA), and conduct an evaluation. The Act appropriated $33 million to HUD “to implement projects to demonstrate how a comprehensive approach to serving homeless youth, age 24 and under, in up to 10 communities, including at least four rural communities, that can dramatically reduce youth homelessness,” $5 million to HUD “to provide technical assistance on youth homelessness, and collection, analysis, and reporting of data and performance measures under the comprehensive approaches to serve homeless youth, in addition to and in coordination with other technical assistance funds provided under this title,” and a further $2.5 million to HUD “for homeless youth program evaluations conducted in partnership with the Department of Health and Human Services” (U.S. Department of Housing and Urban Development, Office of Community Planning & Development, 2016).

3. Clearance Request

This submission requests OMB clearance for (1) data collection to evaluate the YHDP and (2) the estimated burden for collecting data under this protocol. The estimated burden is for data collection in a total of 400 CoCs for a web survey and 13 CoCs for an implementation study (10 YHDP sites and three comparison sites).

3. Overview of the Proposed Evaluation

The evaluation design focuses on assessing the system responses in the demonstration sites through longitudinal case studies and examination of patterns of difference between these sites and the three comparison communities, as well as in the context of all CoCs. The YHDP evaluation (YHDPE) will identify and assess the mechanisms and strategies used to implement and expand community approaches to youth homelessness. It will include a process evaluation to understand both the process by which the CoCs develop and roll out coordinated community plans, and their efforts to use monitoring and evaluation to track their progress. It also will include a qualitative exploration of the experiences of homeless and at-risk youth and their interactions with the YHDP plans. Finally, a quantitative assessment of administrative data will measure changes in the number and composition of youth experiencing homelessness. The process evaluation data will be collected during three rounds of site visits in Years 1, 2, and 4 that will include key informant interviews, youth interviews, and youth focus groups. In addition, web surveys of CoC program directors (excluding the CoCs in the YHDP and comparison groups) in Years 1 and 4 will provide data on system developments across the country. Finally, data from Homeless Management Information Systems (HMIS) and other administrative data sets (such as child welfare and education) from the YHDP and comparison communities will be used to examine changes in the size and composition of the homeless youth population.

2. Indicate how, by whom and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

The purpose of the YHDPE is to assess the progress of the YHDP grants and examine whether and how the implementation of the program in the 10 sites is likely to accomplish the goals of the legislation. HUD requires evidence about communities’ implementation and process of developing coordinated approaches to youth homelessness. HUD also wants a qualitative study to better understand and document the opinions, suggestions, and experiences of the youth whom the demonstration was designed to serve. Finally, HUD needs a quantitative assessment that measures any changes in the size of the target population in all 10 YHDP sites relative both to the size of the target population prior to the start of the demonstration and in comparison, to the three comparison communities in which the YHDP was not implemented (U.S. Department of Housing and Urban Development, 2017). The Notice of Funding Availability to which the grantees responded incorporated the requirement that communities fully participate in the national evaluation activities conducted by HUD (U.S. Department of Housing and Urban Development, Office of Community Planning & Development, 2016).

The information collected for this evaluation will be used by HUD, Government Project Officers (GPO’s), grantees, and the homeless service provider community to: (1) fulfill the program’s legislatively mandated requirements to be evaluated, (2) provide essential program management and development information to HUD leadership, and (3) help articulate and understand youth’s housing needs and experiences with the YHDP grants.

Specifically, the findings will be used by HUD for:

• Monitoring the progress and activities of funded projects and the use and effectiveness of program technical assistance,

• Informing both intra- and interagency program and policy planning

• Developing policies and providing guidance regarding CoC development

• Supporting TA activities to help grantees best meet program goals

Some evaluation findings will be used by HUD and grantees for:

• Identifying best practices and effective strategies

• Understanding barriers and facilitators to successful implementation

• Determining whether observable differences in youth outcomes can plausibly be linked to the YHDP approach

• Understanding the development path of the YHDP’s as they move toward offering coordinated and comprehensive services

• Identifying funding sources used by states to sustain or expand CoCs services

• Describing experiences and implementation practices (across all grantees)

• Providing detailed information on how to successfully bring CoC youth programs to scale and sustain them

• Describing youth utilization patterns and experiences with the service supports

Grantees and the homelessness community can use evaluation findings to:

• Improve the implementation of their youth programs and achieve the goals of YHDP

• Improve the quality of the services they provide

• Identify additional opportunities for financing YHDP services with the goal of YHDP sustainability

• Learn what barriers to treatment and other essential services children or youth and their families perceive and work to eliminate such barriers

• Evaluate whether youth experience services as the grantees intended and identify programs strengths and weaknesses

• Identify gaps in system development and barriers to collaboration

• More effectively allocate personnel and funding and prioritize activities

• Provide summary reports to their local steering committees or other advisory boards, support statewide expansion efforts, develop interagency partnerships, and obtain resources to sustain systems with interagency agreements

Legislative Requirements. The Consolidated Appropriations Act, 2016 (Public Law 114-113) appropriated $2.5 million to HUD “for homeless youth program evaluations conducted in partnership with the Department of Health and Human Services” (U.S. Department of Housing and Urban Development, Office of Community Planning and Development, 2016).

HUD’s Strategic Initiative. In 2010 and updated in 2015, the United States Interagency Council on Homelessness (USICH) published Opening Doors: The Federal Strategic Plan to Prevent and End Homelessness (USICH, 2015a). This plan includes youth as one of four special populations and sets forth a goal of preventing and ending youth homelessness by 2020. At its heart, Opening Doors advocates a coordinated community approach and a system-wide prevention crisis response. It also emphasizes the importance of preventing and quickly diverting youth from homelessness and understanding and responding to the needs of youth experiencing homelessness. The Opening Doors strategic plan was expanded by the USICH in the Framework to End Youth Homelessness (USICH, 2013) and Preventing and Ending Youth Homelessness: A Coordinated Community Response (USICH, 2015b). These publications present the components necessary to prevent and end youth homelessness. The YHDP funding is aimed at learning how communities can address youth homelessness by building and expanding CoCs for youth rather than providing disconnected services. YHDP sites are developing comprehensive community plans to design better projects and strong coordination. The primary objectives of the YHDP include building national momentum to end youth homelessness, evaluating the coordinated community approach, expanding capacity to serve homeless youth, evaluating system-level performance measures on outcomes for youth, and establishing a framework for federal program and TA collaboration (U.S. Department of Housing and Urban Development, 2017).

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology.

CoC Web Survey. Information technology assists in the survey data collection in both survey administration and survey data management. In Years 1 and 4 of the evaluation the evaluation team will conduct brief web-based surveys of lead agency directors (or designees) in all 400 CoCs not in the YHDP or comparison communities. The use of web-based surveys and forms decreases respondent burden, as compared to that required for alternative methods, such as a paper format, by allowing for direct transmission of the survey or form. Respondents can complete the survey at a time and location that is convenient for them. In addition, the data entry and quality control mechanisms built into the web-based format reduce errors that might otherwise require follow-up, thus reducing burden compared to that required for a hardcopy data collection.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

There is no existing data source that can be analyzed for this evaluation. This data collection will provide information specific to the YHDP grants awarded under a new HUD funding program. The evaluation will serve as a primary mechanism through which effects of the YHDP will be evaluated. These data are not collected through any other mechanism.

5. If the collection of information impacts small businesses or other small entities describe any methods used to minimize burden.

Respondents for this data collection include grantees who are generally states or counties, not small entities. Every effort has been made to minimize the number of data items collected to the least number required to accomplish the objectives of the effort and to meet evaluation reporting requirements and therefore, there is no significant impact involving small entities.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

The evaluation was designed to keep the burden of data collection to a minimum. Exhibit 1 summarizes the maximum number of times each data collection activity is proposed to be conducted in each site (YHDP communities and comparison communities). The data collections proposed for the evaluation need to be collected more than once to assess change over time. Successful programs are expected to expand homeless youth services and improve youth outcomes. This project is a demonstration and less frequent data collection would severely curb HUD’s ability to adequately evaluate how well it was implemented and how well resulting projects met expectations of increased services and improved outcomes. A consequence of not conducting this collection is to lose the knowledge gained as to what types of efforts work and what don’t and to fail to obtain the information which this demonstration was designed to provide in order to better design interventions for the future.

Exhibit 1. Maximum Number of Administrations

Data Collection Activity	Maximum Administrations per Site
CoC web surveys	2
Lead agency interviews	3
Service provider interviews	3
Local government agency interviews	3
TA provider interviews	3
Youth board member interviews	3
Youth focus group discussions	18
Total	35

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

The proposed data collection activities are consistent with the guidelines set forth in 5 CFR 1320 (Controlling Paperwork Burdens on the Public). There are no special circumstances that require deviation from these guidelines.

• Under this ICR, HUD will not conduct any data collection requiring respondents to report information to the agency more often than quarterly;

• Under this ICR, HUD will not conduct any data collection requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• Under this ICR, HUD will not conduct any data collection requiring respondents to submit more than an original and two copies of any document;

• Under this ICR, HUD will not conduct any data collection requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;

• Under this ICR, HUD will not conduct any data collection in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

• Under this ICR, HUD will not conduct any data collection requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• Under this ICR, HUD will not conduct any data collection that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• Under this ICR, HUD will not conduct any data collection requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping disclosure, or reporting format (if any) and the data elements to be recorded, disclosed, or reported.

Federal Register Notice

In accordance with the Paperwork Reduction Act of 1995, HUD published a 60-Day Notice of Proposed Information Collection in the Federal Register on September 1, 2017. The Federal Register number was 82 FR 41635, and the notice appeared on page 41635-41637. The notice provided a 60-day period for public comments, and comments were due by October 31, 2017. No public comments on the proposed information collection were received. A copy of the notice is included as Attachment 1.

Outside Consultations

Both external and internal stakeholders were consulted in the development of the protocols, data collection methodology, and associated burden. HUD’s Office of Policy Development and Research (PD&R) obtained feedback and consultation regarding the availability of data, methods and frequency of collection, and the appropriateness of data elements. Section B.5 lists the specific experts and consultants with their contact information and area of expertise based on which they provided consultation for the relevant evaluation component. More information describing outside consultations follows.

Federal Consultation. The HUD lead designers for YHDP were Mathew Aronson, Ebony Rankin, and Sarah Hunter. Norm Suchar was the manager on the project.

Expert Consultation. The YHDPE team includes and has consultation agreements with experts in areas relevant to the evaluation, including homelessness research, positive youth development, CoCs, program evaluation, measurement, quantitative and qualitative analysis, economics, web site development and usability testing, and culturally responsive research. These experts are identified in Supporting Statement B, Section 5.

Youth and Family Consultation. Building Changes, a nonprofit based in Seattle, WA that addresses youth and family homelessness, is an integral part of the YHDPE and helped develop data collection procedures and training resources. Staff provide expertise in educational outcomes for youth experiencing homelessness, system responses to homelessness, and cross-system collaboration. Their contributions help ensure sensitivity to youth issues and concerns. Westat’s two consultants on the evaluation, Dr. Paul Toro and Dr. Mark Courtney, are national experts on youth homelessness and were fully involved in the design of the evaluation.

8. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

We will provide each youth who participates in a focus group an incentive of $20 gift cards. We will spend up to $2,000 in each site to assist with the costs of obtaining administrative data such as child welfare or education. No monetary incentives will be provided to other respondents.

8. Describe any assurance of confidentiality provided to respondents and the basis for assurance in statute, regulation or agency policy.

The YHDPE team is in the process of obtaining Internal Review Board (IRB) approval of all data collection tools which operates according to the Common Rule on the Protection of Human Subjects found in Title 45 of the Code of Federal Regulations, Part 46 (45 CFR 46). The team has multiple procedures in place to protect and ensure respondent confidentiality. All data collection efforts have informed consent processes that are being reviewed by the IRB. All project staff, including subcontractors and consultants, are required to sign confidentiality agreements, and all studies are reviewed by the IRB to ensure compliance with the Privacy Act and applicable regulations that protect human subjects. The evaluation team will comply with all requirements identified in the Privacy Act. The evaluation team will provide all involved employees, consultants, and subcontractors information regarding Privacy Act and systems security requirements. The information requested under this collection is protected and held private in accordance with 42 U.S.C. 1306, 20 CFR 401 and 402, 5 U.S.C.552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974) and OMB Circular No. A-130. A Privacy Threshold Assessment was approved by the Department on 12/1/2017. All electronic exchanges of data will be transmitted via a secure website. The facility’s security includes maintaining all electronic data storage media (e.g., data files, recordings), source documents, abstracts, reports, and any forms or lists that contain confidential or private information within secure areas that are locked or password-protected. The evaluation team will not disclose data or information to any person, organization, or agency other than those specifically authorized or prescribed by contractual procedures. Specific rules used to avoid inadvertent disclosure of identity will be determined in consultation with YHDPE statisticians, but typically involve suppressing data where a small number of respondents would otherwise appear in a table shell or text.

Prior to performing any project work or accessing any system, and on an annual basis thereafter throughout the period of the project, all members of the YHDPE team shall receive training on security awareness and human subjects protection commensurate with the responsibilities required to perform the tasks of the project. We will maintain a list of all individuals who have completed these trainings and shall submit this list to the Contracting Officer’s Representative (COR) upon request.

The YHDPE team shall make efforts to guard the names of respondents, all information or opinions collected in the course of interviews, and any information about respondents learned incidentally during the project. Hard copies of interview data and notes containing personal identifiers shall be kept in locked containers or a locked room when not being used. Reasonable caution shall be exercised in limiting access to data to only those persons who are working on the project and who have been instructed in appropriate Human Subjects requirements for the project.

Identifying information such as individuals’ names and addresses will not be part of any machine data record. Electronic files and audio files will be accessible only to project staff and under password-protection. Access to network-based data files is controlled through the use of Access Control Lists or directory- and file-access rights based on user account ID and the associated user group designation, which is maintained by the system administrator. Access control on PC’s is achieved for the most part by sound file management procedures by each user. Staff are instructed on the proper use of PCs for the storage, transfer, and use of sensitive information and the tools available, such as encryption.

YHDPE data collection involves five sources of data: (1) CoC program directors; (2) key stakeholders asked to respond based on their professional roles, not their personal thoughts or feelings; (3) HUD-funded providers of technical assistance (TA) to CoCs in the YHDP communities; (4) youth in the YHDP and comparison communities who have experience with homelessness; and (5) client-level administrative data. Informed consent forms and scripts for youth are included in the youth instruments. Our procedures include limiting the number of individuals who have access to identifying information, using locked files to store hardcopy forms, assigning unique IDs to each participant to ensure anonymity, and implementing guidelines pertaining to data reporting and dissemination. For all data collection activities, respondents will be informed that their participation is voluntary, that they have the right to discontinue participation at any time without impacting any services they receive, and of the risks and benefits of participation. Informed consent will be obtained from all respondents participating in interviews or focus groups.

CoC web survey. Potential web survey respondents will be contacted first by mail, email, or telephone to explain the survey. The explanation will include the voluntary nature of survey completion, treatment of responses, and the risks, benefits, and rights as respondents. When they access the web survey, before they complete and submit the survey, respondents will be asked to indicate by checking a box that they agree to participate in the evaluation. Information about the evaluation and participant rights will be presented in the survey prior to the checkbox indicating consent to participate. The letter and the web survey will also provide contact information if the survey recipient has questions or desires clarification prior to participation. If the individual does not have internet access, alternative administration methods will be used such as (1) a packet sent by regular mail containing a cover letter, an informed consent form, a survey, and a return envelope (the cover letter will indicate that the respondent is to return the informed consent form and the completed survey in separate envelopes enclosed in the packet) or (2) the survey will be administered by telephone interview following the procedures detailed below for TA provider interviews. At the end of the evaluation in 2021, the evaluation team will submit the data to HUD.

Key Stakeholders in their Professional Roles. Many of the data collection activities in the YHDPE involve interviews with key stakeholders such as lead agency staff, service providers, and government agency staff, as well as telephone interviews with TA providers. Interviews will be conducted by YHDPE staff during site visits to each of the 10 YHDP and three comparison sites in Years 1, 2, and 4. Respondents’ identities will be known so to ensure participants’ rights, an active informed consent process will be followed. Potential respondents will be contacted by mail, email, or telephone to explain the evaluation prior to the site visits. The explanation will include the voluntary nature of the interviews, treatment of responses, and the risks, benefits, and rights as respondents. With the consent of the respondents, the interviews will be audio recorded and transcribed. At the end of the evaluation in 2021, the evaluation team will submit the de-identified transcripts to HUD without names or other identifiers.

Youth interviews and focus groups. To enlist youth to participate in interviews and focus groups, we will ask the site leads in the YHDP and comparison communities to work with local organizations to identify potential interview and focus group participants. They will send the youth a letter (prepared by the evaluation team) that will introduce the evaluation and describe its purpose and the importance of including the youth perspective. If the leads prefer the evaluation team to send the letter, the team will follow a “consent to contact” process in which site leads ask youth if they consent to give the evaluation team their names and contact information. If the youth consent, the evaluation team connects with the youth and invites them to participate in the evaluation. At the interviews and focus groups, youth will be asked to sign a consent form that describes their rights as well as risks and benefits. A key provision of the consent process is guaranteeing youth’s confidentiality and ensuring that focus group participants respect each other’s confidentiality. In general, youth under age 18 need parental consent; however, in previous studies of similar populations, homeless youth living independently of their parents and youth in foster care can be considered emancipated and able to consent to participate without parental consent. With the consent of the respondents, the youth interviews and focus groups will be audio recorded and transcribed. At the end of the evaluation in 2021, the YDPE team will submit the de-identified transcripts to HUD without names or other identifiers.

HMIS and other administrative data. The client-level administrative data from each YHDP and comparison site will be stored in site-specific databases. The data will come from the HMIS at each site as well as other sources such as child welfare data and education data. When electronic data are transferred to the YHDPE team, data files will be encrypted to make the information indecipherable during electronic transfer. Data will be transmitted securely and all caution will be used. These records also are covered under the Privacy Act System of Records. Access to them will be password protected and data encryption will be used to enhance security. We will request information such as name, date of birth, and social security number so that we can link the data sets; once the data sets are linked, we will permanently delete the personal information. At the end of the evaluation in 2021, the YHDPE team will submit the data to HUD without names or other identifiers.

8. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

Because this project pertains to services to youth who have experienced homelessness, it is necessary to ask interview and focus group questions that are potentially sensitive as part of the YHDPE. In addition, the administrative data may include sensitive information such as substance abuse and mental health issues. However, only information that is central to the evaluation is being sought. Some questions asked of youth in the interviews and focus groups, as well as variables in the administrative data, are about child welfare and juvenile justice involvement, physical and mental health needs, and substance abuse. The answers to these questions and the analysis of the administrative data will be used to better understand the needs of the youth and the changes in these areas experienced after receiving YHDP services. To encourage candid responses, respondents will be reminded during the interviews that their responses will be kept confidential. Respondents will also be reminded that they can refuse to answer any question.

To avoid disclosing youth’s identities, only aggregated information will be disseminated. The primary dissemination vehicles will be the site visit memos and evaluation reports, though there may be other dissemination efforts as well. The evaluation team will be careful to avoid disseminating demographic information that might be used to deduce the identity of individual respondents. Specific rules used to avoid such dissemination will be determined in consultation with YHDPE statisticians, but typically involve suppressing data where a small number of respondents would otherwise appear in a table shell or text.

12. Provide estimates of the hour burden of the collection of information. The statement should: Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample of potential respondents (fewer than 10) is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices. Note: If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens.

Exhibit 2 shows the estimated annualized burden hours for the respondents’ time to participate in the instrument/data collection activity. Across the instruments, the annualized burden over four years is estimated to be 770 hours. Exhibit 2 also shows the estimated annualized cost burden associated with the participants’ time to take part in this research. The annualized cost burden is estimated to be $9,664.35. Across all years of the study, the total burden hours would be 3,080 and the total cost for the four years to be $38,657.40. The annual cost of information collection from CoC program directors assumes 400 respondents, surveyed on two occasions over the four years of the evaluation, ((400*2)/4=200). Presented as an annualized figure, the 400 respondents would need to provide 0.5 responses each year. It is further assumed that two lead agency staff, six service providers, two government agency staff, and two youth per site will be interviewed. One TA provider in each of the YHDP sites will be interviewed. There will be six focus groups, each with six youth, in each of the 13 sites. The full calculation assumptions are shown in Exhibit 3. Derivations for the column “Hourly Cost Per Response” are explained below.

Exhibit 2. Estimated Hour and Cost Burden of Information Collection

Information Collection	Number of Respondents	Frequency of Response (Per Annum)	Responses Per Annum	Burden Hour Per Response	Annual Burden Hours	Hourly Cost Per Response	Annual Cost
Continuum of Care (CoC) Web Survey (CoC Program Directors)	400.00	0.50	200.00	0.50	100.00	$31.10	$3,110.00
Lead Agencies Interview	26.00	0.75	19.50	2.00	39.00	20.73	808.47
Service Provider Interview	78.00	0.75	58.50	1.00	58.50	20.73	1,212.71
Local Government Agency Staff Interview	26.00	0.75	19.50	0.80	15.60	23.39	364.88
TA Providers Interview	10.00	0.75	7.50	1.00	7.50	20.73	155.48
Youth Board Member Interviews	26.00	0.75	19.50	1.00	19.50	7.25	141.38
Youth Focus Groups	468.00	0.75	351.00	1.50	526.50	7.25	3,817.13
Total	1,034.00		675.50		766.60		9,610.05

Exhibit 3. Estimated Hour Burden of Information Collection Calculation Basis

Information Collection	Number of Respondents	Frequency of Response	Responses Per Annum
CoC Program Directors	400	2	(400x2)/4 = 200
Lead Agencies	2/site, 13 sites = 26	3	(26x3)/4 = 20
Service Providers	6/site, 13 sites = 78	3	(78x3)/4 = 59
Local Government Agencies	2/site, 13 sites = 26	3	(26x3)/4 = 20
TA Providers Interview	10	3	(10x3)/4 = 8
Youth Board Members (Interviews)	2/site, 13 sites = 26	3	(26x3)/4 = 20
Youth Focus Groups	36/site, 13 sites = 468	3	(468x3)/4 = 351
Total	1,034.00		678.00

As summarized in Exhibit 4, we estimated the hourly cost per response using the May 2016 Bureau of Labor Statistics, Occupational Employment Statistics median hourly wages for the labor categories Social and Community Services Manager (11-9151, $31.10) and Social and Community Services Specialist, All Other (21-1099, $20.73). We used the Social and Community Services Manager rate for the CoC Program Directors and Program Administrators. We used Social and Community Services Specialist, All Other rate for YHDP grantee staff, service providers, and TA providers. For the government workers, we used an average of state and local Social and Community Services Specialist, All Other (21-2099, $23.39). The youth hourly wage is based on the federal minimum wage of $7.25/hour.

Exhibit 4: Median Hourly Wages for Respondents

Respondent	Occupation	SOC Code	Median hourly wage
CoC Program Directors	Social and Community Services Manager	11–9151	$31.10
Lead Agencies	Social and Community Services Specialist, All Others	21-1099	$20.73
Service Providers	Social and Community Services Specialist, All Others	21-1099	$20.73
Local Government Agencies	Social and Community Services Specialist, All Others	21-1099	Average of state and local, $23.39
TA Providers	Social and Community Services Specialist, All Others	21-1099	$20.73
Youth	Federal minimum wage	--	$7.25

Source: Bureau of Labor Statistics, Occupational Employment Statistics (May 2016), https://www.bls.gov/oes/current/oessrci.htm

13. Provide an estimate for the total annual cost burden to respondents or record-keepers resulting from the collection of information. Do NOT include the labor cost (wage equivalent) of the burden-hours described in item 12 above. The information required here corresponds to that in item 14 on the 83-I (cost to the public).

There are neither capital nor startup costs, nor are there any operations or maintenance costs. There are no additional total annual cost burdens to respondents or record-keepers beyond the labor cost of burden-hours described in item 12 above.

13. Provide estimates of annualized cost to the Federal government.

The estimated total cost to the federal government of task 8: “In-person site visits and data collection activities” of the planned work under the contract awarded for this evaluation is $1,312,084 over a 48-month period of performance. This maximum amount was derived from the contractor price model showing a total dollar amount for labor hours of $1,042,505, plus costs of $269,579 for travel, incentives etc. Thus, the total cost for this task arrived at $1,312,084. This figure was divided over four (4) years to equal the annualized contract cost of $328,021.

Exhibit 5: Calculation of annualized cost to the Federal government

Contract Cost Type	Dollar Amount
Labor Hours	$1,042,505.00
Non-labor costs (Travel, Incentives, etc.)	$269,579.00
Total Cost for Task 8: Site Visits and Data Collection	$1,312,084.00
Task Dollar Amount/Period of Performance of 4 years	=$1,312,084.00/4-years= $328,021.00 a year

13. Explain the reasons for any program changes or adjustments reported in Items 13 and 14 of the OMB Form 83-I.

This is a new study.

13. For collection of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

a. Time Schedule

The time schedule for implementing the evaluation is summarized in Exhibit 5. A 4-year clearance is requested for this project.

Exhibit 5. Schedule of Evaluation Activities

Activity	Projected Date
Data use agreements	January 2018
OMB approval for data collection	June 2018
Year 1 data extractions	June through September 2018
Year 1 site visits	June through September 2018
Year 1 phone interviews with TA providers	April 2018
Interim evaluation report	October 2018
Year 2 site visits	December 2018 through February 2019
Year 2 phone interviews with TA providers	January 2019
Qualitative research report	August 2020
Year 4 site visits	January through March 2021
Year 4 phone interviews with TA providers	February 2021
Final evaluation report	March 2021
Final HUD briefing	April 2021

b. Data Analysis Plan

A description of data analyses for each evaluation component as well as a description of the multi-site comparative analysis is shown below. For further details, the analysis plan from the YHDPE Research Design/Data Collection and Analysis Plan is presented in Attachment 2.

CoC Surveys. Analyses of the CoC survey data will be conducted in Years 1 and 4. Analyses in Year 1 will provide an understanding for the starting points of the demonstration communities relative to other CoCs in the country. The evaluation team will examine the degree to which each community has a coordinated response to youth homelessness, including coordination across homeless service providers and coordination across systems. The team will examine the extent to which each community employs prevention, outreach, coordinated assessment and entry, as well as the availability of housing and services and supports for youth experiencing homelessness. Additionally, the team will examine variations in overall community size, whether they are urban/rural, and the size of the population of youth experiencing homelessness. Year 4 analyses will examine changes over time in the systems and the size and nature of the populations they serve and will provide a basis of comparison among all CoCs for how the demonstration communities changed over time. We will conduct descriptive analyses, including cross-tabs and chi-square analyses for categorical measures and t-tests for continuous variables of interest to assess significant differences between Year 1 and Year 4.

Case Studies. Case studies will be comprised of interviews and focus groups with key stakeholders and youth, reviews of documents and CoC data, and information gathered during routine phone calls with the sites’ lead agencies and TA providers. Analyses for the case studies will be guided by the study protocol. For the initial analysis of the interview and focus group transcripts, the team will generate a codebook and set of a priori codes that mirror the interview protocol. Using the qualitative analysis software program NVivo, the team will code each of the transcripts to measure the occurrence of each of the codes. Staff conducting the coding will meet regularly to discuss the codebook and check reliability. During the initial coding period, the research team will assign the transcripts to pairs of coders, and the lead coder will check to ensure inter-rater-reliability (i.e., that different raters are using codes in the same way). The team will aim for a reliability of 90 percent agreement in codes. Discrepancies between codes will be discussed and reconciled. After the data are coded, the team will then run reports to identify common themes within sites and across methods, and use these themes to develop key findings.

Administrative Data. Analyses of the HMIS and other administrative data will be conducted in Year 1 for the YHDP and comparison communities to understand the nature of the populations at baseline. Descriptive analyses will characterize the size and nature of the baseline target population in each of the demonstration and comparison sites. The evaluation team will determine the total number of youth experiencing homelessness for each site and the size of subpopulations of interest (pregnant and parenting youth, minors, and LGBTQ youth) where they can be identified with available administrative data. The estimates of the number of youth experiencing homelessness in each site will be based on both HMIS and other administrative datasets. Analyses of the HMIS in each community will indicate the number of unduplicated youth who are receiving homeless services (e.g., shelter, transitional housing, rapid re-housing, permanent supportive housing). Analyses of other administrative datasets will help identify the number of youth experiencing homelessness who are not currently receiving services from the homeless service system. For youth connected to the homeless service system, our analyses will include demographic characteristics (i.e., age, race, gender), education and employment status (where available), program type (i.e., shelter, transitional housing, rapid re-housing), length of stay in the system, receipt of assistance from youth-specific programs (as opposed to the family or adult system) and exits to permanent housing. For youth who are not connected to the homeless service system, our analyses will include demographic characteristics, education and employment status (where available), and program information about the system (e.g., child welfare, juvenile justice) with which they are involved, such as length of time in foster care or detention.

In Year 4, we will compare the Year 1 and 4 data sets to analyze changes in the population of youth experiencing homelessness over time within each community. Again, the analyses will draw on the HMIS as well as other administrative datasets. We will conduct descriptive analyses, including cross-tabs and chi-square analyses for categorical measures and t-tests for continuous variables of interest to assess significant differences between the two-time periods. These analyses will include an examination of the overall population size, as well as subcategories of interest (e.g., minors, parenting youth, LGBTQ youth, unsheltered youth). We will compare over time the size of the populations connected to the homeless service system and those not connected. Additionally, we will examine changes overtime in demographic characteristics such as race, education, and employment status of homeless youth. Finally, we will also examine changes over time in program type (i.e., shelter, transitional housing, rapid re-housing), length of stay in the system, whether served by youth system or other homeless service system and exits to permanent housing.

Finally, in Year 4 we will examine how changes over time in the YHDP communities compare to the three comparison communities. To the extent the data across the 13 sites are comparable, we will conduct statistical comparisons on the percentage of the unsheltered youth population, the percentage of youth served in youth-specific programs, the length of time youth are served in the system, and exits to permanent housing. To the extent that the same measures are available across the sites, we will conduct multivariate regression analyses on these same outcomes, nested by site. For this cross-site analysis we will also control for county-level characteristics, such as cost of living, vacancy rates, etc.

Contextual Data. Our analysis comparing changes over time will weave together extant data from multiple sources. In addition to the administrative data, we will also weigh changes in contextual factors that may affect the size of the homeless population. Contextual factors include the unemployment rate, the minimum wage, the cost of living, the vacancy rate, and the fair market rent for a one-bedroom apartment among others.

Multi-site Comparative Analysis. The last set of analyses will be multi-site, pulling across the different qualitative and quantitative data sources to understand how the YHDP sites compare and contrast with the comparison sites and within the context of all CoCs. Guided by the key research questions, the data will be first summarized within site and then examined across sites at baseline, midway through the evaluation, and at the end. Analysis will examine patterns across the sites, particularly as they relate to the sites’ “starting points.” Analysis will also seek to understand the site factors that correlate with system-level changes, such as changes in: coordination across the system, the services and supports available to youth experiencing homelessness, and the size and composition of the population. We will examine the role TA had in shaping the development and implementation of the CoC plans as well as the role that youth played in the planning and implementation processes. Administrative data will be incorporated into the case studies at baseline and in the final set of analyses.

13. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

All data collection instruments will display the expiration date of OMB approval.

13. Explain each exception to the certification statement identified in item 19.

No exceptions are necessary for this information collection.

References:

National Alliance to End Homelessness (2009). Ending Youth Homelessness Before It Begins: Prevention and Early Intervention Services for Older Adolescents. Retrieved from: http://www.endhomelessness.org/page//files/2455_file_Early_Interv_Prevention_Issue_Brief__7_24_09__FINAL.pdf

U.S. Department of Housing and Urban Development (2017). Evaluation of the Youth Homelessness Demonstration Program for HUD. RFP 17-233-SOL-00164.

U.S. Department of Housing and Urban Development (2012). Introductory Guide to the Continuum of Care (CoC) Program. At https://www.hudexchange.info/resources/documents/CoCProgramIntroductoryGuide.pdf

U.S. Department of Housing and Urban Development, Office of Community Planning & Development (2016). FY 2016 Notice of Funding Availability for the Youth Homelessness Demonstration Program (YHDP). Retrieved from: https://www.hudexchange.info/resources/documents/YHDP-NOFA.pdf

United States Interagency Council on Homelessness (2013). Framework to End Youth Homelessness: A resource text for dialogue and action. Retrieved from: http://www.usich.gov/resources/uploads/asset_library/USICH_Youth_Framework__FINAL_02_13_131.pdf

United States Interagency Council on Homelessness (2015a). Opening Doors: federal strategic plan to prevent and end homelessness. Retrieved from: https://www.usich.gov/resources/uploads/asset_library/USICH_OpeningDoors_Amendment2015_FINAL.pdf

United States Interagency Council on Homelessness (2015b). Preventing and Ending Youth Homelessness: A Coordinated Community Response. At https://www.usich.gov/resources/uploads/asset_library/Youth_Homelessness_Coordinated_Response.pdf

Attachments:

1. Federal Register 60-Day Notice of Proposed Information Collection, September 1, 2017

2. Analysis Plan from the YHDPE Research Design/Data Collection and Analysis Plan

12