A. Justification
Circumstances Making the Collection of Information Necessary
The Runaway and Homeless Youth Act (RHYA), as amended by Public Law 106-71 (42 U.S.C. 5701 et seq.), provides for the Transitional Living Program (TLP), a residential program designed to prepare older homeless youth ages 16-21 for a healthy and self-sufficient adulthood. The following amendment was included in the 2003 “Runaway, Homeless Youth and Missing Children’s Assistance Act “ (P.L. 108-96), which reauthorized the” Runaway and Homeless Youth Act”.
“STUDY OF HOUSING SERVICES AND STRATEGIES
The Secretary of Health and Human Services shall conduct a study of programs funded under part B of the Runaway and Homeless Youth Act (42 W.S.C. 5714-1 et seq.) to report on long-term housing outcomes for youth after exiting the program. The study of any such program should provide information on housing services available to youth upon exiting the program, including assistance in locating and retaining permanent housing and referrals to other residential programs. In addition, the study should identify housing models and placement strategies that prevent future episodes of homelessness.”
Purpose and Use of the Information Collection
The proposed collection is a set of surveys to be administered to youth in residential services (Part B, Transitional Living Program or TLP) to measure their long term outcomes and effective service strategies. In addition to collecting information on housing outcomes, the study will also consider the living, employment, education, and family situation of the youth before and after their time in the TLP. Grantees will also be surveyed on service methods and program structure. This information will be used to better understand the most effective practices in improving long-term outcomes of youth in an effort to guide program improvements.
The data collection will provide baseline, service, exit, and follow-up information about the experiences and outcomes of homeless youth in the TLP. Characteristics of programs will also be documented. Baseline and exit surveys will be administered when youth enter and are discharged from the program. Information about services provided to the youth and about the service approach, administrative, and organizational structure of participating grantees will be provided by TLP agency staff. 6 months and 12 months after a youth leaves the program, he or she will complete an online survey about their experiences after they exit.
The study will compare situational and behavioral outcomes among various program models and assess the role played by dosage, demographic or regional factors, along with characteristics of the youth themselves. Statistical methods will enable us to not only document how youth fare after program exit, but also identify promising practices associated with more successful long term living situations. It will document access to housing and other services in various communities.
Use of Improved Information Technology and Burden Reduction
Each of the surveys will be completed online via a secure web page. When youth leave the program, their six and 12 month follow-ups will be via a secure but youth-friendly website.
Efforts to Identify Duplication and Use of Similar Information
This study is a unique, one-time event studying a population that has not previously been surveyed nationally or as intensively and systematically as this research envisions. FYSB collects in-service information on non-identified youth through the Runaway and Homeless Youth Management Information System (RHYMIS). Information from RHYMIS will provide background and context but not in sufficient detail or individual specificity to produce quality research. Moreover, RHYMIS reports on youth services and issues while they are in the TLP. The important living status of youth after they leave the program is not available from RHYMIS, only their immediate destination at exit.
5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize burden.
Most TLP grantees are small entities. The information being requested has been held to the minimum required for the intended use. The information collection will be implemented on a one-time basis. Surveys were designed by experts, and field tested with staff and youth at three TLP sites in different parts of the country to construct the surveys, refine the questions and how they were worded, and assess the time required to answer all questions and the sensitivity of some questions.
There will be no impact on grantees other than the time required by staff or youth to fill out each survey. Grantees may be asked to help contact youth who have not been heard from when their expected follow-up reports are due, but the actual follow-up effort will be implemented by a contractor. Findings from the survey will in no way impact the funding or management of the grantees, although program design improvements may result over the long term as the effectiveness of various models is determined.
6. Consequences of Collecting the Information Less Frequently
This is a one-time collection.
7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5:
requiring respondents to report information to the agency more often than quarterly;
This is a one-time collection.
requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
Information will be collected on each youth as they enter and exit the program, preferably on about the same day. In order for memories to be fresh and information accurate, there should not be major delays in administering surveys during the timeline. For example, the baseline survey should be given to each youth during their early orientation to the residential program. The timing of follow-up surveys calls for youth responding as near as possible after a six month interval for the first follow-up and after another six month interval thereafter. In this case, long delays may not only affect the ability to recall certain factors, but also negatively impact analytical frames dependent upon timing and the spacing of various events, such as the day of discharge and each of the two six month anniversaries.
requiring respondents to submit more than an original and two copies of any document;
Not required.
requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;
Not required.
in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;
Statistical reliability and valid generalization to the universe of study will be part of the inherent design and adhered to in process.
requiring the use of a statistical data classification that has not been reviewed and approved by OMB;
Not applicable.
that includes a pledge of confidentiality that is not supported by authority established in statue or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or
Youth will be promised confidentiality before providing informed consent. The Runaway and Homeless Youth Act at Sec 322, (a)(13) states that grantees must pledge “not to disclose records maintained on individual homeless [TLP] youth without the informed consent of the individual youth to anyone other than an agency compiling statistical records.”
requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.
Not applicable.
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
The 60 day Federal Register Notice published August 25, 2008, with vol. 73 on page number 50022.
In addition to publication of the First Notice in the Register, FYSB emailed copies of all instruments, along with an invitation for comments and input, to the over six hundred RHY programs, not only TLPs, but Basic Centers and Street Outreach Programs. A number of individuals in the research community had expressed interest at various points prior to the release of the instruments and these, too, were contacted. A staffer at the Congressional Research Service who include RHY programs in her assignment area also expressed interest and provided the most extensive comments that were received.
Commenters asked if the surveys were “a lot for youth to fill out.” One commenter wrote: "I feel that the surveys for the youth are very comprehensive and cover a sufficient amount of information to be effective. I do however have concerns about the length of the surveys for the youth. I feel that some of the information should be combined or cut down to allow youth to complete this survey within a 15 minute time frame.... The concern is that the youth will not take it seriously and answer all the questions to their best ability.”
We explained that a group of TLP youth had tested the surveys during site visits and their reactions were taken into consideration. On average, each survey takes about 15-20 minutes, but we used 30 minutes in the burden calculation. We also explained that staff would help youth understand the value of the information and how it would help improve the program.
Two comments asked about gifts to encourage participation, and we explained the incentive approach described in item 9.
Questions were asked about responsibility for tracking and ensuring completion, about the sample size, and whether a web-based response was under consideration. The answers are as they are found in this statement.
Other comments/questions included:
“I understand the challenges associated with random assignment in this type of study, and that such a study would not likely be possible. Does HHS intend to compare the outcomes of youth in this study to the outcomes of youth who are or were eligible to receive TLP services, but did not (even if those youth were not randomly assigned to the control group)?”
Youth on waiting lists generally enter TLP after a short delay and many do receive services in the interim. We are not in favor of denying lifeline services and residencies to a control group. Tracking an eligible youth who decides himself or herself not to participate would present challenges in eliciting cooperation and maintaining contact. The decision not to participate might well represent a factor that would introduce bias into the control group or quasi control group.
“The surveys do not address the type of relationships youth have with their families. It would be useful to learn about family dynamics before and after entry into the TLP. NEO-RHYMIS asks for similar information, but perhaps this survey could ask more probing questions.”
The only outcomes that Congress specifically asked about involved housing. Because sustained independent living and the ability to maintain permanent housing self-sufficiently depend on educational, employment, mental health, risk avoidance, etc., we have added questions in those areas. Relationships with family and others are important, but we have tried to be mindful of the burden of too many questions.
We made changes in response to a number of suggestions as follow:
To living situation prior to services, we added a few settings including: group home, independent living setting, drug treatment facility, mental health/psychiatric residential treatment facility.
To questions about experiences/services in the program we added: recreational services and voluntary community service activities (along with questions about the youth’s involvement in planning the activity, etc.).
For what the youth found helpful in the TLP we included: “Developing skills to live on my own and make positive decisions” and “Gaining leadership and/or decision-making or life skills.”
To the list of what adults in and outside of the TLP provide the youth we added, “[Someone] you can talk to about your goals and help you reach them.”
To healthcare needs we added “I needed prenatal services or care for my child.”
The item “in a friend’s home” under current living situation, we expanded to “In a friend's home, without a written agreement (like a signed lease) and not paying rent or not paying rent regularly.” The choices already include “In a private room, apartment or house, with a roommate/roommates, paying for my share of the rent.”
To the list of substances that youth will be asked about, we added club drugs like ecstasy, psychedelics and “Medicine prescribed for you but in amounts that were much larger or smaller than the doctor ordered (for example, to feel good).”
To the grantee survey, we added: “Does the program serve a particular group of youth, such as migrant youth, immigrant youth, or minority youth (e.g., Native American youth)?”
In the grantee survey under “factors that seem to increase the likelihood that youth will become homeless again or leave the program without completing it,” we added response categories:
Youth struggles with mental health issues
Youth struggles with substance abuse addiction
The program rules are too strict for the youth
There are a lack of educational/social support/vocational resources in the area
Limited public transportation is a challenge for youth to get to work each day
Other (please describe)
Under “services or aspects of the program do you feel most contribute to positive youth outcomes” we added response categories:
Clear code of conduct with consistent enforcement
The role of staff and leadership, including building connections between youth and adults
Service plans for youth
Experiential learning opportunities
Collaboration with other agencies to provide services to youth after they leave the program
Aftercare services
Other (please describe)
We added a question that asks whether youth are involved in developing the service/action plan. We added a question about the role of the program’s board and volunteers. We added a question asking whether youth are provided with incentives to follow the rules, and examples of any such incentives.
Some questions about individual beliefs and attitudes were reworded for clarity, and throughout the document we followed a number of suggestions on format, wording, semantics and stylistic expression to avoid confusion in the respondent.
9. Explanation of Any Payment or Gift to Respondents
An incentive for completing surveys that has been effective for research into the homeless population is a programmable gift card, renewable online. Each youth will receive a card from a national retail store carrying items which most youth find attractive or practical. Retail chains have different gift card programs, and we inquired with Target, one of the largest and most accessible stores nationwide. Youth will be issued a Target gift card upon agreeing to participate in the study, and the card will contain a nominal value ($1). Within 48 hours of completion of a survey, research staff will authorize the addition of $30 to the youth’s Target Card by sending an updated tracking sheet to Target Corporate Card. Within two hours of notifying Target staff, the youth will be able to use the card for purchases at any Target location or online.
These incentives are designed to increase the survey response rates, particularly from youth dispersing widely into the general population as they gain the independence which is the purpose of TLP. We anticipate the need for both passive and active tracking approaches. We will rely on multiple tracking mechanisms to ensure the highest response rate possible, and the type of mechanism utilized will vary depending on the status of youth.
10. Assurance of Confidentiality Provided to Respondents
To relate after-service experiences with program features and individual youth characteristics, identifying information will be collected with each youth’s informed consent and all records will be protected with security systems to guarantee privacy and confidentiality. Each survey will contain an assurance, and data will be securely protected under IRB-certified ethical research methods.
11. Justification for Sensitive Questions
Some questions, such as about recent substance use, sexual behavior, involvement with law enforcement, and health status, are very personal but also essential to understanding a youth’s overall situation. Independence and self-sufficiency, education completion, and employment success are valued outcomes of TLP, but risk reduction, healthy choices, and pro-social adaptation are important as well. Youth will be assured of privacy and told they do not have to answer a particular question if they find it objectionable.
12. Estimates of Annualized Burden Hours and Costs
ANNUAL BURDEN ESTIMATES
The burden estimates are based both on pretests of the survey instruments, as well as experience with similar data collection efforts for other studies.
Instrument |
Number of respondents |
Number of responses per respondent |
Average burden hours per response |
Total burden hours |
Baseline Survey |
760 (youth) |
1 |
.5 |
380 |
Exit Survey |
760 (youth) |
1 |
.5 |
380 |
Service Log |
760 (by staff for each youth) |
1 |
.25 |
190 |
Grantee Survey |
70 (two staff from each grantee in sample) |
1 |
1 |
70 |
Follow-up Survey (6 months) |
760 (youth) |
1 |
.5 |
380 |
Follow-up Survey (12 months) |
760 (youth) |
1 |
.5 |
380 |
Estimated Total Annual Burden Hours: 1780
The dollar value of this time $25 times 1780 which equals $4,250.
13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers
There are no annualized capital/startup or ongoing operation and maintenance costs associated with collecting this information. There are no direct respondent costs associated with this data collection other than opportunity costs of respondents’ time required to complete the surveys. The evaluation does not place any capital equipment, start-up, or record maintenance requirements on respondents.
14. Annualized Cost to the Federal Government
The overall one-time cost of the research, including survey design, information collection, analysis, and reporting will be $700,000 over at least three years, which comes to $233,333 annually. Conceivably, the time frame could run to four years, depending on how long it takes to reach a full sample and how long most of the youth remain in the TLP.* The annualized cost would then be $175,000. The cost estimate is based upon the budget of the project contract and takes into account salaries of research personnel, cost of subcontracts, administrative overhead, etc.
* TLP residencies last an average of 7 or 8 months, but are permitted for “a continuous period not to exceed 540 days, or in exceptional circumstances 635 days.” A youth not yet 18 at that time may remain in the program until their 18th birthday. The timeline must allow for the 2nd follow-up surveys to be completed 12 months after the last youths are discharged.
15. Explanation for Program Changes or Adjustments
This request is for a new information collection.
16. Plans for Tabulation and Publication and Project Time Schedule
When the study is completed, findings will be reported to Congress and then made available to the general public and research community. Detailed data tabulations will be part of the record and available for study. No personal information will be included.
Estimated Timeline
Planning and conceptualization 4th qtr, FY07
Evaluation design and draft survey instruments 1st qtr, FY08
Site visits (3) and field test of instruments 2nd qtr, FY08
Finalization of survey instruments and design 4th qtr, FY08
OMB approval of instruments 2nd qtr, FY09
Site selection (70); 2rd qtr, FY09
Training of survey administrators 2rd qtr, FY09
Intake of participating youth (760) 2rd qtr, FY09 – ~4th qtr FY09
Exit surveys as youth finish TLP ~2nd qtr, FY10 – 2nd qtr, FY11
1st follow-up ~4th qtr, FY10 – ~4th qtr, FY11
2nd follow-up ~2nd qtr, FY11 – ~2nd qtr, FY12
Analysis and final report 3rd qtr, FY12
17. Reason(s) Display of OMB Expiration Date is Inappropriate
Not applicable.
18. Exceptions to Certification for Paperwork Reduction Act Submissions
Not applicable.
B. Statistical Methods (used for collection of information employing statistical methods)
Respondent Universe and Sampling Methods
The sampling frame includes all TLPs funded in FY09, and 35 TLPs will be selected to participate in the study. The selection process will include both a purposive and random approach. We will purposively select the five largest programs in the country for inclusion in the study. Based on FY06 RYHMIS data, the five largest programs represent 2 percent of the total number of TLPs nationwide but serve 16 percent of all youth enrolled in a TLP during that year. The inclusion of these large programs provides “face validity” by ensuring that the study’s results account for the largest TLPs in the country.
The large TLPs also evince high youth entry/exit rates and thus contribute significantly to the ability to recruit enough youth in the study to produce reliable outcome estimates. The remaining 30 TLPs in the sample will be selected randomly. However, the study will stratify the remaining universe of TLPs (i.e., the total number of TLPs after excluding the five largest grantees) by size.
Five grantees will be selected randomly among the universe of TLPs with less than 10 youth served annually. According to FY06 RHYMIS data, about 22 percent of TLPs nationwide served less than 10 youth during a 12-month period, but account for less than 3 percent of all youth served by a TLP during that time period. The balance of programs (25) will be selected randomly for inclusion in the study.
The total youth sample size is 760 and will include all youth who enter and exit from the 35 TLPs in the sample. Based on monthly turnover rates (i.e., the average proportion of youth who enter and exit on a monthly basis), it is estimated that the recruitment period will take approximately 9 months.
The information collected from programs and youth will be supplemented with up-to-date data from RHYMIS. The RHYMIS data will be used to describe the characteristics of TLPs and youth enrolled in TLPs nationwide. The data will also be used to compare the demographic profile of TLP youth nationwide with the profile of youth in the TLP sample.
Procedures for the Collection of Information
The research approach uses a series of web-based surveys to understand TLP program models and track youth experiences during and after program enrollment. A website will serve as the central repository for data collection and will allow research staff to monitor survey completion rates. The website will permit survey respondents (both TLP grantee staff and youth) to log-in using a unique username and password and complete their respective surveys online. Grantees will be asked to complete a survey about their program model and report on the number and types of services provided to each youth participating in the study. Youth who enroll in a TLP will be asked to complete a survey shortly after they enter the program (at “baseline”), at program exit, and at two 6-month intervals following program exit.
Methods to Maximize Response Rates and Deal with Nonresponse
Grantee staff will participate under their statutory obligation to cooperate with HHS research activities, but, as discussed in item A9, grantees will receive a small compensation for their administration assistance and completion of service logs and grantee surveys. Youth will be encouraged and reassured by grantee staff, and the incentive payments for youth are expected to increase the survey response rates. Nevertheless, in anticipation of the need for both passive and active tracking approaches, we will rely on multiple tracking mechanisms to ensure the highest response.
To assist in tracking and follow-up, contact information, last known address, and other data will be collected from youth when they are first surveyed. For the 6- and 12-month follow-up surveys, the contractor will automatically send emails to youth one week prior to the expected survey completion date using email addresses reported by youth on their exit surveys as the first mode of contact. The email will remind youth to complete the appropriate survey, contain a link to the survey, and indicate how they can receive the incentive payment. If a survey is not completed within one week of the initial email invitation, a second email invitation will be sent out as a reminder to complete the survey.
Youth who do not respond to the automated emails will be contacted by telephone two weeks after the initial email invitation date using contact information reported by youth on their exit surveys. Three attempts will be made to prompt youth to complete the surveys on-line, although during any of these attempts, youth who are willing to provide the information directly over the phone will be interviewed and their responses recorded as the interview is being conducted.
The study will also rely on TLP agency staff to help locate youth who have disappeared and are non-responsive to our emails or telephone calls. Case managers develop a rapport with youth during youths’ program stay and they may have information about youths’ whereabouts. TLP staff will be asked to assist in tracking youth as needed.
The surveys ask youth to provide information on 3 persons—i.e., family member, friend, and other trusted adult—who would know about the youths’ whereabouts. This information will help track youth who disappear and are not in contact with TLP staff or the research team. The tracking will rely primary on telephone calls to each of these individuals, as needed, to locate youth in the study and collect the survey information. All survey data will be stored on a secure server and available only to contract project staff.
Test of Procedures or Methods to be Undertaken
A pretest involving fewer than 10 respondents has been conducted during visits to three TLP sites around the US. It was considered adequate for the purposes of the study. Youth in three TLP programs, the same group to be measured, were asked to take and review the surveys.
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
FYSB program manager: Stan Chappell, 202-205-8496
Abt Associates (statistical and research contractor)
Alvaro Cortes (301) 634-1857
Emily Holt, Jill Khadduri
| File Type | application/msword |
| File Title | Information Collection Supporting Statement |
| Author | USER |
| File Modified | 2009-05-04 |
| File Created | 2009-03-31 |