Planning Grants to Develop a Model Intervention for Youth/Young Adults with Child Welfare Involvement At-Risk of Homelessness
OMB Information Collection Request
New Collection
Supporting Statement
Part A
March 2014
Submitted by:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
7th Floor, West Aerospace Building
370 L’Enfant Promenade SW
Washington, DC 20447
Project Officer:
Maria Woolverton
A1. Necessity for the Data Collection
The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval for data collection activities with grantee and partner organizations that are receiving planning grant funds through the Children’s Bureau (CB) in ACF. These activities will include (1) a survey sampling form to gather contact information for survey respondents, (2) a survey instrument, (3) communication related to planning site visits, and (4) individual interviews or focus groups with key informants during site visits to be used for the process study of efforts by grantees to develop model intervention programs during a two-year period.
In order to improve the well-being of youth and young adults with child welfare involvement who are at risk of homelessness, CB is supporting the development of comprehensive, integrated service models based on the youth framework from the U.S. Interagency Council on Homelessness (USICH).
In spite of the recent changes in federal policy intended to assist these youth and young adults, there is currently no solid evidence showing what works to provide stability for this vulnerable population. To date, efforts to address homelessness and unstable housing among former foster youth have been limited and approaches to these problems have not been rigorously tested. The U.S. Department of Housing and Urban Development (2012, p. 28) has noted: “The lack of rigorous evaluations of housing programs for young people who have aged out of foster care means that we do not know whether any of the programs prevent homelessness or otherwise reduce housing instability.”
Experts agree that approaches to the problems of homelessness and housing instability must engage multiple human services systems. To fully address the problem, there must be a comprehensive, multisystem approach—ideally, including the criminal justice system—and long-term programming (Kroner 1999). The evidence to date indicates that approaches to preventing homelessness among former foster youth must be adapted to the difficult array of situations they face.
Policymakers have recently paid more attention to the problems facing former foster youth and have made new program resources available for them. But the field lags behind other efforts launched by HHS and other federal agencies in building and sustaining comprehensive, integrated, evidence-based services and supports. On September 30, 2013, ACF announced the award of planning grants to 18 new grantees to develop model interventions for youth with child welfare involvement who are at risk of homelessness (“YARH grantees”). The grants were awarded under a new CB program, “Planning Grants to Develop a Model Intervention for Youth/Young Adults with Child Welfare Involvement At-Risk of Homelessness” (HHS-2013-ACF-ACYF-CA-0636). YARH grantees are expected to use the two-year planning grant period to develop multi–component interventions that will support better outcomes for this population in four critical domains: (1) stable housing, (2) permanent connections to caring adults, (3) education and employment, and (4) social/emotional well-being. CB may provide funding to a subset of grantees for a second phase of grants. The second phase of grant funding will enable grantees to implement the model interventions developed in the first two-year planning phase and undertake a rigorous evaluation of those model programs. In addition, the Office of Planning, Research and Evaluation (OPRE) has awarded a technical assistance contract to Mathematica Policy Research to assist the planning grantees with the development of rigorous evaluation plans and conduct a process study of the planning processes. The process study is the focus of this Information Collection Request (ICR).
Given the current housing instability of former foster youth and the pressing need to respond more vigorously to this problem, this initiative comes at an important time for advancing efforts to develop evidence-based, comprehensive interventions for youth at risk of homelessness.
The YARH process study data collection is authorized by the Foster Care Independence Act (FCIA) of 1999 (Pub. L. 106-169) amending Title IV-E of the Social Security Act. FCIA requires that funding under the statute be set aside for evaluations of promising independent-living programs (Title IV-E, Section 477 [42 U.S.C. 677], g, 1).
A2. Purpose of Survey and Data Collection Procedures
Overview of Purpose and Approach
This Information Collection Request (ICR) describes the data collection activities for a process study that will document planning activities, technical assistance, and outcomes of the two-year planning period for YARH grantees. The process study will be descriptive, and will provide ACF and the field with a better understanding of the planning activities and supports needed for developing comprehensive, integrated, evidence-based service models that can be rigorously evaluated. The process study will include (1) findings from surveys administered to planning team members and key partners at two points in time, (2) information from interviews and focus groups conducted with planning team members and key partners in 2015, and (3) documents submitted to the federal government to meet the grant-reporting or contractual requirements for the larger YARH project. Data analyses will describe (1) grantee planning activities and outcomes, (2) target populations for model interventions (3) frameworks guiding the system- and practice-level changes, (4) evolution of the frameworks, (5) partnerships, (6) technical assistance activities and outcomes, and (7) the characteristics of all grantees—both those that met the requirements of the planning phase and those that did not.
The process study will provide ACF with an improved understanding of what the grantees have accomplished during the planning period, and the extent to which they are positioned to implement a rigorous evaluation should they be awarded implementation funding. Data collected for the process study will be used for two main purposes: (1) to assess grantees’ organizational capacity and readiness to implement and evaluate the comprehensive services model, and (2) to conduct systematic monitoring of each grantee’s progress toward achieving the goals of the planning period. Researchers will gather data through document review—for example, grant applications and semi-annual progress reports, administer a survey of organizational readiness and partnership at two time points (at the mid-points of grant years 1 and 2), and site visits involving focus groups and interviews to all 18 grantees.
Research Questions
The research questions for the process study address three key areas:
1. Planning Grant Activities and Outcomes
How grantees originally structured the planning activities and how the activities proceeded
The extent to which the original plans reflect ACF’s expectations for the planning period activities as described in the funding opportunity announcement (FOA)
What grantees and partners see as the major outcomes of the planning process
2. Development of Target Populations and Intervention Design
Which target populations and interventions were originally proposed, how these populations were defined, and how they changed during the planning phase
How partner agencies participated in the development of the approach used for identifying the target population and the selection of interventions
Explanations for changes made to proposed interventions
The extent to which the final array of interventions addresses youth outcomes in the four areas of interest
3. Partnerships and Service Integration
Information on organizations that partnered with the grantee in the phase 1 application, changes in partnerships during the planning period, and explanations for changes
The extent to which partners achieved service integration and explanations for changes in services provided
Explanations for not partnering with some community organizations
The survey instruments that are the focus of this ICR will gather information about perceptions of readiness for organizational change and the partnership. This information will help answer questions about partnerships and service integration, as detailed above. The discussion guides that are the focus of this ICR will gather information about experiences with the planning grant that will help answer questions about planning grant activities and outcomes and the development of target populations and intervention design.
Study Design
The process study will include the full population of grantees receiving planning grants. Given that the number of grantees is 18, limiting the process study to a subset of the grantees is not warranted. This descriptive study will use qualitative and quantitative methodologies to meet the information needs described above. There are three sources of data for the process study: (1) document review, (2) web-based surveys, and (3) site visits.
Documents submitted by the grantees as part of the normal grant administration process will be reviewed by the process study team. These documents include the grant application, semiannual progress reports, and presentations, if applicable. Additionally, the process study team will review records of technical assistance provided to grantees. The team’s document review will provide information on what the grantees plan to do, what they accomplish, and the resources provided by the technical assistance contractor that they use.
Data collected using the web-based survey instrument in this ICR will provide critical information on collaboration and partnerships of grantees across several domains, including the context and structure of collaborative efforts and processes. These data will be used to describe the relationship between grantee characteristics and the likelihood of success in the development of a promising model. Collecting these data during year 1 of the grant (Spring 2014, actual start date is dependent on the date of receipt of OMB approval) and at a follow- up point (early 2015) will provide ACF with critical information on the ways in which partnerships evolved, the extent to which collaboration and coordination among partners may have shifted, and what policies were changed during the planning period.
Finally, the process study team will conduct site visits to each of the 18 grantees. Site visits will last approximately two days and include focus groups and interviews. Researchers will gather information to examine the decisions made by grantees and help fill in missing information from the document review and web-based survey.
The process study team will meet the needs of ACF by providing an in-depth study of how the planning grants supported efforts by local communities to plan for systemic change to support coordinated services. The information obtained through the process study can be used to inform decisions related to future government investments in programs seeking to develop comprehensive service models for at-risk youth and young adults. The process study is limited in that it will reflect the experiences of only the 18 communities awarded the initial YARH planning grants. Other communities may undertake similar comprehensive planning efforts in a different manner, and require other technical assistance and support.
Universe of Data Collection Efforts
Current Request
ACF seeks OMB approval for the following data collection activities to inform the previously identified areas:
Survey Sampling Form. Before the first wave of data collection, the project director for each grantee will be asked to complete a sampling form to identify the planning team members and administrators at key community partners who should be asked to complete the Survey of Organizational Readiness and Partnership. Prior to the second administration, the project directors will be asked to update the form they originally provided with information on changes to the planning team. Attachment 1 includes the sampling survey correspondence and form.
Survey of Organizational Readiness and Partnership. The questions in the survey will focus on the partnerships developed as part of the YARH planning grant and assess the organizational readiness to implement systemic change. The survey will capture initial levels of grantee readiness to engage in successful collaboration with partner agencies and to implement system- and practice-level change in the local community. A follow-up administration will gather the same information after approximately 18 months of planning activities. The Survey of Organizational Readiness and Partnership that will be used at both data collection points is included as Attachment 2.
This web-based survey will be administered to planning team members—who are likely to be administrators and directors of the grantee agency or other agencies serving youth and young adults with child welfare involvement and/or at risk of homelessness—and administrators from key community partners at two points in time: spring 2014 (year 1) and prior to site visits in winter 2015 (year 2). The survey, which will be administered in all 18 grantee sites, takes approximately one hour to complete. We will offer a paper-and-pencil, self-administered survey for respondents who do not wish to complete the survey online.
The Survey of Organizational Readiness and Partnership has three sections. Section A includes questions about the respondent and the respective organization. Many of these items are based on items used in the Evidence-Based Home Visiting Partnership Survey (OMB No. 0970-0375). Section B contains questions about partnerships and collaboration, including items from the Collaboration Survey (Chrislip and Larson 1994) that were used in the Evidence-Based Home Visiting Partnership Survey. Section C has questions about organizational readiness for change and includes items from the Management Team Assessment Tool (Lasker and Weiss 2003; Weiss, Anderson and Lasker 2002).
Communications for Site Visit Planning. The study team will work with the project director of each grantee to plan the site visit that will occur in early 2015. We anticipate the scheduling an initial 30-minute conference call with the project director to discuss the purpose of the site visits, identify individuals to participate in individual interviews or focus groups, and begin the process of scheduling the actual site visit. Additional communication may be needed to develop the actual site visit, which we anticipate will require no more than an additional 30 minutes of time from the project director.
Discussion Guide for Interviews and Focus Groups. The same discussion guide will be used for both individual interviews and focus groups during site visits to each grantee. This protocol will ensure that questions are asked systematically across grantee sites in order to ensure that information is collected efficiently and completely, and facilitate comparability of data during analysis. The semi-structured protocols will enable the process study team to collect additional data to help explain variation seen in other data (that is, document review and analysis of the survey data). The discussion will last no more than one and a half hours, depending on the number of participants and their roles on the planning team. A list of topics is included in Attachments 4 and 5. These are draft versions; if any updates are made, revised guides will be submitted to the OMB Office of Information and Regulatory Affairs as a non-substantive change, per the terms of approval.
Site visits will be conducted in the winter of 2015. A two-person team will visit each grantee for no more than two days. The timing of the site visit, and the interviews and focus groups conducted during the site visit, will be coordinated with grantees to ensure minimal disruption to the work of planning team members.
A3. Improved Information Technology to Reduce Burden
The survey will be administered on the web. Paper-and pencil administration will be available, but it is not expected to be the preferred method for many respondents, and it will not be offered as the primary means of responding to the survey.
Planning the site visit will be done collaboratively with the grantee project director. We will use conference calls and emails to the extent possible to minimize burden.
The interviews will be conducted either individually or as a focus group. Due to the nature of the interviews, it is not appropriate to use information technology such as computerized interviewing.
To minimize the burden, we will hold semi-structured group discussions (focus groups), rather than individual conversations, whenever possible. Each group discussion will include staff at the same or similar levels. For example, one group discussion may be held with multiple front-line workers, such as case workers or outreach specialists. A separate group discussion may be held with supervisors of front-line staff. A third discussion group may include staff at the management or administrative level, such as directors of offices or agencies. If there is a single staff member in a particular level, however, an individual discussion will be held. We anticipate that staff at each of these levels will have different perspectives and thus may have different experiences with the planning team. Group discussions will allow us to reduce the length of time spent at the site while still obtaining valuable feedback on the planning grants from staff with a range of experiences.
A4. Efforts to Identify Duplication
The information collection requirements for this process study have been carefully reviewed to determine what information is already available from existing studies and program documents and what will need to be collected for the first time. Although the information from existing sources improves our understanding of the planning process, ACF does not believe that it provides sufficient information on how comprehensive service models are developed. This data collection is intended to yield new and useful information about developing partnerships to plan comprehensive system- and practice-level change. The interviews and focus groups will support a deeper exploration of patterns seen in the survey data or review of documents.
A5. Involvement of Small Organizations
Many of the respondents will be from small entities. The data collection was designed to have as little impact as possible on the entity by requiring that a single respondent from each entity will be surveyed. This will ensure that efforts are not duplicated by staff members within a small entity.
The site visit and interviews will be scheduled in collaboration with the program staff to minimize disruption on daily activities. The site visit team will conduct group discussions to the extent it is feasible to do so. An individual discussion may be necessary if the grantee does not have more than one staff at a particular level or due to scheduling conflicts.
A6. Consequences of Less Frequent Data Collection
Rigorous evaluation of innovative initiatives is crucial to building evidence of what works and how best to allocate scarce government resources. This process study represents an important opportunity for ACF to learn about activities associated with successfully developing comprehensive service models for youth and young adults who have had contact with the child welfare system and are at risk of homelessness, and designing rigorous evaluations to measure the impacts of those services.
Not collecting information for the process study would limit the government’s ability to document the kinds of activities implemented with federal funds and to measure their effectiveness. Data from this initial information collection offers an opportunity to assess the utility of the grantee planning period, and to determine whether the cost and time associated with this phase produces high-quality, comprehensive service models and rigorous evaluation designs.
Without the information collection requested by this clearance package, policymakers and providers of these programs will lack high-quality information about the value of using resources (time, funding, and technical assistance) to support the planning of comprehensive system- and practice-level change in communities serving at-risk youth. There are specific consequences:
If the Survey of Organizational Readiness and Partnership is not administered twice—that is, if detailed information on grantees’ partnerships and readiness for change is not collected in the near the beginning and the end of the project—the evaluation team will be limited in its ability to identify grantee characteristics that are correlated with successful planning projects. In addition, without baseline information on factors that could predict model success, it will not be possible for the contractor to provide adequate technical assistance to grantees in specific planning areas. Also, the contractor will not be in a position to provide ACF with critical information on grantee progress in developing models, policies, and evaluation designs, or on the shifts that may have affected collaboration among grantee partners during the planning period.
The site visit interviews and focus groups are a one-time collection effort. If the site visits are not conducted then the evaluation team will be limited in its ability to examine patterns seen in the survey data or document reviews.
A7. Special Circumstances
There are no special circumstances for the proposed data collection efforts.
A8. Federal Register Notice and Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on Sept 20, 2013 Volume 78FR 58309, Number 2013-22961, pages 58309–58310, and provided a 60-day period for public comment. A copy of this notice is included as an attachment. During the notice and comment period, no comments were received. Two requests for the instruments were received and OPRE provided a copy of the draft instruments in response.
Dr. Allison Metz at the National Implementation Research Network is serving as a consultant to the study. She has provided consultation on instrumentation for the process evaluation and on the use of an implementation science approach to guide grantee planning activities. Discussions about the study have been conducted with federal staff in ACF and HUD as part of agency collaboration on the youth framework model from the U.S. Interagency Council on Homelessness (USICH). ACF recently initiated a monthly advisory call for the initiative with federal staff, including Sarah Hunter and Todd Shenk from the Office of the Secretary, HUD. The ACF team coordinated with Anne Fletcher (Office of Policy Development and Research, HUD) on planning for the HUD-sponsored November 12, 2013 meeting on Housing Supports for Youth Aging Out of Foster Care. This meeting was convened by Mathematica Policy Research (also the contractor for the current ICR), and therefore we were able to combine the housing meeting with a kickoff meeting for our evaluation contractor and for the new grantees under this initiative.
A9. Incentives for Respondents
No incentives for respondents are proposed for this information collection.
A10. Privacy of Respondents
Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law.
Participants in interviews and focus groups will be told that their conversations will not be shared with anyone outside the research team or federal staff and will not be publicly released in a form that identifies them. Each discussion guide includes the statement: “Your responses will be kept private and used only for research purposes.”
The contractor has extensive corporate, administrative, and security systems to prevent the unauthorized release of personal records—including state-of-the-art hardware and software for encryption that meets federal standards, other methods of data protection, such as, requirements for regular password updating, as well as physical security that includes limited key-card access and locked data storage areas. Hosting on a hypertext transfer protocol secure (HTTPS) website ensures that data are transmitted using 128-bit encryption, so that transmissions intercepted by unauthorized users cannot be read as plain text. This security measure is in addition to standard password authentication that precludes unauthorized users from accessing the web application.
As a condition of employment, all contractor staff are required to sign a pledge to maintain the privacy of all data collected, and are informed that failure to comply with the terms of this signed statement will result in severe consequences, including termination of employment and/or legal action.
A11. Sensitive Questions
There are no sensitive questions in this data collection.
A12. Estimation of Information Collection Burden
Newly Requested Information Collections
Table A.1 summarizes the proposed annualized estimated reporting burden for the data collection instruments. The total annualized burden includes an estimate for the Sampling Survey Form, administered to all 18 sites, which is estimated to be 4.5 total burden hours per year. Another element of the total annualized burden is for the Survey of Organizational Readiness and Partnership which will be conducted at all 18 sites and is estimated to be 270 total burden hours per year. We expect 15 directors or staff from grantee and partner agencies at each site to respond to the survey twice (once during grant year 1 and once during grant year 2). The survey will take approximately one hour to complete. A third element of total annualized burden is for Communications for Site Visit Planning, which will be conducted with one individual at each of the 18 sites and is estimated to be 9 total burden hours per year. The fourth element of the total annualized burden is for the Individual Interviews, which will be conducted with five individuals at each of the 18 sites and is estimated to be 68 total burden hours per year. The final element of the total annualized burden is for the Focus Groups – four focus groups with three individuals will be held at each of the 18 sites and is estimated to be 162 total burden hours per year.
Table A.1. Estimate of Burden and Cost for the YARH Process Study—Current ICR
Instrument |
Total Number of Respondents |
Annual Number of Respondents |
Annual Number of Responses Per Respondent |
Average Burden Hours Per Response |
Annual Burden Hours |
Average Hourly Wage |
Total Annual Cost |
Survey Sampling Form |
36 |
18 |
1 |
.25 |
5 |
$27.86 |
$125 |
Survey of Organizational Readiness and Partnership |
540 |
270 |
1 |
1 |
270 |
$27.86 |
$7,522 |
Communications for Site Visit Planning |
18 |
9 |
1 |
1 |
9 |
$27.86 |
$251 |
Discussion Guide: Individual Interviews |
90 |
45 |
1 |
1.5 |
68 |
$27.86 |
$1,894 |
Discussion Guide: Focus Groups |
216 |
108 |
1 |
1.5 |
162 |
$27.86 |
$4,513 |
|
|
Estimated Annual Burden Total |
|
|
514 |
|
|
Total Annual Cost
We estimate the average hourly wage for staff at the grantee organizations, $27.86, to be the average hourly wage of “social and community service managers” as determined by the U.S. Bureau of Labor Statistics National Compensation Survey (2010).
A13. Cost Burden to Respondents or Record Keepers
There are no additional costs to respondents.
A14. Estimate of Cost to the Federal Government
The total cost for the data collection activities under this current request will be $423,120. Annual costs to the Federal government will be $211,560 for the proposed data collection.
A15. Change in Burden
This is a new data collection.
A16. Plan and Time Schedule for Information Collection, Tabulation and Publication
Process Study Analysis Plan
The survey data collection effort is part of the larger process study, a broad descriptive study documenting the design and implementation of the planning grants by all 18 YARH grantees. The first step of this study included a review of information already available on grantees. The next step—which is the focus of this information collection request—will be to administer a web-based survey of organizational readiness and partnership to planning team members and community partners at two time points. A statistical program, such as SAS or STATA, will be used to analyze the survey data. The second survey administration will be followed by site visits that include interviews and focus groups with planning team members. Atlas.ti, or a similar software program, will be used to support analysis of documents and data collected during the site visits.
In the process study report, which will be based on all of these data collection efforts, researchers will document how grantees used their planning time, what kind of evaluation technical assistance they received, and whether they achieved the goals of the planning period. The data will be reported in two ways: First, a summary profile will be created for each grantee that will contain planning process facts—such as the planned activities, the completed activities, and elements of the comprehensive services model, as well as its evaluation as designed. Second, researchers will discuss themes that emerge from responses regarding how and why decisions were made; the degree to which grantees changed their intended planning activities and models; any challenges that emerge during the planning process; the degree to which grantees could address these challenges; and, the characteristics associated with successful planning periods (that is, planning periods that yielded a comprehensive services model and evaluation design that met the requirements stated in the initial FOA).
This study is expected to be conducted during a two-year period beginning on September 30, 2013. Baseline data collection for the web survey is expected to begin in Spring 2014 (actual start date is dependent on the date of receipt of OMB approval). Table A.2 shows a schedule of the process study.
Table A.2. Schedule for the Process Study
Activity |
Date* |
Conduct document review |
September 2013–April 2015 |
Initial administration of survey to grantees and community partners |
Spring 2014 |
Submit OMB package for site visit semi-structured interview and focus groups |
Summer 2014 |
Analyze survey data |
Summer 2014 |
Follow-up administration of survey to grantees and community partners |
November 2014–January 2015 |
Analyze follow-up survey data |
December 2014–January 2015 |
Conduct two-day site visits to grantees |
January 2015–April 2015 |
Analyze data collected during site visits |
February 2015–May 2015 |
Draft process study report |
April 2015–June 2015 |
Revise process study report based on comments from ACF |
July 2015–August 2015 |
*Note that the actual start date is dependent on OMB approval
A17. Reasons Not to Display OMB Expiration Date
All instruments will display the expiration date for OMB approval.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
No exceptions are necessary for this information collection.
References
Chrislip, David, and Carl Larson. Collaborative Leadership: How Citizens and Civic Leaders Can Make a Difference, 1st ed. San Francisco: Jossey-Bass, 1994.
Kroner, Mark. Housing Options for Independent Living Programs. Washington: CWLA Press, 1999.
Lasker, R. and E. Weiss. “Creating Partnership Synergy: The Critical Role of Community Stakeholders.” Journal of Health and Human Services Administration, vol. 26, no. 1, summer 2003, pp. 119–139.
U.S. Department of Housing and Urban Development. “Housing for Youth Aging Out of Foster Care: A Review of the Literature and Program Typology.” Washington, DC: U.S. Department of Housing and Urban Development, April 2012.
Weiss, E., R. Anderson, and R. Lasker. “Making the Most of Collaboration: Exploring the Relationship Between Partnership Synergy and Partnership Functioning.” Health Education and Behavior, vol. 29, no. 6, December 2002, pp. 683–698.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OPRE OMB Clearance Manual |
| Author | DHHS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |