Supporting Statement Part A - SCF Evaluation

Among the key objectives of the American Recovery and Reinvestment Act of 2009 (ARRA) are to preserve and create jobs, promote economic recovery, and help Americans most affected by the recession to recover. These objectives are being addressed by the Department of Health and Human Services’ Office of Community Services, Administration for Children and Families (ACF/OCS) in its new initiative “The Strengthening Communities Fund (SCF),” as part of its place-based strategy to develop strong, healthy and supportive communities, especially in low-income neighborhoods. The SCF is designed to help nonprofit faith-based and community organizations (FBCOs) build their capacity and contribute to the economic recovery by helping low-income individuals access employment and training services as well as state and Federal benefits and tax credits, including those under the Recovery Act.

FBCOs have long been an essential part of the social and economic fabric of local communities. They provide an array of basic services, foster civic participation, help strengthen and enrich community life, and frequently play a role in economic revitalization in distressed communities.

But working with government can sometimes present challenges. FBCOs often lack the capacity to work effectively with government and build collaborations within their communities. Government agencies, on the other hand, may lack the skills and infrastructure to work effectively with FBCOs, particularly smaller ones. In this situation, FBCOs may need assistance to strengthen their organizational infrastructures, improve service delivery programs, build leadership potential, and measure and monitor program progress and client outcomes. Both FBCOs and government agencies need assistance in building collaborations and working effectively with one another.

The SCF is structured to address the challenges faced both by FBCOs and by government agencies as they try to work in partnership to address individual and family needs during the recession. The SCF awarded grants to 35 “lead” organizations to assist FBCOs in local communities build their capacity in five areas: 1) organizational development; 2) program development, 3) collaboration and community engagement, 4) leadership development, and 5) evaluation of effectiveness. Because nonprofits have various capacity building needs, each FBCO will receive training or technical assistance from the lead organization in at least three of the five areas. “Training” refers to group-based educational and skill development activities, and “technical assistance” refers to individual consultation tailored to a particular organization’s needs.

Lead organizations are required to use at least half of their awards to provide financial assistance to selected FBCOs to build their capacity by expanding and strengthening services and programs to assist low-income individuals. These awards are made on a competitive basis. The awards may be used for staff training, to establish procedures that assess eligibility for benefits, develop referral protocols, develop guides to benefits, and build capacity to offer specific types of assistance or services.

The SCF also awarded 48 grants to state, local and tribal governments (or their designees) so they may build their capacity and better partner with local FBCOs and other government agencies. Grants may be used for three purposes: 1) to provide outreach or education to increase the involvement of FBCOs in economic recovery; 2) to provide training and technical assistance to build capacity of FBCOs in the five critical areas noted above; and 3) to develop the capacity of the government agency grantees to enable them to partner with nonprofits and other government agencies in support of economic recovery.

Outreach and education activities may include information about, or facilitate access to ARRA benefits or activities, facilitate partnerships or serve as liaisons between government and FBCOs. Training and technical assistance to FBCOs is expected to focus on the same five areas identified in the nonprofit capacity building program. Additionally, government grantees’ own capacity building efforts are intended to focus on one or more of the same five areas.

The proposed data collection is intended to support an evaluation of the implementation of the SCF-ARRA initiative and measure early outcomes achieved by the grantees and FBCOs to strengthen organizational capacity and build local collaborations. HHS/OCS will use the evaluation findings to understand more fully the implementation issues related to the SCF-ARRA initiative and the early outcomes achieved. Additionally, findings from this evaluation may assist government officials, nonprofit leaders, public policymakers, and other stakeholders in designing and implementing future initiatives that are intended to build FBCO capacity and support stronger communities.

A.2 How the Information Will be Collected, by Whom, and For What Purpose

The proposed information collection will use both quantitative and qualitative methods. The data will be collected through two surveys (one for grantees and one for FBCOs) and telephone interviews with grantees. Data collection will occur near the end of the awards for each type of respondent and will solicit both retrospective and prospective information on the activities undertaken with SCF-ARRA funding, the challenges and successes encountered, and the early outcomes obtained during the 24-month grant period.

The web-based surveys (with hard copy options for those who cannot complete the survey online) will be sent to the SCF grantees and to the FBCOs that received subawards and/or extensive training and technical assistance. The FBCO survey will occur in two rounds: the first round will be for FBCOs that participated during the first year of the SCF-ARRA program, and the second round for those that participated during the second year. (Note: Subawards and extensive training/technical assistance is provided to FBCOs during two 12-month periods. A FBCO is to receive assistance for only 12 months.) The survey instruments will be self-administered, using UI’s Check Box software.

The telephone interviews will obtain in-depth qualitative information to give context and nuance for the quantitative data collected in the web-based surveys. Interviews will be conducted with at least one key informant in each of the 84 grantee organizations. The key informant is likely to be the executive director, program manager of the SCF-ARRA grant, or other individual knowledgeable about the SCF-related activities. The telephone interviews will be conducted by Urban Institute staff who have been trained in the collection of qualitative data.

Further details about the data collection procedures are provided in Section B of this submission.

Research Questions

The web-based surveys and telephone interviews will be used to answer the primary research questions posed by this study. These questions can be described as follows:

What was the socioeconomic context in which the grantees/FBCOs operated?
What were the grantee characteristics and their goals for providing capacity building services?
How were grantee SCF services implemented (i.e., allocation of training and TA services, criteria for selecting FBCOs, challenges encountered, etc.)
For FBCOs, what was the focus of capacity building activities and what outcomes were attained?
What types of collaborations and partnerships were formed; with whom; for what purpose?
Is there evidence that the capacity built or collaborations and partnerships developed can be sustained (e.g., have plans for sustaining what has been developed?
To what extent did SCF grants affect employment (e.g., jobs or consultant positions created or retained by grantee organizations)?
What were grantees’ perspectives about the way the SCF program was administered?

Once the data collection is complete, the information collected will be used to assess change and improvement in various areas of capacity building. The analysis will use breakout categories (such as type of organization, size of organization, geographic location, etc.) to understand differences in outcomes and performance.

A.3 Use of Improved Information Technology to Reduce Burden

To reduce burden, a web-based format will be used to collect survey data. Using a web-based format allows for automated “skip” patterns limiting the number of questions respondents have to read/answer. Any organization that lacks access to the Web or prefers to complete a paper version of the survey will be mailed a hard copy of the survey. UI staff will be available by phone and email to answer questions and facilitate completion of the survey.

A.4 Efforts to Identify and Avoid Duplication

The proposed data collection is unique because the information is specific to the evaluation of the SCF-ARRA program and is not available elsewhere.

A.5 Efforts to Minimize Burden on Small Business or Other Entities

No small businesses will be involved as respondents. Respondents to the surveys will be government and nonprofit faith- and community-based organizations. As noted below, each survey will take less than 30 minutes to complete and will not have a significant economic impact on a substantial number of small entities. The telephone interviews will take between 60 and 90 minutes to complete, depending on the respondent’s experiences. This is a one-time interview and will not have a significant economic impact on a substantial number of small entities.

A.6 Consequences of Less-Frequent Data Collection

The data collection efforts are essential to conducting the evaluation of the SCF-ARRA program. Each SCF grantee will be asked to complete one online survey and to participate in one telephone interview. FBCOs will be asked to complete an online survey one time. Less frequent data collection would jeopardize HHS’s ability to assess the SCF-ARRA implementation process and to measure increased organizational capacity of participating organizations.

A.7 Special Circumstances Requiring Collection of Information in a Manner Inconsistent with Section 1320.5(d)(2) of the Code of Federal Regulations

There are no special circumstances associated with this data collection.

A.8 Federal Register Comments and Persons Consulted Outside the Agency

In accordance with the Paperwork Reduction Act of 1995, ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of data collection activities. The notice was published on November 10, 2010, in volume 75, number 217, page 69088, and provided a 60-day period for public comments. No public comments were received during this time. A copy of the Federal Register notice for this information is included in Appendix D.

The study design and data collection instruments were developed by the Urban Institute research team, comprised of Drs. Carol J. De Vita and Elaine Morley, Co-Principal Investigators; Mary K. Winkler and Robin Koralek, senior team leaders, and Timothy Triplett, project statistician. Feedback on the design and instruments was obtained from our internal UI advisory group, namely, Elizabeth T. Boris, director of UI’s Center on Nonprofits and Philanthropy; Ajay Chaudry, director of UI’s Center on Labor, Human Resources, and Population; Olivia Golden, UI Institute Fellow and expert on children and family issues; and Harry Hatry, UI Institute Fellow and expert outcomes and performance measurement.

Drafts were also reviewed by program staff from the SCF-ARRA staff in the Administration on Children and Families. Based on these reviews one question was added to the survey of faith-based and community organizations (now, Q9) to obtain a baseline number for the average number of people served before the FBCO received SCF assistance.

The Urban Institute Institutional Review Board (IRB) reviewed and approved the data collection instruments before a pretest was conducted. A pretest of the survey instruments provided feedback from 6 SCF grantees and 4 FBCOs that received subawards and/or extensive training and technical assistance from SCF grantees.

A.9 Payments to Respondents

We do not plan to offer payments to respondents.

A.10 Assurance of Confidentiality

Every effort will be made to maintain the privacy of respondents, using several procedural and control measures to protect the data from unauthorized use. No personal identifying information is proposed to be collected. The procedures to protect data during information collection, data processing, and analysis activities are as follows:

All respondents included in the study will be informed that the information they provide will be used only for the purpose of this research. Individuals will not be cited as sources of information in prepared reports. Prior to collecting data, grantees and FBCOs will receive a letter (or email) that explains the nature of the study and how privacy of information will be handled.

All respondents will be informed that “Responses to this data collection will be used only for statistical purposes. We will treat the information in a private manner and will not identify you or your organization to anyone outside the study team, except as required by law.”

To ensure data security, all staff that have permission to access the data are required to adhere to strict research standards and sign an oath of confidentiality as a condition of working on the study.

The UI will maintain restricted access to all data-related tasks (i.e., survey preparation, follow-up, receipt, coding, and data entry). Electronic data files will be maintained on a secure, password protected drive. Hard-copy data forms will be maintained in a locked filing cabinet. All data files on multi-user systems will be under the control of a database manager, with access limited to project staff on a “need-to-know” basis only.

Individual identifying information will be maintained separately from completed data collection forms and from computerized data files used for analysis. No respondents identifiers will be contained in public use files made available after the study, and no data will be released in a form that identifies individuals.

A.11 Questions of a Sensitive Nature

The questions included on the data collection instruments for this study do not involve sensitive topics. The surveys request information about the organization; no personal information is asked.

A.12 Estimates of Respondent Burden

The total burden across all data collection instruments is estimated to be a maximum of 657 hours, valued at $17,313. The table below presents the assumptions that were used in deriving this estimate.

ANNUAL RESPONSE BURDEN ESTIMATES

Instrument	Number of Respondents	Responses Per Respondent	Average Hours Per Response	Total Burden Hours	Estimated Cost Per Hour^*	Cost per Respondent	Total Cost Burden
SCF grantee survey	84	One	0.25 hrs.	21 hrs.	$26.75	$6.69	$562
FBCO survey	1,000	One	0.50 hrs.	500 hrs.	$26.75	$13.38	$13,380
SCF grantee telephone interview	84	One	1.5 hrs.	126 hrs.	$26.75	$40.13	$3,371

^*Estimated cost per hour is based on the 2010 median income for nonprofit social services executive directors per http://www.payscale.com/research/US/Industry=Social_Services/Salary

Estimated Total Annual Burden Hours and Estimated Costs:

Total burden for SCF grantees = 147 hours $ 3,933

Total burden for FBCOs = 500 hours $13,380

Grand total = 647 hours $17,313

A.13 Estimates of the Cost Burden to Respondents

There are no costs to respondents other than their time needed to respond.

A.14 Estimates of Annualized Government Costs

This information collection activity has been developed in the performance of the U.S. Department of Health and Human Services Contract Number: HHSP23320095654WC. The total cost to the Federal government for the Strengthening Communities Fund Evaluation (SCF) Performance Management and Evaluation Support is $1,443,178.00, for the period September 30, 2009 to September 29, 2012 (i.e., $ $481,059.33 per year). The annualized cost ($481,059.33) includes the survey activities and other tasks. An estimate of total data collection costs alone, including development of data collection instruments and actual data collection, is $461,381 – or one-third of the total budget. These costs will be borne by the HHS/ACF.

A.15 Changes in Hour Burden

This is a new data collection.

A.16 Time Schedule, Publication, and Analysis Plan

The schedule shown below displays the sequence of activities required to conduct our information collection activities. It includes key dates for activities related to data collection, analysis and reporting.

Activity and Deliverable Planned Date to Initiate Activity

Survey of Round 1 FBCOs Immediately after receiving OMB clearance

Survey of SCF grantees Late Spring (May) 2011

Survey of Round 2 FBCOs Summer (July) 2011

Phone interviews of SCF grantees Early Summer (June) 2011

Data Analysis Late fall/winter (November/December) 2011

Deliver Final Report and Data Sets September, 2012

Each survey will be in the field for approximately 5 to 8 weeks to ensure sufficient time for SCF grantees and FBCOs to respond.

It should be noted that the Round 1 FBCOs are likely to have a longer recall period than the Round 2 FBCOs to answer questions about their experiences with capacity building activities. However, Round 1 FBCOs may have more experience in answering questions about the extent to which SCF capacity-building was helpful and their plans to sustain their newly acquired capacity/collaborations than the Round 2 FBCOs. Some items in the questionnaire may be time sensitive – that is, the accuracy of a response may be affected by a longer recall period. To detect differences that may be affected by recall, we will compare the responses of Round 1 and Round 2 respondents and, if it appears that recall factors are influencing the responses, we will report the findings both combined and separately.

Our analysis will use both the quantitative data obtained primarily from the surveys of FBCOs and SCF grantees, and the qualitative data obtained primarily from telephone interviews of SCF grantees. We will also rely extensively on the administrative data provided in the SCF Performance Progress Reports (PPR) that SCF grantees submit on a quarterly basis.

Data analysis for the study will focus primarily on assessing the implementation efforts of SCF grantees; the capacity building efforts and accomplishments of FBCOs; and collaborations and partnerships that emerged from the SCF program. We will also document the early outcomes of the grantees, effects on employment and identify good practices in implementing capacity building services. In particular, we will explore differences by organizational characteristics such as size and type of grantee/FBCO; geographic location; number or types of collaborations/partnerships formed, etc.

Various types of data display, primarily in the form of tables, matrices, and graphics, will be used to present information collected and develop formats that enhance the analysis and presentation.

The Matrix of Research Questions and Sources of Data (Appendix E) shows how the surveys, telephone interviews and review of secondary documents will contribute to a better understanding of the major evaluation questions posed by this research.

A.17 Display of Expiration Date of OMB Approval

The OMB approval number and expiration date will be displayed on all survey instruments and discussion guides.

A.18 Exceptions to Certification Statement

There are no exceptions to the certification.

File Type	application/msword
File Title	Grantee Early Site Visit Discussion Guide
Author	admined
Last Modified By	Department of Health and Human Services
File Modified	2011-06-27
File Created	2011-06-27