2ndFRNotice-OMBSupportingStatement_11-3-2010_Version_5

The Supporting Statement For New Data Collection Request Package

The Office of Community Services (OCS) Evaluation Initiative

Lynda E. Pérez

Director

Division of Community Discretionary Programs (DCDP)

Office of Community Services

Administration for Children and Families

370 L'Enfant Promenade, S.W. - 5th Floor

Washington, DC 20447,

(202) 401-9365

Fax (202) 401-4687

Main Office Number (202) 401-9333.

[email protected]

Submitted to:

Lynda E. Pérez, OCS, Director

Rafael J. Elizalde, Team Leader

Division of Community Discretionary Programs

Submitted by:

Rose Ann Rentera, PhD

Jesse Gelwicks, MA

AED

Reviewed by:

Julie Jakopic, ICF International

Pooja Patel, ICF International

Table of Contents

Table of Contents 2

Introduction 3

Part A. Justification 3

Part B. Statistical Methods 3

Introduction

For the last five years, the Office of Community Services (OCS) has administered annual surveys (approved by the Office of Management and Budget (OMB)) to grantees of the Community Economic Development (CED) and Job Opportunities of Low-Income Individuals (JOLI) programs in order to collect quantitative performance data. OCS used this data to develop comprehensive performance snapshots on the programs, as well as to track program performance over time and to conduct cohort analyses. These reports were also used to inform annual reports to Congress on each program’s performance.

Now OCS would like to adopt a standardized semi-annual reporting format based on the surveys, Performance Progress Report (PPR). CED and JOLI grantees would be required to use the PPR to submit data twice a year. This format will not require grantees to collect additional data and submit yet another form, as the PPR will replace the grantees’ current semi-annual reports as well as the annual surveys. OCS adapted the new format to fit the OMB-approved PPR forms. Within the adaptation process, OCS also revised the questions to improve their clarity and utility.

The PPR forms collect both quantitative and qualitative data. Most of this data is already collected either through the grantee’s semi-annual report and/or the annual survey. This format will reduce the burden on grantees by consolidating all reporting requirements into the PPR forms. The quantitative data elements deal with program outputs and outcomes, while the qualitative questions provide space for grantees to discuss their successes and challenges, as well as changes to their grant that might not be captured in the quantitative section.

The new format also increases the required level of grantee accountability, a priority under the Obama administration. By requiring grantees to report on a consistent set data elements on a twice-annual basis, OCS will improve its understanding of grantee success, strengthen its understanding and knowledge of each grantees’ progress, and ultimately enhance the efficacy and effectiveness of CED and JOLI grant dollars.

Overall, the new standardized reporting format should improve the quality and clarity of the data collected by OCS, while providing a singular, clear data collection tool for grantees.

Part A. Justification

I.Circumstances Making the Collection of Information Necessary

One of the current priorities at the Office of Community Services (OCS) is to improve performance and accountability. Monitoring the Community Economic Development (CED) and Job Opportunities of Low-Income Individuals (JOLI) programs more closely in order to better measure success and to understand and replicate the programs that excel has been and continues to be a priority for OCS. OCS works in partnership with states, communities, and other agencies to provide a range of human and economic development services and activities, which ameliorate the causes and characteristics of poverty and otherwise assist persons in need. OCS continues to have great success in the realm of poverty reduction and community development. However, measuring that success in a systematic way has not always been easy. OCS has learned from past data collection efforts which questions provide valuable program data, and is in the process of creating a more performance-based environment with greater emphasis on accountability and achieving results.

The new Performance Progress Report (PPR) forms are a critical part of the evaluation strategies for the CED and JOLI programs. The legislative requirement for the CED program is in Title IV of the Community Opportunities, Accountability, and Training and Educational Services Act (COATS Human Services Reauthorization Act) of Oct. 27, 1998, Pub. L. 105-285, section 680(b) as amended:

“The Secretary shall require all activities receiving assistance under this section to be evaluated for their effectiveness. Funding for such evaluations shall be provided as a stated percentage of the assistance or through a separate grant awarded by the Secretary specifically for the purpose of evaluation of a particular activity or group of activities.”

Under Title V, section 505, of the Family Support Act of 1998, Pub. L. 100-485, section 505(f), JOLI was initially a demonstration program that required local evaluations of each project. After JOLI was reauthorized in 1996 (Pub. L. 104-193--Aug. 22, 1996), it no longer had demonstration status and evaluation requirements. As a result, a formal evaluation for the JOLI programs has not been conducted since the 1996 Pub. L. reauthorization.

Currently, both the CED and JOLI programs require semi-annual reporting from their grantees. However, the reporting format and instructions allow the grantee to determine what program data is included and how that data is presented. A review of grantees’ annual reports revealed that the reports are not uniformly submitted and that the data therefore varies among reports. Standardized reporting is necessary to compile accurate program data. The new reporting format will clarify the data elements and time periods involved in each data collection effort.

For the last five years, OCS has augmented the semi-annual reports with annual OMB-approved surveys designed to collect program outputs and outcomes. The questionnaires were first approved in 2006 and were given OMB Control Number 0970-0317. The data collection period for the surveys was extended in 2008. OCS used this data to develop comprehensive performance snapshots on the programs, as well as to track program performance over time and to conduct cohort analyses. The response rates were high for the annual survey— above 80 percent for the last two years, but fell short of the desired 100 percent response rate. Further, some respondents did not answer all of the survey questions. Adopting a new semi-annual reporting format, the PPR, will ensure that all the grantees provide the relevant data needed in order to comply with their grant requirements.

II.Purpose and Use of the Information Collection

The primary purpose of the PPR forms is to establish a semi-annual reporting format to collect data in a uniform and systematic manner for two OCS discretionary grant programs (CED and JOLI). The new reporting format will gather uniform program performance data from each grantee and ultimately inform reporting on program-level outputs and outcomes. Once the forms are approved, OCS will build an online reporting system that will automatically aggregate grantee data and allow for snapshot-, cohort-, and progress-over-time reporting. This will allow for improved monitoring and evaluation of the CED and JOLI programs. In addition, OCS will be able to use the resulting data to target technical assistance based on individual grantee outcomes, and to make adjustments to the requirements based on the grantees capabilities.

While OCS has used data from OMB-approved surveys for the last five years, comprehensive programmatic analysis has not been possible given the optional nature of the questionnaire. The new required reporting format will allow for the 100 percent response rates that are necessary to execute comprehensive cohort and longitudinal analyses of program outcomes.

The PPR is categorized by the following programmatic goals:

• To create new businesses to employ low-income people

• To expand existing businesses to employ low-income people

• To create positions to employ low-income people

• To create full-time positions with benefits

• To prepare low-income individuals for employment

• To employ low-income individuals in positions created

• To create full-time positions with opportunity for advancement,

• To help low-income people retain new jobs

• To leverage additional funds to increase project success

• To generate revenue through CED-created businesses

Each programmatic goal contains a set of associated questions to track program outputs and, where appropriate, outcome measures. In most cases, outcomes are measured by the retention of an output for either six or 12 months. By measuring program outcomes, OCS will be prepared to demonstrate the programs’ success in future Federal-level evaluations, such as the Annual Report to Congress.

III.Use of Improved Information Technology and Burden Reduction

Once the PPR forms are approved, OCS will assess the feasibility of an online data collection system to facilitate the data submission process for grantees. The online reporting system could reduce the burden on grantees by establishing clear reporting and data collection expectations, allowing grantees to access the system anywhere with an internet connection, and streamline the data submission and revision process. Until an online data collection system is completed, the PPR will be distributed and collected as email attachments.

IV.Efforts to Identify Duplication and Use of Similar Information

This new information collection process will eliminate unnecessary duplication that exists in the current process. The PPR forms will replace the current semi-annual reports and the annually administered OMB-approved surveys. No similar data available are available from other studies because this evaluation is specific to the two OCS discretionary grant programs (CED and JOLI) and their grantees. Although some of the grantee performance data requested is similar to the data gathered from both the existing evaluation questionnaires and the grantee annual reports, the new reporting format condenses the data collected into one instrument. Further, the new instrument clarifies the data elements requested from grantees and sets more specific guidelines for data collection. Ultimately, it will provide a significantly more detailed and accurate picture of how the grantees are implementing and performing programs.

V.Impact on Small Businesses or Other Small Entities

The proposed information collection process reduces the burden on small entities, which constitute the bulk of CED and JOLI grantees. The evaluation has been designed to assure minimal burden on grantees by reducing the number of times OCS requests data from three times to twice annually. In addition, the new reporting format also establishes clear expectations for data collection and reporting — potentially reducing the burden for grantees that were collecting data that was not being used by OCS, or who were surprised by data elements requested and had to piece together information from varied sources.

VI.Consequences of Collecting the Information Less Frequently

Respondents will be required to submit the PPR forms twice a year. Consistent and timely reporting is required in order to track grantees’ progress over the course of their grants, which last from three to five years. It also is required for tracking the progress of annual cohorts of grantees.

Regular semi-annual reporting is the current reporting requirement for the grantees and allows OCS staff to identify grantees in need of assistance, and to respond to those needs in time to keep the projects on track to meet their timelines and goals. Reducing the reporting requirement to an annual basis would diminish the opportunity for OCS staff to manage programs and help under-performing grantees. OCS would only become aware of program challenges at the end of the fiscal year, which in most cases represents one-third of a grantee’s grant period. Additionally, the data collected in the semi-annual reports is included in Annual Reports to Congress.

Grantees are only required to submit semi-annual reports while their grants are active. An active grantee is defined as a grantee with currently open program dates, meaning that the grantee’s projects are operational.

VII.Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This data collection fully complies with the guidelines of 5 CFR 1320.5.

VIII.Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

The 60 day Federal Register notice published 11/16/2010, vol 75, page number 7008.
OCS made every effort to obtain feedback on the PPR forms from current grantees. In addition to presenting the proposed forms at the 2010 Annual Grantee Conference, OCS also sent an email to current CED and JOLI grantees notifying them of the proposed changes to the semi-annual reporting format and directing them to the Federal Register Notice. OCS did not receive any comments from the Federal Register, nor from the grantees directly.

IX.Explanation of Any Payment or Gift to Respondents

The respondents are grantees and will not require an additional payment or gift.

X.Assurance of Confidentiality Provided to Respondents

The grantee data collected from the CED and JOLI PPR forms will not be treated in a confidential manner. The topics focus on the outputs and outcomes of programs funded with Federal dollars. There are no questions of an inherently sensitive nature in the questionnaire. OCS will not use the term “confidential” in its correspondence with respondents and will use other appropriate language (such as “kept private to the extent permitted by law”) as necessary.

XI.Justification for Sensitive Questions

There are no questions of an inherently sensitive nature included in the PPR forms.

XII.Estimates of Annualized Burden Hours and Costs

The PPR forms are based on the previously administered OMB-approved surveys. Based on grantee responses to the annual survey administered in 2010, the survey took respondents an average of one and half hours to complete. As such, the PPR is estimated to result in a total of 630 annual burden hours.

Table 1. Annual Burden Estimate
Instrument	Number of Responses	Number of Responses Per Respondent	Average Burden Hours Per Response	Total Burden Hours
PPR Forms for current OCS JOLI grantees	40	2	1.5	120
PPR Forms for current OCS CED grantees	170	2	1.5	510
Estimated Annual Burden Hours	…	…	…	630

The project’s cost burden is presented in Table 2, which assumes a 100 percent response rate or level. The primary grant contacts for OCS grantees – the anticipated respondents – are generally senior executives in community-based organizations. Therefore the average hourly rate used for the respondents was the mean hourly rate for senior executives in community-based organizations ($36.22). The hourly wage rate is derived from statistics provided by the 2005 National Compensation Survey (Department of Labor, 2005). The estimated annual cost is the product of total burden hours and the mean hourly rate for senior executives in community-based organizations, which results in $22,818.60.

Table 2. Annual Cost Estimate
Instrument	Total Burden Hours	Estimated Hourly Wage*	Total Cost
PPR Forms for current OCS JOLI grantees	120	$36.22	$4,346.40
PPR Forms for current OCS CED grantees	510	$36.22	$18,472.20
Estimated Burden Hours	…	…	$22,818.60
*Source: Department of Labor; U.S. Bureau of Labor Statistics (2005).

XIII.Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There is no other burden on respondents.

XIV.Annualized Cost to the Federal Government

There is no cost to the Federal Government.

XV.Explanation for Program Changes or Adjustments

This is a new information collection request.

XVI.

XVII.Plans for Tabulation and Publication and Project Time Schedule

OCS will assess the feasibility of an online data collection system. Until this online data collection system is built, the PPR forms submitted by grantees will be collected electronically and analyzed using the current data analysis process. Grantees will be required to submit the PPR at the end of April and October. The data will be cleaned and analyzed once all grantee forms have been submitted. Each PPR submission will be reviewed and used for data analysis for overall program reports as well as individual grantee monitoring reports. Quantitative and qualitative content analysis will be conducted to determine key themes rising from respondent comments regarding program impact and experiences.

XVIII.Reason(s) Display of OMB Expiration Date is Inappropriate

The OMB Control Number and expiration date will displayed on the bottom right corner of each page of the PPR forms (Attachments A and B).

XIX.Exceptions to Certification for Paperwork Reduction Act Submissions

OCS is not seeking exception to certification for the Paperwork Reduction Act Submission for this data collection.

Part B. Statistical Methods

This submission requests OMB clearance to enable information gathering from all of OCS grantees whose CED and JOLI programs were funded with start dates of fiscal year 2006 and later. The intent of this project is to collect program progress report data from all OCS grantees; thus, no statistical sampling methods are employed. Quantitative and qualitative grantee performance data will be collected first by email, then later by online form. Descriptive statistics and content and thematic analysis will be used to conduct quantitative and qualitative analysis of the data. Both statistical and non-statistical information on the respondent and the information collection procedures for projects are described below.

XX.Respondent Universe and Sampling Methods

This data collection effort does not employ statistical methods, as all active CED and JOLI grantees with start dates of fiscal year 2006-2007 or later will be required to complete the PPR forms. In this context, an active grantee is defined as one with currently open program dates, which means that their projects are operational.

XXI.Procedures for the Collection of Information

Data will be collected by distributing the PPR forms via email to each lead grantee contact person. Grantees will fill out the PPR on their computers and return the forms as email attachments. Within one to two reporting cycles, OCS anticipates completion of an online reporting system that will allow the grantees to enter, update, and submit the PPR through an automated online process.

XXII.Methods to Maximize Response Rates and Deal with Nonresponse

In order to ensure compliance with the reporting requirements, grantees will receive a letter from OCS requesting their participation in webinars that will be conducted prior to the implementation of the PPR forms. The webinars will outline the new data collection and reporting format and tools. As needed, OCS will schedule telephone contact and an interview with grantees to encourage completion of the PPR and to provide one-on-one technical assistance. All active grantees will be required to submit their semi-annual reports in this format. Lack of compliance could affect the distribution funds from OCS.

XXIII.Tests of Procedures or Methods to be Undertaken

The PPR forms were reviewed and approved by Dr. Yolanda Butler, Acting Director of the Office of Community Services (OCS) and key OCS staff, including Lynda Pérez, Director of the Division of Community Discretionary Programs, and Rafael J. Elizalde, Team Leader of the Division of Community Discretionary Programs. The CED and JOLI grantees have completed an evaluation survey specific to their programs for the last five years. Additionally, OCS has provided CED and JOLI grantees the opportunity to respond to the proposed data collection and reporting format by providing a link to the first Federal Register Notice notification and the proposed forms to all current grantees.

XXIV.Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

This data collection does not employ statistical methods. As needed, the Office of Community Services will enlist the help of its current contractors to collect and/or analyze data using existing contracts/grants.

11