IC_-_OMB_Supporting_Statement_with_NOA_Summary_Followed_-_Revised_FEBRUARY_8-2008_v41

This Supporting Justification for OMB Package is using the OMB Control Number of 0970-0317 – for the OCS Evaluation Initiative: Community Economic Development (CED) and Job Opportunities for Low-Income (JOLI) Individuals. This package returned to the final NOA Summary and updates core information, however, the package continues to ensure that OCS addresses core issues from the perspective of the OMB. The NOA Date was 02/20/2007 with the Expiration Date of 02/29/2008. It had the Approved initial Responses of 172 with Approved Hours of 258 and Approved Cost of $6,230. With this in mind, ACF current submits the following information, and ensures that the package continues to be directed with OMB considerations: 1) This is a data collection which ACF anticipates completing by the end of FY 2012; when the ICR is expected to expire, ACF will submit a revision request if it wishes to continue using this IC and if it is making any substantive changes to the survey instruments or study design (e.g. new questions, different populations, etc.). There are no changes to the two survey instruments, which were approved in March 2007, by the OMB. 2) These ICs will continue to be used to evaluate CED and JOLI programs only. 3) The supporting statement continues to clarify that the 1.5 hour burden estimate is based on the actual 2007 time burden with 1 hour for the survey instrument and 30 minutes for reading the instructions and collecting pertinent data. 4) The supporting statement and instruments continue to state that respondents should take as much time as needed to complete the survey instruments, that they should keep track of how long it takes them to complete the instrument, and to solicit comment on how long it takes to complete this survey. In January 2008, ACF has taken these responses and reports the average burden continues to be 1.5 hour to OMB. 5) Because it is unclear whether ACF has the statutory authority to provide assurances of confidentiality for this ICR, ACF will not use the term “confidential” in its correspondence with respondents and will use other appropriate language (such as “kept private to the extent permitted by law”) as necessary. 6) ACF will account for non-response bias in its analysis and reporting of results. Specifically, ACF will stratify results according to where the grantees fall in their grant cycles, if necessary; ACF will conduct secondary analysis of non-responders based on semi-annual reports submitted by grantees and based on the administrative data ACF already has for each grantee, as necessary. ACF will also include a discussion of study limitations when it reports results from this study. 7) ACF will take special effort in following up with grantees with multiple grant awards (i.e. those respondents who will be completing multiple survey instruments), to ensure that they understand what is being asked of them, and to encourage their participation in this evaluation to reduce non-response bias. 8) ACF will continue to use two separate cover letters for grantees in the CED program and the JOLI program and has revised them to include Ms. Lynda E. Pérez’s signature, as director of OCS.

PART A. JUSTIFICATION

A1. Circumstances Making the Collection of Information Necessary

This supporting justification for the OMB package was originally approved in March 2007 with the following OMB Control Number: 0970-0317. For the most part, the items presented below remain consistent with the data collection period expanded to fiscal years 2008 to 2012; and, the CED and JOLI grantee cover letters, questionnaires, mode of data collection, and burden estimated remain identical from those approved in March 2007.

One of the current priorities at the Office of Community Services (OCS) is to improve performance and accountability. OCS leadership has been very clear about monitoring the programs more closely in order to better measure success and to understand and replicate the programs that excel. OCS works in partnership with states, communities, and other agencies to provide a range of human and economic development services and activities, which ameliorate the causes and characteristics of poverty and otherwise assist persons in need. OCS continues to have great success in the realm of poverty reduction and community development. However, measuring that success systematically has not always been easy. Thus, OCS is in the process of creating a more performance-based environment, with greater emphasis on accountability and achieving results.

Legislative Requirement

These questionnaires are part of evaluation strategies for two programs administered by OCS: Community Economic Development (CED) and Job Opportunities for Low-Income Individuals (JOLI). The Legislative requirement for the CED program is in Title IV of the Community Opportunities, Accountability, and Training and Educational Services Act (COATS Human Services Reauthorization Act) of Oct. 27, 1998, Pub. L. 105-285, section 680(b) as amended. This legislative directive states that “The Secretary shall require all activities receiving assistance under this section to be evaluated for their effectiveness. Funding for such evaluations shall be provided as a stated percentage of the assistance or through a separate grant awarded by the Secretary specifically for the purpose of evaluation of a particular activity or group of activities.” Under Title V, section 505, of the Family Support Act of 1998, Pub. L. 100-485, section 505(f), JOLI was initially a demonstration program that required local evaluations of each project. When JOLI was reauthorized in 1996 (Pub. L. 104-193--Aug. 22, 1996), it no longer had demonstration status and evaluation requirements. As a result, a formal evaluation for the JOLI programs has not been conducted since the 1996 Pub. L. reauthorization.

A2. Purpose and Use of Information Collection

As stated in the original OMB package in February 2007, the primary purposes are to document and systematically evaluate the program performance of two OCS discretionary grant programs in qualitative and quantitative terms. The survey data analyzed in September 2007 will serve as base-line data as the evaluation initiative moves forward. Both CED and JOLI programs will be assessed using qualitative and quantitative evaluation methods that capture key information about program and grantee-level performance in four general areas: (1) program purpose and design; (2) strategic planning; (3) program management; and (4) program results. The current evaluation activities will build upon spring and summer 2007 data collection and analysis to improve the validity and generalizability of evaluative and program impact findings. The current data collection request collected in fiscal years and up to fiscal 2012—will provide guidance for OCS in creating future program announcements with new evaluation definitions and expectations so that future CED and JOLI grantees will have evaluation plans that will produce PART level outcomes. With the results from the survey OCS will be able to target evaluation training at specific levels of grantees and to make adjustments to the requirements based on the grantees capabilities. The questionnaire data provides the baseline data for future evaluations.

A3. Use of Improved Information Technology and Burden Reduction

The questionnaires will continue to be administered using web-based questionnaires for electronic submission. This technique does not require paper from the respondents. To reduce burden on respondents, the questionnaire continues to be anticipated to take 1.5 hours to complete as the spring-summer 2007 burden equaled 1.5 hours, including 30 minutes to read the instructions and assemble the necessary materials to fill out the survey, and one hour to answer the questions. However, there is no time limit for how long the grantees can take to fill out the survey. The subcontractor (Academy for Educational Development, AED), will assist respondents that encounter information technology barriers to reduce paper submission and will conduct telephone interviews as requested; the questionnaire only will be used to collect the survey data during this telephone conversation as no other interviewing steps will be employed.

A4. Efforts to Identify Duplication and Use Similar Information

The collection of information avoids unnecessary duplication. There are no similar data available from other studies because this evaluation is specific to OCS programs and OCS grantees. Although some of the grantee performance data requested is similar to the data included in the grantee annual reports, the questionnaire contains specific performance indicators and measures than currently required by OCS. It will provide a significantly more detailed and accurate picture of how the grantees are implementing and performing programs. A review of grantee’s annual reports revealed that the reports are not uniformly submitted and therefore the data varies between reports. Standardized reporting is necessary to compile accurate program data.

A5. Impact on Small Businesses or Other Small Entities

The collection of information reduces burden on small entities. The evaluation has been designed with the minimal burden on the grantees by creating an electronic form and requiring no new data collection. Grantees will be instructed to only gather data from their current and accessible files and not to call participants or new business to gather new information, but report only on the data they have already collected. The questionnaires are a snapshot revealing both how many jobs the grantees have created and how many grantees are already collecting the data.

A6. Consequences of Collecting the Information Less Frequently

Respondents will be asked to participate in a once-a-fiscal year questionnaire (thus, a one-time only data collection per fiscal year). One of the benefits of using close- and open-ended questionnaires is assuring that all pertinent information is gathered at one-time. Respondent’s responses will provide comprehensive information on program impacts and experiences during the anticipated 1.5 hours questionnaire administration, eliminating the need for multiple data collections thereby reducing the burden on the respondent. There are no legal obstacles to reducing the burden.

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This data collection fully complies with the guidelines of 5 CFR 1320.5.

A8. Comments in Response to the Federal Register Notice and Effort to Consult Outside the Agency

a. Federal Register Notices

For this extension request, the first notice required in 5 CFR 1320.8 (b)(3) was originally published in the Federal Register, Volume 72, Number 218, page 63911, on November, 13, 2007. The second notice was published in the Federal Register, Volume 73, Number 20, Pages 5573-5574, on January 30, 2008. A copy of these notices is provided in Attachments A and B. Three public comments were received, one on February 4, 2008; another on February 6, 2008, both requesting a copy of both questionnaires. These requests were fulfilled on February 8, 2008. A third comment was received on January 30, 2008. This third public comment is beyond the scope the evaluation initiative as it deals with an opinion of an individual in regards to the use of Federal resources by the U.S. Government. See Attachment G for all comments.

b. Effort to Consult Outside the Agency

Efforts were undertaken to consult a CED and JOLI content and program expert, Dr. Mark Lelle, who has worked with AED for two years, to obtain his views on the availability of data, the clarity of instructions, disclosure, and the development of the proposed information collection.

A9. Explanation of Any Payment or Gift to Respondents

The respondents will not require a gift or payment.

A10. Assurance of Confidentiality Provided to Respondents

The grantee data collected from the CED and JOLI questionnaires will not be treated in a confidential manner. The topics focus on the program outcomes and grantee experiences. There are no questions of an inherently sensitive nature in the questionnaire. ACF will not use the term “confidential” in its correspondence with respondents and will use other appropriate language (such as “kept private to the extent permitted by law”) as necessary.

A11. Justification for Sensitive Questions

There are no questions of an inherently sensitive nature included in this data collection (See the final CED and JOLI questionnaires with the OMB Number).

A12. Estimates of Annualized Burden Hours and Costs

The project’s current hour burden and cost burden are presented in Table 1, which works with a 100% response rate or level. The hourly wage rate is derived from statistics provided by the 2005 National Compensation Survey (Department of Labor, 2005). The OCS grantees are mostly all executives working as directors of a community-based organization. The mean hourly rate for the respondents was therefore calculated as the overall mean of the mean hourly rate for senior executives in community-based organizations (mean of $36.22).

Table 1—Annualized Burden Hours and Cost Burden

Type of Respondent	Number of Respondents	Response per Respondent	Average Burden Hours	Total Burden Hours	Hourly Wage Rate	Total Respondent Cost
CED Grantees	147	1	1.5	220.5	$36.22	$7.987
JOLI Grantees	25	1	1.5	37.5	$36.22	$1,358
Total	172			258		$9,345

Source: Department of Labor; U.S. Bureau of Labor Statistics (2005).

A13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no additional costs to the respondents.

A14. Annualized Cost to the Federal Government

The cost to the Federal Government is $70,500. This includes both the Federal staff participation and the contract support participation in collecting the information.

Federal Staff GS- 14.3	Cost per Hour	Total Federal Staff Cost Per Year	Contract Support Cost Per Year	Federal staff + Contract Cost Per Year	Cost for 3 Years
16 hours	50	$800	$23,100	$23,900	$71,700

A15. Explanation for Program Changes or Adjustments

This project is a continuation of a 2007 ACF data collection with OMB Control Number: 0970-0317. The evaluation method, burden estimates, instrumentation, cover letters, and mode of data collection remain constant from the OMB package approved in March 2007 with the stated control number.

A16. Plans for Tabulation and Publication and Project Time Schedule

a. Tabulation

Data Entry. Each questionnaire submitted by the respondent will be coded into an Excel spreadsheet, which will be automatically updated by the AED information technology staff. An electronic Excel spreadsheet with survey data will be delivered to the AED research and evaluation staff by the AED information technology staff. Upon receipt, data from questionnaires will be logged in and then stored in a secure data storage room. Electronic questionnaires are already created in a standard on-line survey program with appropriate code (HTML). The respondent’s will be provided with unique passwords and user names in the ACF cover letter in order to complete the individual program questionnaire. The resulting electronic files will be saved on a secure directory available only available to the AED research staff. Each electronic questionnaire submission will be reviewed and used for data analysis for the final report.

Data Analysis. Quantitative and qualitative content analysis will be conducted to determine the key themes rising from respondent’s comments around program impact and experiences. Toward this end, all survey data will be uploaded into Excel and exported into SPSS, a quantitative data analysis software package, and qualitative data will be coded into relevant summary themes or codes. The initial coding structure uses descriptive codes based on the questions in the questionnaire. Each grouping of comments from a respondent will therefore be identified as relating to the plan for dissemination and translation of their research results.

b. Plan for Publication

One type of formal documentation will be required. This will consist of a detailed report, with an associated executive summary, that can be shared with OCS. This report the executive summary will have as their primary focus concrete findings on key performance indicators or measures.

c. Project Time Schedule

Table 2—Project Time Schedule presents the projected timeline for scheduling the distribution and administration of the questionnaires, conducting encouragement to participate, data analysis of the survey data, and preparation of reports.

Table 2—Proposed Project Time Schedule*
Activity	Number of Months After OMB Approval
Mail final cover letters and schedule telephone interviews using the questionnaire to collect survey data from grantees with information technology barriers	September-October
Field questionnaires Conduct telephone calls to encourage participation	October-November December
Data analysis	January
Production of reports for OCS	February-March

*This is a data collection for two questionnaires which will be completed by in fiscal years 2008 to 2012.

A17. Reason(s) Display of OMB Expiration Date is Inappropriate

The data collection instruments will continue to display the OMB Control Number 0970-0317and the new dates will be posted.

A18. Exception to Certification for Paperwork Reduction Act Submissions

We are not seeking exception to certification for the Paperwork Reduction Act Submission for this data collection.

PART B. Collection of Information Employing Statistical Methods

This submission requests OMB clearance to enable the gathering of information from the OCS grantees whose CED and JOLI programs were funded with start dates between fiscal years 2001 and 2007. It is the intent of this project to survey as many of these OCS grantees and, thus, no statistical sampling methods are employed. Information will be collected via the use of a closed- and open-ended web-based questionnaires. Descriptive statistics and content and thematic analysis will be used to conduct quantitative and qualitative analysis of the data. Thus both statistical and non-statistical information on the respondent and the information collection procedures for the project are described below.

B1. Respondent Universe and Sampling Methods

This data collection does not employ statistical methods as all active CED and JOLI grantees with start dates between 2001 and 2007 will be surveyed. This project will attempt to survey the active OCS grantees (i.e., lead grantee contact persons identified by OCS) of all CED and JOLI programs with start dates between fiscal years 2001 and 2007. It is preferable to survey all of the grantees rather than a sample for several reasons. First, since OCS funding amounts and project time lines change from year to year and from project to project, there is a great variety of CED and JOLI programs that make up the currently active CED and JOLI grantee sites. Second, for CED projects, there is a desire to survey over 100 grantee leads to strengthen the statistical methods (e.g., descriptive statistics) employed. Third, for JOLI projects, there are a small number of active grantees that can respond to the questionnaire, making it desirable for researchers to survey as many JOLI grantees leads, with the desire to survey 60-70 percent or more grantees to truly understand the full range of experiences.

B2. Procedures for the Collection of Information

This data collection does not employ statistical methods as all active CED and JOLI grantees with start dates between 2001 and 2007 will be surveyed. In this context, “active,” means grantees with programs with currently open Program Dates, which means that their projects are operational and have not officially ended based on OCS grants management. Data will be collected through the close- and open-ended questionnaire, which will be administered on the Internet web or by US mail distribution. For surveys administered via the web, the web site for the OCS evaluation initiative will be used, which has been developed by the Academy for Educational Development (the evaluation contractor to OCS) with guidance from key OCS staff. The OCS grantee lead per CED and JOLI projects will receive a cover letter—specific to CED or JOLI grantee work—from OCS requesting their involvement in the project (See final cover letters in the Appendices). Each cover letter will direct respondents to the OCS evaluation initiative web site and provide unique user names and passwords that ensure that each questionnaire completed represents one OCS program. Grantees with more than one action OCS program will receive the appropriate number of user names and passwords as well as cover letters to specify how the on-line questionnaires must be completed on a one-to-one basis per program. OCS and AED will work together to contact non-response grantees after the second cover letter is distributed to secure the response rate of 60-70 percent per OCS program. For grantee leads unable to use web and/or information technology, an AED evaluator (Dr. Rose Ann M. Rentería) will schedule a telephone interview time to administer the questionnaire and to collect the data. In such cases, all interview data will be entered in the web-based questionnaires by AED staff. Each section of the questionnaire corresponds to several core topics such as CED and JOLI programmatic impacts and outcomes, performance measurement and reporting, and a self-assessment regarding grantee organizational capacity.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

In order to encourage as many grantees as possible to participate in this project, lead grantees will receive a letter from OCS requesting their involvement in the evaluation with user-friendly instructions on the http address, password usage, and how to begin the questionnaire. If a lead grantee does not respond to the initial communications, AED staff will send the letter a second time to the lead grantee, encouraging his or her participation in the project. As needed, OCS and AED will schedule a telephone interview with lead grantees to encourage greater participation towards the completion of the questionnaires. OCS will account for non-response bias in its analysis and reporting of results. Specifically, OCS will stratify results according to where the grantees fall in their grant cycles, if necessary; AED with OCS support will conduct secondary analysis of non-responders based on annual and/or semi-annual reports submitted directly to OCS by the CED and JOLI grantees. AED and OCS will include a discussion of study limitations when it reports results from the study.

B4. Tests of Procedures or Methods to be Undertaken

The questionnaires were reviewed and approved by key OCS staff, including Lynda Perez, OCS Division of Community Discretionary Programs Director and Rafael J. Elizalde OCS Division of Community Discretionary Programs Team Leader. The questionnaires were internally and externally pre-tested on March 31, 2006, by an AED information technology and web design specialist. The pre-testing was done with internal AED volunteers until no additional usability enhancements could be suggested, and no usability concerns were raised.

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

This data collection does not employ statistical methods. AED, under the project direction of Dr. Rose Ann Rentería, has been granted the task of helping with the design and data collection, conduct the follow-up e-mails and telephone interviews, and assist with the analysis of the data. Dr. Rentería can be reached as (202) 884-8608. Ms. Lynda Pérez will be responsible for receiving and approving AED work products. She can be reached at Lynda Pérez, Director, Division of Community Discretionary Programs, Office of Community Services, Administration for Children and Families, 370 L’Enfant Promenade, S.W. - 5th Floor, Washington, DC 20447, (202) 401-9365; fax (202) 401-4687; and Main Office Number (202) 401-9333.

REFERENCE

Department of Labor; U.S. Bureau of Labor Statistics (2005). National Compensation Survey: Occupational Wages in the United States, July 2004. [Summary 05-04.].

FOUR APPENDICES

(Submitted Under Separate Cover)

Attachment A: Final Federal Register Notice No. 1
Attachment B: Final Federal Register Notice No. 2
Attachment C: Cover letter to CED Grantees on OCS Initiative
Attachment D: Cover letter to JOLI Grantees on OCS Initiative
Attachment E: Final CED Questionnaire with OMB Number
Attachment F: Final JOLI Questionnaire with OMB Number
Attachment G: Comments to Federal Register Notice No. 2

Attachment G: Comments to Federal Register Notice No. 2

From: [email protected] [mailto:[email protected]]
Sent: Wednesday, February 06, 2008 12:17 PM
To: InfoCollection (ACF)
Subject: OCS Evaluation Initiative Questionnaire

Dear ACF Reports Clearance Officer,

We are requesting a copy of the proposed collection, per the instructions of the January 30, 2008 Federal Register (pages 5573-6674, Department of Health and Human Services, Administration for Children and Families, Submission for OMB Review; Comment Request), as specified below:

Title: Office of Community Services (OCS) Evaluation Initiatives: Community Economic Development (CED) and Job Opportunities for Low-Income (JOLI) Individuals

OMB No.: 0970-0317

Let us know if you have any questions regarding this information request.

Thank you,
Jen Brandwein

Policy and Management Associates, Inc.
45 Newbury Street, Suite 310
Boston, MA 02116
Tel 617-266-1600
Fax 617-262-4580
[email protected]

From: Theresa Howe [mailto:[email protected]]
Sent: Monday, February 04, 2008 12:26 PM
To: InfoCollection (ACF)
Subject: OCS CED and JOLI Individuals

Can I please receive a copy of the proposal for the CED and JOLI grantees questionnaire?

Thank you

Theresa Howe

Director of Fund Development

Community Housing Partnership

415.929.2470 x 304

[email protected]

From: [email protected] [mailto:[email protected]]
Sent: Wednesday, January 30, 2008 7:26 AM
To: InfoCollection (ACF); [email protected]; [email protected]; [email protected]; [email protected]
Subject: public comment on federal register of 1/30/08 vol 73 #20 pg 5573 dhs

attn jeanann chambers - office of community services evaluation community economic development and job opportunity for loc income individuals

this whole program and bureaucracy should be shut down. we need to cut the spigots spending us tax dollars since so much of what tax dollars come in go into the pockets of opportunist profiteers who accomplish nothing with it. i think an audit of this program should be done before it is shut down to see who is pocketing the money. I ask the inspector general to do an audit of the spending here - an independent inspector general.

we have a labor dept in charge of jobs for low income and all people. let those who need jobs use that dept. we DO NOT NEED TO FUND DUPLICATION OF SERVICES IN EVERY SINGLE DEPT.
B. SACHAU
15 ELM ST
FLORHAMP ARK NJ 07932

**************

File Type	application/msword
File Title	IC - OMB Supporting Statement with NOA Summary Followed - Revised_FEBRUARY 8 2008 v2
Subject	IC - OMB Supporting Statement with NOA Summary Followed - Revised_FEBRUARY 8 2008 v2
Author	Rafael J. Elizalde
Last Modified By	sxp1
File Modified	2008-02-29
File Created	2008-02-29