Part A CDFI Fund_ Rev_10-18-10

This supporting statement provides detailed information on proposed information collection activities associated with a four-year evaluation of the New Markets Tax Credit (NMTC) program. The program is administered by the U.S. Department of the Treasury’s Community Development Financial Institutions (CDFI) Fund.

The evaluation is being done by the Urban Institute under contract to the CDFI Fund. It includes a one-time information collection effort involving participants and stakeholders in the program, and is intended to describe and assess program activities, identify project-specific outputs and outcomes as well as community outcomes, and address the issue of the need for NMTC investment.

A1. Circumstances that make the collection of information necessary

The NMTC program was established by the Community Renewal Tax Relief Act of 2000 to encourage private capital investment in low-income communities that are, or are perceived to be, high investment risks.^¹ It permits individual and corporate taxpayers to receive a credit against Federal income taxes for making qualified equity investments in designated Community Development Entities (CDEs). Substantially all of the qualified equity investment must in turn be used by CDEs to provide investments in Qualified Active Low-Income Community Businesses (QALICBs), which include both operating businesses and real estate developments. The federal tax credits are allocated competitively by the CDFI Fund to CDEs; CDEs, in turn, select projects in which to invest.

There is considerable flexibility and, consequently, variability with respect to the types of projects that can be supported by NMTC-stimulated investments. They can include for-profit and non-profit businesses involving commercial, industrial, retail, manufacturing or mixed-uses, for example, as well as community facilities such as childcare or health facilities or charter schools.

The data collection described in this document will provide critical information for an independent evaluation of the NMTC program. In contracting with the Urban Institute, the CDFI Fund expects that the evaluation will help to satisfy the Office of Management and Budget’s (OMB’s) requirements that federal agencies undertake independent evaluations of program performance. To date, no such evaluation has taken place. Instead, using a web-based system, CDE tax credit allocatees are required to submit standardized data to the Fund for compliance monitoring purposes. Such data, however, provide limited information on individual project activities or outcomes. While industry organizations conduct surveys of their membership and disseminate information on successful NMTC-financed projects, these initiatives do not provide a comprehensive picture, or an evaluation, of the NMTC program.

A2. How, by whom, and for what purpose the information is to be used

The Urban Institute and its subcontractors are responsible for the design, data collection and analysis for the NMTC program evaluation. The Institute will report its findings to the CDFI Fund, which, in turn, will report them to the Congress, the community development industry, and general public. The research will contribute to ongoing policy discussion regarding the outcomes and effectiveness of the NMTC program.

As context for understanding the scope of the evaluation, it is useful to refer to Bartik and Bingham’s conceptualization of a program evaluation continuum that roughly corresponds to a program’s life-cycle sequence and consists of six different levels—each of which builds on the previous.^² Points on the evaluation continuum are:

Assessing

Monitoring Assessing Costs Impact on

Daily Program Enumerating Measuring and the

Tasks Activities Outcomes Effectiveness Benefits Problem

Process/Formative Evaluation O u t c o m e / S u m m a t I v e E v a l u a t I o n

The continuum begins with two levels generally termed process or formative evaluations; these focus on how a program is delivered. The first such level involves examination of the internal workings of a program (like whether daily tasks are being carried out efficiently or contractual obligations are being met) and the next involves assessment of program activities (like what activities are taking place or how well the program is being implemented). A third level, enumerating outcomes, involves determining whether a program’s objectives are being achieved; it is the initial stage of what are called outcome or summative evaluations. One step further along the continuum is effectiveness measurement, which inquires as to whether a program’s goals have been accomplished and if the program is working; and the next is cost-benefit analysis, which inquires as to whether a program’s costs outweigh its benefits. A final level, impact assessment, seeks proof that a program is having a measurable impact on the problem to which it responds. Each successive level presents increasingly difficult challenges—particularly in establishing what would have happened in the absence of the program and, the ultimate challenge, determining whether a program caused particular outcomes to occur.

While the plan for evaluating the NMTC program contains elements from various levels, it focuses primarily on the middle of the continuum. In part this is because the program is beyond its formative phase but still at a point where some of its outcomes are not yet fully evident and, in part, this because the program’s impacts (like those of many community and economic development programs) are inherently difficult to demonstrate. The reasoning is as follows:

There is a growing NMTC literature dealing with formative program issues that provides a basic understanding of the program and its process. Contributors include academic researchers, program practitioners, program advocates, and the GAO. It is, therefore, neither appropriate nor necessary to duplicate that literature by concentrating on formative or process questions.

In terms of timing, it is not yet appropriate to attempt a summative evaluation. The NMTC program requires further seasoning before warranting a full benefit-cost analysis or assessment of the extent to which there has been a diminution of capital flow deficits to the nation’s economically distressed communities.
Apart from whether it is sufficiently mature for a summative evaluation, impact assessment of the NMTC program is an extremely challenging undertaking, requiring establishment of causality—i.e., evidence that observed outcomes are explicitly brought about by the program. Many serious academic and government researchers, however, question the ability to rigorously determine whether NMTC-type investments, in fact, cause community outcomes to occur, since this requires having a counterfactual—identically matched neighborhoods/communities or some form of controlled experimentation—to know what would have happened in the absence of program interventions.^³

The plan, therefore, emphasizes learning what is knowable at this point about the NMTC program’s investment effects and project-specific outputs and outcomes, and laying the groundwork for continued identification and tracking of longer-term outcomes beyond that. However, it stops short of being a full-fledged impact or summative evaluation. The purpose is to identify data collection methods and analyses that will produce valid evidence either consistent or inconsistent with the program’s objective of attracting capital to low-income, economically distressed communities to further their economic and community development.

The evaluation design flows from a general logic model for the NMTC program, shown on the following page. Logic models are tools to identify outputs, intermediate outcomes, end outcomes, and outcome indicators. These models visually diagram the steps that lead to the desired goals of the program. Logic models begin with “inputs” (in this case, the tax credits) that lead to “outputs” (such as the extent and types of incentives provided to improve the flow and access to private capital). These lead to “intermediate outcomes” (events or results expected to lead to end outcomes) and “end outcomes” (the ultimate goal of the particular program or service). End outcomes may include results that are expected in the relatively short term (such as an increase in the number of low-income communities served) as well as long-term outcomes, such as an increase in the number of successful businesses owned by residents of eligible communities. Distinguishing between intermediate and end outcomes is important. It is very important to measure both intermediate and end outcomes, although some end outcomes (such as increased wealth among community residents) may occur too far in the future to be addressed even in a comprehensive evaluation. The logic model lists examples of outputs, outcomes, and measures of these, which can include a mix of job creation or retention, physical improvement, tax revenue, service development or expansion, institutional capacity building, etc., and would vary by project and type of project.

INTERMEDIATE OUTCOMES

Improving Flow/Access to Private Capital

Performance of Allocatees

Community Level

OUTCOMES

(longer term)

Improving Flow/Access to Private Capital

Performance of Allocatees

Community Level Outcomes

OUTPUTS

INPUTS

Tax Credits

External/

Pre-existing Community Factors

EXAMPLES

Economic and business conditions
Population demographics
Geographic location
State/local policies
Existing financial and community development institutions/ resources
Other tax credits available
Role of local government

EXAMPLES

Value of NMTC allocations
Applicant characteristics
Selection criteria
Designated CDEs and their characteristics
Other investments: sources and amounts

EXAMPLES

Lower rates of return
Below market interest rates
Lower fees
Non-traditional forms of collateral
Financial counseling
Other uses of capital (flexibility, innovation)
Role of investors and CDEs in determining investment strategies
Characteristics of communities that received NMTC investments

EXAMPLES

Investment timeframe
Types of investments

Consistency between investments and activities proposed in applications
CDE success in putting NMTC-related dollars into qualified investments
Characteristics of investors; # new investors; increases in private investment

Influence on mission of CDEs, their affiliates or their parent corporations
Fee generation by CDFIs and other organizations
% of eligible census tracts being served by CDEs

EXAMPLES

Jobs created; types of jobs’ “quality” of jobs
Local employment
Sq. ft. of commercial real estate rehabilitated or developed
# and % of affordable housing units developed (for rent; for sale)
# of businesses financed that are owned by residents
Additional investments leveraged
Local residents’ perceptions of neighborhood changes
Business owners’ perceptions
Increase in non-NMTC transactions by CDEs

Following from the logic model, the evaluation design involves multiple research methods to produce valid evidence for describing and assessing program activities, outputs and outcomes. The focus is on the program’s primary objective of attracting capital to low-income, economically distressed communities so as to further their economic and community development. The evaluation is intended to inform an array of important policy questions, including:

How and in what manner the program is affecting the flow of new private capital to low-income communities.
Where, in what timeframe, and how that capital is invested to achieve community and economic development objectives.
The quantity, nature, and quality of the benefits to low-income communities and persons that can potentially be attributed to the NMTC program.

To answer such questions, the following will be undertaken:

A description of program activities—to provide a thorough description of projects implemented using the NMTC program, their capital flows, and project financing.

An assessment of project outputs and outcomes—to provide a comprehensive description of the outputs and outcomes associated with projects that include NMTC investments, quantify such outputs and outcomes to the extent possible, and gain knowledge that will establish results expectations with respect to different types (i.e., categories) of NMTC projects. Because NMTCs support a wide range of projects, differentiation is necessary to be able to identify appropriate metrics and fairly evaluate the program as a whole.

An assessment of community level outcomes—to learn about broader and longer-term community level results associated with NMTC investments, including additional investment, physical improvements, and benefits for community residents.

An assessment of the extent to which NMTC investment is needed—to learn about the efficiency of the credits in stimulating community and economic development.

In addition to the information collection described in this submission, the evaluation also includes: (a) a review of relevant economic development, performance measurement, and tax credit literature; (b) informal discussions with key NMTC stakeholders; (c) an analysis of existing NMTC administrative data; (d) development of a typology of NMTC projects; and (e) an examination of secondary public and private data. To address the key evaluation issues noted above, new data must be gathered about NMTC projects beyond existing data and what is being collected by the CDFI Fund for program administration purposes. Paperwork Reduction Act clearance is requested for the following new information collection efforts:

Case-style data collection related to 80 sampled NMTC projects that will consist of semi-structured telephone interviews with 80 CDEs, 80 QALICBs, 80 investors, and 80 other key project stakeholders.
An online, predominantly closed-ended survey of representatives of 380 QALICBs.
An online, predominantly closed-ended survey of 380 local community/economic development officials.

Toward this end, four topical discussion guides, an information sheet for CDEs, an information sheet for QALICBs, and two survey instruments are included in this submission. Table 1 shows how each of these data collection instruments, along with existing data, will be used to address the four key study clusters. While most data collection addresses project-specific implementation and outcomes, the role of federal programs generally in local community and economic development is important to understanding the role of NMTCs in this broader context. The survey of local community/economic development officials addresses this issue. Please note also that in Table 1 the use of existing NMTC administrative data (called the Community Investment Impact System, CIIS) does not constitute a new data-collection effort for this evaluation; the authority for the CDFI Fund to collect those data has previously been approved by OMB.

The remainder of this section provides a more detailed description of the use of each of these data sources and the plans for collection.

Case-style semi-structured telephone interviews with CDEs and other key project stakeholders will supplement available administrative data CIIS) for the NMTC program. Eighty (80) NMTC projects will be randomly selected, comprising a sample of diverse business, geographic, and financial characteristics. An average of four interviews will be conducted for each project, one each with a CDE official, investor, QALICB representative, and another stakeholder (as appropriate for each particular project, such as an attorney, accountant, local official, or representative of community group).

Case-style telephone interviews allow for exploring project initiation, financing alternatives, and outcomes to date with several respondents for each project. For example, the interviews will include a line of questioning about market conditions, selection of the project site, and the role of NMTCs in the project that will shed light on the issue of the need for NMTC investments. The interviews will also address actual, rather than projected, project outcomes. For outcomes such as employment, the questions will probe more deeply to ascertain the types of jobs created or retained and whether these jobs are filled by residents of the disadvantaged communities targeted by NMTCs. Finally, the interviews will include a discussion of a broader range of potential outcomes at the project and community level.

Appendix 1 contains topical guides that will be used for the telephone discussions with CDEs, QALICBs, investors, and other local stakeholders. Since some information requested of CDEs and QALICBs will require checking existing reports or records, a brief information sheet will be sent to CDEs and QALICBs in advance of their respective interviews—to be completed and returned in advance of the interviews. These information sheets are also included in Appendix 1.

An online survey of QALICBs. An online survey of 380 QALICBs allows examination of the characteristics and outcomes of a larger number of recipients of NMTC investments than will be included in the case-style interviews associated with 80 projects. The survey will provide quantifiable outcome data that can be disaggregated by categories of projects—facilitating analysis by project focus as well as other project characteristics. A copy of the QALICB survey instrument is included in Appendix 2.

An online survey of local community/economic development officials. A survey of 380 local (town, city, or county) community or economic development officials in places with NMTC projects provides the opportunity for more generic understanding of local officials’ perceptions of the role, if any, that a program like NMTC can play in their economic and community development efforts. Of interest is how NMTC projects fit into local economic development planning and the extent to which local officials are involved in the process of initiating or shaping NMTC-supported investments. This topic is neither well understood nor readily discernable from administrative data or other existing information on the program: local officials that plan and are involved in economic development activities in low-income communities have not previously been surveyed or specifically sought out for their knowledge of, or response to, the NMTC program.

A copy of the local community and economic development officials’ survey instrument is included in Appendix 3.

Table 1: Summary of Research Clusters and Data Sources
Research Questions and Data Collection Topics	Data Sources/Respondents*
	Use of Existing NMTC Administrative Data** Years 2003-2007	Local CD/ED Officials Survey Sample		QALICB Survey Sample	Case-style Interview Sample Projects
					CDE	QALICB	Investor(s)	Local CD/ED official, or other local stakeholder
Sample Size	2,030	380		380	80	80	80	80
Estimated # of Respondents	2,030	285		285	80	80	80	80
Program and Project Description
Project Initiation and Implementation	X				X	X	X	X
Capital Flows/Project Financing	X			X	X	X	X
Investor Information					X	X	X
Project Outputs and Outcomes
Enhanced Business Soundness				X	X	X	X
Increased Employment				X	X	X		X
Improved Physical Development				X	X	X		X
Expanded Housing Opportunities				X	X	X		X
Expanded Community/ Public Services				X	X	X		X
Enhanced Commercial/Retail Activity				X	X	X		X
Leveraged additional investment				X	X	X		X
Community Outcomes
Improved Economic Status/Quality of Life				X	X	X		X
Enhanced local or institutional capacity				X		X		X
Leveraged additional investment				X	X	X		X
Need for MNTC Investment
Role of NMTCs in the project				X	X	X	X
Market conditions				X	X	X	X	X
Community Economic Development Context
Role of Federal Programs in Local Economic Development			X					X

* Will be supplemented by data from secondary sources.

**Existing data—does not require new data collection or OMB clearance.

Given that this data collection constitutes the first comprehensive evaluation of the NMTC program, it is reasonable that a small portion of it be exploratory in nature, to serve as a basis for future research efforts. That is the case for several items in the two online surveys that explicitly ask respondents to speculate about certain phenomena. The items generally begin with words such as, “to the best of your knowledge,” or “based on your assessment.” These types of items are valuable where (a) one cannot expect respondents to have or report more objective information, (b) objective data are inherently difficult to come by, and/or (c) there is little opportunity (or too much burden) for seeking open-ended explanation in an online mode. This is the case, for example, for questions relating to ease of access to capital—given that there are few independent quantitative data sources for assessing the investment environment/climate at the community level. That information can better be obtained through in-depth case studies that track investment flows in selected sites, but that is not the approach being taken in the current evaluation. For other questions, such as the extent and quality of improvement of exterior, physical appearance or access to services, there are a number of possible measures that have been suggested in the community/economic development literature but no consensus as to which is best.

Subjective questions included in the online surveys should be viewed in the context of the overall evaluation plan in that they are intended to solicit the assessments of a reasonably large sample of QALICB and local community/economic development NMTC stakeholders. The opportunity to probe in greater detail with respect to how such stakeholders arrive at their assessments is provided through the smaller number of in-depth case-style interviews that are also being conducted as part of the evaluation. The latter information will be used (along with secondary data on economic context and CDFI Fund administrative data) to consider the evidentiary basis for the online responses. Triangulation of data sources will facilitate better understanding and categorization of more subjective responses, and will assist in identifying measures and indicators for future evaluation research. In sum, the few subjective questions on the online surveys are designed to be a valuable part of the evaluation; responses to them will be appropriately reported for what they are, what they add to an understanding of the NMTC program, and what they contribute to a next stage evaluation.

A3. Use of automated electronic, mechanical or other technological collection techniques to reduce burden

Silber & Associates Inc., a professional survey research firm and subcontractor to the Urban Institute, will conduct the two web-based surveys—the survey of QALICBs and the survey of local community/economic development officials. Respondents will respond online to an electronic version of the survey. Based on the research team’s experience with these respondent groups, it is expected that all of the respondents will have online access and Internet experience.

The web-based versions of the surveys will be created in Microsoft Visual Studio 2008.Net, a professional edition suite of tools to develop full-featured ASP.NET web applications. The advantage of the online survey is the automatic tabulation of responses that reduces both the hours of staff time needed for survey processing and the possibilities for introducing errors into the data. The automated skip patterns of the on-line survey also place less of a burden on the respondent than the customary “if-then go to” instructions of a mail questionnaire.

Silber & Associates makes full use of the latest methodological and technical developments in electronic surveys, including proprietary software for the distribution of mass e-mail and website designs that are compliant with Section 508 of the Rehabilitation Act.

A4. Efforts to identify duplication

Neither the CDFI Fund nor the Urban Institute is aware of any other national evaluation of the New Markets Tax Credit program. An extensive review of the literature by the Urban Institute revealed no other current studies collecting the same information evaluating the NMTC program. In the authorizing language for the NMTC Program, the Government Accountability Office (GAO) was required to report to Congress on the NMTC Program by January 31 of 2004, 2007, and 2010. Urban Institute researchers have reviewed the 2004 and 2007 reports and have met with GAO staff members who are working on GAO’s 2010 report in an effort to identify and prevent duplication of data collection efforts.

A careful review of data reported to the Fund and entered into its administrative CIIS has been done to ensure that data collection instruments will not repeat or duplicate that system.

A5. Methods to minimize the burden on small businesses or other small entities

Some QALICBs that will be surveyed will be small businesses and some stakeholders that will be surveyed will be small nonprofit and/or community-based organizations. Efforts have been taken to minimize the reporting burden on these entities. When applicable, Information will be collected from CDFI Fund administrative (CIIS) data so it does not have to be collected again for the evaluation. Financial statements and additional information on non-NMTC financing will be requested first from the CDE rather than from the QALICB or other stakeholders. All survey questions and discussion guides will use terminology and definitions that are consistent with CIIS reporting guidelines (e.g., defining full-time employment as 35 hours or more per week) in order to minimize confusion and eliminate the need for recalculation on the part of respondents.

A6. Consequences if data are not collected

This will be the first comprehensive evaluation of the NMTC program. Failure to collect these data will result in insufficient information to evaluate the performance and outcomes of the program. This information is critical to ongoing assessment, including identifying appropriate output and outcome measures and refining existing reporting requirements. While CDFI Fund administrative (CIIS) data provide valuable information on the NMTC program, descriptions of program activities and actual, rather than projected, outputs and outcomes are limited. These need to need to be expanded and probed in greater detail in order to complete a systematic and rigorous evaluation.

A7. Special circumstances

The proposed data collection activities are consistent with the guidelines set forth in 5 CFR 1320.6 (Controlling Paperwork Burden on the Public—General Information Collection Guidelines). There are no special circumstances that require deviation from these guidelines.

A8. Federal Register Notice and consultations with persons outside the agency

8a. Federal Register Notice

In accordance with 5 CFR 1308.8 (d) a Notice was published in the Federal Register on February 10, 2010 (page 5848) announcing the CDFI Fund’s intention to request OMB review of this data collection effort and soliciting public comments. No public comments were received.

8b. Consultations with Persons Outside the Agency

Consultations with persons outside the agency (U.S. Department of the Treasury, CDFI Fund) have included a series of discussions with program stakeholders and regular contact (for advice and review) with a panel of academic and program-specialist consultants—convened especially for this evaluation by the Urban Institute.

Stakeholder discussions. Between March and May 2008, Urban Institute researchers held a series of informal telephone and in-person consultations with selected NMTC program stakeholders—including those who had been involved in the original planning and design of the program, such as: Congressional and GAO staff; program administrators; and prominent practitioners, advocates and researchers. Stakeholders were selected to allow for a broad range of perspectives. They were identified through the suggestions of CDFI Fund staff, the Urban Institute’s consultants for the evaluation (see below), and the discussants themselves. The purpose of the consultations was to learn about: the program’s intent as articulated by executive and legislative branch officials and key program observers; the standards of success that such stakeholders apply to the program; and stakeholders’ expectations and interests with respect to a program evaluation. In total, approximately 25 persons were consulted. Their observations provided important and relevant background information that helped to shape the design of the evaluation.

Advisory and review consultations. The panel of consultants established by the Urban Institute specifically for the evaluation first met as a group in December 2007 to discuss the research design and, subsequently, have reviewed and commented on all key project deliverables. They reviewed an earlier draft of this Supporting Statement as well as draft data collection instruments, and their comments and suggestions have been incorporated in this document. The consultant panel consists of:

John P. Caskey, Ph. D

Professor of Economics

Swarthmore College

Swarthmore, PA

Kenneth Temkin, Ph. D

Principal

Temkin Associates

Rockville, MD 20851

Robinson Gill Hollister, Jr., Ph.D.

Professor of Economics

Swarthmore College

Swarthmore, PA

Sean Zielenbach, Ph. D.

Principal

SZ Consulting, LLC

Arlington, VA 22201

Michael J. Rich, Ph.D.

Associate Professor of Political Science

Director, Office of University-Community Partnerships

Emory University

Atlanta, Georgia

Signe-Mary McKernan, Ph. D.

(Internal Advisor)

Senior Research Associate / Economist

Center on Labor, Human Services, and Population

The Urban Institute

Julia Sass Rubin, Ph. D.

Assistant Professor

Edward J. Bloustein School of Planning and

Public Policy

Rutgers University

New Brunswick, NJ

A9. Remuneration to respondents

No payments are being made to respondents, who voluntarily agree to participate in this data collection.

A10. Assurances of confidentiality

As previously indicated, data collection and analysis for the evaluation of the New Markets Tax Credit program will be done by the Urban Institute under contract to the CDFI Fund. The Institute maintains an Institutional Review Board (IRB) to ensure that research practices and procedures effectively protect the rights and welfare of human subjects, consistent with the requirements set forth in Title 45, Part 46 of the Code of Federal Regulations (45 CFR 46). The Institute’s policy is that all research involving human subjects, not just research sponsored by federal government agencies that have adopted the Common Rule under 45 CFR 46, must adhere to the following principles, among others:

Risks to human subjects from research must be reasonable in relation to anticipated benefits, and must be minimized to the extent possible;
Human subjects must be fully and accurately informed of the nature of the research in which they will be involved, whether their participation is mandatory or voluntary, any consequences of non-participation, any risks associated with their participation, and how the research will be used;
Adequate provision must be made to protect the privacy of human subjects and to maintain the confidentiality of data that are collected, where promised and as appropriate; and

Prospective survey and interview respondents for the present data collection will be informed, through introductory communications and as part of the survey and interview process, about the purpose of the data collection, its sponsorship, and that participation is voluntary. They will also be informed that the data being collected by the Urban Institute, as a contractor to the CDFI Fund, are the property of the Fund, which does not have statutory authority to protect the confidentiality of the data. For that reason, respondents cannot be assured of confidentiality. That notwithstanding, the CDFI Fund has informed the Urban Institute that its intention is to use the information collected only in aggregate form and only for research purposes. Accordingly, reports produced by the Urban Institute for the CDFI Fund will not include the names of organizations, projects, respondents, or other identifying information that may be used to link respondents with particular information collected. Survey instruments and introductory scripts for telephone interviews will explicitly inform prospective participants of the terms and limits related to confidentiality, including the CDFI Fund’s intentions for use of the information and the Urban Institute’s reporting plans—as illustrated by the following excerpt from the case-style telephone discussion guide for CDEs:

The information you give us will be combined with information we receive from all others we talk with, and the Urban Institute will not cite or report it in any way that would identify you, your organization, or your project. However, because the information we collect is the property of the CDFI Fund, I need to tell you that we cannot promise you confidentiality. We have been informed by the Fund, nevertheless, that it is their intention to use the information only in aggregate form and only for research purposes. Likewise, all requests for information from the public (through the Freedom of Information Act [FOIA] or otherwise) will be examined by the CDFI Fund on a case-by-case basis to ensure that, where possible, personally identified information is protected from release.

Within the Urban Institute, identifying information of respondents will only be shared with Urban Institute staff who have signed a Data Confidentiality Pledge and who need the information for research purposes. All such staff, as well as consultants to the Urban Institute for the evaluation, have signed this pledge. Hard-copy materials containing respondent identifying information will be locked up when not in use, and electronic materials will be stored in a secured server in password-protected and/or encrypted files, where appropriate.

A11. Questions of a sensitive nature

The questions being asked of CDEs, QALICBs, investors, community/economic development officials, or community stakeholders are not considered personally sensitive. Personally sensitive questions are defined as those whose answers, if made public, could cause physical, mental, emotional, economic, or other harm to an individual. However, some of the entities to be interviewed will be private-for-profit businesses or non-profit organizations that may consider some information about their businesses or organizations to be proprietary. Respondents will be informed that participation is voluntary and that they can decline to answer any question—without consequence.

A12. Estimates of the burden of the collection of information

12a. Estimate of respondent burden hours

The data collection efforts will involve online surveys and telephone interviews. Online survey respondents will be QALICBs and community/economic development officials in communities with NMTC projects. Telephone interview respondents will be stakeholders in projects that use NMTCs—including CDE officials and staff; representatives of QALICBs (may include operating businesses, real estate developers, nonprofit service providers); investors (may include bankers, insurance company financial managers, individual investors); local economic or community development officials; and other stakeholders such as community organizations, attorneys and accountants. There will be no overlap between the survey and telephone interview respondents. CDE representatives are the only respondent group that may be requested to participate in more than one interview, as some CDEs may have provided NMTC allocations for more one than project in the evaluation sample. The interviews, therefore, will not be duplicative since each will focus on a unique project.

It is estimated that the evaluation will collect data involving approximately 1,080 responses. This estimated number represents the sum of 760 web-based survey responses and 320 telephone interview responses (Table 2).

The times estimated in Table 2 are based on a previous study conducted by the Urban Institute using the case-style telephone discussion method to collect information about five NMTC projects,^⁴ and a pretest (conducted in October and November 2009) of each of the data-collection instruments appended to this Supporting Statement. Fewer than 9 respondents were included in the pretesting of each instrument (further detail about the pretest is provided in Part B of this Justification).

QALICB survey burden. The estimated average response time for the online survey of QALICBs is 30 minutes (1/2 hour) based on a pretest of the survey with 3 QALICBs. There will be 380 QALICBs in the survey sample. As discussed further in Section B1.3, a 75 percent response rate of is assumed. Each respondent will be surveyed only once.
CD/ED survey burden. The estimated average response time for the online survey of local community/economic development officials is 15 minutes (1/4 hour) based on a pretest of the survey with 4 community/economic development officials. There will be 380 community/economic development officials in the survey sample. As discussed further in Section B1.3, a 75 percent response rate of is assumed. Each respondent will be surveyed only once.
Case-style telephone interview burden. The estimated average response time for each telephone interview is 1 hour and 15 minutes (1 ¼ hours) for CDEs and QALICBs and 30 minutes (1/2 hour) for investors and other community stakeholders. This is based on pretest telephone interviews with: 4 CDEs of the CDE topical guide; 3 QALICBs of the QALICB topical guide; 3 investors of the investor topical guide; and 2 community stakeholders of the community stakeholder topical guide. The estimate for CDEs and QALICBs includes interview time as well as time required to access reports or records and complete the information sheet. The information requested should be available in financial statements routinely provided by the QALICB to the CDE, so no extraordinary information collection is required of respondents. It is estimated that approximately 17 CDEs will have more than one project to report on. As discussed further in Section B1.3, it is assumed that interviews will be completed with virtually all of the projects.

Table 2: Respondent Burden Estimates

Description	No. of Respondents	No. of Responses per Respondent	Estimated Total Responses	Estimated Hours per Response	Total Burden Hours
QALICB Survey	285	1	285	0.50	142.5
QALICB Survey-non-respondents	95	1	95	0.03	2.85
Local CD/ED Agency Survey	285	1	285	0.25	71.25
Local CD/ED Agency Survey-non-respondents	95	1	95	0.03	2.85
QALICB Interview*	80	1	80	1.25	100
Investor Interview	80	1	80	.50	40
Local CD/ED Agency Interview	80	1	80	.50	40
CDE Interview**	80	1	80	1.25	100
TOTALS			1080		499.45

* Includes time to retrieve records and complete the information sheet.

** Includes time to retrieve records and complete the information sheet. Some CDEs will be asked to respond on more than one project, but different CDE staff may respond for each project, so this table uses the total number of projects (80) that will be reported on.

12b. Total annual cost burden to respondents

Potential respondents for the online surveys and telephone interviews associated with this data collection range widely in position and industry. The median hourly wages of selected occupations (classified by Standard Occupational Classification, SOC, codes) was compared using Occupational Employment Statistics from the U.S. Department of Labor’s Bureau of Labor Statistics. Potentially relevant occupations and their median hourly wages include:

Occupation	SOC Code	Median Hourly Wage Rate
Financial Analyst	13-2051	$35.17
Loan Officer	13-2072	$26.30
Chief Executive	11-1011	$76.23
Financial Manager	11-3031	$47.76
Social and Community Service Manager	11-9151	$26.92

Source: Occupational Employment Statistics, May 2008, accessed online at June 17, 2009 at http://www.bls.gov/oes/current/oes_stru.htm

The median hourly wages for these occupations ranges from $26.30 to $76.23. Among the occupations, “financial manager” was selected as most representative of the type of respondent expected for the online surveys and telephone interviews included in this data collection. The 2008 median hourly wage for financial managers was $47.76, roughly in the middle of the range of the selected occupations. Based on this assumption, the estimated total respondent costs are: 499.45X $47.76 = $ 23,853.73 (Total Respondent Burden Hours X Median Hourly Wage Rate = Total Respondent Costs).

A13. Total annual cost burden to respondent or record keepers

There are no capital/start-up or ongoing operation/maintenance costs associated with this information collection.

A14. Estimate of annual cost to the government

The total cost for this study, including but not limited to the data collection activities described in this submission, is $1,756,044 over a 48-month period. Included are costs associated with background research, evaluation design, development of data collection instruments, data collection activities, analysis, and reporting. Study costs by year are:

Year 1, October 2007-September 2008, $267,254
Year 2, October 2008-September 2009, $222,954
Year 3, October 2009-September 2010, $821,842
Year 4, October 2010-September 2011, $443,994

A15. Reasons for any program changes or adjustments

This submission is a new request for approval; there is no change in burden.

A16. Plans for tabulation, analysis, and publication

The online survey and telephone interview data collection will begin after OMB approval. The QALICB and local community/economic development officials’ surveys will be fielded from May 2010 through July 2010. Case-style telephone interviews will be conducted from June 2010 through October 2010.

Silber & Associates, a survey research subcontractor, will provide the Urban Institute with an electronic analysis file of all survey responses. For the case-style interviews, Institute staff are preparing a standardized format to be used by researchers in documenting responses. This will produce electronic files suitable for analysis of qualitative as well as quantitative data obtained in the interviews.

Descriptive analysis of survey and interview data will consider key variables such as allocation round, project size, and location. A typology of projects has been developed to allow analysis of project outcomes by the focus of project activities. One of the challenges in evaluating the NMTC program as a whole, as distinct from evaluating individual projects, is that the program includes quite diverse types of investments. The intent is to be sensitive to the possibility of a range of potential outputs and outcomes (identified from an initial assessment of the NMTC program and from a review of the community and economic development literature), and consider the extent to which any given output or outcome is likely to be associated with a particular focus of NMTC project activities. Since the evaluation will collect and analyze a good deal of data beyond that collected by the CDFI Fund through its administrative systems, it is not limited to information and categories available in the CIIS. Initially, the evaluation will begin to distinguish among anticipated outputs and outcomes for each of the following foci:

Restaurants, hotels, food services, or other retail space investments
Financial, professional, scientific, management, business, or other office space investments

Industrial, manufacturing, transportation logistics, or warehousing space investments
Housing investments
Health, human and social service facilities investments
Educational and community facilities investments
Facilities or space for the performing arts, cultural activities, entertainment, or other amenities investments
Business operations (e.g., start-up, working capital, equipment) investments

The intersection of the focus of project activities and outputs/outcomes will then be considered, including those related to business soundness, employment, physical development, housing opportunities, expanded community/public services, etc. Two types of analysis will be done. First, the outputs/outcomes of all projects will be aggregated to estimate the extent of their occurrence in the program as a whole. So, for all 80 projects to be reviewed, the extent to which employment was increased, housing opportunities were expanded, local institutional capacity was enhanced, etc. will be assessed. Based on this information and extrapolation to the universe of NMTC projects, the extent to which each of these output/outcome categories is associated with the NMTC program will be estimated.

However, since not all projects would be expected to produce all such outputs/ outcomes, the typology allows for clustering of projects by the focus of their activities and, then, for separate assessment of each activity type where it is hypothesized that particular outputs/outcomes are likely to occur. According to this reasoning, for example, investments in community facilities would not be likely to result in enhanced business soundness but could produce better access to community services. Likewise, investments in industrial, manufacturing, transportation and logistics, or warehousing space would not be likely to result in expansion of housing opportunities but could result in increased employment. Therefore, projects involving community facilities would be evaluated against the extent to which they resulted in better access to community services while projects involving industrial, manufacturing, or warehousing space would be judged against the extent to which they resulted in increased employment. This provides a vehicle for evaluating outcomes for diverse types of projects that are reasonable with respect to their attributes.

The evaluation will also incorporate data collected by the CDFI Fund through the CIIS as well as secondary data on employment, property values, and demographic characteristics of communities with NMTC-supported projects. The final report for this evaluation will bring together and synthesize all such relevant data—including from the literature review, administrative data, and data from the proposed new information collection. The report will be organized to address the four issue clusters discussed above: program descriptions and trends; project-specific outputs and outcomes; community outcomes; and the need for NMTC investment.

A17. Approval to not display the OMB expiration date

Not Applicable. The CDFI Fund plans to display the expiration date for OMB approval of the information collection on all instruments and correspondence with prospective respondents.

A18. Exception to the certification statement

This submission, describing data collection, requests no exceptions to the Certificate for Paperwork Reduction Act (5 CFR 1320.9).

1 P.L. 106-554 was signed into law on December 21, 2000. It defines low-income communities as consisting of census tracts with a poverty rate of at least 20 percent, a median family income of less than 80 percent of the metropolitan areas or statewide median (whichever is greater) or, for non-metropolitan census tracts, a median family income of less than 80 percent of the statewide median. Census tracts typically contain between 1,500 and 8,000 people, with an average size of about 4,000 people.

2 Timothy J. Bartik and Richard D. Bingham, Can Economic Development Programs be Evaluated? Upjohn Institute Staff Working Paper 95-29 (pages 2-3), prepared for Richard D. Bingham and Robert Mier, Significant Issues in Urban Economic Development (Newbury Park, CA: Sage).

3See, for example, U.S. General Accounting Office, New Markets Tax Credit: Status of Implementation Related to GAO’s Mandated Reports, 2002; Julia Sass Rubin and Gregory M. Stankiewicz., “The New Markets Tax Credit Program: A Midcourse Assessment, Community Development Investment Review, 2005; and Dan Immergluck. What Might We Know? Research Design Issues for Measuring CDFI Subsector Impacts, 2006, http://www.prism.gatech.edu/~di17/Macathur.pdf.

4 Martin Abravanel, Nancy Pindus and Brett Theodos, Analysis of Selected New Markets Tax Credit Projects, Urban Institute, June 2007.

File Type	application/msword
File Title	Table of Contents
Author	admined
Last Modified By	US Department of Treasury
File Modified	2010-10-19
File Created	2010-10-19