4 Survey of Nonprofit Comparison Group

SIF National Assessment Survey
Version 4: 2015 survey of nonprofit comparison group

Introduction

The Corporation for National and Community Service (CNCS) funds nonprofit intermediaries under the Social Innovation Fund (SIF). These SIF intermediaries make grants and provide other support to nonprofit organizations to implement programs with evidence of effectiveness, to build the evidence base for programs through rigorous evaluation, and to share results and lessons learned.

CNCS has contracted with ICF International to conduct a National Assessment of SIF. To understand the SIF program in the larger context of grantmaking in the U.S., ICF is conducting a survey of both SIF intermediaries and a national sample of nonprofit organizations that make grants to other U.S. nonprofits.

Your organization was selected for the survey sample because it is a nonprofit organization that makes grants to nonprofit community organizations to carry out programs. On behalf of CNCS and the SIF program, you are asked to participate in this survey. Participation is voluntary, but we hope your organization will participate because your organization’s experience and perspective are extremely valuable for understanding grantmaking in the U.S.

The survey asks about selection of grantees, support for grantees, evaluation, scaling up of programs, and collaboration. For purposes of understanding change in grantmaking over recent years, the survey will ask about your organization both in 2009 and five years later, in 2014.

The survey is sent to you as the contact person for your organization. If you need to involve someone else in your organization to respond to the survey questions – either because of the topic or because it asks about 2009 and 2014, please ask that person or persons to respond to questions where they have the needed knowledge. We are requesting that we receive one completed survey for the organization.

The survey will take about 20 minutes to complete. Data will be reported in aggregate; reports of survey findings will not identify individual persons or organizations. If some comments by respondents would be helpful to present with the organization identified, we will check with respondents and only identify the source with the respondent’s permission.

If you have any questions about the survey, please contact Elyse Goldenberg ([email protected]; 703-225-2426) or Whitney Marsland ([email protected]; 703-225-2247) at ICF International. 

Background

1) Does your organization make grants to any U.S. nonprofit organizations to carry out programs or interventions in communities?

( ) Yes

( ) No

Does your organization fund nonprofit organizations in the U.S. to carry out programs or interventions in any of the following areas? (please mark all that apply)

[ ] Community health or healthy futures – Promoting healthy lifestyles and reducing the risk factors that can lead to illness (for example, disease prevention, low-income healthcare/outreach)

[ ] Youth development (includes early childhood development) – Preparing America’s youth for success in school, active citizenship, productive work, and healthy and safe lives (for example, summer or after-school programming, delinquency prevention, youth mentoring/tutoring/volunteers)

[ ] Economic opportunity – Increasing economic opportunities for economically disadvantaged individuals (for example, job or career training or readiness for adults)

2) Thinking of your organization's funding of nonprofit organizations for carrying out programs, is this funding (please mark all that apply):

[ ] Geographically-based (fund organizations within a community, state or other specified geographic region)

[ ] Issue-based (fund organizations whose programs focus on specific issue areas, but may be located in a wide variety of geographic locations around the U.S.)

[ ] Other (specify): _________________________________________________

3) What level of knowledge do you have about the Corporation for National and Community Service (CNCS) Social Innovation Fund?

( ) None – not aware of CNCS Social Innovation Fund

( ) Some knowledge – have heard of it and know something about it

( ) Know quite a lot about the CNCS Social Innovation Fund

( ) Know a great deal about it – very familiar with the program

Selection of Grantees to Fund

This section asks about your organization’s approach to selecting grantees to fund to carry out programs in communities, in both 2009 and 2014. It also asks about changes your organization may have experienced over this period and (when applicable) the major factors that contributed to these changes.

In responding to these questions, please think about your overall funding to nonprofits that conduct programs in communities (not about individual programs or grant portfolios). 

4) To what extent did your organization do the following in selecting nonprofits to fund in 2009? To what extent did your organization do these things in 2014?

For each thing that changed between 2009 and 2014, please indicate the factors that contributed to this change in the rightmost column. If there was no change between 2009 and 2014, mark “Not applicable (no change).” If you report that "other" major factors contributed to a change, please describe those factors below.

	To what extent did your organization do this in 2009?
	Always	To a very large extent	To a large extent	To a moderate extent	To a small extent	To a very small extent	Not at all
A. Used an open competitive process to solicit and review applications and to make selection decisions
B. Required applicant organizations to provide evidence of intervention effectiveness to be eligible for funding (includes pre- and post-test outcome data or other evidence based on evaluation studies)
C. Required applicants to submit a plan for rigorous evaluation of intervention to be eligible for funding (that is, quasi-experimental designs with a comparison group, experimental designs or other similarly rigorous designs)

	To what extent did your organization do this in 2014?
	Always	To a very large extent	To a large extent	To a moderate extent	To a small extent	To a very small extent	Not at all
A. Used an open competitive process to solicit and review applications and to make selection decisions
B. Required applicant organizations to provide evidence of intervention effectiveness to be eligible for funding (includes pre- and post-test outcome data or other evidence based on evaluation studies)
C. Required applicants to submit a plan for rigorous evaluation of intervention to be eligible for funding (that is, quasi-experimental designs with a comparison group, experimental designs or other similarly rigorous designs)

	If applicable: Major factors that contributed to the change (mark all that apply)
	Trends in the larger grantmaking world	Specific requirements attached to the funding your organization receives	Your organization’s board/leadership directed organization to implement change	Other (please specify below)	Not Applicable (no change)
A. Used an open competitive process to solicit and review applications and to make selection decisions	[ ]	[ ]	[ ]	[ ]	[ ]
B. Required applicant organizations to provide evidence of intervention effectiveness to be eligible for funding (includes pre- and post-test outcome data or other evidence based on evaluation studies)	[ ]	[ ]	[ ]	[ ]	[ ]
C. Required applicants to submit a plan for rigorous evaluation of intervention to be eligible for funding (that is, quasi-experimental designs with a comparison group, experimental designs or other similarly rigorous designs)	[ ]	[ ]	[ ]	[ ]	[ ]

If you indicated that other major factors contributed to change in your organization relating to selection of nonprofits to fund between 2009 and 2014, specify those factors below:

	Other major factor(s) that contributed to change between 2009 and 2014
A. Used an open competitive process to solicit and review applications and to make selection decisions
B. Required prior evidence of intervention effectiveness as basis for funding
C. Required plan for rigorous evaluation of the intervention as a basis for funding

Support for Grantees

Grantmaking organizations vary in the extent to which they provide support to grantees to carry out their work and to develop their capacity to do the work. This section asks about financial assistance your organization may provide to assist your grantees in conducting evaluations of their programs and about non-financial support you may provide to grantees to carry out their work.

The question uses the term “training and technical assistance” to refer to a variety of kinds of non-financial support to help grantees implement their programs and achieve program goals. Examples of training and technical assistance include:

• training and coaching, whether provided in-person or remotely (e.g., webinars)

• technical assistance, including activities such as consultation, problem solving or facilitation

• the provision of handbooks, tools, templates or other resources for grantees to use to carry out their work

• bringing grantees together (in person or remotely) to share problems and solutions

• other similar non-financial support to assist grantees to implement their programs

5) To what extent did your organization provide support for your grantees in the following areas, in 2009 and 2014?  If the support varied by grantee or grant program, please think about your grantees or programs overall or on average.

For each thing that changed between 2009 and 2014, please indicate the factors that contributed to this change in the rightmost column. If there was no change between 2009 and 2014, mark “Not applicable (no change).” If you report that "other" major factors contributed to a change, please describe those factors below.

	To what extent did your organization do this in 2009?
	Always	To a very large extent	To a large extent	To a moderate extent	To a small extent	To a very small extent	Not at all
A. Provided funding to carry out an evaluation or hire an external evaluator (as part of a grant or through other means)
B. Provided training or technical assistance (by your staff, consultants or other means) to conduct rigorous evaluation
C. Provided training or technical assistance (by your staff, consultants or other means) to support implementation of the program

	To what extent did your organization do this in 2014?
	Always	To a very large extent	To a large extent	To a moderate extent	To a small extent	To a very small extent	Not at all
A. Provided funding to carry out an evaluation or hire an external evaluator (as part of a grant or through other means)
B. Provided training or technical assistance (by your staff, consultants or other means) to conduct rigorous evaluation
C. Provided training or technical assistance (by your staff, consultants or other means) to support implementation of the program

	If applicable: Major factors that contributed to the change (mark all that apply)
	Trends in the larger grantmaking world	Specific requirements attached to the funding your organization receives	Your organization’s board/leadership directed organization to implement change	Other (please specify below)	Not Applicable (no change)
A. Provided funding to carry out an evaluation or hire an external evaluator (as part of a grant or through other means)	[ ]	[ ]	[ ]	[ ]	[ ]
B. Provided training or technical assistance (by your staff, consultants or other means) to conduct rigorous evaluation	[ ]	[ ]	[ ]	[ ]	[ ]
C. Provided training or technical assistance (by your staff, consultants or other means) to support implementation of the program	[ ]	[ ]	[ ]	[ ]	[ ]

If you indicated that other major factors contributed to change in your organization relating to support for grantees between 2009 and 2014, specify those factors below:

	Other major factor(s) that contributed to change between 2009 and 2014
A. Provided funding to carry out an evaluation or hire an external evaluator
B. Provided training or technical assistance to conduct rigorous evaluation
C. Provided training or technical assistance to support implementation of the program

Evaluation

Evaluation is a systematic process to address such issues as the extent to which a program or intervention achieves its intended outcomes and impacts and how it can be improved. Organizations differ in their use of evaluations of programs that address community needs. In addition, the importance of evaluation as part of an organization's practice may change over time.

This section asks about your organization’s use of evaluation in 2009 and 2014 and its evaluation resources and infrastructure. In addition, we are interested in changes your organization may have experienced over that time, and the factors that contributed to those changes.

6) To what extent did your organization do the following regarding evaluation in 2009 and 2014?

For each thing that changed between 2009 and 2014, please indicate the factors that contributed to this change in the rightmost column. If there was no change between 2009 and 2014, mark “Not applicable (no change).” If you report that "other" major factors contributed to a change, please describe those factors below.

	To what extent did your organization do this in 2009?
	Always	To a very large extent	To a large extent	To a moderate extent	To a small extent	To a very small extent	Not at all
A. Conducted rigorous evaluations of programs funded by your organization
B. Used evaluation findings to improve programs funded by your organization
C. Used evaluation findings to demonstrate and communicate effectiveness of programs funded by your organization

	To what extent did your organization do this in 2014?
	Always	To a very large extent	To a large extent	To a moderate extent	To a small extent	To a very small extent	Not at all
A. Conducted rigorous evaluations of programs funded by your organization
B. Used evaluation findings to improve programs funded by your organization
C. Used evaluation findings to demonstrate and communicate effectiveness of programs funded by your organization

	If applicable: Major factors that contributed to the change (mark all that apply)
	Trends in the larger grantmaking world	Specific requirements attached to the funding your organization receives	Your organization’s board/leadership directed organization to implement change	Other (please specify below)	Not Applicable (no change)
A. Conducted rigorous evaluations of programs funded by your organization	[ ]	[ ]	[ ]	[ ]	[ ]
B. Used evaluation findings to improve programs funded by your organization	[ ]	[ ]	[ ]	[ ]	[ ]
C. Used evaluation findings to demonstrate and communicate effectiveness of programs funded by your organization	[ ]	[ ]	[ ]	[ ]	[ ]

If you indicated that other major factors contributed to change in your organization relating to evaluation between 2009 and 2014, specify those factors below:

	Other major factor(s) that contributed to change between 2009 and 2014
A. Conducted rigorous evaluations of programs funded by your organization
B. Used evaluation findings to improve programs funded by your organization
C. Used evaluation findings to demonstrate and communicate effectiveness of programs funded by your organization

7) Did/does your organization have the following?

	2009	2014
Staff position(s) or group within your organization dedicated to evaluation	Yes/No	Yes/No
External evaluation partner(s) -- consultant(s) or organization(s) that provide your organization with evaluation services	Yes/No	Yes/No
Part of the organizations' budget dedicated to evaluation	Yes/No	Yes/No

8) For this question, think about your organization's total evaluation budget in 2014 in comparison to 2009 (both in terms of dollars and as a percentage of the organization's total budget). Was your 2014 evaluation budget...

	Total evaluation budget in dollars	Evaluation budget as percentage of organization’s total budget for year
Substantially higher than the evaluation budget in 2009	[ ]	[ ]
Somewhat higher than the evaluation budget in 2009	[ ]	[ ]
About the same as the evaluation budget in 2009	[ ]	[ ]
Somewhat lower than the evaluation budget in 2009	[ ]	[ ]
Substantially lower than the evaluation budget in 2009	[ ]	[ ]

9) If you can access or estimate your organization's budget and staffing for evaluation in 2009 and 2014, please provide the following information:

	2009	2014
Total annual evaluation budget (in $) – for in-house evaluators or external partners
Annual evaluation budget as % of the total organization budget
Number of full time equivalent (FTE) staff with primary responsibility for evaluation

Scaling up of evidence-based programs: Increasing the impact of programs within the community or in other communities

A number of grantmakers are interested in scaling up programs that have shown evidence of effectiveness -- increasing the impact of a program within the community or expanding it to other communities or populations. This section asks about your organization’s involvement in efforts to scale programs.

10) To what extent did your organization do the following regarding scaling up of programs in 2009 and 2014?

For each thing that changed between 2009 and 2014, please indicate the factors that contributed to this change in the rightmost column. If there was no change between 2009 and 2014, mark “Not applicable (no change).” If you report that "other" major factors contributed to a change, please describe those factors below.

	To what extent did your organization do this in 2009?
	Always	To a very large extent	To a large extent	To a moderate extent	To a small extent	To a very small extent	Not at all
A. Undertook efforts to scale up existing program(s) – i.e., to expand the program(s) within the community or to other communities or populations
B. Selected programs for scale-up based on rigorous evaluation that shows them to be effective

	To what extent did your organization do this in 2014?
	Always	To a very large extent	To a large extent	To a moderate extent	To a small extent	To a very small extent	Not at all
A. Undertook efforts to scale up existing program(s) – i.e., to expand the program(s) within the community or to other communities or populations
B. Selected programs for scale-up based on rigorous evaluation that shows them to be effective

	If applicable: Major factors that contributed to the change (mark all that apply)
	Trends in the larger grantmaking world	Specific requirements attached to the funding your organization receives	Your organization’s board/leadership directed organization to implement change	Other (please specify below)	Not Applicable (no change)
A. Undertook efforts to scale up existing program(s) – i.e., to expand the program(s) within the community or to other communities or populations	[ ]	[ ]	[ ]	[ ]	[ ]
B. Selected programs for scale-up based on rigorous evaluation that shows them to be effective	[ ]	[ ]	[ ]	[ ]	[ ]

If you indicated that other major factors contributed to change in your organization’s approach to scaling up programs between 2009 and 2014, specify those factors below:

	Other major factor(s) that contributed to change between 2009 and 2014
A. Undertook efforts to scale up existing program(s
B. Selected programs for scale-up based on rigorous evaluation that shows them to be effective

Collaboration to Address Community Needs

11) To what extent did your organization participate in collaborations to support implementation of programs in communities, in 2009 and in 2014?

For each thing that changed between 2009 and 2014, please indicate the factors that contributed to this change in the rightmost column. If there was no change between 2009 and 2014, mark “Not applicable (no change).” If you report that "other" major factors contributed to a change, please describe those factors below.

	To what extent did your organization do this in 2009?
	Always	To a very large extent	To a large extent	To a moderate extent	To a small extent	To a very small extent	Not at all
A. Participated in funding alliance(s) with other nonprofit sector organizations. (For example, co-funding programs through joint funding; providing or receiving matching funds; or other collaboration)
B. Participated in collaborations with other nonprofit organizations to share knowledge
C. Collaborated with other organizations for purposes of advocacy – to advocate for or develop public support for programs or approaches to addressing social problems

	To what extent did your organization do this in 2014?
	Always	To a very large extent	To a large extent	To a moderate extent	To a small extent	To a very small extent	Not at all
A. Participated in funding alliance(s) with other nonprofit sector organizations. (For example, co-funding programs through joint funding; providing or receiving matching funds; or other collaboration)
B. Participated in collaborations with other nonprofit organizations to share knowledge
C. Collaborated with other organizations for purposes of advocacy – to advocate for or develop public support for programs or approaches to addressing social problems

	If applicable: Major factors that contributed to the change (mark all that apply)
	Trends in the larger grantmaking world	Specific requirements attached to the funding your organization receives	Your organization’s board/leadership directed organization to implement change	Other (please specify below)	Not Applicable (no change)
A. Participated in funding alliance(s) with other nonprofit sector organizations. (For example, co-funding programs through joint funding; providing or receiving matching funds; or other collaboration)	[ ]	[ ]	[ ]	[ ]	[ ]
B. Participated in collaborations with other nonprofit organizations to share knowledge	[ ]	[ ]	[ ]	[ ]	[ ]
C. Collaborated with other organizations for purposes of advocacy – to advocate for or develop public support for programs or approaches to addressing social problems	[ ]	[ ]	[ ]	[ ]	[ ]

If you indicated that other major factors contributed to change in your organization between 2009 and 2014, specify those factors below:

	Other major factor(s) that contributed to change between 2009 and 2014
A. Participated in funding alliance(s) with other nonprofit sector organizations
B. Participated in collaborations with other nonprofit organizations to share knowledge
C. Collaborated with other organizations for purposes of advocacy

Federal Funding

12) Did your organization receive any federal government funding in 2009?

( ) Yes

( ) No

13) Did your organization receive any federal government funding in 2014?

( ) Yes

( ) No

We may want to contact your organization again in the future to ask your participation in more in-depth studies of topics relating to grantmaking.

Are you willing to be contacted again for more in-depth follow-up studies? (Even if you agree now to be recontacted, you can agree or refuse to participate if you are contacted at a later time.)

( ) Yes – I am willing to be recontacted for possible follow-up studies

( ) No – I am not willing to be recontacted

If yes:

Name: _________________________________________________________________________________________________

Position/title: ____________________________________________________________________________________________

Organization: ____________________________________________________________________________________________

Email address: ___________________________________________________________________________________________

Telephone number: ________________________________________________________________________________________

If you are interested in receiving survey results, please indicate below and, if interested, provide contact information to send reports to.

( ) Yes – I am interested in receiving survey results

( ) No – I am not interested in receiving survey results

If yes:

Name: __________________________________________________________________________________________________

Position/title: ____________________________________________________________________________________________

Organization: ____________________________________________________________________________________________

Email address: ___________________________________________________________________________________________

Telephone number: _______________________________________________________________________________________

Thank You!
Thank you for taking our survey.

Your response is very important to the Corporation for National and Community Service.