Evaluation of the Rural Innovation Fund 1046-000/ DU205NC-13-T-0007 |
This section provides a facsimile of the RHED Telephone Survey Instrument as well as the relevant material that will precede or accompany the survey. We will conduct this survey with a purposive sample of approximately 50 RHED grantees that received awards between 2005 and 2009 and will not overlap with the RIF grantee sample.
For the RHED telephone surveys, it will be necessary to identify a “primary respondent” from each grantee who will respond to the survey. We will begin the process by identifying the grantee coordinator or administrator from the administrative files, grant applications, and more recent reports from the grantee. These files are available in hard copy at HUD Headquarters and we will develop a list of these individuals prior to beginning the telephone interviews. We will then ask the RHED Program Office to initiate contact with the grantee to schedule an appropriate time to hold the interview. The local HUD Field Office will also be notified at this time.
For the RHED telephone survey, we will initially use the selection criteria spelled out below and in Section 4.B. to identify a purposive sample of recent RHED grant recipients. As mentioned in Section 1.3.2, there are 141 unique RHED grantees who received awards between 2005 and 2009. After removing the 23 RHED grantees from that period who also received a RIF grant, 118 recent RHED grantees remain to select our RHED sample from using the criteria below. These grantees who received both an RHED and RIF grant will be removed from this RHED survey sample to avoid confusion over exactly which program we are discussing.
RHED Telephone Interviews: Selection Criteria
Received a RHED grant after 2005 (310 grants).
Received multiple RHED awards (141 unique, repeat RHED grantees).
Did not receive a RIF grant (118 unique, repeat, RHED-only grantees).
Sample will include similar numbers of the four disadvantaged communities.
Total number of grantees interviewed will be no less than 50 and no more than 75. Final numbers will depend on staff availability and organizational permanence.
After we select our RHED survey sample, we will ask the RHED Program Office and the HUD Field Office to establish contact with the grantee. From this communication and a review of the program files, we will determine which grantee staff member had or has primary responsibility for managing the RHED grants. We will then contact the staff member directly and schedule an appropriate time for the interview at the convenience of the grantee.
We will attempt to complete most of the logistical preparation while awaiting OMB clearance. After we receive OMB approval, we will notify the RHED Program Offices that it is time to contact the grantees and introduce the Econometrica team. Econometrica will then schedule the interviews directly with the grantees. In Section 2.2, we have provided an email template introduction to the RIF Evaluation Project, the Econometrica team, and the telephone survey.
Dear [insert name of RHED project contact here]:
The U.S. Department of Housing and Urban Development (HUD) is seeking your help. HUD has contracted with Econometrica, Inc.—a research firm located in Bethesda, MD—to conduct a study of the Rural Innovation Fund (RIF) program. The purpose of the study is to explore how the larger grant amounts available through the RIF have changed program activities, impacts, and outcomes relative to its predecessor, the Rural Housing and Economic Development (RHED) program.
As part of their study, Econometrica will conduct a series of telephone interviews with representatives from a sample of RHED grantees. This information is necessary to evaluate the impact of the RIF program relative to the RHED program and will help HUD develop policy approaches to rural housing and economic development issues in the future.
We have identified [insert name of RHED project contact here] as the Project Coordinator for [insert name of grantee organization]’s [insert RHED grant number]. If this information is incorrect or out of date, please respond with the person currently responsible for [insert RIF/RHED grant number]. If no available staff members have actively managed this grant, we would like to interview the current Executive Director of [insert name of grantee organization].1
This email serves as an introduction to key project staff including the HUD Representative, Ndeye Jackson, and the Econometrica Project Lead, James Hedrick. Econometrica staff will be working with you directly to schedule a time to conduct the telephone interview. They will be following up with you to determine your availability within the next 2 weeks. Please work with them to establish a time to conduct the interview.
The interview should take approximately 45 minutes to complete. Due to the open-ended nature of some of the questions, the interview may last up to 1 hour. Please allow for extra time when scheduling.
If you have any questions about the telephone survey or the overall evaluation, please do not hesitate to contact Ndeye Jackson, GTR/COTR, in HUD’s Office of Policy Development and Research (PD&R) at (202) 402-5737. She will be pleased to talk with you.
Thank you for your hard work and we look forward to your participation in this research effort.
Sincerely,
Jackie Williams
Director, Office of Rural Housing and Economic Development (RHED)
Department of Housing and Urban Development
This survey concerns the RHED grant for project number ________________, originally approved in _______ (fiscal year); known as the ______________________________ (project name); with the following purpose:
________________________________________________________________________________________________________________________________________________________________________________________________________________ __.
The purpose of this evaluation is to explore how the larger grant amounts available through the RIF have changed program activities, impacts, and outcomes relative to the RHED.
First and foremost, this is not an audit or a monitoring exercise. Econometrica is performing a programmatic evaluation of the RIF and RHED programs. We are interested in hearing from you about how you operated your grants, what partners you engaged, what resources you leveraged, how the grants affected your organization’s capacity, and the overall impact of the project on the community. No information you provide will be used to evaluate your organization specifically.
Additionally, your responses will be anonymous and will not be attached to you specifically or to your organization. We may quote some passages or responses to open-ended questions, but we will not identify the source of the comment.
Please answer honestly and fully to the best of your knowledge. If you are unsure of any question or believe that someone else would be better able to answer any question, please let us know and we will make a note of it in your response.
Thank you for your participation. If you have any questions about the survey or any concerns you would like to express to HUD, you may contact Ndeye Jackson, GTR/COTR, in HUD’s Office of Policy Development and Research (PD&R) at (202) 402-5737.
If you are ready, we can begin.
Organization of the Survey
General Information
Grant Use and Capacity Building
Grant Application and Management
Leveraging
Partnering-Linkages
Program Outcomes and Impact
Conclusion
Grantee Entity: (Organizational entity submitting grant application and winning grant award.)
Respondent: (Organizational official currently responsible for managing RHED grant activities or follow-up to grant activities. Additional staff may participate in interview if requested. Confirm positions and names at beginning of interview.)
Date of Interview: (mm/dd/yy)
Project Team Interviewer: ________________________________________________________
For any RHED grant, were the grant funds used in any manner that differed from your application? Please describe differences.
________________________________________________________________________
Did the number of your FTEs changed due to the funding you received via any RHED grant?
Yes No
If yes, to what extent do you think any RHED grant contributed to your organizational growth?
Growth attributed almost solely to RHED.
Growth attributed largely to RHED.
Growth somewhat attributed to RHED.
RHED one of many factors leading to growth.
If staff has decreased since the completion of your RHED grant, what factors led to the decrease?
________________________________________________________________________
An often-used measure of an organization’s basic financial strength is assets divided by liabilities. How would you characterize your financial strength today compared to when you received your first RHED grant?
Financial strength is better than when grant was awarded.
Financial strength is about the same as when grant was awarded.
Financial strength is worse than when grant was awarded.
How would you characterize the impact of your RHED grants on your organization’s financial strength? Did any RHED grant help your organization improve your financial strength in any way? If so, how?
________________________________________________________________________
_______________________________________________________________________
Did your RHED grant(s) help your organization moved into areas of activity where you had not previously been active?
Yes No
If yes, what were the new areas of activity?
________________________________________________________________________
If yes, how did the RHED program help your organization to move into this/these new area(s)?
________________________________________________________________________
________________________________________________________________________
In your opinion, how important were your RHED grant(s) in enabling your organization to address the issues – such as high unemployment, low education levels, etc. – that your organization listed in your grant application?
________________________________________________________________________
________________________________________________________________________
Did your organization consider any other grant sources to address the issues you included in your application?
Yes No Unsure
If yes, what other sources?
________________________________________________________________________
________________________________________________________________________
How would you characterize the current status of your RHED-funded project(s)?
________________________________________________________________________
________________________________________________________________________
How would you characterize the status of your RHED-funded project(s) relative to the original schedule from your application?
Completed ahead of schedule.
Completed on schedule.
Still in progress.
Completed behind schedule.
Experienced significant delays or project was not completed.
If any of your RHED project(s) were not completed as scheduled or failed to be completed, what barriers did you encounter when implementing the project?
________________________________________________________________________
________________________________________________________________________
How would you characterize the impact of the RHED grant(s) on your organization’s overall technical capacity? Has administering your RHED grants increased your organization’s ability to implement certain types of projects?
________________________________________________________________________
________________________________________________________________________
How would you characterize the impact of the RHED grant on your organization’s overall administrative capacity? Has it improved your organization’s ability to handle federal grants as well as other sources of funding?
________________________________________________________________________
________________________________________________________________________
How would you characterize the RHED’s administrative and reporting requirements compared to other Federal programs from which you receive funding?
Requirements are more burdensome than other Federal funding sources.
Requirements are similar to other Federal funding sources.
Requirements are less burdensome than other Federal funding sources.
Did you in any way see the RHED grant as complementary to funding you were receiving or could receive from other Federal agencies?
Yes No
If Congress were to authorize a new round of RHED grants, what would be the likely response of your organization?
Would almost certainly apply for a grant.
Would probably apply for a grant.
Would consider applying after looking at organizational priorities.
Likely would not apply for a RHED grant.
In your original grant application, you indicated (retrieve from application) as a source(s) for obtaining leveraged fund support for your RHED project(s). Is this the funding source you actually used at project start-up?
Yes No
If no, why were you were unable to access those planned-for funds?
________________________________________________________________________
________________________________________________________________________
At project start-up, were you able to put in place any additional leveraged funds other than those mentioned in your grant application?
Yes No
If yes, please specify the additional source(s) and amounts of funding.
________________________________________________________________________ ________________________________________________________________________
Also, if yes, did this additional funding allow you to expand or qualitatively enhance your project design?
Yes No
If yes, what changes did you make in your project design?
________________________________________________________________________________________________________________________________________________
Did winning any RHED grant enable you to access funding sources that you otherwise may not have been able to access?
Yes No
If yes, please specify the new sources, and explain how successful you have been in obtaining funds from one or more of these sources.
________________________________________________________________________
________________________________________________________________________
How would you characterize the impact of your RHED grant(s) on your ability to leverage funds for ongoing or future projects?
Has significantly increased the range of funding options open to us.
Has somewhat increased the range of funding options open to us.
Has had minimal impact on our ability to obtain additional funding.
How many formal partnerships (characterized by exchange of resources and/or signed agreement) were created primarily because of your RHED grant(s)?
________________________________________________________________________
How important was securing an RHED grant in establishing these partnerships?
Could not have had relationship without RHED.
RHED was very important in establishing relationship.
RHED was a factor in establishing relationship.
RHED was not important in establishing relationship.
Will your organization be able to build upon the linkages established on the RHED project for further collaboration?
Already involved in further collaboration.
Further collaboration has been planned but not implemented.
There has been no further collaboration since our RHED grant was completed.
Overall, how would you characterize the impact of your RHED grant(s) in enabling you to establish productive working relationships with other groups in your community? Has it helped you establish new partnerships with other local organizations?
________________________________________________________________________
________________________________________________________________________
What individuals or groups did you originally intend to target with your RHED project(s)? What outcomes did you expect for these recipients?
________________________________________________________________________
________________________________________________________________________
How would you characterize your progress toward achieving these outcomes for recipients?
________________________________________________________________________
________________________________________________________________________
Did your organization use any indicators to measure the progress of your RHED-funded projects toward completion?
Yes No
If yes, what data indicators have you used (are you using) to
measure the progress of your RHED-funded projects toward completion?
________________________________________________________________________
________________________________________________________________________
If yes, how effective have these indicators been in allowing you to manage the implementation of your project?
Very effective.
Somewhat effective.
Not very effective.
If yes and indicators were “not very effective,” have you looked at different indicators?
Yes No
If yes, what indicators are you using now or will be using in the future?
________________________________________________________________________
Currently, how would you characterize the capacity of your organization to collect and analyze data for planning and management purposes?
High level of technical capability.
Adequate level of technical capability.
Marginal level of technical capability.
Less than adequate technical capability.
If you have a “high” or “adequate” level of capability, how important were your RHED grant(s) to improving your capabilities in this area?
Extremely important.
Somewhat important.
Not important.
Did you have to develop the organizational capacity to obtain and analyze both project implementation data and impact data?
Yes No
If yes, were you able to increase your staff capability to collect and analyze data during the implementation of your RHED grant(s)?
Yes No
How would you characterize the overall impact of your RHED project(s) so far? How has it impacted the community as a whole?
________________________________________________________________________
________________________________________________________________________
When you received your first RHED grant, did your organization have a strategic plan or other statement of organizational goals?
Yes No
If yes, how did you integrate your RHED project(s) into this plan? How did your RHED project(s) support your organization’s overall goals?
________________________________________________________________________
________________________________________________________________________
Overall, what has your impression been of the RHED program? Do you have anything to add about your experience that we have not covered here?
_______________________________________________________________________________________________________________________________________________________________________________________________________________________
1 Since all RIF projects are still active, we believe it is unlikely that we will need to include this sentence in the introductory email to RIF grantees. However, many RHED grantees may have closed their projects years ago and high staff turnover in rural organizations means it is likely that we will have to include this or a similar sentence in the introductory email to RHED grantees.
Page
Econometrica, Inc. May 27, 2014
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | James Hedrick |
| File Modified | 0000-00-00 |
| File Created | 2021-01-27 |