Project Director S Project Director Survey

SAMHSA’S Multiplier Surveys

Supporting Statement

Attachment A:

Project Director Introductory Email

Project Director Survey

SAMHSA’s Multiplier Surveys

Project Director Survey Introductory Email

Dear SAMHSA Project Director:

Policy Research Associates, Inc. is conducting a study on behalf of the Substance Abuse and Mental Health Services Administration (SAMHSA) to learn what broader impacts programs may have and what happens to programs after Federal funding ends. SAMHSA wants to identify factors and characteristics associated with sustainability and grant impacts on local or state service delivery, systems change or infrastructure development. The findings of this study will inform the structure and implementation of future SAMHSA funding.

To do this, we are studying a number of SAMHSA-funded grants that have ended in the last several years. Please note that we are asking about programs where Federal funding ended by September 30, 2008, including any no-cost extensions. You have been identified by SAMHSA as a project director or other contact for the following program, which is included in our study:

[NAME OF GRANT] program,

Announcement Number

Award number

Dates of award

If you believe you have been selected in error or if you feel that you are not familiar enough with this program, please contact me so we may identify a more appropriate respondent.

We would like to set up a convenient time to conduct an in-depth interview about this project. We anticipate the interview will last about an hour. Please let me know, by responding to this email, that you are willing to participate in this important study.

If you have questions about this study, please contact me.

Sincerely yours,

Susan Becker

OMB Number: 0930-XXXX

Expiration Date: MM/DD/YYYY

SAMHSA Multiplier Surveys

Project Director Interview

Public Burden Statement

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  The OMB control number for this project is 0930-xxxx.  Public reporting burden for this collection of information is estimated to average 75 minutes per respondent per year, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, Maryland, 20857.

SAMHSA Multiplier Surveys

Project Director Interview

Introduction

Thank you for agreeing to complete this survey. Your responses will assist SAMHSA in understanding the broader impacts of their funded programs and what happens to programs after Federal funding ends. In particular, we are interested in post-grant changes in local or state service delivery, systems change or infrastructure development which might have been influenced by this program.

We will be talking about [Fill in specific grant name and RFA#] that was funded by the Center for [fill in SAMHSA center] from [date] to [date] including any no-cost extensions. Responses will be reported in the aggregate only. Responding to this survey will not affect you, your agency or applications for any current or future SAMHSA grants. We expect that the interview will last about an hour. We appreciate your time.

Project Intent and Design

1. What was the impetus for applying for this grant? What were you hoping to achieve? [RECORD VERBATIM. PROBE.]

2. What was the target population for your grant?

3. Did it change over the life of the grant? IF YES, please explain. [RECORD VERBATIM. PROBE.]

4. Had your organization received a SAMHSA grant before this grant?

• Yes

• No (go to Q. 6)

• DON’T KNOW (go to Q. 6)

5. If yes, from which Center(s)? [CHECK ALL THAT APPLY.]

   • CMHS

   • CSAP

   • CSAT

   • DON’T KNOW

Project Implementation

6. Describe, in brief, the project that was funded. [RECORD VERBATIM. PROBE.]

7. What, if any, Evidence-Based Practices (EBPs) were implemented as part of this grant?

8. Who were the stakeholders in the project? Describe any State or community participation or buy-in. [RECORD VERBATIM. PROBE.]

9. Was their leadership key to project implementation? Who in particular? Why?

10. [IF PROJECT WAS A COOPERATIVE AGREEMENT, ASK:]

(a)Describe the role of the GPO in your project. [RECORD VERBATIM. PROBE.]

(b)How often did you have contact with him/her? [RECORD VERBATIM. PROBE.]

(c) Did s/he play a major role? [RECORD VERBATIM. PROBE.]

11. What were the three (3) greatest challenges encountered in implementing the grant? [RECORD VERBATIM. PROBE.]

a.

b.

c.

12. How were these challenges addressed? [RECORD VERBATIM. PROBE.]

a.

b.

c.

13. To what extent was each of these challenges overcome? Would you say . . . . [ASK ABOUT EACH SEPARATELY]

a. [Insert Challenge A]  Almost completely  Somewhat  Very little or not at all  DK

b. [Insert Challenge B]  Almost completely  Somewhat  Very little or not at all  DK

c. [Insert Challenge C]  Almost completely  Somewhat  Very little or not at all  DK

14. [Ask only if a match was required. If not, go to Q. 16.] How did you meet the grant’s match requirement?

15. Was there any difficulty meeting the match?

• Yes – IF YES, can you tell me what was difficult about meeting the match?

• No

• DON’T KNOW

16. Some SAMHSA grants do not fund direct services to individuals. Instead, their focus is on infrastructure development, meaning service delivery change or systems change. Did this grant fund…? [CHECK ONE]

• Service delivery to individuals only [SKIP TO Q. 19]

• Services or systems change only

• Both

17. What were the main goals of systems change component?

18. To what extent was this component implemented? Would you say… [CHECK ONE]

• Almost completely

• Somewhat

• Very little or not at all

• DON’T KNOW

19. If direct services were included in this grant, were they provided by your organization (GRANTEE) or by a subcontractor (SUBGRANTEE) or both?

• Grantee

• Subcontractor or subgrantee

• DON’T KNOW

20. Did the grant require or provide for a planning period before services could begin?

• Yes – For how long?____________________________

• No [GO TO Q. 22]

• DON’T KNOW [GO TO Q. 22]

21. [IF YES to Q. 19, ASK] To what extent did the planning component contribute to successful program implementation?

[IF NO TO Q. 19, ASK] To what extent do you think a planning component would have contributed to successful program implementation? [READ CHOICES.]

• A lot

• Somewhat

• A little or not at all

• DON’T KNOW

22. What types of technical assistance were available to you from SAMHSA? [READ CHOICES. CHECK ALL THAT APPLY.] Which of these did you actually use or receive? [READ CHOICES. CHECK ALL THAT APPLY]

Available Received/Used

  Program-specific SAMHSA technical assistance center

  Evaluation technical assistance

  Formal evidence-based practice (EBP) training or materials

  On-site visit(s) with GPO or expert consultants

  Phone consultation

  Contact (phone or in-person) with other grantees

  Program-specific SAMHSA website

  Program-specific SAMHSA-sponsored list serv

  Any other?_____________________________________

  NONE

Project Impact

23. How important was the technical assistance you received to the impact of your project? [READ CHOICES.]

• Very important

• Somewhat important

• Not at all important

• DON’T KNOW

24. Was an evaluation of the project conducted during the grant period? [IF YES] Was it a…

• Formative or process evaluation?

• Outcome evaluation?

• Both

• NO EVALUATION [GO TO Q. 26]

• DON’T KNOW [GO TO Q. 26]

25. To what extent did the evaluation contribute to the impact of the project? [READ CHOICES. CHECK ONE.]

• A lot

• Somewhat

• Very little or not at all

• NO EVALUATION

• DON’T KNOW

26. To what extent did GPRA measures contribute to the impact of the project? [READ CHOICES.]

• A lot [ASK HOW.]

• Somewhat [ASK HOW.]

• Very little or not at all

• DON’T KNOW

27. What systemic or other changes resulted from this project? [READ CHOICES. CHECK ALL THAT APPLY.]

• State policy changes

• Local policy changes

• Other programs started statewide

• Expanded geographic service area

• Changes in other agency programs that serve same population

• Increased use of evidence-based practices

• Improved collaboration among community agencies

• Planning groups established

• Other. Please describe.

28. What changes occurred as a result of this project at the: [READ CHOICES.]

• Consumer level (for example service delivery, outcomes, number of clients)?

• Agency level (for example, use of staff, program improvement, streamlining)?

• Community level (for example, awareness of the issue, local government funding, collaboration)

• State level (e.g. funding, policy or programmatic changes)?

Project Sustainability

29. Approximately when did Federal funding end for this grant?

30. Was the project continued after Federal funding ended? [IF YES,] please tell me how?

• Yes, pretty much as originally funded

• Yes, in part

• No (go to Q. 35)

• DON’T KNOW (go to end of survey)

31. What, if anything, about the project changed after Federal funding ended?

32. What funding source(s) made it possible for the project to continue in whole or in part? [READ CHOICES. CHECK ALL THAT APPLY.]

• State agency

• Local agency

• Medicaid

• Blended

• Private

• Other (Describe________________________________)

33. [IF YES TO ANY IN Q. 32, ASK] Can you tell me how this was accomplished?

34. How important were the following factors to the project’s continuation? [READ EACH FACTOR. INDICATE WHETHER EACH IS VERY, SOMEWHAT OR NOT AT ALL IMPORTANT. THEN RANK THE TOP 3 FOR IMPORTANCE TO PROJECT CONTINUATION.]

Importance (Check one)
Very	Some-what	Not at All	Rank
				Planning process
				Availability of on-going EBP training or materials
				Administrative or fiscal technical assistance
				Evaluation technical assistance
				Evaluation findings or outcomes
				Fit into mission of organization
				Served intended population more effectively
				Having widespread community support
				Having a champion
				Other. Please describe.

[RECORD COMMENTS:]

35. If the project did not continue, how important were the following factors to the project’s discontinuation? Please indicate which was the most important. [READ EACH FACTOR. INDICATE WHETHER EACH IS VERY, SOMEWHAT OR NOT AT ALL IMPORTANT. THEN RANK THE TOP 3 TO THE PROJECT’S DISCONTINUATION.]

Importance (Check one)
Very	Some-what	Not at All	Rank
				Lack of leadership
				Problem was not significant in the community
				Staff turnover
				People busy with other tasks
				Not a priority
				Additional funding wasn’t found
				Another agency is providing services to the intended population
				The program is being continued in a reduced/more targeted form
				Other. Please describe.

[RECORD COMMENTS:]

Conclusion

Thank you for participating in this survey. Your responses have been very helpful. If you have any questions about this project, please contact Sue Becker at [email protected].