ATTACHMENT A1a: SPF SIG I & II
Sustainability Interview
Form Approved OMB
No. XXXX-XXXX Expiration
Date XX/XX/XXXX
Strategic Prevention
Framework
State Incentive Grant
(SPF SIG)
Cohorts I & II
Cross-Site Evaluation
Sustainability Interview
Guide
June 2010
Substance Abuse and Mental
Health Services Administration
Center for Substance Abuse
Prevention
Burden Statement: Public reporting burden of this collection of
information is estimated to average 1.5 hours per response.
Send all comments regarding this burden estimate or any other aspect
of this collection of information to SAMHSA Reports Clearance
Officer, Room 7-1045, Choke Cherry Road, Rockville, MD 20857.
An agency may not sponsor, and a person is not required to respond
to a collection of information unless it displays a currently valid
OMB control number. The control number for this project is
XXXX-XXXX.
DATE: |___|___| / |___|___| / |___|___|___|___|
INTERVIEWER NAME:_____________________________________________________
RESPONDENT NAME:______________________________________________________
RESPONDENT TITLE/POSITION:____________________________________________
RESPONDENT ORGANIZATIONAL AFFLIATION:_____________________________
STATE: |___|___|
INTERVIEW START TIME: |___|___| : |___|___|
OPENING SCRIPT
Hello, this is (Name) from the SPF SIG cross site evaluation team. Is this still a good time to conduct our interview? Joining me for this call is one other member of the cross-site evaluation team, (name of note-taker). We appreciate your taking the time to talk with us today. As we indicated in our email, we want to talk about (State’s) prevention system, including how it was influenced by the SPF SIG. We are interested in speaking with you because of your expertise about (State)’s prevention system.
We first conducted an interview on this topic with representatives from your state in (2006/2007), followed by another interview in (2008/2009). With this interview, we want to learn about the current status of your prevention system and about changes in the system that have occurred since the last interview. To do so, we will ask a series of questions that is similar to the questions from the previous interviews. In some cases, we may probe a bit further to make sure we fully understand the current system, whether changes have take place over time, and whether the SPF SIG contributed to those changes. Do you have any questions so far?
Before we proceed with the interview, I’d like to cover a few important issues. First, the interview will last about 90 minutes. Does this still fit your schedule? (If yes, continue. If no, ask how much time s/he has and say “We’ll cover what we can in that time.”)
Second, your participation in this interview is completely voluntary. If at any time you wish to end the interview, you may do so. Similarly, if you are uncomfortable with any question, you may pass and we can move on to the next question. If you choose not to answer questions or you wish to stop the interview, there will be no effect on your position.
Third, I want to assure you that we will make every effort to keep your answers confidential. We will not connect your name or position with any of your responses. We will only report your responses in combination with the responses of others—individual responses will not be identified.
If you have no objections, we’d like to record this conversation. This will help us check our notes and ensure that we’re accurately capturing your responses. We will not transcribe the recordings and will erase them once all the interviews have been analyzed. Is it OK if we turn on the recorder? (If yes, proceed with interview. If no, say “That’s OK, we’ll proceed without the recorder.”) If at any time you have trouble understanding us due to technical problems or indistinct speech, please let us know right away.
As we begin the interview, let me remind you that we are talking about the overall (State) prevention system, not just the SPF SIG project. We’re defining the prevention system as “the entire set of agencies, organizations, and persons that contribute to efforts to prevent substance abuse and related problems within the state.” Do you have any questions about this?
OK, let’s begin.
Names/Titles of Interviewees:
State:
Interviewer:
Note-taker:
Date of Interview:
Interview start time:
State organization
[8-10.] Is there an agency or body that has the authority to make programmatic, policy, or resource allocation decisions for the entire statewide ATOD prevention system?
Yes
No
Don’t know
Other, specify
Not applicable, because
Not asked, because
Probe: What funding streams (e.g., SDFS, SAPT, Synar) are governed by this agency or body?
Is the group that served as the SPF SIG advisory council still active?
Yes (whether it is the same group or it merged with another one)
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
Probe: If no, why is the group no longer active?
[11.] [Other than the SPF SIG advisory council]…Does a group of state-level decision-makers convene to integrate and/or coordinate ATOD prevention efforts across the entire state system?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
[Probe, if not made clear from previous responses]….How do these two groups relate to one another?
[19.] Is there any group (or forum) in which state-level and sub-state level stakeholders work together to improve or better integrate state and sub-state-level ATOD prevention efforts?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
Probe: If yes, how does the group of stakeholders work to improve or better integrate state and sub-state ATOD prevention efforts?
[2.] Is there a line item (or items) in the state’s general revenue funds budget for ATOD primary prevention?
Yes
No
Other, specify
Don’t know
Not applicable, because
Not asked, because
Did the SPF SIG directly contribute to any changes to the state prevention organizational structure—including anything we’ve already discussed—(e.g., new groups formed)?
NEEDS ASSESSMENT and CAPACITY ASSESSMENT
Management/Organization of Data
Is your SEOW still active?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
[Probe, if not made clear]….What are its main goals now and/or examples of its recent or planned activities?
[51.] Is there any other state-level group that is responsible for organizing and maintaining data (for purposes of state-level needs assessment and planning related to substance abuse prevention)?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
[Probe, if not made clear from previous responses]….
How do these two groups relate to one another?
How do they interact with the state-level advisory council or decision-making group?
Epidemiological Data
Since R2, have there been any changes to the types of ATOD-related epi data your state collects (e.g., new data sources/data collection, no longer collecting certain data)? [Interviewer will have data and levels of collection in front of him/her.]
Will you routinely go through an epidemiological, data-driven process to identify state needs/priority issues?
Probe: Have you gone through this process since the SPF SIG ended?
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
10. Data-driven process to identify priority issue. |
There is no process in place to identify state needs/priority issues. |
There is currently a process in place to identify state needs/priority issues; however, the process does not take place regularly. |
There is currently a regularly occurring process in place (at least every two or three years) to identify state needs/priority issues. |
Will the state routinely produce a state-level epidemiological profile?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
Epidemiological Data Sharing and Guidance
[60.] Does the State routinely share epidemiological data with sub-state level prevention stakeholders (e.g., occurs on a regularly-scheduled basis, such as monthly, semi-annually, annually)?
Epi data are distributed
Epi data are not distributed
Other, specify
Don’t know
Not applicable, because
Not asked, because
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
12. [60.] State distributes epi data to sub-state agencies. |
There is little evidence of the State distributing epi data to sub-state agencies. |
There has been at least one instance in which the State distributed epi data to sub-state prevention stakeholders, but data sharing is not routine. |
The State routinely distributes Epi data to sub-state agencies in periodic reports, web sites or data transfers. |
[61.] What guidance does the State provide to sub-state stakeholders about how to use and interpret epi data?
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
13. [61.] Guidance about interpreting epi data. |
The state does not play a role in providing guidance about how to interpret epi data. |
There is evidence that the state has engaged in some activities that provide guidance to sub-state stakeholders about interpreting epi data, but guidance is not routine. |
The state provides substantial and continuing guidance to sub-state stakeholders about interpreting epi data. |
Prevention Capacity Data
[65.] Please explain how, if at all, the adequacy and needs of the substance abuse prevention workforce at the community level are assessed across the state (e.g., this might include size or composition of the workforce, and the need for training, technical assistance, or continual education).
Workforce adequacy and needs are assessed
Workforce adequacy and needs are not assessed (If not assessed, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
Probe: Describe the workforce capacity data, how and how often they are collected, and how they are used to establish workforce development priorities.
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
14. [65, 65a, 65b.] Assessment of the adequacy and needs of the prevention workforce. |
There is no process in place to assess the adequacy and needs of the statewide substance abuse prevention workforce. |
There is currently a process in place to assess the adequacy and needs of the workforce; however, the assessment does not take place regularly, and/or uses methods or data that are not ideal. |
There is currently a regularly occurring process in place (at least every two or three years) to assess the adequacy and needs of the statewide prevention workforce; the assessment is based on timely, accurate data and sound methods. |
[50.] Are data on other elements of prevention systems capacity at the community level collected (e.g., data collection capacity, cultural competency, coalition capacity/readiness, planning capacity)?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
Probe: Describe the capacity data, how and how often they are collected, and how they are used to establish capacity building priorities.
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
15. [50g.] Comprehensiveness of data collected about prevention systems capacity (i.e., data collection capacity, cultural competency, coalition capacity/readiness, planning capacity). |
The state collects no (or very little) data about systems capacity. |
The state collects some data about systems capacity, but the data are not very comprehensive. |
The state collects a comprehensive set of data about the prevention systems capacity. |
How is the capacity of the state-level prevention system assessed (e.g., workforce needs, data collection capacity, cultural competency, planning capacity of STATE staff)?
Probe: Describe the capacity data, how and how often they are collected, and how they are used to establish capacity building priorities.
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
16. Comprehensiveness of data collected about state-level prevention systems capacity (i.e., data collection capacity, cultural competency, coalition capacity/readiness, planning capacity). |
The state collects no (or very little) data about state-level systems capacity. |
The state collects some data about state-level systems capacity, but the data are not very comprehensive. |
The state collects a comprehensive set of data about the state-level prevention systems capacity. |
Did the SPF SIG contribute directly to any capacity gains in the area of data management, needs assessment, or capacity assessment—including anything we’ve already discussed—that you believe will be sustained as part of your prevention system? Please provide some concrete examples.
STRATEGIC PLANNING
SPF SIG Strategic Plan
What is the status of the strategic plan you developed as part of the SPF SIG (e.g., still in use as a separate plan, merged with another strategic plan)?
Are you still addressing the priority or priorities identified as a result of the SPF SIG needs assessment?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
How would you characterize the impact of the SPF SIG state strategic plan on….
a. your ability to address your SPF SIG priority outcomes?
b. your ability to enhance prevention capacity across the state?
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
20a. Impact of strategic plan on priority issues. |
The SPF SIG strategic plan has had no or little influence in addressing the SPF SIG priority issues. |
The SPF SIG strategic plan has had a notable influence on the SPF SIG priorities, but the impact of the plan was constrained in some way (e.g., due to aspects of the plan not being implemented as intended). |
The SPF SIG strategic plan was successfully implemented and resulted in a substantial impact on the SPF SIG priorities. |
20b. Impact of strategic plan on prevention capacity. |
The SPF SIG strategic plan has had no or little influence in enhancing prevention capacity. |
The SPF SIG strategic plan has had a notable influence in enhancing prevention capacity, but the impact of the plan was constrained in some way (e.g., due to aspects of the plan not being implemented as intended). |
The SPF SIG strategic plan was successfully implemented and resulted in a substantial impact on prevention capacity. |
Broader Strategic Plan
Our past interviews indicated that you DID/DID NOT have a state strategic plan for substance abuse prevention, beyond the strategic plan for the SPF SIG project.
[IF DID NOT HAVE]….Was a plan developed since our last interview?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
[Probe for details, if not made clear, including whether the plan is a “stand-alone” plan for prevention or part of a larger state strategic plan.]
[IF NO…] Do you have plans to develop a state strategic plan or use the SPF SIG plan more broadly?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
[Probe for details, if not made clear.]
[IF DID HAVE]…Is that plan still active?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
[30.] How would you characterize the impact of the state strategic plan on…
a. prevention policies (here we mean public policy, including laws and regulations)?
b. funding prevention programming (including standards of care for providers)?
c. prevention workforce development efforts?
[Probe for the extent to which the plan itself led to the changes, rather than simply reflecting changes that were taking place anyway.]
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
22a. [30a.] Impact of strategic plan on prevention policies. |
There is no evidence that the plan has driven the formulation of prevention policies. |
The plan has played a role in formulating some but not all substance abuse prevention policies. |
The plan clearly drives all or nearly all prevention policies. |
22b. [30b.] Impact of strategic plan on funding prevention programs. |
There is no evidence that funding for prevention programming is influenced by the plan. |
The plan has played a role in some but not all funding for prevention programming. |
The plan clearly drives all or nearly all funding for prevention programming. |
22c. [30c.] Impact of strategic plan on prevention workforce development efforts. |
There is no evidence that the plan has driven workforce development efforts. |
The plan has played a role in formulating some but not all workforce development efforts. |
The plan clearly drives all or nearly all workforce development efforts. |
[67.] Is there a written plan in place for statewide ATOD prevention workforce development (either as a separate document or as part of a strategic plan)?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
[Probe, if not made clear]…How or to what extent is the plan used to inform decisions and actions related to workforce development?
[95.] Is there a written plan for addressing cultural competence in the statewide ATOD prevention system (either a separate document or in a strategic plan)?
Yes
No
Other, specify
Don’t know
Not applicable, because
Not asked, because
[Probe, if not made clear]…How or to what extent is the plan used to inform decisions and actions related to cultural competence?
Mechanisms for State and Sub-state Planning
[47] Are there mechanisms for linking state and sub-state planning efforts (e.g., planning templates/logic models issued to sub-state entities, requirements that sub-state plans mirror the state level plan or contain core elements/principles, technical assistance provided by the state for planning at the sub-state level, etc.)?
Yes
No (If no, skip to)
Other, specify
[48.] Please describe the mechanisms for linking state and sub-state planning efforts.
Probe: How “routine” or regular are these mechanisms? Does the state require communities to have logic models? Strategic plans?
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
26. [47, 48.] Mechanisms for linking state and sub-state planning efforts. |
There do not appear to be any mechanisms for linking state and sub-state planning efforts. |
Mechanism(s) exist for linking state and sub-state planning efforts, but they are not used routinely and/or consistently. |
Mechanism(s) for linking state and sub-state planning efforts, and they are used routinely and/or consistently. |
Did the SPF SIG directly contribute to any capacity gains in the area of strategic planning—including anything we’ve already discussed—that you believe will be sustained as part of your prevention system? Please provide concrete examples.
EVIDENCE-BASED PROGRAMS, POLICIES, AND PRACTICES
[88.] Are SAPT block grant recipients required to use a specified percentage
of funds on evidence-based programs, policies, and practices?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
[89.] What percentage of the block grant funds is required to be spent on evidence-based programs, policies, and practices?
Besides SAPT block grant recipients, are other sub-state grantees/contractors required to use a specified percentage of funds on evidence-based programs, policies, and practices?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
What percentage of these funds is required to be spent on evidence-based programs, policies, and practices?
[92.] Please describe the assistance available to program providers in selecting and implementing evidence-based programs, policies, and practices (e.g., training, technical assistance, and materials).
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
32. [92.] |
The state is not prepared to offer any technical assistance to local program providers to select and implement EBPs (including referrals to the program developers). |
The State responds only ad hoc to requests for assistance selecting and implementing EBPs, but does not have a plan for actively supporting local program providers. |
The State has resources in place – contractors, trainers or regional staff – offering technical assistance in selecting and implementing EBPs. |
The next questions are about supporting strategies that are appropriate for the cultural context in which they are being implemented.
[101.]
a. Are there requirements to ensure culturally and linguistically competent programs, policies, and practices, including the use of culturally appropriate prevention materials, at the community level?
Yes
No
Other, specify
Don’t know
Not applicable, because
Not asked, because
Probe, What impact have these requirements had in ensuring cultural competence?
b. [c.] Are there resources provided to sub-state entities to help them select, adapt, and/or implement culturally and linguistically competent programs, policies and practices (e.g., technical assistance and materials)?
Yes
No
Other, specify
Don’t know
Not applicable, because
Not asked, because
[Probe for details, if not made clear.]
Did the SPF SIG directly contribute to any capacity gains in the area of EBPPPs—including anything we’ve already discussed—that you believe will be sustained as part of your prevention system? Please provide concrete examples and also consider any gains in relation to supporting and implementing culturally competent EBPPPs.
EVALUATION
[103.] Is there someone– on staff or through contract – who provides evaluation services or evaluation-related technical assistance to state or community level staff?
Yes
No (If no, skip to question)
Other, specify
Don’t know (If marked, skip to question)
Not applicable, because (If marked, skip to question)
Not asked, because (If marked, skip to question)
[104.] Can you please describe the evaluator’s role and involvement in ATOD prevention evaluation efforts (e.g., Does the evaluator provide TA in accessing or analyzing data or developing evaluation plans and logic models)?
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
35, 36 [103., 104.] Availability of an evaluator and their role and involvement in ATOD prevention evaluation efforts. |
The state does not have an evaluator on staff or under contract to provide prevention evaluation services and technical assistance. |
The state has an evaluator on staff or under contract who can provide prevention evaluation services and technical assistance, but little regular use of evaluation expertise can be documented. |
The state has working or contractual relationships with evaluation experts who play a significant role in prevention evaluation efforts. |
[109.] According to state guidelines for sub-state ATOD prevention funding, what kinds of evaluation are required for the block grant and other substance abuse prevention programs (e.g. outcome evaluation, process evaluation, or fidelity monitoring)?
[Probe for whether requirements differ between BG and other programs.]
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
37. [109.] State guidelines for the kinds of evaluation required of sub-state ATOD prevention fund recipients. |
There are no guidelines that specify evaluation requirements for sub-state recipients. |
At least some evaluation guidelines have been developed, but they include only one or two of the following: a) process evaluation, b) outcome evaluation and c) fidelity monitoring. |
There are evaluation requirements for sub-state recipients, including a) process evaluation, b) outcome evaluation and c) fidelity monitoring. |
[110.] Does the State routinely share evaluation data with other state agencies or sub-state level prevention stakeholders (e.g., occurs on a regularly-scheduled basis, such as monthly, semi-annually, annually)?
Yes
No
Other, specify
Don’t know
Not applicable, because
Not asked, because
|
No/low capacity process (1) |
Moderate capacity process (2) |
High capacity process (3) |
38. [110a.] State sharing of evaluation data with sub-state level prevention stakeholders. |
There is no evidence of evaluation data being shared with state or sub-state level prevention stakeholders. |
There have been some instances of evaluation data being shared with state or sub-state level prevention stakeholders, but this is not done routinely. |
Evaluation data are routinely shared with state or sub-state level prevention stakeholders. |
Does the state issue formal evaluation reports (i.e., containing data and interpretive narrative) on ATOD prevention? That is, reports that provide information about populations served, the programs, policies, and practices implemented, and the outcomes associated with those activities.
Yes
No
Other, specify
Don’t know
Not applicable, because
Not asked, because
Did the SPF SIG directly contribute to any capacity gains in the area of evaluation—including anything we’ve already discussed—that you believe will be sustained as part of your prevention system? Please provide concrete examples.
CAPACITY GAINS IN CULTURAL COMPETENCE
Did the SPF SIG directly contribute to any capacity gains in the area of cultural competence that you believe will be sustained as part of your prevention system? Please provide concrete examples.
EXTERNAL EVENTS
[127.] Finally, we’d like to ask about events that have occurred in your state that may have an impact on the state’s prevention activities.
In the past 12 months, what external events or incidents have taken place that may have an impact on ATOD prevention activities in your state? (These could be things like new legislation, administrative changes, and significant changes in prevention funding sources or levels or even natural disasters.) For each event, please describe how it had an impact or may have an impact on your state's ATOD prevention activities.
WRAP UP QUESTION
Considering all that we’ve discussed, and perhaps things we have not discussed, what do you think are the key elements of the SPF SIG that are being sustained in your state? How and why did you decide to sustain those elements?
CLOSING SCRIPT
May we call you back if we need further clarification on any of the things we have discussed?
Yes
No
Would you like to receive the notes for this interview for your review?
Yes
No
If so, please return that review within one week if you have changes you would like to see. We will be glad to discuss the results of the interview with you.
Thank you for your time.
ATTACHMENT
A1a: Page
|
| File Type | application/msword |
| File Title | SPF SIG National Cross-site Evaluation |
| Last Modified By | Ann Landy |
| File Modified | 2010-07-06 |
| File Created | 2010-07-06 |