Drug Free Communities Support Program National Evaluation

Drug Free Communities (DFC) Coalition Sector Representative Interview Protocol

Interview Introduction

My name is [introduce self and note-taker]. Thank you for agreeing to participate in today’s interview regarding the Drug Free Communities program at [insert name]. The purpose of this interview is to gather information regarding your coalition efforts to reduce youth substance use through community-based collaborative efforts. The case studies will be used to discuss, in-depth, what strategies and activities sites are implementing to increase collaboration and decrease youth substance use through the DFC initiative.

We greatly appreciate you taking time out of your busy schedule to assist with the national DFC evaluation that ICF International is conducting. You were selected to participate in an interview because your individual perspective represents important issues relevant to this evaluation. We are conducting case studies with nine DFC programs this year to gather information about the planning and implementation of DFC, relationships across all stakeholders, and outputs and impacts of your activities to reduce youth substance use. Please feel free to be open and candid in your responses to our questions, as we will keep this information strictly confidential.

Again, this component of our evaluation is completely voluntary; specifically this means that (1) the interview is voluntary; (2) you can decline to answer any questions, or stop the interview at any time; (3) the information will be held in confidence; and (4) only evaluation staff will have access to the interview tape and interview data (if they agree to be tape recorded).

With your permission, we would also like to record the audio of this interview so that we can transcribe the conversation for accuracy in the analysis and interpretation of your comments along with comments of other staff. Nobody outside of the National Evaluation team will have access to this audio recording. Upon transcription of these recordings as appropriate to the evaluation, we will destroy the recordings themselves, maintaining only written records.

Are there any questions before we begin?

(Interviewer: Please remember to ask permission to tape record the interview. They can decline. Ask if they have any questions for you before you begin.)

Background – You and Your DFC Coalition

1. Tell me about the sector/stakeholder group you represent and about your involvement. How long have you been involved?

2. How does your coalition work?

   1. Please describe for us the structure of your DFC partnership? For example, are there any subcommittees, workgroups, teams, etc. charged with specific tasks? What are your primary roles/responsibilitiesin the coalition?

   2. To what extent are the Strategic Prevention Framework elements (assessment, planning, capacity, implementation, and evaluation – cultural competence and sustainability) central to how the coalition works? What do you see as the main vision and goals for your DFC? How have your DFC’s goals, vision, and mission changed over time? Why change?

   3. On average, how much time (in hours) do you spend per month related to DFC (meetings, work outside meetings, attending activities, etc.)?

   4. What other activities related to the mission of DFC, if any, are you involved in that take place outside of coalition meetings?

(Interviewer: Obtain information pertaining to their role in all activities, including assessment, capacity, planning, implementation, evaluation, and administration.)

DFC Assessment, Capacity & Planning

3. What do you see as the main vision and goals for your DFC work? How were the goals, vision, and mission developed? When?

1. Who was involved and at what level?

2. Were youth involved? How?

3. Were local neighborhood leaders (formal and informal) involved? How?

4. How did your community select the target population? How did you identify the needs of this population?

1. What assessment activities were conducted? What role, if any, did you have in the needs assessment process? Did you receive any assistance in these efforts (e.g., local evaluator)?

2. What difficulties, if any, did your coalition experience in conducting assessment activities and selecting a target population (e.g., availability of data, buy-in from participating organizations, turnover)?

3. What specific protective and risk factors, if any, did your community identify to target in order to reduce youth substance use?

4. Given the findings from your assessment activities, and coalition discussions during the planning phase, what do you feel are the strengths of your strategic plan?

5. What do you see, if any, as potential challenges in implementing your strategic plan? (For instance, do you feel that your strategic plan is realistic in terms of goals and the timeline associated with these goals – please explain?)

5. How well do different agencies work together within DFC to meet the needs of your target population?

1. Would you say the right organizations and right people are active and engaged in your DFC partnership? Is there any individual or agency representative missing that might help coalition efforts to reduce youth substance use? Please explain.

2. Are all the key neighborhood leaders involved in the effort? What are the challenges or facilitators of brining in neighborhood leaders? What successes/challenges has your DFC faced in recruiting those who are missing?

3. Are the critical individuals and agencies also represented on workgroups/subcommittees? Do they have a voice in coalition decisions?

4. If you are in a rural community, how have you worked across jurisdictional boundaries to build capacity?

5. If you are in an urban community, how do you avoid competing with other prevention initiatives?

6. Describe how ‘ready’ you believe your coalition was at the start of the DFC program? How about now? How did you define readiness?

1. Do you feel that you or your organization was ready to assist coalition activities and efforts from the beginning? Now?

2. Was anything done to increase coalition readiness and capacity to implement your community-based drug use prevention program?

3. If you received any help from internal or external agencies and organizations, can you describe this assistance? Specifically, how did they help your coalition? Was the outcome of this assistance successful, and, if so, what do you think contributed to this success?

4. What were the challenges encountered in moving your coalition forward to act more effectively in the community? To what extent/how have you been able to overcome challenges?

(Interviewer: If the coalition is a mentor coalition or mentee, please discuss how this relationship has impacted their coalition efforts to reduce youth substance use.)

8. Did the coalition assess the cultural competence of various aspects of your system, such as coalition policies, procedures, and practices; in coalition representatives; in any contracts with service providers; in available services?

1. What, if any, activities have been planned/conducted to increase cultural competence within your coalition (e.g., coalition representation), or throughout your system - including at the level of service availability, service delivery, in staff?

2. What successes/challenges have been encountered?

(Interviewer: Please get name of TTA provider if one was used and details about their assistance.)

10. How does the coalition assess whether and to what extent families and youth from the target population are involved in DFC assessment, capacity, and planning activities and processes?

    1. What successes/challenges has your DFC coalition faced in planning/conducting activities to increase family and youth involvement? How many/to what extent are parents of the target population actively involved? Youth? In what capacity (please describe)??

Local Leader, Community, and Economic Context

10. Has you coalition experienced any major changes in leadership?

1. What has been the impact, if any, on changes in coalition leadership on coalition dynamics and effectiveness in the community? On changes in coalition participants? What are they keys to effective coalition leadership?

2. What has been the impact, if any, on changes in local political and community (please specify which one) leaders? Has this helped or hindered efforts?

3. To what extent has your coalitions been able to sustain momentum for change during any changes in leadership?

10. What is the context in your community for conducting DFC activities/efforts? Has the community focus on substance abuse prevention been impacted by other community contexts?

1. What has been the impact, if any, in changes of local political and community (please specify which one) leaders? Has this helped or hindered efforts? How can coalitions sustain momentum for policy change during transitions in political and local neighborhood leadership?

2. Has the downturn in the national economy impacted DFC activities and efforts? How? How about your local economic context – has it impacted DFC activities and efforts? How?

1. For instance, if the local economy has been struggling, has this taken peoples’ attention away from youth drug abuse? How do you engage the community when there are such overarching concerns? How do you sustain their efforts toward positive community change?

2. What can other communities with limited resources do to improve readiness to begin coalition work?

Implementation

(Interviewer: This section should be pre-filled from the appropriate sections of COMET and verified on-site. Use a separate font color for information obtained on-site. Please make sure to obtain detailed information on the process of planning and implementation that is not covered in COMET – challenges encountered, timeline, critical implementation events, etc.) This is the key section of the protocol. Please review COMET and query respondents specifically about all activities (i.e., ask 15a-15e and 16a-17e for each major activity,) with a focus on the last two COMET reporting periods.

15. Please tell me about key accomplishments since you started participating in the DFC coaltion. Of the seven DFC strategies, were there any particular you felt made the most difference – in terms of increasing collaboration or decreasing youth substance use/abuse? What is something that DFC did in the past year of which you are most proud and why? Upcoming events that coalition is excited about?

    1. Any events that were disappointing? Will you try again making changes based on what you learned or focus on new ideas?

    2. Do you have any recommendations for future DFC communities on strategies or activities that were particularly effective?
       
       

16. What activities has your DFC coalition planned or implemented to increase collaboration between coalition members? Between DFC and the broader community?

1. Can you describe the activity? Was it successful? What challenges did the coalition encounter and how were these resolved?

2. Was your organization involved? How/to what extent? What sectors were represented in this effort? Which agencies took the greatest lead in implementing the activity (please name up to 3)?

3. How many individuals or organizations did the activity impact? Please describe in detail.

4. How did the coalition involve families and youth from the target population in implementation activities and efforts? How many families and youths were involved and what were their roles and responsibilities?

15. What activities has your DFC coalition planned or implemented to decrease youth substance use in the targeted communities?

(Interviewer: Pass out listing and description of the seven DFC strategies. Make sure respondent links activities to specific strategies. This is the key section of the protocol. Please review COMET and query respondents specifically about all activities (i.e., ask 17a-17e for each major activity,) with a focus on the last two COMET reporting periods.)

1. Can you describe the activity? To what extent did this activity contribute to your coalition achieving its objectives and goals (i.e., youth substance use reduction)? What makes/made this activity successful? What challenges did the coalition encounter and how were they resolved?

2. What type(s) of overarching strategy(ies) did your activity target?

3. Was your organization involved? How? What sectors were represented in these efforts? Which agencies took the greatest lead in implementing the activity (please name up to 3)?

4. What percent of time, in terms of coalition time and energy, did your coalition spend on this activity?

5. What adaptations, if any, were necessary to achieve success?

6. How did the coalition involve families and youth from the target population in implementation activities and efforts? How many family and youths were involved and what were their roles and responsibilities?

Evaluation

15. Is the coalition effective in accomplishing activities and/or goals? Are the conversations around goals productive and do they lead to clear action steps?

(Interviewer: If they have a logic model, please review with stakeholder noting what has been accomplished, what they are working on, and what they need to do in the future to reduce youth substance use. Discuss and collect any documents that have documented and/or evaluated their efforts?)

1. How does your collaborative partnership assess its progress toward its goals?

2. Does the collaborative annually develop an action plan? What has been the focus (i.e., main objectives, strategies, and activities) this past year and what is the focus for the upcoming year? What major changes, if any, have there been to the strategic plan?

3. To what extent, if any, are youth and family members involved in evaluation activities? Success/Challenges in involving families/youth?

15. How are data and evaluation findings incorporated into future planning and implementation activities?

DFC Impacts & Outcomes

15. Please tell me about key accomplishments of the DFC coalition since you started participating.

    1. Have core outcomes for your identified target population changed during the DFC initiative? If so, how? What do you feel was responsible for this change in core outcomes?

    2. What community and systems changes (e.g., increased ownership of drug use issues, increase in safety, increase in community identity and sense of community) have resulted from DFC activities? What do you feel was responsible for this change or changes? Why?

    3. Has your DFC coalition generated momentum in the community (i.e., engaging community members)? How (formalized strategy, grassroots campaign)?

Community and DFC Strategies

15. How would you describe your local community – is it safe, is there a good sense of community, are drugs readily available, are there enough recreation areas and other facilities for prosocial activities for youth? Has DFC had any impacts on the local community? Please describe.

16. Of the seven DFC strategies, were there any particular ones you felt made the most difference – in terms of increasing collaboration or decreasing youth substance use/abuse?

1. Do you have any recommendations for future DFC communities on what strategies or activities were particularly effective?

15. Overall, what key events most influenced the activities and impacts of your DFC coalition (critical events)?

1. What are the two or three greatest facilitators to implementation of DFC activities and community change?

2. What are the two or three greatest barriers to implementation of DFC activities and community change?

3. What are the keys to sustaining a coalition over time?

Sustainability

24. What has your coalition done, if anything, to ensure sustainability of your drug use prevention effort when federal funding is terminated?

a. What are the keys to sustaining a coalition over time?

2. What recommendations would you give to other communities with limited resources interested in beginning/sustaining DFC coalition work?

DFC Sector Representative Interview Protocol Page 6