Drug-Free Communities (DFC) Support Program National Evaluation

Attachment 8:

Proposed Changes to DFC Site Visit Protocols

Proposed Changes to:

DFC Site Visit Protocols

Summary of proposed changes:

1. Remove Financial Officer protocol.

2. Remove “REMIND TO TAKE SNA SURVEY!!!” from the following protocols: Key Staff Focus Group; Local Evaluator/Data Collector; Multi-Sector Focus Group; Parent Focus Group; Project Director; Sector Individual Interview.

   1. We are no longer administering the SNA survey.

NOTE: Not all items are asked on all site visits. Some items are modified slightly to indicate focus of visit (e.g., on being an inner-city or LGBT community or focus on a substance such as marijuana or synthetics).

DFC SITE Visit Protocols

The protocols are generally flexible enough to adapt probes. We are proposing eliminating the Financial Officer Protocol. In many cases, the financial officer is not aware of the day-to-day work of the coalition. In addition, grantees participating in the site visits will no longer be asked to complete a social network survey.

Introductory Protocol for Interviews and Focus Groups

This protocol is used to introduce ICF during each new meeting that occurs during the site visit.

For student focus groups, make sure that each focus group participant has turned in signed copies of their student assent AND parent consent form. The forms can be collected by DFC grantee staff, but please review them to make sure they are filled out.

For parent focus groups, obtain signed copies of the parent focus group consent form from each participant prior to the beginning of the focus group.

My name is [introduce self and note-taker]. We are the part of the Drug- Free Communities National Evaluation team which is visiting select DFC coalitions across the country. Thank you for agreeing to participate in today’s focus group/interview regarding the program at [insert coalition name]. We appreciate you taking time to assist with this evaluation effort because your perspective gives us the opportunity to learn from the people who are actually doing coalition work. We are hoping to learn about how you operate as a coalition, your challenges and successes, the strategies you use to accomplish your coalition’s mission and any lessons that can be shared with other DFC funded coalitions.

Before we begin, we want to remind you that your participation in this focus group/interview is voluntary and the information you share with us will be kept confidential. Specifically, this means that (1) you can decline to answer any questions, or leave at any time; (2) we will not connect your name with what was said any written reports ; and (3) only evaluation staff and a transcription vendor will have access to the interview tape and interview data (if you agree to be tape recorded). Confidentiality agreements will be signed by the whole DFC Evaluation Team and the transcription vendor to ensure privacy. In our report on the evaluation of the DFC program we will provide overviews of what was learned during the site visits and will connect anecdotes to other data we have collected. However, we will not report or present the information you share with us in any way that will identify a specific person. Your coalition may be identified in the reports but only if prior permission is given from the coalition leadership.

As a reminder, we are a third party external evaluator. Please understand that the information gathered on this site visit will NOT have an impact on your funding. We are not conducting a compliance check – but are specifically here to learn about the work that you do and your coalition.

Throughout the interview/focus group, please keep in mind that when we use the word “substances” or “drugs”, we are referring to tobacco, marijuana, alcohol, prescription drugs, and any other substance youth use to change a state of mind.

[FOR FOCUS GROUPS ONLY]:

To help the focus group work, we would like to ask each of you to:

1. Be respectful of other participants and the facilitators. This includes being respectful about not sharing outside of this room without the participant’s permission.

2. Fully participate to the best of your abilities by sharing your expertise and experiences with your peers.

3. Ask questions, pose scenarios, and make suggestions that will help everyone.

4. Turn off cell phones and/or pagers or place them on vibrate.

(Facilitator: Ask participants if they would like to add their own/additional ground rules.)

With your permission, in addition to taking notes, we would like to record the audio of this focus group/interview. Nobody outside of the National Evaluation team and transcription vendor will have access to this audio recording, which will be used to make sure we have captured this conversation accurately. If participant does not agree, say “That’s fine. We’ll just take notes.”

Are there any questions before we begin?

Reminders and Tips

-Allow for wait time. Pause in silence is fine.

-Remain in listening mode.

-Avoid leading or reacting (use phrases such as “That is interesting” or “That is helpful to understand.”

Consent Form for Parent Focus Groups

Parental Assent for Participation in a Focus Group

for the Drug-Free Communities National Evaluation

You are being asked to participate in a focus group for the national evaluation of the Drug Free Communities Support Program, a Federal grant program that provides funding to community-based coalitions that organize to prevent youth substance use. This focus group will help National Evaluation staff identify key factors that lead to successful outcomes and understand your community in more detail. The focus group of parents is a group discussion with between 6-8 participants. It should last about an hour.

You will not be paid for your participation, but food and drink (e.g., pizza, soda, water) will be provided.

The information collected during this focus group will be used in the National Evaluation. You are free to withdraw your consent and to discontinue your participation in this study at any time without affecting services that you or your child might receive from DFC.

You may review the questions that will be asked by contacting Allan Porowski, Principal Investigator, at 1.800.532.4783.

Please check one of the boxes below; fill in complete name information and sign the consent form.

I hereby:

• agree to participate in a focus group as part of the Drug Free Communities National Evaluation.

• do not agree to participate in a focus group as part of the Drug Free Communities National Evaluation.

Parent/Guardian Name (Please print): _________________________________________

Parent/Guardian Signature: __________________________ Date: _________________

(signature must be in ink)

Parental Consent for Student Focus Groups

PARENTAL CONSENT FOR PARTICIPATION IN DRUG FREE COMMUNITIES NATIONAL EVALUATION STUDENT FOCUS GROUPS

You are being asked to consent for your child to participate in a focus group for the national evaluation of the Drug Free Communities Support Program, a Federal grant program that provides funding to community-based coalitions that organize to prevent youth substance use. This focus group will ask for your child’s opinion about substance abuse prevention efforts at their school and in your community. By giving consent, you are agreeing to allow your child to participate in a one-hour focus group. There will be approximately 6-8 students in each focus group. Food (e.g., pizza, snacks) and drink (e.g., soda/water) will be provided. We are asking students not to share information about what is discussed during these focus groups sessions. However, we cannot guarantee that all students will keep these discussions confidential.

The information collected during this focus group will be used in the National Evaluation of the Drug Free Communities Program. All information obtained from or about your child will be protected by the evaluation team to the extent allowable by law. No information will be reported that can identify you, your family, or your child.

You are free to withdraw your consent and to discontinue your child’s participation in this study at any time without affecting services that your child might receive from DFC.

You may review the questions that will be asked of your child by contacting Barbara O’Donnel, Principal Investigator, at 1.877.854.0731.

Financial Officer Interview Protocol

CHANGE: Remove Protocol

See Introductory Protocol for Interviews and Focus Groups attachment for guidance on language for the beginning of each session.

Throughout the protocol bullets represent probing questions to use if participants’ initial responses do not generate relevant information.

1. Member Involvement

1. Tell us a little about yourself and your role in the DFC coalition.

• What are your primary responsibilities as the Financial Offer?

• How long have you been involved with DFC?

2. If confirmed as a border town/city: What does being a border town/city mean to you/your coalition?

3. If confirmed as an inner city: What does being an inner city mean to you/your coalition?

2. Coalition Capacity (history of the coalition and level of community involvement)

Suggested transition language: We’re interested in learning about your coalition and its funding streams.

1. Please describe a little about the coalition history.

• What, if any, prior experiences does the community have with substance abuse prevention? (specifically with youth substance abuse prevention and with being a coalition)

• To what extent did these past efforts increase your readiness and capacity for the DFC initiative? (E.g. having the right people, agencies/organizations, knowledge, skills, resources, leadership, and implementation. Please explain.)

1. What is the decision-making process in your coalition?

• How do you determine where to invest funding, resources, and staff?

• What sorts of information is used to make decisions (costs, survey data, public information, feedback from the community, evaluation data)?

1. What are the main expenses that the DFC funding goes towards?

• How have the main expenses of the coalition changed over time?

1. How have your sources of funding changed over the years?

• What are the main businesses, organization, or sector in the community that financially contribute or donate resources to the coalition?

• What are other ways you solicit funds in your coalition?

1. What has your coalition done to ensure sustainability of your substance abuse prevention efforts when federal funding ends?

• What are the keys to sustaining a coalition over time?

• Please describe any committee/task force structures that focus on funding or sustainability.

1. How effective is communication in the coalition?

• How would you describe your communications with those involved in DFC?

• How well are you kept informed about coalition activities and events?

1. Have there been any changes in key personnel in your coalition?

• What was the impact on coalition dynamics and coalition effectiveness?

• How do you handle transitions in leadership in your coalition?

• What do you do when you lose a key ally? How do you work with a new key ally?

  • If confirmed by staff as border/inner-city and not self-generated in earlier responses: To what extent does [being a border town/city or being in the inner-city] affect the coalition capacity?

3. Intervention Strategies and Activities

   1. What general prevention approach does your coalition use?

• In what ways, if at all, do you approach different community problems with different strategies and activities?

• As an example, describe one or two of the coalition activities that you or your organization has been involved with and tell us about the impact. How did the activity contribute to coalition accomplishments or community change?

• What made it successful or unsuccessful?

• How involved were you or your organization in the decision-making process for this activity?

• What might you suggest the coalition does differently if they decide to do the activity again?

• If confirmed by staff as border/inner-city and not self-generated in earlier responses: In what ways, if any, does [being a border town/city or being in the inner-city influence] the strategies the coalition selects/implements?

4. Overall Challenges and Accomplishments

Suggested transition language: Now we want to talk about the impact of the DFC programs/services on your community.

1. What are community changes resulting from your DFC coalition?

• On youth substance use; On community problems related to substance use; On the general public; On community organizations, particularly related to substance use problems and prevention

• What do you feel was responsible for this change?

1. From a service perspective, what are activities that your coalition implements that, when combined, the whole is greater than the sum of the parts?

2. In what ways are you working to sustain this work?

• What are keys to sustainability that you would like to see the coalition address?

1. What are current or anticipated challenges your coalition faces in creating change?

• What recommendations would you offer your coalition to overcome these challenges?

• If confirmed by staff as border/inner-city and not self-generated in earlier responses: Please share any ways that you think [being a border town/city or being in the inner-city] presents challenges to the coalition work? To what extent does it create opportunities for success?

5. Overall Comments and Recommendations

   1. What recommendations or advice would you give to financial officers in other [inner-city or border, as relevant to the coalition] communities interested in beginning DFC coalition work?

   2. Is there anything else that you would like to share about your coalition that we haven’t covered?

Thank you for participating in this interview and for contributing to the evaluation of DFC.

Key Staff Focus Group Protocol

See Introductory Protocol for Interviews and Focus Groups attachment for guidance on language for the beginning of each session. Throughout the protocol bullets represent probing questions to use if participants’ initial responses do not generate relevant information.

1. Member Involvement

1. Tell us a little about yourself and your role in the DFC coalition.

• What are your primary responsibilities as a staff member?

• How long you have you been involved with the coalition?

• How did you get involved with DFC (e.g., recruited, pro-actively involved)?

• What was your motivation for being on the staff?

1. Community Context

Suggested transition language: We would like to learn more about your community in relation to the coalition.

1. What are important contextual factors that would help us better understand the community here?

• If confirmed as a border town/city: What does being a border town/city mean to you/your coalition?

• If confirmed as an inner city: What does being an inner city mean to you/your coalition?

1. In what ways does the community influence the coalition’s work?

• In what ways does the community support the coalition’s work?

• In what ways does the community present challenges to the coalition’s work?

• In what ways has the coalition been able to address challenges over time?

3. Coalition Capacity (history of the coalition and level of community involvement)

Suggested transition language: We are also interested in learning about the level of community involvement in your coalition.

1. Please describe a little about the coalition history.

• What, if any, prior experiences does the community have with substance abuse prevention? (specifically with youth substance abuse prevention and with being a coalition)

• To what extent did these past efforts increase your readiness and capacity for the DFC initiative? (E.g. having the right people, agencies/organizations, knowledge, skills, resources, leadership, and implementation. Please explain.)

• How has the coalition changed over time?

1. How does the coalition get the right people at the table to create momentum in your community in preventing youth substance abuse?

• In what ways does coalition membership represent the diversity within the community?

• What are the sectors/players that are highly active?

• Are there certain sectors that need to work together in order for your coalition to be effective?

• What are other organizations/businesses you would like to see get involved?

• Please describe any committee/task force structures exist in the coalition.

1. What is the decision-making process in your coalition?

• To what extent do coalition members agree on the major objectives of the coalition?

• To what extent do coalition members agree on how to achieve goals?

• To what extent does resolving disagreements move your coalition forward or impede its progress?

  • What are the major areas of disagreement?

  • How are disagreements resolved when they occur? How is agreement established?

• To what extent are leadership decisions centralized within the staff or distributed among coalition members?

• How do you determine where to invest resources and time?

1. How effective is communication in the coalition?

• How would you describe your communications with those involved in DFC?

• How well are you kept informed about coalition activities and events?

1. What has your experience been with training to prepare you for your role in the coalition?

2. Please describe any changes in key personnel or political leadership that the coalition has faced.

• What was the impact on the coalition?

• How do you handle transitions in leadership in your coalition?

• What do you do when you lose a key ally?

• How do you work with a new key ally (e.g., in your local government)?

• If confirmed as border/inner-city and not self-generated in earlier responses: To what extent does [being a border town/city or being in the inner-city] affect the coalition capacity?

4. Intervention Strategies and Activities

   1. What general prevention approach does your coalition use?

• In what ways, if at all, do you approach different community problems with different strategies and activities?

1. Please give a brief description of the population and substances the coalition targets.

• How has the target population and targeted substances changed over the years?

• What factors have contributed to these changes?

1. As an example, describe one or two of the coalition activities and tell us about the impact.

• How did the activity contribute to coalition accomplishments or community change?

• What made it successful or unsuccessful?

• What might you do differently if you did the activity again?

• If confirmed by staff as border/inner-city and not self-generated in earlier responses: In what ways, if any, does [being a border town/city or being in the inner-city influence] the strategies the coalition selects/implements?

5. Challenges and Accomplishments

Suggested transition language: Now we want to talk about the impact of the DFC programs/services on your community.

1. What are any community changes resulting from your DFC coalition?

• Do you see changes on youth substance use; community problems related to substance use; the general public; community organizations, particularly related to substance use problems and prevention?

• What do you feel was responsible for this change?

1. In what ways is the coalition working to sustain this work?

2. What are current or anticipated challenges your DFC coalition faces in creating change?

• What recommendations would you offer your coalition to overcome these challenges?

• If confirmed by staff as border/inner-city and not self-generated in earlier responses: Please share any ways that you think [being a border town/city or being in the inner-city] presents challenges to the coalition work? To what extent does it create opportunities for success?

6. Overall Comments and Recommendations

   1. What recommendations or advice would you give to other [inner-city or border, as relevant to the coalition] communities interested in beginning DFC coalition work?

   2. Is there anything else that you would like to share about your coalition that we haven’t covered?

Thank you for participating in this interview and for contributing to the evaluation of DFC.

DELETE: REMIND TO TAKE SNA SURVEY!!!

Local Evaluator / Data Collector Protocol

See Introductory Protocol for Interviews and Focus Groups attachment for guidance on language for the beginning of each session. Throughout the protocol bullets represent probing questions to use if participants’ initial responses do not generate relevant information.

1. Member Involvement

   1. Tell us a little about yourself and your role in the DFC coalition.

• What are your primary responsibilities as a local evaluator/data collector (e.g., data collection, data analysis)?

• How long have you been involved with DFC?

• In what ways, if any, do you serve in a similar capacity for other DFC grantees or other coalitions? If confirmed as border/inner-city and not self-generated in earlier responses: Are these also border towns or inner city?

2. Coalition Capacity (history of the coalition)

   1. Please describe a little about the coalition history.

• What, if any, prior experience does the community have with substance abuse prevention? (specifically with youth substance abuse prevention and with being a coalition)

• To what extent, did these past efforts impact your readiness and capacity for the DFC initiative? (e.g., having the right people, agencies/organizations, knowledge, skills, resources, leadership, and implementation. Please explain.)

3. Intervention Strategies and Activities

Suggested transition language: We’re interested in the various strategies you implement to evaluate your data and your thoughts on DFC Me (the new grants management system) (NOTE: Questions on DFC Me will be asked ONLY if the local evaluator has interacted with the DFC Me environment).

1. What general intervention approach does the coalition use?

• Does the coalition approach different community problems with different strategies and activities?

• How/to what extent does your coalition incorporate data and evaluation findings into future planning and implementation activities?

2. Did your coalition follow a logic model?

• If so, how helpful has it been in focusing coalition activities and the evaluation?

3. Please describe the population and substances the coalition targets.

• To what extent were you involved in identifying these targets?

• How has the target population and targeted substances changed over the years?

4. What are the specific surveys, tools, and methods that are used to collect and analyze data for needs assessment?

• Who or what are the main sources of information?

• Tell me about other consequences data that you are collecting.

• What resources or tools would be helpful in tracking the coalition’s progress?

• In what ways are these data also used for DFC and other reports?

5. What, if any, concerns do you have about the quality of the data being entered into DFC Me?

• How does the coalition ensure that there is a representative sample in their data collection?

6. Have you used the DFC Me system? If yes, tell us about your experiences with it.

• What do you like and dislike about DFC Me?

• Who enters the data into DFC Me?

• What are some recommendations on how DFC Me can more user-friendly and meaningful for coalitions?

• What data reports that should be incorporated into DFC Me?

• [For data collector only] On average, how many labor hours does it take to enter DFC Me data for one reporting period? When do you begin collecting the data for DFC Me?

7. Were you provided any training to prepare you for your role in the coalition?

4. Overall Challenges and Accomplishments

Suggested transition language: Now we want to talk about the impact of the DFC programs/services on your community.

1. Tell me about some of your key findings.

• What are major coalition successes?

• What are some of the coalition’s challenges?

2. What are any community changes you have seen resulting from your DFC coalition?

• Do you see changes on youth substance use; community problems related to substance use; on the general public; community organizations, particularly related to substance use problems and prevention?

• What do you feel was responsible for this change?

• In what ways have other coalition influenced the work?

5. Overall Comments and Recommendations

1. What are some strategies, resources, or activities that you would like your coalition to provide?

2. What recommendations or advice would you give to organizations like yours in other [border or inner-city] communities interested in beginning DFC coalition work?

3. Is there anything else that you would like to share about your coalition that we haven’t covered?

Thank you for participating in this focus group and for contributing to the evaluation of DFC.

DELETE: REMIND TO TAKE SNA SURVEY!!!

Multi-Coalition Focus Group Protocol

See Introductory Protocol for Interviews and Focus Groups attachment for guidance on language for the beginning of each session. Throughout the protocol bullets represent probing questions to use if participants’ initial responses do not generate relevant information.

1. Member Involvement

1. Tell us your name, the coalition you work with, and your role in that coalition.

• What are your primary responsibilities in your coalition?

• How long you have you been involved with your coalition?

2. Community Context

Suggested transition language: We would like to learn more about your community in relation to the coalitions.

1. In what ways is the community your coalition serves similar to or different from [site visit coalition name or community]?

2. How would you describe the larger region or area that this group of coalitions spans?

• In what ways does this context support the coalition’s work?

• In what ways does this context present challenges to the coalition’s work?

3. If site visit coalition is confirmed as a border town/city: Would you also describe your coalition as serving a border town/city and if so, what does that mean to you/your coalition?

• How does serving a border town/city impact your collaboration with [site visit coalition name]?

4. If site visit coalition is confirmed as an inner city: Would you also describe your coalition as serving an inner city and if so, what does that mean to you/your coalition?

• How does serving an inner city impact your collaboration with [site visit coalition name]?

3. Coalition Capacity (history of the coalition and level of community involvement)

Suggested transition language: We are also interested in learning about the way your coalitions work together.

1. Please describe a little about the history of your collaboration with [site visit coalition name].

• How did you make the decision to work together?

• Why did you initially decide to pursue this collaboration?

• What, if any, prior experiences does your coalition have working with other DFC grantees?

• How has the collaboration changed over time?

2. What is the decision-making process in any joint projects or shared initiatives?

• To what extent do coalitions agree on the major objectives?

• To what extent do coalitions agree on how to achieve shared goals?

3. How effective is communication among the coalitions?

• How would you describe your interactions with other DFC grantees?

• How well are you kept informed about other coalition’s activities and events?

• Do you hold regular meetings or communicate informally via phone and email?

4. Please describe any ways you work together when faced with changes in key personnel or political leadership.

• In what ways do you leverage this collaboration when facing transitions in your own coalition’s leadership?

• How do you work together to reach out new key ally (e.g., in your local government)?

4. Intervention Strategies and Activities

1. To what extent does your coalition have a similar prevention approach as [site visit coalition name]?

• Is there a similar focus on the population and substances that your coalitions target?

2. In what ways do you collaborate and work together?

• To what extent do you share/brainstorm ideas to implement individually?

• To what extent do you implement joint activities to cover a larger catchment area?

• If they engage in joint activities: As an example, describe one or two of the activities you have worked on together.

• What made it successful or unsuccessful?

• What might you do differently if you did the activity again?

5. Challenges and Accomplishments

Suggested transition language: Now we want to talk about the impact of collaborating with each other.

1. What are any benefits from collaborating with [site visit coalition name]?

• Are there ways that you see this collective effort to be greater than the sum of its parts?

• Has this collaboration been a useful way to attain funding, including DFC or other sources?

• In what ways, if at all, have you found cost savings from joint purchases or economies of scale?

2. In what ways does this help to sustain the work of your coalition?

3. What are challenges in this partnership?

• What recommendations would you offer to overcome these challenges?

• To what extent do you face challenges in competing for the same funding sources?

6. Overall Comments and Recommendations

1. What would you like to see come from this collaboration going forward?

2. What recommendations or advice would you give to other [inner-city or border, as relevant to the coalition] communities interested in working with other DFC grantees?

3. Is there anything else that you would like to share about your coalition that we haven’t covered?

Thank you for participating in this interview and for contributing to the evaluation of DFC.

Multi-Sector Focus Group Protocol

See Introductory Protocol for Interviews and Focus Groups attachment for guidance on language for the beginning of each session. Throughout the protocol bullets represent probing questions to use if participants’ initial responses do not generate relevant information.

1. Member Involvement To start, tell us a little about who you are and your role with the DFC coalition.

1. How did you get involved with DFC (e.g., recruited, pro-actively involved) ?

• What is your motivation for involvement?

• Why are you interested in coalition work?

2. Please identify the sector that you represent and tell us about your organization.

• What sector does your organization represent (business, media, school, youth-serving, law enforcement, religious/fraternal organization, civic/volunteer group, health, government)?

• What is the organization’s background regarding substance use?

2. Community Context

Suggested transition language: We would like to learn more about your community in relation to the larger coalition.

1. What are important contextual factors that would help us better understand the community here?

• If confirmed as a border town/city: What does being a border town/city mean to you/your coalition?

• If confirmed as an inner city: What does being an inner city mean to you/your coalition?

1. In what ways does the community influence the coalition’s work?

• In what ways does the community support the coalition’s work?

• In what ways does the community present challenges to the coalition’s work?

• In what ways has the coalition been able to address challenges over time?

3. Do you think the coalition is focusing on the critical issues for youth in your community?

3. Coalition Capacity (history of the coalition and level of community involvement)

Suggested transition language: We are also interested in learning about the level of community involvement in your coalition.

1. How does the coalition get the right people at the table to initiate discussion and create momentum in your community on targeting substance abuse among youth?

• Do you think all of the important players are at the table?

• Which sectors usually work together?

• Are there certain sectors that need to work together in order for your coalition to be effective?

• What are other organization/businesses you would like to see get involved?

• Please describe any committee/task force structures you are involved in.

• To what extent do you use your organization to reach out to the community and promote the coalition?

1. What has your experience been with any training to prepare you for your role in the coalition?

2. How effective is communication in the coalition?

• How would you describe your communications with those involved in DFC?

• How well are you kept informed about coalition activities and events?

• To what extent do you use your organization to reach out to the community and promote the coalition?

4. What is the decision-making process in your coalition?

• To what extent do coalition members agree on the major objectives of the coalition?

• To what extent do the coalition members agree on how to achieve goals?

• What are the major areas of disagreement?

• How are disagreements resolved when they occur? How is agreement established?

• If confirmed as border/inner-city and not self-generated in earlier responses: To what extent does [being a border town/city or being in the inner-city] affect the coalition capacity?

4. Intervention Strategies and Activities

1. As an example, describe one or two of the coalition activities that you or your organization has been involved with and tell us about the impact.

• How did the activity contribute to coalition accomplishments or community change?

• What made it successful or unsuccessful?

• How involved were you or your organization in the decision making process for this activity?

5. Overall Challenges and Accomplishments

Suggested transition language: Now we want to talk about the impact of the DFC programs/services on your community.

1. What are any community changes you have seen resulting from your DFC coalition?

• Do you see changes on youth substance use; community problems related to substance use (e.g., access to alcohol, laws, policies); the general public; community organizations, particularly related to substance use problems and prevention.

• What do you feel was responsible for this change?

• To what extent have other coalition influenced the work?

2. How has your involvement in the coalition benefited you and your organization/sector?

3. What are current or anticipated challenges faced by your DFC coalition (i.e., funding, community involvement)?

• What recommendations would you offer your coalition to overcome these challenges?

• If confirmed as border/inner-city and not self-generated in earlier responses: Please share any ways that you think [being a border town/city or being in the inner-city] presents challenges to the coalition work? To what extent does it create opportunities for success?

6. Overall Comments and Recommendations

1. What are some strategies, resources, or activities that you would like your coalition to provide?

2. How can the coalition provide you or your sector more opportunities to be involved?

3. What recommendations or advice would you give to organizations such as yours in other [border town/city or inner-city] communities interested in beginning DFC coalition work?

4. Is there anything else that you would like to share about your coalition that we haven’t covered?

Thank you for participating in this interview and for contributing to the evaluation of DFC.

DELETE: REMIND TO TAKE SNA SURVEY!!!

Parent Focus Group Protocol

See Introductory Protocol for Interviews and Focus Groups attachment for guidance on language for the beginning of each session. Throughout the protocol bullets represent probing questions to use if participants’ initial responses do not generate relevant information.

1. Member Involvement

   1. To start, tell us a little about who you are and the age of your children.

   2. What is your role with the DFC coalition?

• How did you get involved with DFC (recruited, pro-actively involved)? What is your motivation for involvement?

3. Why do you think some parents might not be involved in DFC? What do you think coalitions could do to get more parents involved?

2. Community Context

Suggested transition language: We would like to learn more about the DFC coalition in relation to the community.

1. In what ways does the community influence the coalition’s work?

• In what ways does the community support the coalition’s work?

• In what ways does the community present challenges to the coalition’s work?

1. What are important contextual factors that would help us better understand the community here?

• If confirmed as a border town/city: What does being a border town/city mean to you/your coalition?

• If confirmed as an inner city: What does being an inner city mean to you/your coalition?

1. To what extent is substance use a problem for youth in the community?

• How would you describe the awareness on the issues/need for change in your community?

• Do you think parents generally agree on the changes needed in your community?

1. What is the <COALTION NAME> approach to combating youth substance abuse in your community?

• Do you think the coalition is focusing on the critical issues for youth in your community?

1. Please describe the conversations you have had with your kids about substance abuse.

• What do you talk about?

• How does your child typically respond?

3. Overall Challenges and Accomplishments

Suggested transition language: Now we want to talk about the impact of the DFC programs/services on your community.

1. What are some of the major challenges faced by your DFC coalition (i.e., funding, community involvement)?

• What recommendations would you offer your coalition to overcome these challenges?

1. In what was has youth involvement with coalition impacted your child? How has it impacted you as a parent?

2. What are any community changes resulting from your DFC coalition?

• Do you see changes on youth substance use; community problems related to substance use; the general public; community organizations, particularly related to substance use problems and prevention?

• What do you feel was responsible for this change?

If confirmed as border/inner-city and not self-generated in earlier responses: Please share any ways that you think [being a border town/city or being in the inner-city] presents challenges to the coalition work? To what extent does it create opportunities for success?

4. Overall Comments and Recommendations

   1. What are some strategies, resources, or activities that you would like your coalition to provide?

   2. How can the coalition provide you and other parents more opportunities to be involved?

   3. Is there anything else that you would like to share about your coalition that we haven’t covered?

Thank you for participating in this focus group and for contributing to the evaluation of DFC.

DELETE: REMIND TO TAKE SNA SURVEY!!!

Project Director Interview Protocol

See Introductory Protocol for Interviews and Focus Groups attachment for guidance on language for the beginning of each session. Throughout the protocol bullets represent probing questions to use if participants’ initial responses do not generate relevant information.

1. Member Involvement

   1. Tell us a little about yourself and your role in the DFC coalition.

• What are your primary responsibilities as Project Director?

• How long have you been the Project Director?

2. Community Context

Suggested transition language: We would like to learn more about your community in relation to the coalition.

1. What are important contextual factors that would help us better understand the community here?

• If confirmed as a border town/city: What does being a border town/city mean to you/your coalition?

• If confirmed as an inner city: What does being an inner city mean to you/your coalition?

1. In what ways does the community influence the coalition’s work?

• In what ways does the community support the coalition’s work?

In what ways does the community present challenges to the coalition’s work? Has the coalition been able to address challenges over time?

3. Coalition Capacity (history of the coalition and level of community involvement)

   1. Please describe a little about the coalition history.

• What, if any, prior experiences does the community have with prevention? (specifically with youth substance abuse prevention and with being a coalition)How did your coalition learn about DFC funding/decide to purse DFC funding?

• To what extent did these past efforts increased your readiness and capacity for the DFC initiative? (E.g. having the right people, agencies/organizations, knowledge, skills, resources, leadership, and implementation. Please explain.)

• How has the coalition changed over time?

1. How/to what extent does the coalition get the right people at the table to create momentum in your community in preventing youth substance abuse?

• Who are the sectors/players that are highly active?

• What role does each sector play in contributing to the work of the coalition? Are there certain sectors that need to work together in order for your coalition to be effective?

• To what extent do coalition members represent the diversity within the community?

• What are other organization/businesses you would like to see get involved?

1. How do you recruit new members or leadership within the coalition?

2. Please describe any committee/task force structures that members are organized in.

3. What is the decision-making process in your coalition?

• To what extent do coalition members participate in decision-making?

• How does your coalition determine where to invest resources and time?

• To what extent do coalition members agree on the major objectives of the coalition?

• To what extent do the coalition members agree on how to achieve goals?

• To what extent does resolving disagreements move your coalition forward or impede its progress?

• What are the areas of disagreement? How are disagreements resolved? How is agreement established?

1. How effective is communication in the coalition?

• How would you describe your communications with those involved in DFC?

• How does the coalition keep members informed about coalition activities and events?

1. What has your experience been with training to prepare you for your role in the coalition?

• Please describe any changes in key personnel or political leadership that the coalition has faced. What was the impact on the coalition?

• How do you handle transitions in leadership in your coalition?

• What do you do when you lose a key ally?

• How do you work with a new key ally (e.g., in your local government)?

1. Please describe any interactions you have with other coalitions.

• What kind of information/resources do you share?

• Do you collaborate with other DFC coalitions? Which ones?

• In what ways, if any, does that benefit your coalition?

• What kind of collaborations with other coalitions would you find helpful?

• If confirmed as border/inner-city and not self-generated in earlier responses: To what extent does [being a border town/city or being in the inner-city] affect the coalition capacity?

4. Intervention Strategies and Activities

1. How were the goals, vision, and mission of your coalition developed?

• When were they developed?

• Who was involved and at what level?

2. Please give a brief description of the population and substances the coalition targets.

• How has the target population and targeted substances changed over the years?

• What factors have contributed to these changes?

3. What general prevention approach does your coalition use?

• Does the coalition approach different community problems with different strategies and activities?

• What past or upcoming activities have been planned to increase cultural competence within your coalition? What was the impact of these activities?

4. As an example, describe one or two of the coalition activities that you are proud of and tell us about the impact of these activities.

• How did the activity contribute to coalition accomplishments or community change?

• What made it successful or unsuccessful?

• What might you do differently if you did the activity again?

• If confirmed as border/inner-city and not self-generated in earlier responses: In what ways, if any, does [being a border town/city or being in the inner-city influence] the strategies the coalition selects/implements?

5. Overall Challenges and Accomplishments

Suggested transition language: Now we want to talk about the impact of the DFC programs/services on your community.

1. What are any community changes resulting from your DFC coalition?

• Do you see changes in youth substance use; community problems related to substance use; the general public; community organizations, particularly related to substance use problems and prevention?

• What do you feel was responsible for this change?

2. In what ways is the coalition working to sustain this work?

• What are the keys to sustaining a coalition over time?

3. What are current or anticipated challenges your coalition has faced in creating change?

If confirmed as border/inner-city and not self-generated in earlier responses: Please share any ways that you think [being a border town/city or being in the inner-city] presents challenges to the coalition work? To what extent does it create opportunities for success?

6. Overall Comments and Recommendations

   1. What is your vision for the future of your DFC coalition?

   2. What recommendations or advice would you give to other [inner-city or border, as relevant to the coalition] communities interested in beginning DFC coalition work?

   3. Is there anything else that you would like to share about your coalition that we haven’t covered?

Thank you for participating in this interview and for contributing to the evaluation of DFC.

DELETE: REMIND TO TAKE SNA SURVEY!!!

Sector Individual Interview Protocol

See Introductory Protocol for Interviews and Focus Groups attachment for guidance on language for the beginning of each session. Throughout the protocol bullets represent probing questions to use if participants’ initial responses do not generate relevant information.

1. Member Involvement

1. Tell us a little about who you are and your role with the DFC coalition

• How did you get involved with DFC (e.g., recruited, pro-actively involved)?

• How long have you been involved in the coalition?

• Why are you interested in coalition work?

2. Please tell us about the organization you represent.

• What sector does your organization represent (business, media, school, youth-serving, law enforcement, religious/fraternal organization, civic/volunteer group, health, government)?

• If known, how did the organization become a coalition member?

• What is the organization’s background regarding substance use?

• What role does your organization have in prevention? How has DFC impacted this?

• To what extent does your organization work with youth? How has DFC influenced this?

• How do you work with other youth serving community resources, drug prevention programs, or drug treatment programs that youth are referred to help them with their alcohol or substance issues? Has DFC membership impacted these activities?

2. Community Context

Suggested transition language: We would like to learn more about your community in relation to the coalition.

1. What are important contextual factors that would help us better understand the community here?

• If confirmed as a border town/city: What does being a border town/city mean to you/your coalition?

• If confirmed as an inner city: What does being an inner city mean to you/your coalition?

1. In what ways does the community influence the coalition’s work?

• In what ways does the community support the coalition’s work?

• In what ways does the community present challenges to the coalition’s work?

• In what ways has the coalition been able to address these challenges over time?

3. How does substance abuse affect the population your organization serves?

4. SPECIFIC SECTOR QUESTIONS

Law

• What are the local laws and consequences regarding underage drinking, substance possession, and underage smoking?

  • What changes, if any, should be made to these laws?

• What type of public ordinances does your agency enforce (e.g. Alcohol compliance checks, shoulder-tap operations, Smoke-free Air Law)?

  • What are the key challenges of enforcement?

  • How has coalition helped with this?

Judicial

• Please describe any Teen Courts or Drug Courts that exist in the community. In what ways has the coalition been involved in this?

  • What impact has this had? If no teen/drug court, to what extent do you see that as something useful for the community?

• What are the legal consequences for underage drinking, underage smoking, and youth substance abuse/possession?

• To what extent are they effective?

• How frequently do court cases related to youth substance abuse occur?

• What community resources, prevention programs, or treatment programs that youth are referred to?

• How effective are they?

• To what extent is there a need for more resources?

School

• What are the consequences for students when they get caught in possession or under the influence of substances at school?

• What resources or programs are students referred to?

• What are the major substance use problems in your school (types, populations)?

• Please describe any substance abuse prevention clubs or drug-free programs at the school.

• What are some of the activities?

• Who participates? What is the level of involvement?

3. Coalition Capacity (history of the coalition and level of community involvement)

Suggested transition language: We are also interested in learning about the level of community involvement in your coalition.

1. How does the coalition get the right people at the table to create momentum in your community in targeting youth substance abuse?

• Do you think all of the important players from your sector are at the table?

• Which sectors do you usually work with? Which ones would you like to work with more?

• Are there certain sectors that need to work together in order for your coalition to be effective?

• What are other organizations/businesses you would like to see get involved?

• Please describe any committee/task force structures you are involved in.

1. How effective is communication in the coalition?

• How would you describe your communications with those involved in DFC?

• How well are you kept informed about coalition activities and events?

• To what extent do you use your organization to reach out to the community and promote the coalition?

1. Have there been any changes in key personnel or political leadership?

• What was the impact on coalition dynamics and coalition effectiveness?

• How do you handle transitions in leadership in your coalition?

• What do you do when you lose a key ally?

• How do you work with a new key ally (e.g., in your local government)?

1. What has your experience been with training to prepare you for your role in the coalition?

2. What is the decision-making process in your coalition?

• To what extent do coalition members agree on the major objectives of the coalition?

• To what extent do the coalition members agree on how to achieve goals?

• To what extent does resolving disagreements move your coalition forward or impede its progress?

• What are the major areas of disagreement? How are disagreements resolved when they occur? How is agreement established?

4. Intervention Strategies and Activities

   1. As an example, describe one or two of the coalition activities that you or your organization has been involved with and tell us about the impact.

• How did the activity contribute to coalition accomplishments or community change?

• What made it successful or unsuccessful?

• How involved were you or your organization in the decision making process for this activity?

1. SPECIFIC SECTOR QUESTIONS

• Business: What policies do you have in place to address substance abuse issues in your business this impact (drug testing, employee assistance program, etc.)?

• Media: Tell me about the content of any media campaigns or messages you have worked on with the coalition (main messages, target audience, social media).

5. Overall Challenges and Accomplishments

Suggested transition language: Now we want to talk about the impact of the DFC programs/services on your community.

1. What are any community changes resulting from your DFC coalition?

• Do you see changes in youth substance use; community problems related to substance use; the general public; on community organizations, particularly related to substance use problems and prevention?

• What do you feel was responsible for this change?

1. What are current or anticipated challenges faced by your DFC coalition (i.e., funding, community involvement)?

• What recommendations would you offer your coalition to overcome these challenges?

1. How has your involvement in the coalition benefited you and your organization?

2. SPECIFIC SECTOR

• Healthcare: In what ways, if any, has being part of the coalition increased your ability to address their substance-related issues?

  • To what extent have you seen changes in the severity or prevalence of substance abuse problems?

  • How much would you attribute these changes to the coalition’s efforts?

• Law: What changes in policies or law enforcement have been effective in preventing youth substance abuse?

  • To what extent do you think the coalition actions have actually reduced substance use in your community?

• Media: What has been the impact of the coalition’s media campaigns?

  • What is the key to effective messaging about youth substance use?

  • What messages resonate with youth?

6. Overall Comments and Recommendations

   1. What are some strategies, resources, or activities that you would like your coalition to provide?

   2. What recommendations or advice would you give to organizations like yours in other [inner-city or border, as relevant to the coalition] communities interested in beginning DFC coalition work?

   3. Is there anything else that you would like to share about your coalition that we haven’t covered?

Thank you for participating in this interview and for contributing to the evaluation of DFC.

DELETE: REMIND TO TAKE SNA SURVEY!!!

Youth Focus Group Protocol

See Introductory Protocol for Interviews and Focus Groups attachment for guidance on language for the beginning of each session. Throughout the protocol bullets represent probing questions to use if participants’ initial responses do not generate relevant information. For students, this will be a brainstorming session. Give students time to write down on notecards or a piece of paper their thoughts on a question then ask to start contributing. Call on those you see writing but not participating to encourage participation (not punishment or have to but encourage). Plan for about seven minutes on activity (introduce question one minute, two minutes brainstorm 4-5 minutes discussion per section)

Materials: Name tags, pens, chart paper markers, notecards

1. Member Involvement

   1. Please briefly introduce yourself with your first name and age.

   2. How did you get involved in the DFC program (e.g., recruited, pro-actively)?

   3. What is your motivation for involvement?

For the next question, provide these instructions: Please tell us about <COALITION NAME> and we will have <COORDINATOR NAME> list your ideas on the chart paper.

4. When someone mentions <COALITION NAME>, what do you think of? What activities/events have you attended or participated related to reducing youth substance abuse?

• Has anyone else participated in this?

• What about this was challenging?

• What about this was successful?

2. Community Context

Suggested transition language: Tell us about the community you live in.

1. How would you describe life as a kid in <City/town name>?

• Do you feel safe in your community or school?

2. If confirmed as a border town/city: What does living near Mexico/Canada mean to you?

3. If confirmed as an inner-city: What does living in a big city mean to you?

4. What are some issues in your community that are related to substance abuse?

For the next question, provide these instructions: Take a piece of paper and write down who influences you. Ask youth to read their responses aloud.

5. Who influences you about using alcohol, marijuana, cigarettes, prescription pills, and other substances?

• What do they say about it?

• How does this influence you?

• Does anyone else feel similar or different about this?

• In what ways do your parents or friends influence you?

• Who has a bigger influence on you: your friends or parents? Why?

1. What are common substances that your peers use?

• How easy/hard is it to get these substances?

• Why do you think kids try cigarettes, alcohol, marijuana, prescription drugs, or other substances in the first place?

• If kids get caught with substances or using substances at school, what happens?

3. Coalition Capacity

Suggested transition language: We are also interested in learning about the level of youth involvement in your coalition.

1. How might the DFC coalition get youth to be more involved in the coalition?

2. How does your coalition involve youth in their activities, services or decision-making processes?

3. Does your school have any substance abuse prevention programs for youth other than DFC?

• Are you involved?

• What motivated you to participate?

• If confirmed as border/inner-city and not self-generated in earlier responses: To what extent does [living near a border or in a big city] affect the coalition?

4. Overall Challenges and Accomplishments

Suggested transition language: Now we want to talk about the impact of the DFC programs/services on your community.

1. Tell us about DFC programs, events, and services that you thought were most effective.

• What made the activity/event successful?

1. Is there anything that the DFC coalition has done that hasn’t worked?

   • What made the activity/event unsuccessful?

For the next question, provide these directions: Take a piece of paper and write down some of the changes you have seen in the community from <COALITION NAME>. Ask youth to read their responses aloud.

3. What have been some of the changes you have seen in the community as a result of participating in the DFC coalition?

• If confirmed as border/inner-city and not self-generated in earlier responses: Please share any ways that you think [being near a border or in a big city] presents challenges to the coalition work? To what extent does it create opportunities for success?

5. Overall Comments and Recommendations

   1. What are some strategies, resources, or activities that you would like your coalition to provide?

   2. What would you recommend to other youth who have a DFC coalition in their community?

   3. Is there anything else that you would like to share about your coalition that we haven’t covered?

Thank you for participating in this focus group and for contributing to the evaluation of DFC.