Form Grantee-Level Inte Grantee-Level Inte Grantee-Level Interview

Program Evaluation for Prevention: Strategic Prevention Framework for Prescription Drugs

Grantee Interview Protocol

PROGRAM EVALUATION FOR PREVENTION: STRATEGIC PREVENTION FRAMEWORK FOR PRESCRIPTION DRUGS

Attachment 5: State/Tribal Grantee Interview Protocol

SAMHSA’s Strategic Prevention Framework for Prescription Drugs (SPF-Rx) - OMB No. 0930-0377, expiration date August, 31, 2020. SAMHSA funded the Program Evaluation for Prevention Contract (PEP-C), which supports the cross-site evaluation activities for SPF-Rx.

Burden Statement

This information is being collected to assist the Substance Abuse and Mental Health Services Administration (SAMHSA) for the purpose of program monitoring of the Strategic Prevention Framework Prescription Drugs (SPF-Rx) grant program. This voluntary information collection will be used at an aggregate level to determine the reach, consistency, and quality of the SPF-Rx Program. Under the Privacy Act of 1974 any personally identifying information obtained will be kept private to the extent of the law. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid Office of Management and Budget (OMB) control number. The OMB control number for this project is 0930-0377. Public reporting burden for this collection of information is estimated to average 1.5 hours and per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 5600 Fishers Ln, Room 15 E57B, Rockville, MD 20857.

OVERVIEW AND PURPOSE

The PEP-C team conducted initial telephone interviews with each SPF-Rx grantee Project Director or designated staff in fall 2017and 2018. Follow-up interviews will be conducted using a similar protocol in Year 5 of the project. The primary purpose of the interviews is to obtain the perspective of the implementing Project Directors or their staff on several important topics, including infrastructure and capacity, collaboration, leveraging funding and resources, selection and implementation of interventions, monitoring and evaluation, and health disparities. This information is critical for the SPF-Rx cross-site evaluation reports and understanding of how SPF-Rx is being implemented around the country. These semi structured interviews will last approximately 90 minutes and will proceed using the included guide.

PREPARATION FOR INTERVIEWS

Before each interview:

• The interviewer will arrange and confirm the interview time and telephone number with each grantee Project Director or designated staff using an email template approved by SAMHSA and will obtain written (email) consent to record the interview.

• The interviewer will review grantee-specific information, including quarterly reports, proposal information, annual implementation instrument, and other documents as applicable.

INTERVIEWS

Data collection will consist of individual telephone interviews with the Project Director or designated staff for the state-level (or tribal-level) SPF-Rx Program.

The interviewer will document the interviews through notes taken on the interview forms in this document. The files containing the completed interview forms will be placed in the relevant grantee’s folder under the Data Collection folder on Abt’s internal shared drive. With participant consent, the interviews may be audio-recorded to facilitate reporting after the interview.

POSTINTERVIEW ACTIVITIES

Within a week after each interview, the interviewer will email the Project Director or participating staff member to thank him or her for taking the time to be interviewed. The interviewer will follow up by telephone within 2 weeks for any outstanding information or document requests (e.g., contact information, plans, reports, local analyses).

The interviewer will complete the interview process by:

• Filing the completed interview forms (see remaining sections of this document);

• Providing a draft copy of the interview notes to the interviewee for review;

• Obtaining any feedback from the interviewee and making any additions to the notes; and

• Filing any written materials obtained from the grantee.

1. Introduction

INTERVIEW GUIDE FOR STATE/TRIBAL GRANTEES

Hello, my name is and I work for the Program Evaluation for Prevention Contract (PEP-C). PEP-C is part of the team conducting an evaluation of the Strategic Prevention Framework for Prescription Drugs (SPF-Rx), which is sponsored by SAMHSA’s Center for Substance Abuse Prevention (CSAP). The overall goal of the cross-site evaluation is to document and assess the effectiveness of SPF-Rx in preventing prescription drug misuse. We also want to gain insights to improve SPF-Rx and future efforts to reduce prescription drug misuse.

This interview is one of a two of interviews. The first was conducted during the first or second year of your SPF-Rx grant and the second will be conducted during the fifth year of the grant. You are one of 25 grantees who will participate in this study. We are contacting grantees to get your unique perspectives on several important topics related to the implementation of SPF-Rx.

The interview will help the PEP-C team understand your approach to planning and implementing SPF-Rx and the progress of your SPF-Rx efforts in preventing prescription drug misuse and abuse in your (state/tribe/jurisdiction). Through these interviews, we want to understand what kinds of challenges and barriers grantees are experiencing. We also want to hear about success stories and lessons that could help other grantees. If you or your subrecipients have developed any innovative approaches, please share them.

Information gathered during these interviews will be summarized in the SPF-Rx cross-site evaluation reports. Selected information from this interview may also be used for brief summaries of each grantee’s program as a resource for SAMHSA project officers. As part of the SPF-Rx grant, SAMHSA requires grantees’ participation in the cross-site evaluation, including this interview. Therefore, data are not confidential or anonymous. We will provide you with a copy of the interview notes to review for accuracy.

All grantees will be asked the same questions. As you answer the questions, try to focus on the role and contributions of your SPF-Rx strategies rather than on activities your state (or tribe) was doing before the grant. What is the value added of SPF-Rx?

This interview will take about 90 minutes of your time. In addition, we may contact you after this interview if necessary to clarify responses or to obtain outstanding information.

In preparation for this interview, you responded positively to an email requesting your permission for audio recording. We are audio recording this interview as a backup to our written/typed notes. This recording is only for the use of our evaluation team and will be destroyed after interview notes are finalized. Is it OK if I begin recording now?

Thank you for agreeing to participate in this interview. Do you have any questions for me before we begin?

2. Interview Questions

   1. Infrastructure and Capacity

The PEP-C team and CSAP want to understand the many ways in which the SPF-Rx program may affect prevention capacity and infrastructure at the grantee and community levels.

You can think of infrastructure as the foundation that supports a system, community, or society in achieving desired outcomes.

For SPF-Rx grantees, key infrastructure components include groups such as an advisory body, epidemiological outcomes workgroup (EOW), and evidence-based practice workgroup (EBPW). Infrastructure can also include collaborative relationships, workforce development policies, and monitoring and evaluation systems. The following questions are intended to help us understand processes affecting your capacity and infrastructure for preventing prescription drug misuse.

Organizational infrastructure

1. Now that you’re near the end of the grant, how has the SPF-Rx initiative contributed to your state’s (or tribe’s) planning and decision-making process for prevention of prescription drug misuse? This process might involve your Advisory Council, EOW, and EBPW.

2. How has the operation of your Advisory Council shifted to address SPF-Rx priorities?

3. How has the Epidemiological Outcomes Workgroup contributed to your SPF-Rx efforts?

4. How has the Evidence-Based Practice Workgroup contributed to your SPF-Rx efforts?

2. Prescription Drug Monitoring Program Data

Now I’d like to ask you a few questions related to your use of data from the prescription drug monitoring program, also called PDMP.

[INTRO IF FOLLOW-UP] Since the baseline interview in (2017/2018),

What has changed in your capacity, understanding, or ability to use PDMP data for prevention?

PROBES

1. Have there been any significant changes in the quality of the PDMP data? [IF YES] Please summarize the changes.

2. Have there been any changes in your access or capacity to use PDMP for your prevention efforts?

For example, this could relate to using PDMP data in new ways to monitor prescribing trends, or making more use of unsolicited reports.

• [IF YES] In what ways has your capacity improved at the grantee level?

• [IF YES] How were you able to accomplish these improvements in capacity for using PDMP data?

5. How has your use of PDMP data as part of the SPF-Rx effort affected your prevention efforts at the state (or tribal) level?

5. How have you been facilitating your community subrecipients’ use of PDMP data reports for prevention?

PROBES

1. Did you encounter any major challenges or barriers to using PDMP data reports at the subrecipient community level?

Yes No

• [IF YES] What were the major challenges or barriers?

2. How were you able to address the main challenges or barriers to using PDMP data reports at the community level?

3. Since the start of SPF-Rx, how has capacity improved to use PDMP data at the community level?

5. Do you have any lessons to share for other grantees or communities related to building capacity to use PDMP data for prevention?

5. As part of SPF-Rx, have you been able to do any workforce development for your or your subrecipients’ prevention staff related to PDMP use?

Yes No

[IF YES] Please tell me about some of your workforce development efforts at the grantee level and for community subrecipients.

[IF NO] What were the main challenges or barriers?

3. Collaboration

Now I would like to find out more about the role of your collaborations with other organizations in implementing SPF-Rx.

12. At the grantee level, what progress did you make in your goals in working with the medical and pharmaceutical communities in terms of policy or practice?

PROBES

Please talk about your SPF-Rx’s…

1. …progress in having health care providers use PDMP data when prescribing opioids.

2. …progress in safe prescribing (adhering to CDC guidelines for prescribing opioids for chronic pain).

12. At the grantee level, what have been factors for success in collaborating with the medical and pharmaceutical communities to achieve your SPF-Rx goals?

PROBE

1. Do you have any other promising approaches or innovations to share that might be helpful to other grantees?

[IF R. ASKS WHAT WE MEAN BY INNOVATIONS, WE MEAN PROMISING NEW, ENHANCED OR ADAPTIVE APPROACHES IN YOUR STATE]

12. At the grantee level, what have been challenges or barriers to successful collaboration with the medical and pharmaceutical communities to achieve your goals?

12. Now, turning to your subrecipient communities, please describe some of the successes and challenges or barriers your communities have experienced in collaborating with the medical and pharmaceutical communities to address prescription drug misuse.

PROBES

1. For subrecipients, what have been major factors for successful collaboration with the medical or pharmaceutical communities?

PROBE

• Have you observed any promising approaches that might be helpful for other communities?

2. For subrecipients, what have been major challenges or barriers to successful collaboration with the medical or pharmaceutical communities to achieve their goals?

12. Last, I would like to ask you about your collaboration with other state agencies and partner organizations to achieve SPF-Rx goals.

    1. First, what are some of the ways that you have been successful in advancing collaboration?

PROBE

• What has been helpful to your progress?

2. What are some of the challenges or barriers you have been experiencing?

12. Finally, what highlights or themes can you identify regarding any prominent successes or challenges your subrecipients are encountering in their collaborative partnerships?

1. What are some of the key factors for success or examples of successes that you have observed so far?

2. What are biggest challenges that you hear about?

4. Leveraging Funds and Resources

Leveraging resources describes the process of combining SPF-Rx-funded resources with other resources, including non-SAMHSA funds, to enhance prevention strategy delivery (i.e., to do more together than with SPF-Rx resources alone).

Neither of these definitions includes using SPF-Rx funds to “free up” previously programmed funds for other uses, as this is not permitted under the grant.

18. What strategies did your jurisdiction employ for leveraging prevention funds and resources (e.g., staff) in support of your SPF-Rx efforts?

IF NEEDED, EXAMPLES YOU CAN SHARE WITH THE GRANTEE INCLUDE…

• Building off other prevention funding streams like CDC Prevention for States

• Conducting joint trainings with other agencies on PDMP use and access

• Jointly implementing (or coordinating) interventions with other agencies

• Engaging prevention providers or coordinators from other funding sources

• Collaborating with other agencies to develop reports using PDMP data

18. What strategies did your jurisdiction employ for redirecting or realigning prevention funds and resources in support of your SPF-Rx efforts? Redirecting or realigning resources describes the process of shifting resources (fiscal and nonfiscal) from one focal area to a focal area that complements SPF-Rx efforts.

IF NEEDED, EXAMPLES YOU CAN SHARE WITH THE GRANTEE INCLUDE…

• Redirecting training to high-priority areas for prescription drug (including opioid) misuse or diversion

• Realigning focus from solely individual services to also include population- focused practices for prescription drug misuse or diversion.

18. How has your jurisdiction encouraged the use of local funds to supplement resources provided by SPF-Rx?

IF NEEDED, EXAMPLES YOU CAN SHARE WITH THE GRANTEE INCLUDE…

• Providing training and technical assistance on obtaining local funds

• Encouraging subrecipients to apply for locally available grant funding

5. Interventions

Now I’d like to ask you about your selection and implementation of interventions.

21. How are you selecting your SPF-Rx interventions? PROBES

    1. What was helpful to you in identifying promising or evidence-based interventions for this grant?

2. What challenges have you faced in finding promising or evidence-based interventions to meet the goals of the SPF-Rx grant to prevent prescription drug misuse?

[IF RESPONDENT ASKS: Criteria for a promising or evidence-based intervention might be publication in a scientific journal, inclusion in government lists of evidence-based interventions, review by a panel of informed expertise and key community leaders, and so on.]

21. To what extent have you or your agency been involved with your community subrecipients in identifying promising or evidence-based interventions, policies, or practices for SPF-Rx?

PROBE

1. Please describe any resources or technical assistance available to assist subrecipients in the selection, implementation, and adaptation of promising or evidence-based approaches for the focal areas of SPF-Rx.

21. How is SPF-Rx contributing to your progress in planning or implementing any legal or policy changes to reduce prescription drug and opioid misuse? For example, legal or policy changes could relate to PDMP use, opioid prescribing practices, or health care provider training.

What is the value added of SPF-Rx beyond what the state/tribe was already doing?

PROBES

1. What factors have been most helpful to you in making progress in planning or implementing policy or legal changes?

2. Do you have any other lessons to share that might be helpful to other grantees in terms of policy or legal change?

21. What is the unique contribution of SPF-Rx to your state’s (or tribe’s) social marketing campaigns to prevent prescription drug and opioid misuse?

6. Monitoring, Performance Measurement, and Evaluation

Now I’d like to turn briefly to the subject of performance measurement.

25. Have you been using PDMP data to monitor the progress of your SPF-Rx efforts? Yes No [IF NO, SKIP a. and b.]

[PROBE FOR ALL] What challenges or barriers have you faced in trying to use PDMP data for monitoring and performance measurement?

1. [IF YES] How have you been using the PDMP data for monitoring and performance measurement?

2. [IF YES] Were you able to combine PDMP data with other data, such as opioid overdose morbidity or mortality data?

Yes No

[IF YES, PROBE]:

• Please discuss what capabilities you developed in this area. For example, did you conduct geographic analysis at the aggregate level or merge individual-level data for analysis?

• How did you use these combined data?

1. As part of your SPF-Rx effort, has your (state/tribe) developed any promising innovative practices using PDMP data for monitoring that might be helpful to other grantees?

7. Concluding Questions

Now, I have a couple of last questions to wrap up the interview.

26. In your view, what is the most important thing that SPF-Rx is helping your state (or tribe) to do above and beyond your other funding resources? What is the unique contribution of SPF-Rx?

1. What aspects of your SPF-Rx effort and infrastructure do you think will be most sustainable after SPF-Rx funding ends?

[IF NEEDED, EXAMPLES YOU CAN SHARE WITH GRANTEE INCLUDE…]

• Regular access to PDMP data for prevention

• Workforce development (e.g., prevention staff, medical community)

• Changes in laws or policies

• Ongoing collaborations (e.g., other state agencies, local communities, medical community)

26. As you are nearing the end of your SPF-Rx grant, what are the most critical gaps that remain in your state’s (or tribe’s) efforts to address prevention of prescription drug, including opioid, misuse? What would be your next priorities?

CLOSING

Thank you for talking with me today. We appreciate your sharing your experiences.

If you have any questions or concerns about this interview, please contact the PEP-C Help Desk through the Management Reporting Tool, by telephone message (at 877-439-1211), or by email (to [email protected]). You can request assistance at any time and someone will respond to you on the next business day.

Do you have any final questions or concerns today? Thanks, again, for your time.

Abt Associates Inc. November 2019, Page 5