Prescription Drug Overdose Prevention for States Program Assessment

Attachment C –Telephone Interview Guide

Form Approved

OMB No. 0920 0879

Expiration Date 03/31/2018

TELEPHONE INTERVIEW GUIDE

KEY:

• Italicized text indicates scripted information the interviewer can read (as appropriate) to facilitate the interview flow.

• CAPITALIZED BOLD TEXT indicates an interviewer instruction.

• Yellow highlighted text indicates segments of information that should be identified in awardee documents and used to prepare questions.

• ADR indicates that the question may potentially be in awardee documents. ADR suggests that the question could be dropped (if comprehensively answered in the awardee documents) or that the question may need to be revised to obtain more detail than what the awardee documents provide, which will be reviewed prior to the telephone interview.

• Turquoise highlighted text indicates that the interviewer will need to use the key informant’s response during the interview to phrase the question.

CORE QUESTIONS: Asked during all key informant interviews

INTERVIEWER: I’d like to record the interview for transcription purposes. Your name will not be used. Do I have your permission to begin recording? Before beginning the interview, I’d also like to thank you for your participation. The data collected in this interview will be used by the CDC to tailor technical assistance to states to specifically address their needs; obtain a valid program assessment about how states operationalized and implemented their activities; ensure a full understanding of the barriers and facilitators to PfS implementation and achieving outcomes; and understand and disseminate which PfS activities are most effective in achieving short-and intermediate-term goals.

CDC estimates the average public reporting burden for this collection of information as 120 minutes per response, including the time for reviewing instructions, searching existing data/information sources, gathering and maintaining the data/information needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing burden to CDC/ATSDR Information Collection Review Office, 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA (0920-0879).

To begin the interview, I’d like to start with general questions about your role in the PfS program.

Interviewer note: If you have more than one informant on the call. Be certain to ask the roles of each individual on the phone. You can adapt the text to indicate your intention to do so. “Thank you for your consent. I’d like to start with general questions about your role in the PfS program. Let’s start with [PERSON 1].”

1. We know that you are [LIST JOB TITLE] at [AWARDEE ORGANIZATION]. Could you start by telling us a little more detail about your role in the organization?

2. What was/is your role in the implementation of the PfS program?

INTERVIEWER: Now, I’d like to ask about your state and program more generally.

3. Prior to the PfS, what gaps or needs existed in terms of your state’s ability to address the opioid overdose epidemic?

• 3a: Does the PfS address these gaps or needs? If so, how?

4. ADR Do your PfS activities address needs in your whole state or in particular regions? If regions, which regions? How were those selected?

Interviewer note: For Q5, the interviewer should review the State Activities Inventory to determine which strategies the awardee works on and adjust the question accordingly. Further, the interviewer should ask about each strategy individually to avoid a double-barreled question.

Be certain to cover all probes in Q5; they are important.

INTERVIEWER: The context in which people work can have an impact on how programs are put into place. In these interviews we are trying to understand some of the external factors that have hindered or helped your work. These factors can occur at any level—the national, state, or local levels.

5. Thinking about all your PfS strategies and activities, are there any major external factors that have affected- positively or negatively- your ability to implement your PfS strategies or activities? INTERVIEWER PROBE FOR EACH LEVEL: FEDERAL, STATE, OR LOCAL. ALSO, PROBE FOR POLICIES THAT MAY HAVE HAD AN IMPACT.

   • 5a: What are they? How did they help or hinder your implementation?

   • 5b: Which strategies or activities did these [FACTORS] affect?

   • 5c: How did you manage challenges?

   • 5d: Which individuals or organizations are most actively engaged in implementation of your PfS strategies? [INTERVIEWER: CLARIFY WHETHER PARTNER IS INTERNAL OR EXTERNAL IF NOT CLEAR IN RESPONSE. WE ALSO WANT THE TOP 1-3 PARTNERS.]

• 5d1: How is the [PARTNER] you mentioned involved in implementation?

• 5d2: What successes have you encountered in working with [PARTNER]? What if anything helped to facilitate those successes?

• 5d3: Have you encountered any barriers with [PARTNER] to date? Could you tell me about that? How did you deal with it?

• 5e: What non-PfS activities are you aware of in your state related to addressing the opioid overdose epidemic? [INTERVIEWER: THIS MAY BE COVERED IN Q3. YOU CAN FOLLOW-UP ON THE PROBES ON INTERVENTIONS, REGULATORY ACTION, COALITIONS, DIRECT SERVICE ORGANIZATIONS, OTHER STATE AGENCY ACTIVITIES. AS TIME ALLOWS. PROBE FOR DURATION OF ACTIVITIES (i.e., how long have these been in place?).

• 5f: Some states have higher-level government entities, such as task forces or advisory councils and strategic plans, that help guide and coordinate work related to addressing the opioid overdose epidemic from a state- or county-level perspective. Does your state have a higher-level government entity that coordinates the opioid overdose epidemic work from a state level? If so, could you describe it for me? [INTERVIEWER: PROBE FOR MEMBERSHIP, ROLES, AND ACTIONS TO DATE.]

• 5f1: Does your state have a strategic plan for addressing the opioid overdose epidemic? If so, could you describe, in general, the components of the plan?

• 5f2: Do any counties involved in the PfS have a higher-level government entity like this? Could you describe them?

• 5f3: Do any counties involved in the PfS have a strategic plan? If so, could you describe, in general, the components of the plan?

6. Since 2015 [or start of funding as appropriate] your state has engaged in [list programs/practices]. To what extent were these activities prompted by research or best practices observed in other states?

7. Is identification and response to overdose clusters part of your state’s PfS work? When an overdose cluster is identified, what is the time between identification and implementation of a response?

INTERVIEWER: We’ve talked about work going on in the state; now, I’d like to shift to your organization and would like to start at the top of the organizational chart with your organization’s leadership. Leadership is defined as the person(s) the Principal Investigator or Project Manager of PfS reports to. They should be in a position of authority and have the power to make resource allocation decisions. Examples of this person may be the State Health Director for the Department of Health or the Attorney General for the Department of Criminal Justice.

8. How involved are your organization’s leadership in the PfS work? What does the leader do for the PfS (e.g., providing in-kind support or FTEs (staff), talking with state decision makers, etc.)? Could you give an example of that?

9. Are there any organizational factors (e.g., administrative processes, organizational structures, staffing) that affect your ability—positively or negatively—to implement your PfS activities? What are they?

Optional Module 1 for Strategy 1/Activity 1: Expand and improve proactive reporting

Documents awardees may consider sharing: screenshots of internal notifications, screenshots of external notifications

Additional documents to review prior to interview: Annual progress reports, Pew Report on best practices, PDMP Quick Reference Summary Table

Interviewer note: Although we provide a general script below, please revise the scripted information for each awardee based on document review to demonstrate that you have read the materials they provided. For example, “We’d like to learn more about Proactive PDMP Data Analysis and Reporting. Thank you for sending examples of your screenshots for internal and external notifications. For your internal notifications, we noticed [DESCRIBE CONTENT]; we have a few other questions related to how you generated parameters and how stakeholders use that content. For your external notifications, we noticed [DESCRIBE CONTENT]; we have a few other questions related to how you generated parameters and how stakeholders use that content.”

INTERVIEWER: In this module we’d like to learn more about Proactive PDMP Data Analysis and Reporting; this is commonly called proactive reporting. Within proactive reporting, we are interested in internal PDMP notifications and external PDMP notifications. To make sure we are on the same page, I’ll briefly summarize what I mean by internal and external notifications. Internal PDMP Notifications occur within the PDMP. The intended recipient or entity of the notification must log in to the PDMP to receive the actual report. An example includes notifications that “pop-up” when a prescriber logs into the PDMP. External PDMP Notifications occur outside of the PDMP. No action by intended recipient is necessary to receive the report. These include reports/alerts sent to prescribers, dispensers, others via mail, fax, and email. I’ll start with a general question about your work, and then we’ll delve into some specifics related to internal and external notifications.

Now, I’d like to focus on internal notifications.

1. ADR Does your PDMP include internal notifications—also known as alerts (i.e., sent to prescribers within the PDMP system)?

• ADR 1a: How are these notifications presented in the PDMP (e.g., message box, red flag, etc.)?

2. ADR What are the parameters used to generate an internal notification (e.g., multiple prescribers within a given time frame, >90 MME, GIS mapping of pharmacies and prescriber clinics)?s

• 2a: How did you decide on the parameters?

• 2b: Are different internal notifications generated for different stakeholders (e.g., providers, prescribers, law enforcement, licensing boards)? If so, how do the reports differ?

• 2c: Who is able to receive internal notifications (e.g., types of providers, prescribers, law enforcement, licensing boards)?

3. What stakeholder groups receive internal notifications? [INTERVIEWER: IF INFORMANT RECEIVES Q3 WITH THE PROBE ON THE NOTIFICATIONS FOR DIFFERENT STAKEHOLDERS, THEN ASK, “A moment ago you mentioned [USER GROUPS] as receiving notifications. Were there any others that we did not discuss?” ALSO, FOR THIS QUESTION, PROBE FOR HOW INFORMATION IS USED BY EACH STAKEHOLDER GROUP IDENTIFIED.]

   • 3a: What is the protocol /course of action for a [STAKEHOLDER (i.e., PRESCRIBER, DISPENSER, LAW ENFORCEMENT)] when they receive a report about a [PATIENT, PRESCRIBER]?

• 3b: Can users within your PDMP contact one another (e.g., if doctors who share a patient can communicate with one another via email internal to the system)? Can you provide some examples of this?

• 3c: How has usage by different stakeholders been assessed?

4. How do you get feedback from people who have access to internal notifications?

• 4a: What kind of feedback have you received to date? What do people like the best? What do they like the least?

• 4b: How do you use that information?

INTERVIEWER: Now, I’d like to cover your external notifications.

5. ADR Does your PDMP include external notifications (i.e., reports sent to prescribers outside of the PDMP system)?

6. ADR What parameters are used to generate an external notification (e.g., multiple prescribers within a given time frame, >90 MME, top 10% of high prescribers by type receive a report, GIS mapping of pharmacies and prescriber clinics)?

• 6a: How did you decide on the parameters?

• 6b: Are different external notifications generated for different stakeholders? If so, how do the reports differ?

• 6c: Who receives external notifications (e.g., types of providers, prescribers, law enforcement, licensing boards)?

• 6d: Who can send external notifications (e.g., types of providers, prescribers, law enforcement, licensing boards)?

• 6e: How do your recipients receive the notification (e.g., e-mail, dashboard)?

7. What stakeholder groups receive external notifications? [INTERVIEWER: IF INFORMANT RECEIVES Q7 WITH THE PROBE ON THE NOTIFICATIONS FOR DIFFERENT USER GROUPS, THEN ASK, “A moment ago you mentioned [USER GROUPS] as receiving external notifications. Were there any others that that we did not discuss?” ALSO, FOR THIS QUESTION, PROBE FOR HOW INFORMATION IS USED BY EACH STAKEHOLDER GROUP IDENTIFIED.]

   • 7a: What is the protocol /course of action for a [STAKEHOLDER (i.e., PRESCRIBER, DISPENSER, LAW ENFORCEMENT)] when they receive a report about a [PATIENT, PRESCRIBER]?

• 7b: How has usage by different stakeholders been assessed?

8. How do you get feedback from people who have access to external notifications?

• 8a: What kind of feedback have you received to date? What do people like the best? What do they like the least?

• 8b: How do you use that information?

INTERVIEWER: Some states share the notifications with various state licensing boards. The licensing boards include boards for physicians, nurses, dentists, and pharmacists.

9. Do you share information with state licensing boards? If so, which ones? If not, why not? [INTERVIEWER: ONLY ASK THIS QUESTION IF LICENSING BOARDS DID NOT COME UP IN PREVIOUS QUESTIONS, Q3, Q4, Q7, Q8]

10. How do you share information from [INTERNAL AND/OR EXTERNAL] notifications with your state licensing boards?

• 10a: What information is shared with the licensing board (e.g., overprescribing provider name, patient information)? Does the information shared vary by board? If so, how?

• 10b: Does the licensing board receive copies of each notification? Do they receive annual reports?

• 10c: What does the licensing boards do with the information you provide? How have you assessed that? Does this vary by board? If so, how?

11. What are some of the lessons learned with regards to this activity?

• 11a: What are some of the successes you would like to highlight?

• 11b: What barriers have you encountered? How have you dealt with them?

12. Is there anything else that you’d like to add about your state’s PDMP reporting or alert/notification system that we didn’t talk about?

Optional Module 2 for Strategy 1/Activity 2: Conduct public health surveillance with PDMP data and publicly disseminate reports

Documents awardees may consider sharing: Links to dashboards, examples of surveillance reports (e.g., reports by state epidemiological outcomes workgroups who do the State Epidemiological Profiles), scorecards (VT and CT), specs for dashboard in development (NV only)

Additional documents to review prior to interview: Annual progress reports, review state websites with data dashboards (NM, VA, RI, WA, MA, CA, OR, CO)

INTERVIEWER: Now, I’d like to talk about your work conducting public health surveillance with PDMP data and disseminating reports.

1. ADR How are you using PfS resources for this activity (conducting public health surveillance with PDMP data and publicly disseminating reports)?

2. ADR With respect to your PfS work, what types of PDMP surveillance activities are supported (e.g., data briefs, dashboards)? [INTERVIEWER: THE PRODUCTS OF THIS ACTIVITY ARE BROADER THAN REPORTS, BE PREPARED TO ADAPT THIS QUESTION AS NEEDED TO ACCOMMODATE THE PRODUCT. IF WE DO NOT HAVE A SAMPLE SURVEILLANCE PRODUCT FOR THIS ACTIVITY, ASK:] What is covered in these [TYPE MENTIONED IN MAIN QUESTION—data brief, report type, etc.] (i.e., content)? Could you describe a typical report?

• ADR 2a: What other datasets do you use, in combination with PDMP data, for publicly disseminated reports (e.g., National Violent Death Reporting System [NVDRS] data, Hospital Inpatient Discharge Data [HIDD], Emergency Department [ED] data)?

• 2b: From whom did you get input in developing these [TYPE MENTIONED IN MAIN QUESTION—data brief, report type, etc.]? Did your state epidemiological workgroup provide input on these reports? If so, how?

• 2c: Who are the intended audiences of the [TYPE MENTIONED IN MAIN QUESTION—data brief, report type, etc.] (e.g., Local health departments, law enforcement, providers, media, coalitions, etc.)?

• 2d: Does the [TYPE MENTIONED IN MAIN QUESTION—data brief, report type, etc.] content vary by intended audience? If so, how are the reports tailored for different audiences?

• 2e: For each audience identified, how regularly are the [TYPE MENTIONED IN MAIN QUESTION—data brief, report type, etc.] provided?

• 2f: For each audience identified, how are [TYPE MENTIONED IN MAIN QUESTION—data brief, report type, etc.] disseminated?

• 2g: [INTERVIEWER: PROBE FOR EACH AUDIENCE/INTENDED USER] How do [AUDIENCES] use the [TYPE MENTIONED IN MAIN QUESTION—data brief, report type, etc.]? How are you assessing this?

3. ADR Do you have a data dashboard? I’d like to ask a few questions about it. How was the dashboard developed? [INTERVIEWER: NOTE ABOVE WHICH STATES HAVE DASHBOARDS. ONLY STATES WITH DASHBOARDS SHOULD RECEIVE THIS QUESTION]

• 3a: Who provided input on the development?

• ADR 3b: What is the content of the data dashboard?

• ADR 3c: Where is the dashboard housed (e.g., web-based, within an EHR)?

• 3d: How often are data updated? How are end users notified about updates?

• 3e: Are there any data linkages (e.g., hospital discharges, emergency departments)? What are they?

• 3f: Who are your intended users?

• 3g: How often does the typical end user log in? What types of information do they seek?

• 3h: Has the dashboard evolved since receiving PfS resources? If so, how?

4. Has dissemination of public health surveillance products (e.g., reports, briefs, etc.) changed as a result of your work on PfS? If so, how?

5. Some PfS states provide technical assistance (TA) on the reports or other products they generate with PDMP data. Does your state offer that? Could you tell me about the TA you provide on this?

   • 5a: Who provides the TA? To whom is it provided? [INTERVIEWER: PROBE FOR EACH AUDIENCE TYPE AND WHETHER FREQUENCY, MODES, OR METHODS VARIES BY AUDIENCE TYPE]

   • 5b: How frequently is it provided? [INTERVIEWER: IF MULTIPLE TA PROVIDERS ARE INVOLVED, PROBE TO DETERMINE WHICH TA PROVIDER PROVIDES EACH MODE AND WITH WHAT FREQUENCY.]

   • 5c: What modes have been used? (e.g., webinars, peer-to-peer learning/learning collaboratives, call centers)

   • 5d: What have [AUDIENCES] needed the most help with understanding these reports? How did you ascertain that?

   • 5e: How has that TA enabled [AUDIENCES] to make sense of the data? How have you assessed that?

6. [INTERVIEWER: IF STATE HAS DATA DASHBOARD, ASK]: Some PfS states provide TA on using their data dashboard. Does your state offer that? Could you tell me about the TA you provide on your data dashboard?

   • 6a: Who provides the TA? Who receives it? [INTERVIEWER: PROBE FOR EACH AUDIENCE TYPE AND WHETHER FREQUENCY, MODES, OR METHODS VARIES BY AUDIENCE TYPE]

   • 6b: How frequently is it provided? [INTERVIEWER: IF MULTIPLE TA PROVIDERS ARE INVOLVED, PROBE TO DETERMINE WHICH TA PROVIDER PROVIDES EACH MODE AND WITH WHAT FREQUENCY.]

   • 6c: What modes have been used? (e.g., webinars, peer-to-peer learning/learning collaboratives, call centers)

   • 6d: What have [AUDIENCES] needed the most help with on understanding the dashboard? How were you able to make this determination?

   • 6e: How has that TA enabled [AUDIENCES] to make sense of the data? How have you assessed that?

   • 6f: How has the TA enabled [AUDIENCES] to select appropriate interventions? How have you assessed that?

7. What are some of the lessons learned with regards to this activity?

• 7a: What are some of the successes you would like to highlight?

• 7b: What barriers have you encountered? How have you dealt with them?

8. Is there anything else that you’d like to add about your work on conducting public health surveillance with PDMP data and publicly disseminating reports that we didn’t talk about?

Optional Module 3 for Strategy 2: Identify and provide technical assistance to high-burden communities and counties, especially efforts to address problematic prescribing

Documents awardees may consider sharing: TA plans (if developed), list of TA types/events (if developed), funding announcement/request for proposals for external TA providers

Additional documents to review prior to interview: APRs (partnership section, Cohort 1)

Interviewer note: Awardees vary on how they operationalize TA for this PfS activity. For example, some have a TA call center to answer questions, and others do this via coalitions. Be prepared to adapt these questions to accommodate how the awardee operationalizes TA.

INTERVIEWER: Now, I’d like to shift gears a little to talk about the Strategy 2 activity, identifying and providing technical assistance to high-burden communities and counties. We know that awardees have different models of TA provision and hope to learn more about each of the models.

1. How are you using PfS resources for this activity (providing TA to high burden communities)?

2. ADR How is TA provided to high burden communities/counties? [INTERVIEWER: PROBE FOR BOTH COMMUNITIES/COUNTIES AND WHICH PEOPLE/ORGANIZATIONS IN COMMUNITIES/COUNTIES RECEIVE PARTICULAR TA TYPES AND TOPICS. ALSO, BE SURE TO DETERMINE WHETHER AUDIENCE/REGION DIFFERS FOR TOPICS, MODES, TIMING]

   • 2a: What topics have been covered so far (e.g., addressing problematic prescribing)?

   • 2b: How did you select these topics for TA?

   • 2c: Who provides the TA? To whom is it provided?

   • 2d: What topics are planned for the future?

   • 2e: What modes have been used? (e.g., webinars, peer-to-peer learning/learning collaboratives, call centers) [INTERVIEWER: IF MULTIPLE TA PROVIDERS ARE INVOLVED, PROBE TO DETERMINE WHICH TA PROVIDER PROVIDES EACH MODE AND WITH WHAT FREQUENCY.]

   • 2f: How frequently is TA provided?

   • 2g: How have [AUDIENCES] used the TA? How have you assessed that?

3. [INTERVIEWER: ASK THIS QUESTION IF THE KEY INFORMANT MENTIONS OTHER TA PROVIDERS OUTSIDE THE AWARDEE ORGANIZATION] A moment ago, you mentioned [ORGANIZATION(S)] was/were providing TA to high burden communities. How did you select this/these organization(s) to provide TA?

   • 3a: How do you coordinate with these organizations on TA provision?

4. Some awardees work with a coalition, multi-sector group, or advisory group to provide TA to high burden communities. Are you working with a group like this? If so, could you tell me about your [COALITION, MULTI-SECTOR GROUP, OR ADVISORY GROUP]?

   • 4a: Does the state provide resources for this group? If no, who does provide resources?

   • 4b: How did this [COALITION, MULTI-SECTOR GROUP, OR ADVISORY GROUP] form? When did it form?

   • 4c: Was there a certain model or another coalition that you know of that guided your coalition’s work (e.g., “collective impact model”, coalition on another public health issue)?

   • 4d: What organizations and sectors does your coalition comprise? What does each organization do?

   • 4e: What is your shared vision for addressing the opioid overdose epidemic? What are the coalition’s goals? How is this documented (e.g., strategic plan, document outlining shared vision, organization, and initiatives)?

   • 4f: What are the major initiatives of the [COALITION, MULTI-SECTOR GROUP, OR ADVISORY GROUP]?

   • 4g: How does the [COALITION, MULTI-SECTOR GROUP, OR ADVISORY GROUP] plan to engage organizations in the local health system around reducing opioid misuse? What roles will different organizations on the coalition play in engaging representation from the local health system?

5. How are you evaluating the TA provided?

   • 5a: What measures are you using?

   • 5b: How have you obtained feedback from [NAME EACH AUDIENCE MENTIONED IN Q2]?

   • 5c: How have you used that feedback?

   • 5d: Which topic was most helpful topic for each audience type? What makes these topics helpful?

   • 5e: Which topic was least helpful topic for each audience type? What makes these topics less helpful?

   • 5f: What do you see as potential gaps in the TA provided so far? How will you address that gap?

6. What are some of the lessons learned with regard to this activity?

• 6a: What are some of the successes you would like to highlight?

• 6b: What barriers have you encountered? How have you dealt with them?

6. Is there anything else that you’d like to add about your work on identifying and providing technical assistance to high-burden communities and counties that we didn’t talk about?

INTERVIEWER: If you have examples of TA plans, list of TA types/events and funding announcement/request for proposals for external TA providers, we would love to see them if you can and are willing to share. 

CORE QUESTIONS: Asked during all key informant interviews

INTERVIEWER: Thank you for taking the time to share your experiences today. I have only a few more questions to round out the conversation.

1. During your PfS experience, have there been any unexpected successes or benefits? What were they?

• 1a: What strategies (or major activities) were involved?

2. Did you encounter any unexpected issues or unanticipated challenges? What were they?

• 2a: What strategies (or major activities) did those issues affect?

• 2b: How were they addressed or overcome?

• 2c: Did you expect to encounter any challenges that did not materialize?

3. During the interview, we only had time to focus on a few of your activities. Thinking across your PfS activities, are there other accomplishments or lessons learned that you think are important to share? What are they?

4. Is there anything you wanted to discuss that did not come up during the interview?

Thank you for your time!