Form 1 Environmental Scan Semi-Structured Interview Guide

Attachment A

Environmental Scan

Semi-Structured Interview Guide

Form Approved

OMB Control Number: 0935-XXXX

Expiration Date MM/DD/YYYY

Thank you for taking the time to talk with me. As part of this project, we would like to learn about the methods your organization uses to evaluate your communication with and support for patients and their families. Specifically, we are interested in understanding the ways you assess how you are doing in five areas. Those are:

1. improving communication with patients;

2. making it easier for patients to navigate your organization’s physical environment and health care system;

3. improving patient engagement and self-management;

4. implementing organizational policies and structures to enhance patient understanding of health information, navigation, engagement, and self-management; and

5. engaging organizational leaders in establishing an organizational culture committed to helping patients in these ways.

As we talk, please keep in mind that there are no “right” or “wrong” answers to any of the questions I will ask you. We are simply interested in learning about how you measure your progress in these areas. When I ask about the measures you use, I am interested in both measures you are using right now and measures that you no longer use. Don’t worry if you are not formally evaluating your progress in all five of the areas I mentioned. That’s not a problem.

Before we begin, I want to mention two important things about the wording we will use today. First, although we will refer to patients throughout our conversation, we are interested in hearing about your efforts to help both patients and their families. Second, when we use the word “measure,” we are referring to a variable that can be computed to assess how your organization is doing in a given area. For example, you might compute the percentage of clinical encounters during which teach-back was used or the average reading level of the educational materials you provide to patients. Although we are interested in learning about the measures you use and how you compute them, we would also like to hear about other methods you use to assess your progress.

Public reporting burden for this collection of information is estimated to average 120 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to:

AHRQ Reports Clearance Officer

Attention: PRA, Paperwork Reduction Project (0935-XXXX)

AHRQ

5600 Fishers Lane, # 07W41A

Rockville, MD 20857.

Interviewer: Throughout the interview, probes may be refined to be appropriate to the discussion of measures or other methods of assessment. For instance, whereas questions about specific computational methods (e.g., numerator) may be relevant to computed variables, they may not be appropriate for some other methods of assessment described by interview participants.

This interview should take no more than 2 hours. We won’t share your responses outside of our research team. Although we may report your experiences in presentations and publications, we will not publicly attribute any of your responses to you without your permission. I will be recording this interview, so that I won’t miss anything you say. After our meeting today, we will transcribe the recording of our conversation. However, no names or identifying information will be included in the transcript. Can I confirm that you are okay talking with me about these topics today and having our conversation recorded? (Wait for verbal consent.)

Interviewer: By checking this box  the person consents to this interview.

Great. I will now turn on the digital recorders as we begin the interview. Are you ready?

____________________________________________________________________________

First, I would like to ask you some questions about any methods you use to assess how well your organization is doing with regard to communicating clearly with patients and their families, including those from diverse cultural and educational backgrounds. When I refer to communication, I am speaking very broadly. For example, communication can include verbal communication, written communication, electronic communication (e.g., website content, emails, texts), or any other sort of communication between the organization and its staff and patients. Likewise, your efforts related to these types of communication could focus on improving communication with English-speaking patients or patients with limited English proficiency.

1. Do you assess how well your organization does at communicating clearly with patients and their families, including those from diverse cultural and educational backgrounds?

   1. If no, probe to confirm that they don’t assess any of the types of communication noted: Is that true with regard to each of the types of communication I mentioned before, for instance verbal, written, and electronic? How do you know if you’re doing a good job communicating with patients? Skip to Question 10.

   2. If yes, continue to Question 2.

2. Are there specific measures or other assessment methods you use to assess how well you are communicating with patients and their families, including those from diverse cultural and educational backgrounds?

   1. If no, skip to Question 10.

   2. If yes, probe for details: What measures have you used?

Interviewer: Questions 3-9 should be asked for each measure or other assessment method identified.

Now, I would like to ask you for specific information about each measure or other assessment method you have mentioned. Let’s start with [measure/method]: _________________.

3. Can you tell me exactly how the measure is calculated? (Probe for details: What data are used? What specific data items or elements are used? How are these data collected? How are the numerator and denominator defined? Are there any exclusions made when calculating the measure, for example, omitting patients who do not have a specific diagnosis? Do you have written documentation you can share that would provide us with specific details about computation of this measure?)

4. How do you use the measure to assess your communication with patients? (Probe for details: How often do you compute the measure? Is it possible to compute the measure often enough to be useful for quality improvement work? How do you review the measure? Do you check for disparities between different groups of patients? Do you check for disparities between different groups of patients? Does the measure get reported? If reported: To whom? How does that person use the measure? Is there any accountability for achieving certain measurable changes?)

5. Have you examined the relationship between change on this measure and other things, like changes in processes of care, patient experience, or patient outcomes? (Probe for details: Have you seen any important relationships? Can you tell me about that? Have there been any unintended consequences of implementing this measure? Have any of these findings been documented? Can we obtain a copy of any written documentation?)

6. Has this measure been tested to ensure that it measures what it is meant to measure and that it does so consistently? (Probe for details: Did you do the testing? Can you tell me what testing has been done? In what settings and with what populations has the measure been tested? Do you think the measure could be used in other settings? What have the results of the testing been? Can we have a copy of any written description of these findings?)

7. Do you have any evidence that the measure is considered important by your clinicians and staff? (Probe for details: If yes, what evidence do you have? If not, are clinicians and staff unsupportive of this measure? Can we have a copy of any written description of these findings?)

8. Do you have any evidence that the aspect of care delivery that is being measured is considered important by your patients? (Probe for details: Can you tell me about that? Can we have a copy of any written description these findings?)

9. Has this measure been difficult or expensive to compute or to use to guide efforts improve your organization’s communications with patients? (Probe for details: Why is that?)

Okay, great. Thank you. Now, I would like to ask you some questions about any methods you use to assess how well patients are able to navigate your system. When I refer to navigation, I am thinking of a wide range of things, including patients being able to find their way around the physical environment of a health care organization and patients being able to manage tasks like making appointments, following through on referrals, getting health insurance coverage, and making successful care transitions (both within and across organizations).

10. Do you assess how easy it is for patients and their families to navigate your facilities and your health care system?

1. If no, probe to confirm that they don’t assess navigation of the physical environment or the health care system: Is that true with regard to both of the types of navigation I mentioned before (i.e., navigation of the physical environment and the health care system)? How do you know that patients are getting where they need to go, get timely care, and don’t fall between the cracks? How do you know that they are accessing services and managing transitions successfully? Skip to Question 19.

2. If yes, continue to Question 11.

11. Are there specific measures or other assessment methods you use to assess how easy it is for patients and their families to navigate your facilities and your health care system? (If no, probe for details: Are there other ways you have been trying to assess patient navigation? If yes, probe for details: What measures have you used?)

Interviewer: Questions 12-18 should be asked for each measure or other assessment method identified.

Now, I would like to ask you for specific information about each measure or other assessment method you have mentioned. Let’s start with [measure/method]: _________________.

12. Can you tell me exactly how the measure is calculated? (Probe for details: What data are used? What specific data items or elements are used? How are these data collected? How are the numerator and denominator defined? Are there any exclusions made when calculating the measure, for example, omitting patients who do not have a specific diagnosis? Do you have written documentation you can share that would provide us with specific details about computation of this measure?)

13. How do you use the measure to assess how easy it is for patients to navigate your facilities and your system? (Probe for details: How often do you compute the measure? Is it possible to compute the measure often enough to be useful for quality improvement work? How do you review the measure? Do you check for disparities between different groups of patients? Does the measure get reported? If reported: To whom? How does that person use the measure? Is there any accountability for achieving certain measurable changes?)

14. Have you examined the relationship between change on this measure and other things, like changes in processes of care, patient experience, or patient outcomes? (Probe for details: Have you seen any important relationships? Can you tell me about that? Have there been any unintended consequences of implementing this measure? Have any of these findings been documented? Can we obtain a copy of any written documentation?)

15. Has this measure been tested to ensure that it measures what it is meant to measure and that it does so consistently? (Probe for details: Did you do the testing? Can you tell me what testing has been done? In what settings and with what populations has the measure been tested? Do you think the measure could be used in other settings? What have the results of the testing been? Can we have a copy of any written description of these findings?)

16. Do you have any evidence that the measure is considered important by your clinicians and staff? (Probe for details: If yes, what evidence do you have? If not, are clinicians and staff unsupportive of this measure? Can we have a copy of any written description of these findings?)

17. Do you have any evidence that the aspect of care delivery that is being measured is considered important by your patients? (Probe for details: Can you tell me about that? Can we have a copy of any written description of these findings?)

18. Has this measure been difficult or expensive to compute or to use to guide efforts to improve patient navigation? (Probe for details: Why is that?)

Perfect. Thank you. Now, I would like to ask you some questions about any methods you use to assess how well your organization is doing with regard to encouraging patient engagement and helping patients to manage their health successfully. When I talk about supporting engagement and self-management, I am thinking about activities that involve patients as active participants in health care decision making, provide patients with the guidance they need to prevent illness, maintain their functioning, manage their health conditions, and improve their health, and connect patients to community resources that address their non-medical needs (e.g., transportation, housing low-cost medications, adult basic education programs).

19. Do you assess how well your organization engages patients and supports patient self-management?

    1. If no, probe to confirm that they don’t assess the organization’s approach to engagement or self-management: Is that true with regard to both engagement and self-management support? How do you know if you’re doing a good job of encouraging patient and family engagement? How do you know if you are doing a good job of supporting patient self-management? Skip to Question 28.

    2. If yes, continue to Question 20.

20. Are there specific measures or other assessment methods you use to assess how well your organization supports patient engagement and self-management? (If no, probe for details: Are there other ways you have been trying to assess these things? If yes, probe for details: What measures have you used?)

Interviewer: Questions 21-27 should be asked for each measure or other assessment method identified.

Now, I would like to ask you for specific information about each measure or other assessment method you have mentioned. Let’s start with [measure/method]: _________________.

21. Can you tell me exactly how the measure is calculated? (Probe for details: What data are used? What specific data items or elements are used? How are the data collected? How are the numerator and denominator defined? Are there any exclusions made when calculating the measure, for example, omitting patients who do not have a specific diagnosis? Do you have written documentation you can share that would provide us with specific details about computation of this measure?)

22. How do you use the measure to assess how well you are supporting patient engagement and self-management? (Probe for details: How often do you compute the measure? Is it possible to compute the measure often enough to be useful for quality improvement work? How do you review the measure? Do you check for disparities between different groups of patients? Does the measure get reported? If reported: To whom? How does that person use the measure? Is there any accountability for achieving certain measurable changes?)

23. Have you examined the relationship between change on this measure and other things, like changes in processes of care, patient experience, or patient outcomes? (Probe for details: Have you seen any important relationships? Can you tell me about that? Have there been any unintended consequences of implementing this measure? Have any of these findings been documented? Can we obtain a copy of any written documentation?)

24. Has this measure been tested to ensure that it measures what it is meant to measure and that it does so consistently? (Probe for details: Did you do the testing? Can you tell me what testing has been done? In what settings and with what populations has the measure been tested? Do you think it could be used in other settings? What have the results of the testing been? Can we have a copy of any written description of these findings?)

25. Do you have any evidence that the measure is considered important by your clinicians and staff? (Probe for details: If yes, what evidence do you have? If not, are clinicians and staff unsupportive of this measure? Can we have a copy of any written description of these findings?)

26. Do you have any evidence that the aspect of care delivery that is being measured is considered important by your patients? (Probe for details: Can you tell me about that? Can we have a copy of any written description of these findings?)

27. Has this measure been difficult or expensive to compute or to use to guide efforts to improve patient engagement and self-management? (Probe for details: Why is that?)

OK, great. Now, I would like to ask you some questions about any methods you use to evaluate organizational policies and structures that may influence patient communication, navigation, engagement, and/or self-care. For example, you might examine whether there are policies related to the understandability of written patient materials or an office of patient education that ensures that written patient materials are understandable.

28. Do you assess whether your organizational policies and structures are set up in a way that is likely to enhance patient communication, navigation, engagement, and self-management?

    1. If no, probe to confirm that they don’t assess their policies or structures: Is that true with regard to policies and organizational structures? How do you know if your policies and organizational structures are likely to enhance patient communication, navigation, engagement, and self-management? Skip to Question 37.

    2. If yes, continue to Question 29.

29. Are there specific measures or other assessment methods you use to evaluate your organizational policies and structures? (If no, probe for details: Are there other ways you have been trying to assess your policies and structures? If yes, probe for details: What measures have you used?)

Interviewer: Questions 30-36 should be asked for each measure or other assessment method identified.

Now, I would like to ask you for specific information about each measure or other assessment method you have mentioned. Let’s start with [measure/method]: _________________.

30. Can you tell me exactly how the measure is calculated? (Probe for details: What data are used? What specific data items or elements are used? How are these data collected? How are the numerator and denominator defined? Are there any exclusions made when calculating the measure, for example, omitting patients who do not have a specific diagnosis? Do you have written documentation you can share that would provide us with specific details about computation of this measure?)

31. How do you use the measure to assess your policies and structures? (Probe for details: How often do you compute the measure? Is it possible to compute the measure often enough to be useful for quality improvement work? Do you check for disparities between different groups of patients? Does the measure get reported? If reported: To whom? How does that person use the measure? Is there any accountability for achieving certain measurable changes?)

32. Have you examined the relationship between change on this measure and other things, like changes in processes of care, patient experience, or patient outcomes? (Probe for details: Have you seen any important relationships? Can you tell me about that? Have there been any unintended consequences of implementing this measure? Have any of these findings been documented? Can we obtain a copy of any written documentation?)

33. Has this measure been tested to ensure that it measures what it is meant to measure and that it does so consistently? (Probe for details: Did you do the testing? Can you tell me what testing has been done? In what settings and with what populations has the measure been tested? Do you think it could be used in other settings? What have the results of the testing been? Can we have a copy of any written description of these findings?)

34. Do you have any evidence that the measure is considered important by your clinicians and staff? (Probe for details: If yes, what evidence do you have? If not, are clinicians and staff unsupportive of this measure? Can we have a copy of any written description of these findings?)

35. Do you have any evidence that the policies or structures are aimed at areas that patients think are important? (Probe for details: Can you tell me about that? Can we have a copy of any written description of these findings?)

36. Has this measure been difficult or expensive to compute or to use to guide efforts to improve your organization’s policies and structures? (Probe for details: Why is that?)

Great. Thank you. Now I would like to ask you about organizational leadership. Some people who are engaged in work to improve patient communication, navigation, engagement, and/or self-management in their health care organizations have told us that it is important for senior leaders within the organization to show their support for these efforts and their commitment to creating an organizational culture that supports patients in these ways.

37. Are you aware of any measures that might help organizations to measure the level of support and commitment of their senior leaders? (If yes, probe for details about the computation and source of the measures. If no, are there ways you think it would be possible to measure these things? Can you tell me more about that?)

38. Do you think that any of the measures or other assessments that you’ve mentioned would help persuade the leadership of your organization to more actively support improvement efforts in the areas we’ve discussed? (If yes, probe for details: Which ones? If no, Can you think of any measures or assessments that would motivate your leadership?)

Throughout our efforts to identify ways that organizations can evaluate their performance in the areas we have discussed, we have heard about some other suggested measures. Specifically, the following measures and approaches have been identified: __________.

39. What do you think about the potential value of these methods for monitoring an organization’s progress? (Probe for details: What strengths do these approaches have? What weaknesses?)

Good. Thank you. I have just a few final questions for you.

40. What other measures or assessments do you think would be helpful in evaluating your own organization’s improvement efforts in the areas of patient communication, navigation, engagement, self-management, and organizational policies, structures, and leadership? (Probe for details: What would those measures capture? How might you compute them? Would it be easy to collect the data you would need to compute those measures? How would you use the data?)

41. Are there other organizations you know to be involved in these sorts of efforts? We would be interested in talking with them, too. (Probe for details: What organizations? What kinds of work have they done? Is there someone you would recommend we talk to at that organization?)

Thank you so much. That concludes my questions. However, before we finish, I would like to know if there is anything I have missed. Is there anything else that you think I should know about your organization’s experience with measuring the things we talked about today? (If not: Thank you again for your time!)

Interviewer: Before ending the call, remind the participant of the written documentation he/she indicated a willingness to share. Please reiterate specific documents discussed.