CMS-10550 Interview Topic Guide for Semi-Structured Interview

Attachment VIII. — Interview Topic Guide for Semi-Structured Interview of Hospital Quality Leaders

Qualitative Interview Guide for Hospitals

Organization Name:

Respondent Name:

Respondent Position:

Interviewer Name:

Interview Date:

INTRODUCTION AND PURPOSE OF THE INTERVIEW

Thank you for agreeing to participate in the interview today. I’d like to briefly review the purpose of this interview and the confidentiality provisions that were described in the email we sent you.

• We are conducting interviews with hospitals on behalf of the Centers for Medicare & Medicaid Services (CMS).

• CMS implements a variety of performance measures in the hospital setting to assess the quality and efficiency of care provided to Medicare beneficiaries. Hospitals report on measures included in the Hospital Inpatient Quality Reporting (IQR) and Outpatient Quality Reporting (OQR) Programs. Hospital performance on these measures is publicly reported on the CMS Hospital Compare website, and a subset of the measures is used in the CMS Hospital Value-Based Purchasing Program and the Hospital-Acquired Condition Reduction program.

• The purpose of today’s interview is to learn about your perspective of your hospital’s experiences in reporting and working to improve performance on the CMS measures and your efforts to improve the quality and efficiency of care at your hospital.

CONSENT/CONFIDENTIALITY PROVISIONS

• All of your responses are confidential.

• No one outside of the project will have direct access to the information you provide. The evaluation team will only produce summary information from the set of interviews. You will not be identified by name or hospital affiliation.

• You do not have to participate in the interview. You can stop at any time for any reason. Your decision regarding whether to participate will not affect your institution’s Medicare reimbursement or quality scores.

• You can decline to discuss any topic that we raise.

Do you have any questions? (YES/NO)

Do you agree to participate in the interview? (YES/NO)

As we mentioned in our email, we would like to tape the interview if that is all right with you.

Do you give your permission to record this interview? (YES/NO)

If yes: Great. Let’s get started. I’ll start the recording.

If no: That’s fine. We will take notes – and not tape the discussion. Let’s get started.

Hospital Interview Protocol

[Note to interviewer: An asterisk (*) indicates a question that should be skipped

if there will not be sufficient time to complete all questions.]

I’d like to start by asking you about your background and a few characteristics of your hospital.

Respondent Background

1. We understand that you are the _____________ [title/position] at __________ [hospital name]. Is that correct?

1. [If not correct:] What is your job title or position?

2. What is your professional background?

1. [If physician:] What is your specialty?

Hospital Characteristics

3. Is your hospital affiliated with a hospital system? (YES/NO)

4. How many competitor hospitals exist within your hospital’s service area? [If needed: Are there 0, 1, 2, 3 or more hospitals that are considered competitive?]

1. Compared to your competitors, how well does your hospital perform on the CMS quality and efficiency measures? [Better than, comparably, worse than, don’t know]

5. Do you face a shortage of physicians in your area?

6. Does your hospital employ most of the physicians who practice in the hospital? [If needed, the interviewer can reframe the question to: Are most of your physicians in community practice?]

7. Approximately what percentage of the patients your hospital admits are covered under the following (should total 100%):

Type of Insurance	Percent
Medicare
Commercial insurance
Medicaid
Private pay/other
Uninsured
	100%

Innovations in the Delivery of Care

I’d like to talk now about CMS measurement programs and innovations or changes your hospital may be making in the way care is being delivered. In some cases, I will ask you to specifically reflect on your experiences related to the CMS quality and efficiency measures and measurement programs. These programs include the Inpatient Quality Reporting Program, Outpatient Quality Reporting Program, Hospital Value-Based Purchasing, Hospital Readmissions Reduction Program, and the Hospital-Acquired Condition Reduction Program.

I’d like to make sure you have the full list of the most recent CMS quality and efficiency measures. They should have been sent to you by email in advance of our interview. Do you have the list?

[If yes:] Great, can you take a minute or two to review the list of measures?

[If no / doesn’t have the list:] I will send you the list by email right now. Once you receive it, please take a minute or two to review the measures.

8. Are you familiar with these quality and efficiency measures? (YES/NO)

1. [If no:] Can you help me identify the senior leader within your hospital who would be most familiar with your hospital’s efforts to improve performance on these measures? Thank you for your time. [Then end the current interview.]

9. In your experience, have the CMS measures and measurement programs led your hospital to change anything about how it delivers care? (YES/NO)

[If yes:]

1. Please describe the changes your hospital has made.

1. Which of the changes that you’ve mentioned have been implemented hospital-wide (i.e., across your entire organization)?

2. Have any of the changes focused on addressing specific CMS measures? (YES/NO)

1. [If yes:] Which measures?

[If no, go to Q9:]

2. Why do you believe that CMS measurement programs have not led to changes in care delivery? [If needed, prompts include improvement not needed, lack of resources, incentives too small, quality initiative fatigue]

10. Now, I would like you to think about your frontline physicians and other clinical staff. Have you observed frontline clinical staff initiating any changes in response to quality measurement programs? (YES/NO)

[If yes:]

1. Please describe the changes the frontline clinical staff has made.

2. Do you think the changes frontline clinical staff has made have affected your hospital’s performance specifically on the CMS performance measures? (YES/NO)

1. [If yes:] Which of the actions taken by frontline clinical staff do you believe have had the largest impact on your performance on the CMS measures?

3. Are there areas captured in the CMS measures where it has been difficult to get physicians and other clinical staff to change their behavior? (YES/NO)

[If yes:]

1. If yes, please describe which areas.

2. Why do you think it was difficult to achieve changes in frontline staff behavior?

3. How did your hospital address these barriers?

11. Do you think any of the changes your hospital has made have affected your hospital’s performance specifically on the CMS performance measures? (YES/NO)

1. [If yes:] Based on your experience, which of the changes have had the largest impact on your performance?

12. Have the changes your hospital has made in response to the CMS measures led to improvements in quality of care outside of the clinical areas that the CMS measures cover (i.e., spillover effects)? [Example, if needed: For example, CMS measures you on heart attack, pneumonia, and heart failure care, but delivering better quality care in these areas might positively affect care for patients undergoing hip replacement.] (YES/NO)

1. [If yes:] What measures has your hospital used to track improvements in other areas?

Factors Associated with Change in Quality Performance

I’d like you to look at the measures list that we sent you. On this list you will see the individual measures and your performance score for those measures for the most recent reporting year.

13. For the CMS performance measures where your hospital is performing well [interviewer to have hospital performance list ready], what factors do you think help your hospital perform highly? [If needed, examples include overall resources, data systems, the organization’s culture, internal incentives, leadership engagement, frontline staff engagement, clinical champions, investments in care redesign]

14. For those measures where your hospital’s performance is lagging [interviewer to have hospital performance list ready], what factors do you believe inhibit higher performance? [If needed, examples include overall resources, data systems, the organization’s culture, insufficient internal incentives, lack of leadership or frontline staff engagement, lack of clinical champions, few investments in care redesign]

15. From your perspective, is it harder to improve scores on some CMS measures than others? (YES/NO)

1. [If yes:] Which measures, and why?

16. Thinking about the full list of CMS measures we are discussing, do you think the CMS measures are clinically important?

1. Why or why not?

17. Do you think hospitals have sufficient control over care to be held responsible for performance on these measures? (YES/NO)

[If no:]

1. Who do you think should be responsible?

2. Are there other areas where CMS should consider measures to gauge your hospital’s quality performance?

Many factors may influence your hospital to invest in improving performance on the CMS measures.

18. What do you see as the most important driver of your hospital’s investments to improve performance at your hospital?

[Drivers of improvement might include public reporting of quality scores on Hospital Compare, the potential to receive financial incentives for improvement or high performance, the threat of financial penalties for poor performance, receipt of technical assistance¹ related to quality improvement from CMS, a Quality Improvement Organization, Hospital Engagement Network, or community collaborative.]

18. *How important are each of the following as drivers of improvement in your hospital? (very important, somewhat important, not important)

1. Public reporting of quality scores on Hospital Compare?

2. The potential to receive financial incentives for improvement or high performance?

3. The threat of financial penalties for poor performance?

4. Receipt of technical assistance² related to quality improvement from CMS, a Quality Improvement Organization (QIO), Hospital Engagement Network, or community collaborative?

18. Does your hospital participate in any non-CMS quality measurement reporting programs? Please specify. [Prompts: private sector programs sponsored by commercial insurers, employers, or multi-stakeholder collaboratives, Medicaid]

1. [If yes:] How important are the CMS measurement programs as drivers of improvement relative to other programs that measure the quality and efficiency of care in your hospital (e.g., Leapfrog, private payer programs)? (more important, as important, less important)

Organizational Structure and Delivery System Reform Initiatives

18. Does your hospital participate in an Accountable Care Organization (ACO)?² (YES/NO/DON’T KNOW)

[If yes:]

1. Are you part of one of the Medicare ACOs? (prompts: Medicare Shared Savings Program, Pioneer ACOs, Advanced Payment) (YES/NO/DON’T KNOW)

2. Does your hospital participate in a Medicaid ACO? (YES/NO/DON’T KNOW)

3. Are you part of an ACO arrangement with a private commercial insurer? (YES/NO/DON’T KNOW)

1. [If yes:] How many different private insurer ACOs is your hospital part of?

4. Is your hospital financially at risk (i.e., downside risk) in any of these ACO arrangements? (YES/NO/DON’T KNOW)

18. Is your hospital part of an integrated delivery system?³ (YES/NO/DON’T KNOW)

18. Is your hospital participating in any other type of alternative payment model (Prompts: global budgets, bundled payments for selected procedures)? (YES/NO/DON’T KNOW)

[If yes:]

1. Are these shared savings models? (YES/NO/DON’T KNOW)

2. Is your hospital at risk financially in any of these arrangements? (YES/NO/DON’T KNOW)

I’d next like to discuss how changes to improve care are managed at your hospital.

18. What is the position of the person who directs quality improvement activities in your hospital? [If respondent says that more than one person directs such activities, ask for all positions.]

1. Does this person [or these persons] report to an executive manager, such as the Chief Medical Officer (CMO) or Chief Executive Officer (CEO)?

18. Does your organization have a quality improvement department? (YES/NO/DON’T KNOW)

18. *Has your hospital used any of the following care redesign methods to improve quality?⁴ (YES/NO/DON’T KNOW)

1. Deming/lean processes (i.e., constantly improve the system of production and service to improve quality and decrease cost)? (YES/NO/DON’T KNOW)

2. Six Sigma (i.e., measurement-based strategy/data-driven approach for eliminating defects; focuses on process improvement and variation reduction)? (YES/NO/DON’T KNOW)

3. Plan, Do, Study, Act (PDSA) improvement cycles? (YES/NO/DON’T KNOW)

4. Other methods? Please specify.

18. Does your hospital have an electronic health record (EHR)? (YES/NO)

[If yes:]

1. Is your hospital’s EHR able to exchange information electronically within all departments in the hospital? (YES/NO/DON’T KNOW)

1. [If “don’t know”:] Is there someone we can speak with in your organization who might be able to answer some of these questions about your EHR?

2. Is your hospital’s EHR able to electronically exchange information with providers in your community (e.g., ambulatory physicians and nursing homes)? (YES/NO/DON’T KNOW)

1. [If “don’t know”:] Is there someone we can speak with in your organization who might be able to answer some of these questions about your EHR?

3. Does your EHR have clinical decision support⁵ functions embedded in the system to assist clinicians and providers? (YES/NO/DON’T KNOW)

1. [If yes:] For what clinical areas or functions?

2. Do any of the clinical decision supports address CMS quality and efficiency measure areas? Which areas?

4. Does your EHR help your doctors and other health care providers monitor the quality of care they are providing (Prompts: changes in patient functioning, summary results)?

5. Does your hospital use the EHR system to report the CMS quality and efficiency measures? (YES/NO/DON’T KNOW)

18. Does your hospital provide physicians, nurses, and other clinical staff (such as pharmacists and physical therapists) with information about your hospital’s performance on the CMS quality and efficiency measures? (YES/NO/DON’T KNOW)

1. [If yes:] How often do physicians and nurses receive feedback on their performance on the measures?

Perspectives of Different Stakeholders and Leaders

We’re interested in how different leaders and groups within your hospital have viewed and approached CMS quality and efficiency measures and related public reporting and payment programs. We’d specifically like to ask about support from four general groups of stakeholders within the hospital—executive management, physicians, nursing, and the board of directors. We are interested in the degree to which they have acted to enable and provide assistance for your hospital's successful performance on CMS measurement programs.

18. *On a scale of 0 to 10, with 0 being extremely unsupportive and 10 being extremely supportive, how would you characterize the executive management team’s support of the CMS measurement programs?

1. Has this support changed over time?

18. *On a scale of 0 to 10, with 0 being extremely unsupportive and 10 being extremely supportive, how would you characterize the physicians’ support of the CMS measurement programs?

1. Has this support changed over time?

18. *On a scale of 0 to 10, with 0 being extremely unsupportive and 10 being extremely supportive, how would you characterize the nursing leadership’s support of the CMS measurement programs?

1. Has this support changed over time?

18. *Finally, on a scale of 0 to 10, with 0 being extremely unsupportive and 10 being extremely supportive, how would you characterize the hospital board of directors’ support of the CMS measurement programs?

1. Has this support changed over time?

2. Is performance on the CMS quality and efficiency measures on the board agenda at each board meeting?

Challenges to Implementing CMS Measures

I’d like to talk about what you see as challenges to reporting the data/measures and improving your hospital’s performance on the CMS measures.

18. Have you experienced difficulties in reporting the CMS measures? (YES/NO/DON’T KNOW) [If needed, prompts include measure specifications, challenges with CMS reporting tools, difficulty capturing or extracting the data, insufficient resources, programming new measures each year.]

[If yes:]

1. Please describe the difficulties.

2. How did you address these difficulties?

18. Have you experienced difficulties with improving performance on the CMS measures? (YES/NO/DON’T KNOW) [If needed, prompts include difficulty identifying appropriate improvement strategies or what processes need to be improved, inadequate IT capabilities, need for provider training, staff turnover, lack of leadership support, lack of clinician support or time, insufficient resources]

[If yes:]

1. Please describe these difficulties.

2. How did you address these difficulties?

Effects of Performance Measurement Programs

Some providers have expressed concern that CMS measurement programs might lead to undesired effects at times. CMS is interested in learning about possible undesired effects so it could modify the measurement and reporting programs to minimize these effects. All of the responses you provide are confidential. Your candid feedback will be especially important in helping CMS improve these programs so that they work well for providers and patients.

18. *Are you aware of any undesired effects in your hospital that stem from the CMS measurement program and the use of the measures in public reporting and payment/value-based purchasing efforts? (YES/NO/DON’T KNOW)

1. [If yes:] Please describe these undesired effects.

1. Why do you think these undesired effects have occurred?

2. What do you think could be done to mitigate those undesired effects?

[If no, or respondent is vague on specific undesired effects:]

2. One problem identified by hospitals was related to a pneumonia measure that assessed whether pneumonia patients were given initial antibiotic(s) within 6 hours after arrival. The concern was that hospitals might over-prescribe antibiotics, by providing antibiotics to patients with low probability of pneumonia in order to ensure the metric is met. Do you think such inappropriate changes in treatment occur as a result of the CMS measures? (YES/NO)

1. [If yes:] Can you give any other examples of measures where this might occur?

3. Some are also concerned that hospitals might focus a great deal of effort on data coding to increase reimbursement or exclude sicker patients from the measure calculation. In your opinion, do you think the CMS measurement programs incentivize an increased focus on coding? (Clarify that we are not talking about more accurate coding)

1. [If yes:] Have you seen or heard of a particular emphasis on coding for any measures in particular?

4. Hospitals may also avoid sicker or more difficult patients in order to achieve higher scores on the quality and efficiency measures. Have you heard of this happening? (YES/NO/DON’T KNOW)

1. [If yes:] Have you heard it in association with any measures in particular?

5. Hospitals might focus all their improvement efforts on areas of care where performance is being measured or financially incentivized and ignore or pay less attention to areas of care that are not measured. Do you think this happens? (YES/NO/DON’T KNOW)

1. [If yes:] Does it happen with any specific measures in particular?

18. CMS has been working to evolve the design of its measurement programs. For example, CMS has considered incorporating more outcome measures. Do you think this is a positive change?

1. Do you believe it will be more difficult for your hospital to achieve high performance on outcome measures?

1. [If yes:] Why?

2. In your opinion, will inclusion of outcome measures result in any additional undesired consequences? (YES/NO/DON’T KNOW)

1. [If yes:] What could be done to mitigate the problem?

Closing

18. Based on your experience to date using CMS hospital measures, what changes to the measures would you recommend?

1. What changes to the measurement programs would you recommend?

18. Is there anything else that I did not ask that you would like to comment on?

Thank you very much for your time.

1 Technical assistance refers to support provided to help hospitals with approaches to and ideas for improving the quality of care they deliver. It may take such forms as webinars, toolkits, or coaching.

2 Accountable Care Organizations are networks of health care providers and organizations (usually hospitals and physician groups, and possibly including nursing homes, home health, and hospice organizations), that agree to take some financial responsibility for reducing the costs and improving the quality of care of enrollees.

3 Integrated Delivery System is a network of health care providers and organizations (i.e., hospital, primary and specialty care, rehabilitation, home health care, hospice) that provides or arranges to provide a coordinated continuum of services to a defined population. It may own or be closely aligned with an insurance product, usually using a form of managed care.

4 Some organizations utilize specific methodologies or frameworks to guide and insure consistency in improvement activities throughout the organization. Examples include the Model for Improvement or Plan, Do, Study, Act (PDSA); Cycle or Deming Cycle; Lean Improvement adapted from the Toyota Production System; Six-Sigma DMAIC (which stands for define, measure, analyze, improve, control); and the Seven-Step Method Problem-Solving Model. Other organizations have not adopted a specific improvement methodology.

5 “Clinical decision support” (CDS) provides clinicians, staff, patients or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care. CDS encompasses a variety of tools to enhance decision-making in the clinical workflow. These tools include computerized alerts and reminders to care providers and patients; clinical guidelines; condition-specific order sets; focused patient data reports and summaries; documentation templates; diagnostic support, and contextually relevant reference information, among other tools.” (Source: http://www.healthit.gov/policy-researchers-implementers/clinical-decision-support-cds)

Page 3 of 19