Consumer Research on Public Reporting of Hospital Quality Measures (CMS-10360) - Initial Screening Round 3 Cognitive Interviews

DRAFT Interview protocols for consumer interviews

Consumer Testing Measures for Hospital-Acquired Conditions (HACs), Healthcare Associated Infections (HAIs), and Emergency Department (ED) Throughput

Dates TBD

Testing materials checklist

Verify video/audio recording equipment and DVDs

Desktop for testing

Laptop for note-taking

Interviewer clock

Pens and notepads in each testing room

Website/Mockups of all materials

Protocols

Procedures for obtaining informed consent

As interviewees arrive, greeter should have them read and sign the informed consent form (if not enough time, interviewer should do this prior to starting). Give each person an unsigned copy of the form to keep.

Testing goals

The testing materials will include:

• The current Hospital Compare tool, available on the medicare.gov website

• Paper based mock-ups of the HAC, HAI, and ED measures, table and/or graphic displays, and explanatory text.

The interview will focus on the following:

• Assessing participants’ understanding of the content and language of the new measures, focusing specifically on:

  • The alternative displays and language describing the HAC, HAI, and ED measures. The team will test the domain descriptions, labels, tables, graphs and explanatory language measures to determine whether consumers are able to interpret the displays as shown.

Welcome and Overview of Discussion

Welcome

• Thank you for agreeing to participate in this discussion today.

• My name is {NAME} and I’ll be leading our discussion. I work for a company called {CONTRACTOR NAME}, a health policy research firm.

Background—explain purpose of the discussion

• We’ve been hired by the Department of Health and Human Services (HHS) to help them understand how to get useful information to people who may need hospital services or who are helping to care for someone who may need hospital services.

• We are helping HHS make changes to a website they already have about hospital services. This website, called Hospital Compare, provides information on how well hospitals care for their patients. You can use this website to find information about hospitals in your area or in other areas.

• When HHS puts information about hospitals on the website, they want to make sure that it’s clear, understandable, and as useful as possible. That’s why we’re here talking to you and other people; we would like to hear your opinions about the information on the website.

• Everything you tell us will be confidential. To protect your privacy, we won’t connect your name with anything that you say.

Ground rules

• Please be open and frank. We’re not looking for “right” answers. If something is unclear or confusing to you, it’s bound to be confusing to other people too.

• We’re recording these conversations, for research purposes, to help us remember everything that’s said, but we will not use anyone’s name or share this information beyond our research group.

• Some people working on this project are listening and watching behind the glass, and we are recording this so we can make sure we don’t miss anything. Because we’re recording, please try to speak in a voice at least as loud as the one I’m using now so that we can make sure the tape is picking up our voices.

• We’ll be here about an hour and a half. Any questions before we get started?

Warm-up

Introductions

• Tell me about your experience using the Internet.

• Probe: Do you use the Internet? How often? What kinds of things do you use it for? Have you ever looked for health information on the Internet? What kind of health information?

Section I: Introduction to Hospital Compare

• As I mentioned earlier, we are helping HHS make improvements to the Hospital Compare website. Hospital Compare is a tool you can use to search for information about hospitals based on how well they provide care. Have you ever visited Hospital Compare?

• I’m now going to briefly show you the website to give you an idea of what it looks like and what kinds of information you can find.

{Interviewer will provide a brief overview of Hospital Compare to give participant a general idea of how the tool can be used to search for information about hospitals, using local geographic area as an example.}

• Let’s pretend today that you have recently moved to a new location and you are interested in learning about the hospitals in that area.  One of things you might do is go to the Hospital Compare tool to look at information about the hospitals available in your area.

Now we are going to stop using the current site and start to use some paper printouts of what the screens might look like.

{Note: In order to mitigate possible order bias, interviewers will systematically alternate the order in which the measure sets are presented to respondents for review. This order will be captured in note-taking templates and protocols.}

Section II: Introduction to HAC, HAI, and ED Measures

After going through the tool as we just did, you get to this page with information for hospitals in the location you selected.

{Interviewer hands respondent mockup of Hospital Compare ‘Results’ page, listing hospitals in selected geographic area and data available.}

Take a minute to look over this information and then we’ll talk about it.

In your own words, what kind of information is available, based on the information on this page? What is this table showing you?

Which, if any, terms are you unfamiliar with? What, if anything, is confusing or unclear?

If you were looking at this page on the web site, what topics would you be interested in? What kind of information do you think you would find under these topics?

Right now, the information on Hospital Compare is organized and labeled a different way, but a lot of the information about heart attack, heart failure, pneumonia, surgical care, pediatric asthma, and medical imaging is already available. You can also compare hospitals on these measures, and see how different states compare.

Some of the new information I’d like us to focus on would fall under the Safety and Timeliness categories.

Earlier, I showed you the Hospital Compare website and some of the information that’s available there about hospitals. Right now, it includes some very specific information about what hospitals do for patients who have surgery or certain conditions (like heart attack, heart failure, and pneumonia). It’s organized a little differently, but the information that’s there now would fall in the “Effectiveness” category.

Today, I want to get your feedback about some new information that will be posted in the “Safety” and “Timeliness” Categories.

What do you think about the term “safety” when you think about health care?

• What does that mean to you, in terms of hospital care?

• Can you think of any examples of hospital care that might not be safe?

Section III: HAC Measures

For our discussion today, I’d like you to pretend that you selected the following 3 hospitals to compare: Edgefield Hospital, Mitchell Hospital, and River View Hospital. Please note that these are not real hospitals. And let’s pretend you selected a tab that showed Safety measures for those three hospitals.

Then you might come to pages that looks like this.

{Interviewer hands respondent first page of mock-up for HAC measures, showing general definition of HACs, plus table showing results for 3 hospitals.}

HAC Measures: Definition

• In your own words, what is this telling you?

• What kind of information do you expect to see, based on this?

• Is this important information, in your view?

• Is there anything here that you think might be unclear or confusing to some people?

HAC Measures: Measure Labels

• First, tell me in your own words what each of these measure labels mean to you.

• {Probe: What are the measures about?}

• Is there anything here that might be unclear or confusing to some people?

• {Probe: How would you re-write or change these labels?}

• Is this important? Are some of these more important than others?

• What does this information tell you? How, if at all, would you use the information?

HAC Measures: Table Display, Rates

• Based on what you see here, how often would you say these kinds of things happen in hospitals?

{probe: Do respondents understand that the numbers are rates per thousand discharges?}

• Based on what you see here, what do you think about safety at Edgefield Hospital?

• Which of the items shown here is a more serious area of concern for Edgefield Hospital? Why? How serious are they?

{probe: based on numbers of patients affected; based on comparisons to national rates}

Now, I’d like to get your feedback on some slightly more detailed explanations about what these measures are about (for example, if you clicked on the “Read more . . .” link).

{Interviewer hands respondent narrative explanation about HACs and definitions of HAC measures.}

HAC Measures: Explanatory Text

• How important is this information?

• Who do you think would be interested in this information?

• Is there anything here that you think would be confusing or hard to understand?

HAC Measures: Measure Definitions

The next pages show definitions of each of the measures you saw in the table. Take a minute to look these over.

• Are these helpful definitions? Is there anything here that you think is unclear?

• Does this add information that you think is important to know? What information is particularly important?

Section IV: HAI Measures

Now, I’d like to show you some other information you might find under the Safety tab for those three hospitals.

{Interviewer hands respondent first page of mock-up for HAI measures, showing general definition of HAIs, plus table showing results for 3 hospitals.}

HAI Measures: Definition

• In your own words, what is this telling you?

• What kind of information do you expect to see, based on this?

• Is this important information, in your view?

• Is there anything here that you think might be unclear or confusing to some people?

HAI Measures: Measure Labels

• First, tell me in your own words what this measure label mean to you.

• {Probe: What is this measure about?}

• Is there anything here that might be unclear or confusing to some people?

• {Probe: How would you re-write or change this label?}

HAI Measures: Graphic Display

• Based on what you see here, how often would you say this happens in hospitals?

{Probe: What is meant by “rates per thousand central line days”?}

• Based on what you see here, what do you think about Mitchell Hospital?

• How does River View Hospital compare to the U.S. average?

Now, I’d like to get your feedback on some slightly more detailed explanations about what this measure (for example, if you clicked on the “Read more . . .” link).

{Interviewer hands respondent narrative explanation about HAIs and definitions of HAI measures.}

HAI Measures: Explanatory Text

• How important is this information?

• Who do you think would be interested in this information?

• Is there anything here that you think would be confusing or hard to understand?

HAI Measures: Measure Definition

The next pages show a definition of the measure you saw in the graph. Take a minute to look these over.

• Is this helpful? Is there anything here that you think is unclear?

• Does this add information that you think is important to know? What information is particularly important?

Section V: ED Throughput Measures

We’re getting near the end – but now, I’d like to show you some information you might find under the Timeliness tab for those three hospitals.

{Interviewer hands respondent mock-up for ED measures.}

Time Spent in ED: Definition

• In your own words, what is this telling you?

• Is this important information, in your view?

• Is there anything here that you think might be unclear or confusing to some people?

ED Measures: Measure Labels

• First, tell me in your own words what these measures mean to you.

• {Probe: What is the difference between them?}

• Is there anything here that might be unclear or confusing to some people?

• {Probe: How would you re-write or change these labels?}

ED Measures: Tabular Display

• Based on what you see here, what do you think about ED wait times in the U.S.?

• Which hospital is doing the best job? What makes you say that?

• How does River View Hospital compare to the U.S. average?

ED Measures: Explanatory Text

• How important is this information?

• Who do you think would be interested in this information?

• Is there anything here that you think would be confusing or hard to understand?

ED Measures: Measure Definitions

Take a minute to look over the definitions.

• Does this add any important information? What information do you think is particularly important?

• Is this helpful? Is there anything here that you think is unclear?

Section VI: Closing

Those are all the questions I have for you.

Do you have any final thoughts they would like to add?

If you’ll give me just a moment, I’ll check in with my colleagues.

{See checks to see if observers have anything else they wish to probe.}

Thank you very much for taking the time to meet with me today. Your insights have been very helpful!

14