Form CMS-10310 Attachment C2 - Cogntive Interview Guide Professionals

Interview protocols for professional interviews:

Consumer Testing Hospital Outpatient Measures

January 27 & 28 2010

Objectives

The interview will focus on the following issues:

• Assessing participants’ understanding of the content and language on the site, focusing specifically on:

  • Pathways for the outpatient measures.

  • The displays and language describing the surgical care measures.

  • The displays and language describing the chest pain and heart attack measures.

  • The displays and language describing the medical imaging measures.

Welcome and Overview of Discussion

Welcome

• Thank you for agreeing to participate in this discussion today.

• My name is {NAME} and I’ll be leading our discussion. I work for a company called {L&M Policy Research Policy Research/Mathematica Policy Research}, a health policy research firm.

Background—explain purpose of the discussion

• We’ve been hired by the Department of Health and Human Services (HHS) to help them understand how to get useful information to people who may need hospital services or who are helping to care for someone who may need hospital services.

• We are helping HHS make improvements to a website they have about hospital services. This website, called Hospital Compare, provides information on how well hospitals care for their patients. You can use this website as a tool to gather information about hospitals in your area, or when making decisions about where to get hospital care.

• When HHS puts information about hospitals on the website, they want to make sure that it makes sense and helps people when making decisions about hospital care. That’s why we’re here talking to you and other people – we would like to hear your opinions about the information on the website.

• Everything you tell us will be confidential. To protect your privacy, we won’t connect your name with anything that you say.

Ground rules

• Please be open and frank. We’re not looking for “right” answers. If something is unclear or confusing to you, it’s bound to be confusing to other people too.

• We’re recording these conversations, for research purposes, to help us remember everything that’s said, but we will not use anyone’s name or share this information beyond our research group.

• Some people working on this project are listening and watching behind the glass, and we are recording this so we can make sure we don’t miss anything. Because we’re recording, please try to speak in a voice at least as loud as the one I’m using now so that we can make sure the tape is picking up our voices.

• We’ll be here about an hour. Any questions before we get started?

Warm-up and Introduction

Warm-up

• Before we get started, can you tell me a bit about you and your work?

  • How long have you been in practice?

  • Can you tell me a little but about the services you provide? What kinds of patients you work with?

  • How much of your time is spent providing services in a hospital setting?

[For PCPS]

• Do you ever order imaging tests (such as MRIs, mammograms or CT scans) for your patients?

• Can you tell me a little bit about this process?

  • How, if at all, do you interact with the radiology facilities who provide those services?

[For Radiologists]

• Can you tell me a little bit about this process for ordering radiology services?

  • Who, typically, orders these services?

  • How, if at all, do you interact with the providers who order these services?

Section I: Introduction to Hospital Compare

• As I mentioned earlier, we are helping HHS make improvements to the Hospital Compare website. Hospital Compare is a tool you can use to search for and compare hospitals based on how well they provide care. Have you ever visited Hospital Compare?

• Let’s pretend today that you are a physician interested in seeing quality of care measures about your hospital. One of things you might do is go to the Hospital Compare tool to find that information, to check on the performance of your own hospital.

• I’m now going to briefly show you the website to give you an idea of what it looks like and what kinds of information you can find.

• Interviewer should open the Medicare homepage and take the participants to the Hospital Compare tool.

• On Step 1, explain how users can search by Hospital name, zip code, city, state/territory, or county, but use the city search, selecting Baltimore.

• On Step 2, explain how users can then choose to search by General Search, Medical Condition Search, or Surgical Procedure Search, but use the General Search.

• On Step 3, let participants look over the information for a minute or two, then explain how users then can select up to 3 hospitals to compare.

Now we are going to stop using the current site and start to use some paper printouts of what the screens might look like.

{Moderator will discuss the following sections with PCPS AND Radiologists}

Section II: Your Selected Hospitals, Collapsed

After going through the tool as we just did, you get to this page with information for the 3 hospitals you selected. For our discussion today, I’d like you to pretend that you selected the following 3 hospitals to compare: Edgefield Hospital, Mitchell Hospital, and River View Hospital. Please note that since this is new information that is not yet on the website, these are not real hospitals. Please keep in mind as we review this information that you are interested in checking on the performance of the hospital where you provide services.

{Hand participant Pages 1 & 2 of the mock up of Your Selected Hospitals, Collapsed} Take a minute to look over this information and then we’ll talk about it.

• In your own words, what kind of information is available on this page?

• If you were looking at this page on the web site, what, if anything, would you do next? {Probe: What topics would you be interested in? Why do you say that? How would you get more information about those topics? What would you do?}

• What does the label, “Measures Based on Standards of Care” mean to you?

{Probe: What do you think is meant by standards care?}

• What does the label, “Surgical Care” mean to you? {Probe: What types of things do you think would be included here?}

• Did you notice the “…Learn more about surgical care and why these measures are important” link? How likely, if at all, would you be to click on this link? Why? What do you think you will see if you click on it?

• What does the label, “Heart Attack or Chest Pain” mean to you?

{Probe: What types of things do you think would be included here?}

• What does the label, “Appropriate Use of Medical Imaging (tests like Mammograms, MRIs, CT Scans)” mean to you?

[Probe: What types of things do you think would be included here?}

{Moderator will discuss the following sections with PCPS Only}

SECTION III: SURGICAL MEASURES

Let me show you what you would see if you clicked on the {Show Information} button next to the Surgical Care. {Hand participant the mockups for the Surgical Care Measures} Take a minute to look this over and then we’ll talk about it.

• In your own words, what is this page about?

• Who do you think this information is for? {Probe: Are you interested in this information? Why or why not? Are there particular topics that you are interested in? Why?}

• Do you have a preference for which information appears first, the information about inpatient surgery or the information about outpatient surgery? Why do you say that?

• Did you notice the text in the blue box that says “Not all the data in this measure has been checked for accuracy”?

• In your own words, what does that information mean?

• Is this information important to you? If so, why? If not, why not?

• Does this change the way you view this information? If so how?

• Do you have any preference for where this information should appear? {Probe: would you prefer to see this information on the home page or first page of the website? Would you prefer to see it on the first page we looked at? Would you prefer to see this information here on this page? If so, where?}

Now let’s take a look at these measures.

Percent of outpatient surgery patients who were given an antibiotic at the right time (within one hour before surgery) to help prevent infection

Percent of outpatient surgery patients who were given the right kind of antibiotic to help prevent infection

• Tell me in your own words what these labels mean to you. {Probe: How would you describe this information to patients?}

• What, if anything, is unclear or should be changed? {Probe: How would you re-write or change these labels?}

• Did you notice the footnote for Edgefield Hospital?

• If yes, what do you think this footnote means? What does that tell you about these hospitals? How would you describe that to patients? [For PCPs: Would that affect your willingness to use the information on HospitalCompare.gov in selecting which hospitals to refer your patients to?]

• If no, please take a minute to read the footnote now and tell me what you think it means. How would you describe that to patients? What could make this footnote stand out more?

Now I’d like to have you take a closer look at what you would see if you chose to view these measures in the form of a table. {Hand participant the Surgical Measures Table Display.}

• What does this information tell you? How, if at all, would you use the information?

• Did you notice the column with the US Average and the State Average? What do you think about that information? How does this information affect your overall impression of these 3 hospitals, if at all?

Now I’d like to have you take a closer look at what you would see if you chose to view this information in the form of a graph. {Hand participant the 1^st page of the Surgical Measures Graph Display}

• What do you think about this graph? {Probe: What were your first impressions of this graph? When you first saw it what was your reaction?)

• In your own words, what does this text at the top of the page (Why is this important?) mean? {Probe: how would you describe that to patients?}

• How useful, if at all, did you find this information?

• How would you re-write or change this text?

SECTION IV: HEART ATTACK MEASURES

Let me show you what you would see if you clicked on the {Show Information} button next to Heart Attack or Chest Pain. {Hand participant pages 1 & 2 of the Chest Pain/Heart Attack Measures} Take a minute to look this over and then we’ll talk about it.

• In your own words, what is this page about?

• Who do you think this information is for? {Probe: Are you interested in this information? Why or why not? Are there particular topics that you are interested in? Why?}

Now let’s take a look at some of these measures.

Percent of outpatients with heart attack who got drugs to break up blood clots within 30 minutes of arrival (a higher percentage is better)

{Point to the 1^st measure on page 1}

• Tell me in your own words what this label means to you. {Probe: How would you describe this to patients?}

• What, if anything, is unclear or should be changed? {Probe: How would you re-write or change this label?}

• Did you notice the footnote for Edgefield hospital? Is it clear what “0 patients” means in this case?

• If no, please take a minute to read the footnote now and tell me what you think it means. How would you describe that to patients? What could make this footnote stand out more?

Now I’d like to have you take a closer look at what you would see if you chose to view this measure in the form of a table. {Hand participant the Heart Attack Table Display, on page 5, and point to the 1^st measure.}

• What is different on this page than the last page we looked at? {Probe: Did you notice the column with the US Average and the State Average? What do you think about that information?}

• Did you notice that the footnotes have been omitted? What do you think about that?

• What does this information tell you? How, if at all, would you use the information?

Now I’d like to have you take a closer look at what you would see if you chose to view this information in the form of a graph. {Hand participant page 6 of the Heart Attack Graph Display}

• In your own words, what does this text at the top of the page (Why is this important?) mean? {Probe: how would you describe that to patients?}

• How useful, if at all, did you find this information?

• How would you re-write or change this text?

• Did you notice that the footnotes have been omitted? What do you think about that?

Now let’s go back and take a look at some of the other measures. {Return to page 1 of the Heart Attack Table Display and point to the 2^nd measure}

Average number of minutes before outpatients with heart attack who needed specialized care were transferred to another hospital (a lower number of minutes is better)

• Tell me in your own words what this label means to you. {Probe: How would you explain that to patients?}

• What, if anything, is unclear or should be changed? {Probe: How would you re-write or change this label?}

Now I’d like to have you take a closer look at what you would see if you chose to view this information in the form of a graph. {Hand participant page 7 of the Heart Attack Graph Display}

Average number of minutes before outpatients with heart attack who needed specialized care were transferred to another hospital (a lower number of minutes is better)

• In your own words, what does this text at the top of the page (Why is this important?) mean? {Probe: how would you describe that to patients?}

• How useful, if at all, did you find this information?

• How would you re-write or change this text?

Let’s look at one last measure in this section. {Return to page 1 of the Heart Attack Display and point to the 4th measure}

Average number of minutes before outpatients with heart attack (or with chest pain that suggests a possible heart attack) got an ECG (a lower number of minutes is better)

• Tell me in your own words what this label means to you. {Probe: How would you explain that to patients?}

• In your own words, what do the numbers in the cells mean? {Probe: What does 16 minutes mean for River View Hospital? What does 0 minutes for Mitchell Hospital mean?}

• What, if anything, is unclear or should be changed? {Probe: How would you re-write or change this label?}

Now let’s take a look at what you would see if you chose to view this information in the form of a table. {Go to page 5 of the Heart Attack Table Display and point out the last measure.}

• What do you think about this information?

• What, if anything, is unclear or should be changed?

Now let’s take a look at what you would see if you chose to view this information in the form of a graph. {Go to page 10, the last page, of the Heart Attack Graph Display}

• In your own words, what does this text at the top of the page (Why is this important?) mean? {Probe: how would you describe that to patients?}

• How useful, if at all, did you find this information?

• How would you re-write or change this text?

{Moderator will discuss the following sections with PCPS AND Radiologists}

SECTION V: MEDICAL IMAGING MEASURES

Let me show you what you would see if you clicked on the button next to Appropriate Use of Medical Imaging (tests like Mammograms, MRIs and CT scans). {Hand participant the Medical Imaging Measures, pages 1 & 2} Take a minute to look this over and then we’ll talk about it.

• In your own words, what is this page about?

• Did you read the text in the blue box? Tell me in your own words what that means to you. {How would you explain this information to patients?}

• How useful, if at all, did you find this information?

• How would you re-write or change this text?

Now let’s take a look at some of these measures.

Percent of outpatients with low back pain who had an MRI without trying recommended treatments first, such as physical therapy.

• Tell me in your own words what this label means to you. {Probe: How would you explain that to patients?}

• What, if anything, is unclear or should be changed? {Probe: How would you re-write or change this label?}

Percent of outpatients who had a follow-up mammogram or ultrasound after a screening mammogram.

• Tell me in your own words what this label means to you. {Probe: How would you explain that to patients?}

• What, if anything, is unclear or should be changed? {Probe: How would you re-write or change this label?}

{Moderator will only discuss one of the measures around CT scans, alternating between the CT Scan of the Chest and the CT Scan of the Abdomen.}

Percent of all outpatient CT scans of the chest/abdomen that were “combination” (double) scans.

• Tell me in your own words what this label means to you. {Probe: How would you explain that to patients?}

• How would you explain to patients the difference between a double scan and a single scan?

• What, if anything, is unclear or should be changed? {Probe: How would you re-write or change this label?}

Now I’d like to have you take a closer look at what you would see if you chose to view this information in the form of a table. {Hand participant the Medical Imaging Table Display, pages 3 & 4}

• What do you think of this information?

• Does this change the way you look at the measures?

Now I’d like to have you take a closer look at what you would see if you chose to view this information in the form of graphs. {Hand participant the Medical Imaging Graph Displays, pages 5-13}

Percent of outpatients with low back pain who had an MRI without trying recommended treatments first, such as physical therapy.

• What do you think of this information?

• In your own words, what does this text at the top of the page (Why is this important?) mean? {Probe: how would you describe that to patients?}

• How useful, if at all, did you find this information?

• How would you re-write or change this text?

{Hand participant the explanatory text for the MRI measure, page 6}

• In your own words, what does this information mean? {Probe: how would you describe that to patients?}

• How useful, if at all, did you find this information? [PCPs: Would you use this information when considering which hospitals to refer your patients to? Why or why not?]

• How would you re-write or change this text?

• What can patients do to ensure hospitals are appropriately using MRIs?

Percent of outpatients who had a follow-up mammogram or ultrasound after a screening mammogram. {Hand participant the Medical Imaging Graph Displays, page 7}

• What do you think of this information?

• In your own words, what does this text at the top of the page (Why is this important?) mean? {Probe: how would you describe that to patients?}

• How useful, if at all, did you find this information?

• How would you re-write or change this text?

• What can patients do to ensure hospitals are appropriately using follow up mammograms?

{Moderator will only discuss one of the measures around CT scans, alternating between the CT Scan of the Chest and the CT Scan of the Abdomen.}

Percent of all outpatient CT scans of the chest/abdomen that were “combination” (double) scans.

• What do you think of this information?

• In your own words, what does this text at the top of the page (Why is this important?) mean? {Probe: how would you describe that to patients? how would you explain “contrast”? “Double” or “combination” scan?}

• How useful, if at all, did you find this information?

• How would you re-write or change this text?

• What can patients do to ensure hospitals are appropriately using CT scans?

Before we finish our conversation today, can you tell me what is your overall impression about the information we just discussed? {Probe: Is this information important/useful when looking at hospitals? Would you use the information? If yes, how would you use it? If no, why not?}

CLOSING

Those are all the questions I have for you.

Do you have any final thoughts you would like to add?

If you’ll give me just a moment, I’ll check in with my colleagues. {See if anyone in the back room or on focus vision has anything else on which to probe.}

Thank you very much for taking the time to meet with me today. Your insights have been very helpful!

12