Qualitative Methods

Eisenberg Center Qualitative Methods

The Eisenberg Center (EC) uses qualitative methods to develop and test information products. Qualitative methods help us gain insights and more in-depth understanding of information seeking and decision making about treatment s. The Eisenberg Center uses qualitative methods to elicit meaning about both individual and group experience of the topic. The two primary qualitative methods used in product development at the EC are focus groups and cognitive interviews. The goal of our qualitative methods is to understand audience information needs and responses to messages and materials.

Focus Groups

The group interview (or focus group) is a qualitative method conducted with 7-10 participants and facilitated by a skilled moderator (Krueger, 1994; Morgan, 1997). Focus groups are well suited to formative research because of their ability to generate new ideas through individuals’ perspectives and opinions as well as through group interaction and brainstorming (Patton, 2002). In addition, focus groups are an efficient method of data collection because many opinions can be gathered at one meeting (Krueger, 1994). A weakness of this method is that findings may not be generalizable to larger populations. However, they do have the benefit of providing rich data on one topic for a relatively low cost (Krueger, 1994; Morgan, 1997).

The EC uses focus groups in the formative phase of product development. The goal is to gain insights into how consumers view a topic and how they approach decision making about treatments for a particular condition. We conduct two focus groups for each formative round. We recruit participants from a convenience sample of volunteers. Based on identified audience characteristics, we screen volunteers for inclusion criteria such as experience with the treatments or conditions for which information materials are being developed.

The moderator begins by stating the purpose of the focus group and sharing ground rules for the discussion. The moderator follows a standardized interview guide that has been developed collaboratively by EC researchers. Discussions are audio taped and transcribed. EC investigators/researchers analyze transcripts and field notes for themes important to the development of information materials by EC researchers.

Sample Questions-Consumer Focus Group (Analgesics for Osteoarthritis)

• Let’s begin by talking about your experience with pain medicines you have used. What pain medicines are you currently taking? What kinds of pain medicines have you used in the past? How well have the medicines worked for you?

• How did you choose the pain medicines you have used? Who did you ask for advice about pain medicines? Did you talk with your doctor or pharmacist? How did that go?

• Is anybody looking to change medicine? Yes? How will you decide what to take?

• Based on your experiences, what concerns do you have about the pain medicines you have used in the past or are currently using?

• Overall, what are the most important factors for you personally when choosing pain medicine to treat your osteoarthritis?

• Where would you go to find information when making a decision regarding what pain medicines to take? Have you ever received any information regarding the safety and effectiveness of the pain medicines you have used? Did you use that information when making a decision about which medicine to take? What information sources would you believe to be more (or less) trustworthy?

• If you were to be mailed a brochure regarding the safety and effectiveness of pain medicines for osteoarthritis what information would it absolutely need to include? What information is of little importance to include in a brochure such as this?

Cognitive Interviews

Cognitive interviewing is a qualitative method in which individuals are asked to verbalize thoughts and feelings as they read through text materials (Willis, 2005). This technique is used to discover problems with draft information materials. By asking the interviewee to “think out loud” while reviewing written materials, the interviewer can gain insight into the cognitive processes of the reader (Willis, 2005). By conducting one-on-one interviews, individuals are afforded the privacy and flexibility that are needed to allow them to discuss reactions, reveal interpretations, and communicate how they might use the material in decision making.

The EC uses cognitive interviews to assess the understandability and usefulness of draft material. We conduct two rounds of cognitive testing for each information product. Five to nine people are interviewed per round. After each session, EC investigators/researchers analyze field notes and audio tapes for important themes that emerge across and within the individual interviews. We identify and correct problems with text, graphics and layout. We continue this iterative process until no new problems are identified in the text.

We recruit participants from a convenience sample of volunteers. Interviews last between 30-60 minutes. We briefly instruct interviewees in the cognitive interviewing method of “think aloud” before beginning with text materials. The interviewer then guides the interviewee through the text materials. The interview protocol combines “think alouds” and summary questions. General, open-ended questions are followed by probes. Spontaneous probes are used to elicit maximum feedback. Interview guides are different depending on product and audience but all guides follow this basic format.

Sample Questions – Consumer Interview (Analgesics for Osteoarthritis)

To begin with, I just want you to look at the brochure and I would like to hear your general reactions. What do you think this brochure is about? Anything that comes to your mind; just say it out loud.

• What does that mean to you?

• Tell me a little about the photos here. Do you have any thoughts or feelings about those? Any comments? Anything else?

• How about the color and layout?

• If you saw this sitting in a rack somewhere would you pick it up?

Let’s go onto the next section. What is the first thing that your eyes go to on this page?

• What do you think this section is about?

• Go ahead and read this section and tell me in your own words what you read and what it meant to you?

• Tell me what you were thinking when you said, “Hmmmm.”

• I also heard you say….. (Interviewer paraphrases respondent comments). What did that mean to you?

• What are your thoughts on this?

• Would you prefer to have that information in the front sections or the closer to the back pages?

• Do you have any other feedback on this section?

I noticed your eyes were jumping back and forth on the pages. What were you thinking about while you looked at these sections?

• How about this term right here (points to “Rx”)? Do you recognize this abbreviation? Have you seen this before?

• What about this one (points to NSAIDs)? Have you seen this before?

This process continues until all of the text has been covered or allotted time runs out. We then ask open-ended summary questions and the interview is brought to a close.

Summary Questions

What, if any, of the information in this booklet was new to you?

Was anything missing? Anything you would take out?

Any other advice on how to improve the content or format of this booklet?

How do you think you might use the information in this booklet?

Did anything you read today make you re-think your current strategies for managing your arthritis?

Sample Questions – Clinician Interview (Analgesics for Osteoarthritis)

To start I would like you to just tell me what your overall general impression is of the material. Any thoughts, reactions, questions are fair game. The more frank you are about your reactions the better.

Okay, now let’s talk about the information on the first page. Do you have any comments or feedback about the first page?

• What do you think about the title?

• What is your reaction to the term “Clinician’s Guide”?

• How about the term “Clinical Bottom Line”? What do you think about the information in this box?

• What about the “confidence scale”? Have you seen this kind of thing anywhere else?

• What about layout?

What about the boxed information on page 2? Would this information be useful to you?

• How do you currently talk with your patients about risk?

How about the cost chart on the back? Would that be something that you might find useful?

• Where do you currently get information regarding drug costs?

• Is this information that is important to your patients?

Summary Questions

What would you expect this summary to cover?

Was anything new in this document?

What should be taken out, what should be added?

How would you expect to receive something like this? Mail? Email? Other?

Would you go to a website to get this? Download it to your PDA?

What would you tell a colleague about this summary?

What do you think about the page length, colors, and format?

Any other advice on how to improve the content or format of this summary?

References

Denzin, N.K. & Lincoln Y.S. (Eds). (2000). Handbook of qualitative research (2^nd Ed.). Sage: Thousand Oaks.

Kirk, J. & Miller, M.L. (1986). Reliability and validity of qualitative research (2^nd Ed.). Sage University Paper, Qualitative Research Methods series, Vol 1. Sage: Newbury Park.

Krueger, R.A. (1994). Focus groups: A practical guide for applied research (2^nd Ed.). Sage: Thousand Oaks.

Morgan, D.L. (1997). Focus group as qualitative research (2^nd Ed.). Sage University Paper, Qualitative Research Methods series, Vol 16. Sage: Thousand Oaks.

Patton, M.Q. (2002). Qualitative research and evaluation methods (3^rd Ed.). Sage: Thousand Oaks.

Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Sage: Thousand Oaks.

July 2006 Page 5