Evaluation of AHRQ's Guide to Clincial Preventive Services--Semi-structured Interview

MODERATOR'S GUIDE FOR AHRQ TELEPHONE & IN-PERSON SEMI-STRUCTURED INTERVIEWS

[The moderator’s guide outlines key areas to be probed and issues to be discussed, thus it contains questions which will be asked to participants during the telephone & in-person interviews. Approximately 20-25 open-ended questions should be used for the interview.]

A. Preparation for the Telephone and In-Person Interviews

Prior Preparation. For both the telephone and in-person interviews, have the person’s complete name in front of you along with their survey responses and your script.

Other Considerations. For telephone interviews, since participants are receiving an incentive for taking part in the interview, at the end of the interview, confirm their mailing address and inform them that their incentive will be mailed to them. Reassure participants that their name and address will not in any way be linked to their survey responses or to information they provide during the interview.

For in-person interviews, at the completion of the interview thank the person for their participation and give them their incentive.

Moderating the Telephone and In-Person Interviews

Length of the Interview. The telephone and in-person interviews should last approximately 30 minutes but no longer than 1 hour. The participant will dictate the length of the interview based on the amount of information they provide and their willingness to participate.

Maintain a Neutral Tone. The interviewer will maintain an attitude of acceptance and use emotionally neutral language in order to establish rapport with the participant and encourage the participant’s free expression of his or her positive and negative perceptions and attitudes with respect to the Guide to Clinical Preventive Services (Guide).

B. Preparation for the Interview Session

Room Configuration. If this is an in-person interview, the conference room should be set up and ready for the interview when the participant arrives. The participant should be seated at table with the moderator at the head and a tape recorder placed unobtrusively.

Refreshments. If applicable, refreshments should be placed on a separate table in the room. When the participant enters he/she may take refreshments and be seated at the table.

Other Considerations. Since participants are receiving an incentive for taking part in the interview, the best and easiest way is to provide it right after the completion of the interview session.

Moderating the Interview Session

Keep the Conversation Flowing. The participant is encouraged to talk freely without any fear of disapproval. However, the moderator needs to keep control of the interview session. If the participant gets off track or get ahead of the issue being discussed, the moderator must re-focus the participant. The moderator should try to keep the discussion as informal as possible and should encourage the participant to speak whatever is on his/her mind.

Length of the Interview. The interview session should last approximately 30 minutes but no longer than 1 hour. The participant will dictate the length of the sessions based on the amount of information they have and their willingness to participate.

Be Neutral. One of the benefits of having an external person moderate the interview is that the person can be neutral with respect to whatever perceptions are offered by the participant. Participants should be encouraged to express the full range of their positive and/or negative perceptions and attitudes about the Guide to Clinical Preventive Services (Guide).

C. MODERATOR'S GUIDE

Introduction

Hello. Good morning/afternoon. My name is ___Moderator's name . I’m calling for ___Insert person’s name____________. (If person is not at home/available, inquire when will that person be home/available. If the person is available, then proceed with the interview.) I’m calling on behalf of the Agency for Healthcare Research and Quality (AHRQ) about a survey s/he agreed to participate in to provide us with feedback on the Guide to Clinical Preventive Services. I work for JBS International, a contractor for the Agency for Healthcare Research and Quality (AHRQ), and we are assisting them in the gathering of valuable information about your use of this Guide in clinical practice.

This interview will last approximately 30 minutes. Is this still a good time for you to talk with me about the Guide? (If no, reschedule; if yes, continue). Very good. First, let me start by thanking you for helping us in our efforts. Before we begin, I’d like your consent to record our discussion. I want to assure you that this discussion is completely voluntary and any information you provide will be kept anonymous and confidential. If there are any questions that you are uncomfortable with or find objectionable, you may refuse to answer. You may also end your participation at any time without consequence. If you agree to be recorded, the information from this discussion will be transcribed and then summarized. Any identifiers, which include names, places, organizations, etc., will be removed from the transcripts so that no one (including anyone at AHRQ) will know who provided what comments. AHRQ will use the summarized information to improve the Guide and other related products to better meet the needs of the healthcare community.

Do I have your permission to record this discussion?

[NOTE: For legal and ethical purposes, the moderator must inform participants that they are being recorded during the session and why.]

DISCUSSION GUIDE (30 minutes)

A. Use of the Guide to Clinical Preventive Services

You indicated that you’ve used the following format(s) of the Guide and that the one you use the most is _________. You said that you use this format the most because ______________. Is this still correct? [Note: may need to remind participant of all the versions of the Guide (print, ePSS-PDA, ePSS-online, online) as a context for their response]

1. Please tell me a little about why you selected to use this format of the Guide to Clinical Preventive Services (Guide)?

2. Do you ever refer to other formats of the Guide? [Probe: How did you use each format? Probe regarding setting, audience. What about that format made it seem better to use for this (purpose/setting)? [e.g., Did one seem easier to refer to one while seeing a patient and another seem better for obtaining additional information?]

I’d like to ask you some additional questions about your use of the guide. AHRQ is especially interested in learning more about your use of the Guide, how you have used it, what you’ve done with the information obtained from the Guide, and how to make it even more useful to you and other healthcare professionals

3. I’m interested in how you typically use the Guide. Please step me through a scenario (or several) of how you use the Guide. (Probe: Under what circumstances do you use the Guide? In what setting(s)? With what audiences (patients, colleagues, students)? What parts of the Guide did you use? (Did you use particular parts of the Guide? If so, which ones?) [Possible options: guidelines for screening; chart of recommendations for different groups; vaccination schedules; particular diagnostic categories or risk groups?]

4. Are you aware that the recommendations in the Guide carry ratings to designate the extent (level) of empirical evidence supporting the recommendations and magnitude of net benefit associated with them? (A=Strongly recommended; B=Recommended; C=Not For or Against; D=Recommend Against; I=Insufficient Evidence)?

5. To what extent do you attend to the rating when using the Guide to decide on screening tools to use or recommend to others? How does the rating impact your decision, if at all? (Probe: How does interviewee consider recommendations labeled as C (= not for or against) or as I (= insufficient evidence)?

6. Did you find the Guide easy to use? (Probe: What about it was easy to use? What made it difficult to use? What might make it easier for you to use the Guide?)

7. What do you find most beneficial about the Guide? What do you like about the Guide? Why?

8. How easy or difficult was it for you to implement the recommendations in the Guide? What helped make it easy? What made it more difficult or got in the way of implementing the recommendations?

9. Would you recommend the Guide to a friend? To a colleague? Why? Why not?

10. To your knowledge, to what extent is the Guide being used by other staff at your organization/hospital/practice? [Probe: How can AHRQ interest others in using the Guide?; Is there any special terminology that would help in catching the attention of your peers and getting them to use the Guide?

B. Barriers to Using the Guide, Access to the Guide, and Recommendations for Improving the Guide

11. What, if anything, gets in the way of applying the information from the Guide? (Probe: Any institutional barriers? If so, have you been able to overcome them? How?)

12. What additional support/assistance would you like, if any, to help you to overcome these barriers?

13. What suggestions, if any, do you have for improving the format of the Guide based on your experience with it?

14. What suggestions, if any do you have for improving the content/recommendations in the Guide based on your experience with it?

15. When you agreed to participate in the survey when you ordered the guide, how did you obtain the Guide (e.g., via download, via mail)?

1. Do you have any suggestions for improving how the Guide can be distributed to physicians and other primary healthcare providers?

1. Have you obtained any updates to the Guide? If so, how have you learned about updates to the Guide? (Probe: Are the updates easy to access & incorporate into your clinical practice?) If not, where would you expect to learn about updates to the Guide?

C. General Impression of AHRQ, the U.S. Preventive Services Task Force

3. Before today, were you aware of the U.S. Preventive Services Task Force?

4. What comes to mind when I mention the Task Force?

5. What are your general impressions of the Task Force, its resources and services?

D. Closing (5 minutes)

6. In closing, what other comments or suggestions do you have related to the Guide?

7. What other comments or suggestions do you have related to AHRQ?

8. Are there any final comments you would like to add?

[For Telephonic interview, inform the participant that their incentive will be mailed to the address which they provided. This would be a good time to confirm mailing address.

For In-Person interview, give the participant the incentive at the conclusion of the interview.]

Thank you very much for taking the time to provide AHRQ with your feedback. AHRQ will use the information you provided to make changes to its publications to better serve you and the healthcare community.