Form 3 Attachment D-2 – Post-Choice Test Questionnaire

Form Approved
OMB No. 0935-XXXX
Exp. Date XX/XX/20XX

Attachment D-2

SelectMD 2.0 Consumer Choice Experiment

POST-CHOICE QUESTIONNAIRE

Public reporting burden for this collection of information is estimated to average 20 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-XXXX) AHRQ, 540 Gaither Road, Room # 5036, Rockville, MD 20850.

[TO BE ADMINISTERED ONE WEEK FOLLOWING THE PRE-CHOICE QUESTIONNAIRE]

[DISPLAY]

You're now ready to move on to a website called SelectMD that's designed to help you choose a doctor. The first several screens will ask you some questions about what you prefer in a medical practice and introduceyou to the features of the website. You’ll then be free to roam about the website, examine whatever information looks interesting to you, and select whichever doctor seems the best.

Please be assured that even though you will be connected with a partner survey, your answers will be kept completely confidential as they are in all GfK surveys. Your privacy continues to be our top priority and, as always, you can contact us at 1-800-782-6899 or [email protected]

We’ll ask you to use that information in SelectMD to select one doctor who you think would be the best for you and your health needs. You can assume that you would pay the same amount for your medical care, regardless of which doctor you choose. Although you will not really be selecting a doctor, we’d like you to consider this choice as carefully as if you were making it for yourself. Since this is just an exercise; you will not be contacted by the doctor you “choose.”

After you have made your choice of doctor, you will then be automatically transferred back to the GfK website and will then be asked a set of questions about how you made your choice, the usefulness of the information available for that selection, and your confidence in the choice that you made. Please do not log-off after choosing your doctor; these follow-up questions are very important so that we can understand how you made your choice. And please do not begin this study until you have approximately 30 minutes you can spend, since it’s important that you answer questions about your choice right after picking a doctor.

[REDIRECTION TO EXTERNAL WEBSITE OCCURS AT THIS POINT. ONCE RESPONDENTS HAVE COMPLETED THEIR REVIEW OF THE EXTERNAL WEBSITE, THEY WILL RESUME THE SURVEY STARTING WITH Q13 BELOW.]

[POST-CHOICE SURVEY]

[GRID, SP ACROSS]

Q13. How easy or difficult was it for you to select a doctor?

Very easy	Somewhat easy	Neither easy nor difficult	Somewhat difficult	Very difficult
1	2	3	4	5

[GRID, SP ACROSS]

Q14. How satisfied were you with the choice of doctors available to you?

Very satisfied	Somewhat satisfied	Neither satisfied nor dissatisfied	Somewhat dissatisfied	Very dissatisfied
1	2	3	4	5

[GRID, SP ACROSS]

Q15. Would you recommend that your friends and family use a website like this one when they make their own choices about a primary care doctor?

Definitely recommend	Probably recommend	Not sure	Probably not recommend	Definitely not recommend
1	2	3	4	5

PROGRAMMER NOTE: For Q16 below, respondents would at first see only the three-item scale at the left (note matter much, matter some, matter a lot). If they identify MORE THAN THREE of these attributes as mattering a lot, they will then have a follow-on screen that will show them ONLY the attributes that they reported to matter a lot  they will be asked to select which three of these matter the most.

[GRID, SP ACROSS]

Q16. [C3] Based on the website that you just visited, how much did each of these factors matter in selecting the doctor whom you chose?

Attributes of Doctor [randomized order]	Not Matter Much	Matter Some	Matter A Lot	One of the Three That Mattered Most
Reputation [recommended by others]				
Bedside Manner [warmth, caring, good listener]				
Time Availability [not rushed during visits]				
Patient Complaints [few complaints or malpractice charges]				
Treatment Orientation [how aggressively treats illness]				
Trustworthy [choices on behalf of patient, not insurer]				
Proximity [close to home or work]				
Safety [avoiding medical errors]				
Technical Quality [gives patients best treatments and tests]				
Office Staff [friendly, courteous, helpful]				

[DISPLAY]

We’re now interested in knowing what you remember about the website and how useful the information was for you in choosing a doctor. We understand that you may not have looked at everything on theweb site; please feel free to tell us if you didn’t see some types of information.

[PROGRAMMER NOTE: RANDOMIZE AND RECORD THE FOLLOWING SET OF QUESTIONS:

-Q17/Q18

-Q19/Q20

-Q21/Q22

-Q23/Q24]

[GRID, SP ACROSS]

Q17. This question is about service quality, or what patients typically report when surveyed about their care and experiences with their doctors and office staff. These scores come from a survey of a scientific sample of each doctor’s patients.

Do you remember seeing this information?

Yes	No
1	2

[PROGRAMMER NOTE: Display in box as header on screen for Q18]

Survey results from patients about service quality.

[SHOW ALL Q17=1]

[GRID, SP ACROSS]

Q18. How easy or difficult was it to tell which doctors were best using this information?

Very easy	Somewhat easy	Neither easy nor difficult	Somewhat difficult	Very difficult
1	2	3	4	5

[GRID, SP ACROSS]

Q19. This question is about information on how closely a doctor’s provides the most effective treatments and preventive care.This type of information comes from records of the care doctors have provided to patients with certain common medical conditions.

Do you remember seeing this information?

Yes	No
1	2

[PROGRAMMER NOTE: Display in box as header on screen for Q20]

How often the doctor provides the most effective treatment and preventive care.

[GRID, SP ACROSS. SHOW ALL Q19=1]

Q20. How easy or difficult was it to tell which doctors were best using this information?

Very easy	Somewhat easy	Neither easy nor difficult	Somewhat difficult	Very difficult
1	2	3	4	5

[GRID, SP ACROSS]

Q21. This question is about information on how successful a doctor is in reducing medical errors. This type of information comes from the doctor’s practice.

Do you remember seeing this information?

Yes	No
1	2

[PROGRAMMER NOTE: Display in box as header on screen for Q22]

How successful a doctor is in reducing medical errors.

[GRID, SP ACROSS. SHOW ALL Q21=1]

Q22. How easy or difficult was it to tell which doctors were best using this information?

Very easy	Somewhat easy	Neither easy nor difficult	Somewhat difficult	Very difficult
1	2	3	4	5

[GRID, SP ACROSS]

Q23. This question is about reviews from patients about their experiences with a doctor. These are comments from patients written in their own words.

Do you remember seeing this information?

Yes	No
1	2

[PROGRAMMER NOTE: Display in box as header on screen for Q24]

Comments from individual patients about this doctor, in their own words.

[GRID, SP ACROSS. SHOW IF Q23=1]

Q24. How easy or difficult was it to tell which doctors were best using this information?

Very easy	Somewhat easy	Neither easy nor difficult	Somewhat difficult	Very difficult
1	2	3	4	5

[GRID, SP ACROSS]

Q25 [L1] Based on the website that you just visited, how well were you able to tell which doctors were best in the following ways?

Attributes of Doctor [randomized order for each respondent]	Had A Good Sense of This	Had Some Sense of This	Could NOT Judge This
Training [good medical school; board certification]			
Reputation [recommended by others]			
Bedside Manner [warm, caring; good listener]			
Time Availability [not rushed during visits]			
Patient Complaints [few complaints or malpractice charges]			
Treatment Orientation [how aggressively treats illness]			
Trustworthy [choices on behalf of patient, not insurer]			
Safety [avoiding medical errors]			
Proximity [close to home or work]			
Technical Quality [gives patients best treatment and tests]			
Office Staff [friendly, courteous, helpful]			
Affiliated with Insurance Plan [covered without extra copayments]			

Q26. [L4A – Open ended] Was there any information on the website that you found surprising?

 No

 Yes  Please describe what was surprising

Q27 [L4B – open ended]. Was there anything on the website that led you to think differently about how you should go about assessing or selecting a doctor?

 No

 Yes  Please describe what you learned, as if you were describing it to your family or friends so that they could learn from your experiences

Q28. [T2A --Open-ended]Please describe the way in which you went about selecting the doctor whom you chose? First, we’d like to understand your decision in your own words. Imagine that you were describing this to a family member or friend, so that they could learn from your experience.

[MP]

Q29[T2B]Now consider some specific ways in which you might have gone about making your decision. Which of the following describe how you chose a doctor on this website?

[Check all that apply]

Yes	No
1	2

1. I picked the doctor who seemed most likeable to me

2. I focused on the one quality rating that seemed most important to me

3. I searched for a doctor who was good enough, rather than trying to figure out who was the best

4. I tried to take all of the ratings into account

5. I figured out how much each quality rating mattered to me, then chose a doctor based on the ratings that seemed most important

6. None of the above: I went about choosing in a different way [SP]

[Please describe: ___________________________________________________

__________________________________________________________________

__________________________________________________________________]

[GRID, SP ACROSS]

Q30. [D2] Sometimes making sense of reports on quality can just feel too difficult to be worth the effort. Based on the website that you just visited, how much did each of these sources of information matter in selecting the doctor whom you chose?

Sources of Information [randomized]	Only Source That Mattered	One of Several Sources That Were Important	Took Into Account, But Not A Major Influence	Was Too Difficult to Use
Survey of Patient Experience				
Clinical Records on Treatment/Screening				
Practices Promoting Patient Safety				
Comments from Patients (in their own words)				

[GRID, SP ACROSS]

Q31 [D3] If you were to be faced with a choice of primary care clinician in real life and had a choice of websites reporting information about these doctors, which of the following types of information would you look for on a website?

Sources of Information [randomized]	Absolutely Essential	Important, But Not Essential	Useful, But Not Worth the Effort	Not Useful
Survey of Patient Experience				
Clinical Records on Treatment/Screening				
Practices Promoting Patient Safety				
Comments from Patients (in their own words)				

[GRID, SP ACROSS]

Q32. [L2].Good medical care requires that doctors have a variety of skills, including technical knowledge about treatments, ability to relate to their patients, helping patients select the best treatments and avoiding errors.Based on what you have heard, read or experienced yourself, do you think that doctors who are good at any one of these skills will also be good at the others? Or do some doctors have certain skills, but not so much the others? If you were choosing between the follow pairs of statements, with which do you agree?

A. Doctors who know the technical aspects of treatment are usually not so good relating to their patients			A Bit of Both			Doctors who know technical aspects of medical care are usually also good at relating to their patients
	1	2	3	4	5

B. Doctors who select the best treatments usually are not so good at avoiding medical errors			A Bit of Both			Doctors who can select the best treatments usually are also good at avoiding medical errors
	1	2	3	4	5

[GRID, SP ACROSS]

Q33 [L3] Based on what you have heard, read or experienced yourself, do you think that doctors who provide primary care (check-ups, screening for illnesses, referrals to specialists) are equally good treating all patients, or are some doctors particularly helpful for patients with particular needs or preferences about their medical care? If you were choosing between the follow pairs of statements, with which do you agree?

A. A primary care doctor will be equally good diagnosing and treating all patients, whatever their health problems			A Bit of Both			Primary care doctors are often particularly good with some specific health problems, but not so much others
	1	2	3	4	5

B. When reading comments from patients treated by a primary care doctor, it doesn’t really matter which of them is writing about their care,			A Bit of Both			To really understand patients’ comments about primary care doctors, one has to know what these patients expected from medical care.
	1	2	3	4	5

[GRID, SP ACROSS. RANDOMIZE AND RECORD]

Q34 [T3]How much to you agree or disagree with the follow statements about choosing a doctor?

Completely Disagree	Somewhat Disagree	Neither AgreeNor Disagree	Somewhat Agree	Completely Agree
				

1. I would be willing to accept a higher rate of medical errors in a doctor who was warm and caring

2. I would be willing to accept a doctor who had less time to talk to me about my health care needs if they had a good track record for avoiding medical errors.

3. I would be willing to accept a doctor who was less warm and caring if they had plenty of time to talk with me about my medical care.

[GRID, SP ACROSS. RANDOMIZE AND RECORD]

Q35. [A2] We’d like to ask you some questions about how you approach your health care. Please indicate how much you agree or disagree with each of the following statements.

Completely disagree	Somewhat disagree	Neither agree nor disagree	Somewhat agree	Completely agree
1	2	3	4	5

1. I take primary responsibility for managing my own health

2. I am confident that I know when I need health care

3. If I were taking several medications, I would always bring a complete list each time I went for health care

4. I know how to find information about quality of care for doctors and hospitals.

5. I am confident that I can use the information available to me to choose a good health care provider

6. If something went wrong with my health care, I would make an effort to fix the problem.

[GRID, SP ACROSS. RANDOMIZE AND RECORD]

Q36. [E1] Suppose you were choosing between two doctors, one of whom was in your health plan’s network of providers and could be seen at no cost to you. The second doctor was rated higher on quality-reporting websites, but would cost you $50 out-of-pocket for each visit to their office – and you expected to visit the doctor several times each year. How anxious would you feelabout choosing the lower cost doctor if you knew that:

Not At All Anxious	A Little Anxious	Somewhat Anxious	Very Anxious	Extremely Anxious
				

1. This lower-cost doctor was described by his current patients as having little time to talk with them about their health and treatment choices

2. This lower-cost doctor had an above-average number of patients complaining about being treated impersonally

3. This lower cost doctor had made an above-average number of medical errors

[GRID, SP ACROSS]

Q37A. [E2A] If you were choosing among primary care doctors in real life, how anxious would you feel about making the best choice?

Not At All Anxious	A Little Anxious	Somewhat Anxious	Very Anxious	Extremely Anxious
				

[GRID, SP ACROSS]

Q37B[E2B] If you were choosing among primary care doctors in real life for a family member who was seriously ill, how anxious would you feel about making the best choice?

Not At All Anxious	A Little Anxious	Somewhat Anxious	Very Anxious	Extremely Anxious
				

[GRID, SP ACROSS. RANDOMIZE AND RECORD]

Q38.We’d like to ask you some questions about how you usually make decisions in your life. Please indicate how much you agree or disagree with each of the following statements:

Completely disagree	Somewhat disagree	Neither agree nor disagree	Somewhat agree	Completely agree
1	2	3	4	5

a. Whenever I make a choice, I’m curious about what would have happened if I had chosen differently.

b. Whenever I make a choice, I try to get information about how the other alternatives turned out.

c. If I make a choice and it turns out well, I still feel like something of a failure if I find out that the other choice would have turned out better.

d. When I think about how I’m doing in life, I often assess opportunities I have passed up.

e. Once I make a decision, I don’t look back.

f. When I watch TV, I channel search, often scanning through the available options even while attempting to watch one program.

g. While I am in the car listening to the radio, I often check other stations to see if something better is playing, even if I’m relatively satisfied with what I’m listening to.

h. Renting movies is really difficult. I’m always struggling to pick the best one.

i. No matter what I do, I have the highest standards for myself.

j. I never settle for second best.

k. When making decisions, I rely on my instincts.

l. I often need the assistance of other people when making important decisions.

m. I avoid making important decisions until the pressure is on.

n. I make decisions in a logical and systematic way.

o. I rarely make important decisions without consulting other people.

p. My decision-making requires careful thought.

q. When I make a decision, I trust my inner feelings and reactions.

r. I often procrastinate when it comes to making important decisions.

s. I use the advice of other people in making my important decisions.

[PROGRAMMER NOTE: Show the table for all questions Q30a-d. SHOW GRIDLINES]

[SP]

Q39a. Finally, we’d like to learn more about how you make decisions about consumer products. The information below represents a choice among DVD players. The ratings run from 1=Very Low to 3= Average to 5=Very High, with a higher number signifying better product performance.

			Features
DVD Brand	Picture Quality	Sound Quality	Programming Options	Reliability of Brand	Price
A	2	5	5	1	$199
B	1	2	5	2	$199
C	5	5	4	2	$199
D	3	3	5	3	$199
E	2	5	2	4	$199

If one of your friends told you that he wanted to buy the most reliable DVD of this bunch, which would you recommend that he select?

Brand A 1

Brand B 2

Brand C 3

Brand D 4

Brand E 5

[SP]

Q39b. If another friend told you that she wanted to buy a DVD with the very highest sound quality and, among those with the best sound, the one with the best programming options, which would you recommend that she select?

Brand A 1

Brand B 2

Brand C 3

Brand D 4

Brand E 5

[SP]

Q39c. If a different friend told you that he cared about all these features equally and wanted to buy the DVD that had the best scores overall, which would you suggest that he select?

Brand A 1

Brand B 2

Brand C 3

Brand D 4

Brand E 5

[SP]

Q39d. If one other friend told you that she wanted a DVD that scored no worse than average on every feature, which would you advise her to select?

Brand A 1

Brand B 2

Brand C 3

Brand D 4

Brand E 5