10-28-07 memo

Dear Ms. Matsuoka,

Here are the responses to your comments and questions:

This is a very interesting topic and OMB agrees it is very important. However, we would like more information on how the survey and study design will yield the type of data that is needed to answer the research question. See below.

• Please submit a copy of the recruitment letter

We would like to revise our plans and revert to a mailed survey. We have attached the recruitment letter.

• What is changing from the previously approved collection? Is this revision request a request to implement a “full-scale” study, whereas the previous request was to implement the pilot?

Little will be changed. Our application to you now stems from the fact that we were not able to conduct the approved study earlier and we are attempting to do so now.

• Please provide the results from the pilot and a description of the pilot

As noted above we have not had the opportunity to conduct the study with US physicians. We are attaching the results of the survey done in Europe which may help to give you a clearer idea of the types of results we expect from the US survey.

• Given the complicated nature of this ethical dilemma, is a questionnaire the right mechanism to elicit the data you are after? Wouldn’t in-depth qualitative research provide the degree of nuance this topic requires?

We believe the results of the survey done in Europe which used the identical questions provide evidence that the questionnaire can yield the data we are interested in.

• How did NIH decide to sample 500 doctors? Was a power analysis conducted?

We anticipate that 60% may be eligible and that among those who are eligible we will get a response rate of 50%. This would give us a final sample of 150. For the purpose of determining this sample size, we use the outcome of question 4 (physician self-reported rationing), which is derived from multiple survey questions and has a scale of 0 to 44. We assume the following:

1. Two-sided alpha of .025. This alpha will takes into account testing multiple hypotheses and the potential correlation of this outcome to the other study outcomes.

2. 90% power to detect a difference in mean Physician Reported Rationing between at least two countries.

3. That two countries will yield mean scores of 5 units less than the overall mean across all countries and that two countries will yield mean scores of 5 units more than the overall mean across all countries. An example of this might be: 2 countries with mean scores of 15, one country with a mean score of 20, and 2 countries with a mean score of 25.

4. A standard deviation of 25 within each country.

We used the conservative assumptions of a high variance within each country and of a low difference between the mean scores of each country on the self-reported rationing scale. Assuming differences across the four European countries and the US is tested using anova, and assuming that the standard deviation within physicians in a country is 25 and that the standard deviation of means across countries is 4.5, we require 128 physicians per country. This provides power of 90% with a two-sided alpha of 0.025. As we will need to adjust for other factors (including physician type, age, etc.), we will be performing a multiple linear regression. As a result, the estimate of 128 per group is an underestimate of the true sample size required. However, due to the unknown nature of the relationship between the other covariates and the outcome (and each other), we cannot accurately make further estimates of sample size but have thus conservatively enrolled more than 128 per country to account for the necessary adjustments in the analysis.

• A 70% response rate among physicians is quite ambitious. What kinds of pre-notifications and follow-up will be conducted to maximize response rates? What will NIH do if the response rate is much lower as far as non-response bias?

We agree that 70% is an optimistic expectation regarding the response rate. In our prior survey we got a response rate slightly over 50% and we anticipate the same in the US. We plan to use duplicate mailings and a small incentive of $20.

• The first few questions on the questionnaire appear to be screening questions. First, is there a way to screen for physicians that meet the inclusion criteria directly from the master list, prior to sampling? Second, where is the burden accounted for these screening questions?

The AMA American Medical Association Master List of Physicians and Medical Students for Mailing Purposes does not provide data sufficient to exclude physicians who do not meet the screening criteria. We anticipate that noneligible physicians are likely to take less than 5 minutes to determine that they are not eligible. Assuming that 200 physicians are ineligible and they each take 5 minutes to read the survey before discarding it, this is a response burden of 1000 minutes or 16.6 hours.

• According to the data collection timeline, data collection will take place for 6 months. During these 6 months, if you were to receive a FOIA request, would you be able to withstand the FOIA request and maintain the confidentiality of the data collected?

We data we will be collecting do not contain any personally identifiable information and we will not have performed computer entry and aggregation of the results so we believe that from a practical standpoint we could withstand a FOIA request.

• Conceptually, it is a bit unclear why this study would be appropriate in the US healthcare context. In the UK/European healthcare context, because healthcare spending is capped, physicians may need to allocate resources so that their fixed funds are used in the most cost-effective way. However, in the US healthcare system, spending is not capped. If physicians do deny care, I would imagine this is because of financial incentives or penalties they face from health plans, rather than out of considerations for “scarce resources.” Please clarify.

From the survey results we have collected in Europe we find that scarce resources such as limited ICU beds, nursing home beds, rehabilitation services are rather ubiquitous and we anticipate that this might be the case in the US as well. As you suggest financial incentives, such as DRGs that require hospital discharge in a timely manner, pose substantial pressures on physicians to limit resource use.

• Relatedly, I would imagine that the questionnaire would not yield much variety of response from physicians. For example, does NIH truly think that physicians will answer “yes” to a question like “how often did you refrain from providing care to a patient, even though it would have been the best intervention, because of cost to the health care system?” (question #7) Is it really attitudes towards cost-containment that NIH is looking for, or is it attitudes towards practicing cost-effective medicine? In other words, is the question whether a physician would steer that patient to a less costly but equally effective “next best alternative”?

Please see the attached paper which indicates significant variation in rationing. The attitudes we are looking for are related to the acceptability of bedside rationing – i.e. forging some interventions that may have some marginal value because of cost concerns.

• How will this study separate out any effects from health plans from physician attitudes towards cost-effectiveness? Isn’t the situation you are really interested in the situation where the physician is not restricted by any health plans from offering treatment A or B, treatment B is much more expensive but only marginally more beneficial, and the extent to which physicians will deny treatment B and offer treatment A? If so, it seems like a survey which presented a case study with relevant cost and benefit information and asked respondents to indicate how they would treat this hypothetical patient would yield more useful information.

We agree that the sort of scenario you describe regarding a hypothetical patient might yield very interesting information. However to the extent that we wish to perform a cross-country comparison, we think it is preferable to keep the questionnaire similar to the one used before.

• Is the study team aware of the “Visible Fairness” study on physician attitudes towards cost-effectiveness? Are there questions used in that study that would be useful to use in this study? (see attached)

We are aware of the Visible Fairness study. As we have already noted, we believe it is preferable to avoid modifying the questionnaire since we hope to have comparable results among US physicians and the European physicians who have already taken the survey.

• How are the sub-questions in question #1 relevant to this research question? It seems like only 1h and maybe 1g are relevant.

We include this broad question about ethical dilemmas in general to gain information about the relative frequency and difficulty of rationing issues compared to other ethical dilemmas that physicians encounter.

• As a practical matter, since this is a phone interview, it seems a bit implausible that respondents will remember all items 1a-1l when answering question 2. How will NIH ensure that respondents are not biased towards the last few response options, since those are the freshest in their minds?

As noted above, we have now decided to revert to a mailed survey partly because we think we could get more comparable results to the European survey and because we share the concern you raise here.

• What is question #5 trying to get at? How is this a question about denying costly care to patients?

We ask about availability of ethics support services because we are interested in exploring the feasibility of ethicists assisting clinicians to address difficult rationing issues.

• Question 9: would be useful to know how physicians would respond to a situation where an intervention a) improves quality of life with no impact on life expectancy (e.g. Viagra), b) where the main benefit of an intervention is convenience to the patient (e.g. controlled release medications that patients take once/month rather than daily), and c) where the main benefit of an intervention is convenience to the patient which thereby improves treatment compliance (e.g. insulin pumps for diabetics rather than daily injections).

Thank you for these suggestions. We are adding items regarding issues to question 9. We do believe this will yield interesting information.