Evaluation of the Implementation and Impact of Pay-for-Quality Programs

Appendix to Supporting Statement

1. Survey Cover Letter

2. Physician Survey

3. Key Informant Interview Advance Letter

4. Key Informant Interview Protocol

Dear Colleague:

We seek your participation in a U.S. Department of Health and Human Services study concerning the role of financial incentives in improving quality of care in safety net settings. As you know, there is growing interest among public and private payers to link financial incentives to quality targets. By completing the enclosed questionnaire, you will have a unique opportunity to influence national health policy on this important topic.

The U.S. Department of Health and Human Services has selected faculty from the Boston University Schools of Medicine and Public Health to conduct this survey. In cooperation with Dr. Richard Kalish and Dr. John Robinson at Boston Medical Center Health Plan, we have developed a questionnaire specifically tailored to your practice setting and community. Responses will be used to describe physician views on using financial incentives in safety net settings and to inform policy at Boston Medical Center.

The questionnaire can be completed within 15 minutes. Your responses will be treated with complete confidentiality and will not be shared with your employer. We greatly appreciate your help in what we believe to be a most important effort.

Gary Young, Ph.D. Richard Kalish, M.D. John H. Robinson, M.D.

Principal Investigator Medical Director Chief Medical Officer

Boston University BMC Health Plan BMC Health Plan

Quality Targets and Incentives Survey

Dear Colleague:

• You are being asked to participate in the following survey because you work in a setting that is involved in a project concerning improvement in patient care and the use of financial incentives. We were selected by the Agency for Healthcare Research and Quality to conduct an evaluation of this project. Responses to these questions will allow us to identify factors that contribute to successful patient care improvement efforts. The survey should take no more than 15 minutes to complete.

• The purpose of this study is to describe how financial incentives contribute to patient care improvement efforts and how physicians perceive these incentives.

• You will be one of approximately 300 subjects to be asked to participate at Boston Medical Center. A total of 600 subjects at all institutions will be asked to participate in this study.

• This survey asks for your opinions about a variety of practices in your workplace or in organizations you may work closely with. The survey also includes some demographic questions. Please return the survey in the enclosed, pre-paid envelope. Taking part in this study is voluntary. Completing a survey indicates your consent to participate in the study. You may choose to end your participation in the survey at any time. Your responses to the survey will be strictly confidential. The questions do not focus on highly personal information. Information from this study may be used for research purposes and may be published; however, individual responses will never be reported.

• There are no known risks for participating in this study. The study may inconvenience you in terms of the time or the effort that it takes to read and answer each of the questions. At no time will the responses you give in the survey be shared with your employer. There may be unknown risks/discomforts involved. Study staff will update you in a timely way on any new information that may affect your health, welfare, or decision to stay in this study.

• You will receive no direct benefit from your participation in this study. However, your participation may help the investigators better understand financial incentive programs. We will provide you and leaders in your organization with anonymous feedback from the survey that may be beneficial in planning quality improvement efforts. There are no costs to you or your organization for participating. Your alternative is to not participate in the study. You will not be paid to participate in this research study.

• Each questionnaire is numbered to identify respondents. The identifier will only be used in the data collection phase. We will send a follow-up questionnaire to those who do not return the initial questionnaire. Once we have a complete record for each participant, all identifying information will be removed from the database. All analyses will be completed using private and confidential databases. The only people who have access to these data will be project staff who are required to keep the data private and confidential. Your employer will not have access to your survey answers. Information from this study may be reviewed and photocopied by state and federal regulatory agencies such as the Office of Human Research Protection as applicable, and the Institutional Review Board of Boston University Medical Center. Information from this study may be used for research purposes and may be published; however, your name will not be used in any publications.

• By consenting to participate in this study you do not waive any of your legal rights. Giving consent means that you have heard or read the information about this study and that you agree to participate. You will be given a copy of this form to keep. If at any time you withdraw from this study you will not suffer any penalty or lose any benefits to which you are entitled.

• You may obtain further information about your rights as a research subject by calling the Office of the Institutional Review Board of Boston University Medical Center at 617-638-7207. The investigator or a member of the research team will try to answer all of your questions. If you have questions or concerns at any time, or if you need to report an injury while participating in this research, contact Gary Young at (617) 414-1423 during the day and Karen Sautter at (781) 424-1477 after hours.

• Taking part in this study is voluntary. You have the right to refuse to take part in this study. If you decide to be in the study and then change your mind, you can withdraw from the research. Your participation is completely up to you. Your decision will not affect your being able to get health care at this institution or payment for your health care. It will not affect your enrollment in any health plan or benefits you can get. If you choose to take part, you have the right to stop at any time.

We hope you will take the time to complete this survey. We greatly appreciate your cooperation.

Gary Young, PhD, Karen Sautter, MPH,

Principal Investigator Study Coordinator

Section 1. Quality Targets and Incentives in General

These first questions ask for your thoughts concerning the use of incentives attached to quality targets in health care and performance measures in general.

1. Physicians should be rewarded when they provide higher quality care.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

2. Financial incentives are an effective way to improve the quality of health care.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

3. Financial rewards are more effective as an incentive compared to non-financial rewards such as peer recognition.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

4. Physicians are aware of the financial incentive features of health plan contracts that apply to them.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

5. Financial incentives aside, reaching the quality targets used in incentive programs is good for patients.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

6. Physicians’ efforts to achieve quality targets hinder them from providing other essential medical services.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

7. Physicians have a professional responsibility to measure the outcomes of their care.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

8. Feedback on specific clinical performance measures helps physicians identify opportunities to improve the quality of patient care.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

9. The primary motivation for performance measures is to improve the quality of patient care.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

10. Most physicians are able to achieve the quality targets set by health plans and other payers.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

11. It is informative for physicians to compare their performance with that of their peers.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

12. Financial incentives for achieving quality targets are most effective when linked to the performance of individual physicians (rather than the performance of a team or group practice overall).

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

13. Below please find a list of selected clinical procedures and quality indicators. You or your practice may be eligible for financial incentives for achieving certain specific quality targets related to these procedures and indicators. For each one that is applicable, please indicate whether you are aware of the quality target(s) and any associated individual or group financial incentive.

		Aware of quality target(s)?				Aware that there is a financial incentive?
		Yes	No	Does Not Apply		Yes	No
A.	Asthma medication use	1	2	3		1	2
B.	Well-child visits in the first 15 months	1	2	3		1	2
C.	Diabetes retinal eye examination	1	2	3		1	2
D.	Patient retention	1	2	3		1	2
E.	Appointment availability	1	2	3		1	2

Section 2. Your Experience with a Specific Quality Target

14. To answer the next questions, it will be necessary to focus on a quality target and financial incentive (individual or group) associated with a particular clinical procedure or quality indicator. Detailed knowledge is not necessary in order to answer the questions, however. Please select one of your highest high volume procedures to use as your focus for this section of the survey. If you practice in more than one location, please answer the questions in this section with regard to the medical practice where you received this questionnaire.

• Asthma medication use

• Well-child visits in the first 15 months

• Diabetes retinal eye examination

• N

  If not aware, please skip to Question 46.

  ot aware of both a quality target and the individual or group financial incentive related to any of these procedures/indicators

15. About what percent of your patients have are eligible for the procedure you selected in Question 14?

• Less than 5% of my patients

• 5% to 9% of my patients

• 10% to 14% of my patients

• 15% to 19% of my patients

• 20% to 24% of my patients

• 25% or more of my patients

• Not sure

15. When were you first made aware of the financial incentive associated with the procedure that you selected in Question 14?

• Within the past 6 months

• 7-12 months ago

• 13-24 months ago

• More than two years ago

• I cannot recall

15. Who has given you the most information about the financial incentive associated with the procedure that you selected in Question 14?

• A health plan

• The manager or medical director of my practice

• Other clinicians

• Other - please specify: _____________________________

15. If you achieve the quality target and qualify for the financial incentive, how is the money distributed?

• Entirely to me

• Apportioned between me and the medical practice/IPA

• Entirely to the medical practice/IPA

• Not sure

15. Have you personally received any payment related to the financial incentive you selected in Question 14?

• Yes

• No

• Not sure

Please continue with the one quality target and associated financial incentive that you selected in Question 14 in mind.		Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
20	I have adequate information about the definition of the quality target.	1	2	3	4	5
21	This financial incentive represents an opportunity for me to appreciably increase my income.	1	2	3	4	5
22	This financial incentive is tied to a quality target that is clinically meaningful.	1	2	3	4	5
23	Efforts to obtain this financial incentive hinder me from providing other essential medical services to this group of patients.	1	2	3	4	5
24	The actions necessary to obtain this financial incentive are largely within my control.	1	2	3	4	5
25	Because of the clinical characteristics of my patients, it is more difficult for me to obtain this financial incentive than it is for other physicians.	1	2	3	4	5
26	The data used to assess achievement of the quality target are accurate.	1	2	3	4	5
27	I have adequate information about the scoring system used to compute the incentive amount.	1	2	3	4	5
28	I know the amount of the financial incentive I/my practice will receive if I achieve the quality target.	1	2	3	4	5
29	This financial incentive is sufficiently large to compensate for expenditures that might be necessary in order to meet the quality target.	1	2	3	4	5
30	This financial incentive is tied to a quality target based on sound medical science.	1	2	3	4	5
31	The effort required to obtain this financial incentive has an adverse impact on other types of patients in my practice.	1	2	3	4	5
32	I would be just as focused on this quality target without the financial incentive.	1	2	3	4	5
33	New information technology has been required to obtain this financial incentive.	1	2	3	4	5
34	Physicians are on a level playing field for obtaining this financial incentive.	1	2	3	4	5
35	I get useful feedback regarding my progress toward achieving the quality target.	1	2	3	4	5
36	I invest extra time and effort in the care of those patients who are the focus of this financial incentive.	1	2	3	4	5
37	Obtaining the financial incentive brings me favorable recognition from my colleagues.	1	2	3	4	5
38	The financial incentive aside, reaching this quality target is good for my patients.	1	2	3	4	5
39	I have changed my practice behavior to obtain this financial incentive.	1	2	3	4	5
40	I receive useful assistance in response to my questions or concerns regarding the data related to this quality target.	1	2	3	4	5
41	Overall, my patients who are the focus of this financial incentive are getting better care.	1	2	3	4	5
42	I am able to get the cooperation of other physicians as needed to obtain this financial incentive.	1	2	3	4	5
43	I am able to get the cooperation of support staff as needed to obtain this financial incentive.	1	2	3	4	5
44	The financial incentive is applied fairly to physicians based on their performance.	1	2	3	4	5
45	The quality target helps me focus my time and effort constructively.	1	2	3	4	5

Section 3. Your Experiences with Patients’ Health Plans

Please answer the next questions on the basis of your typical experience with your patients’ health plans during the past six months.

46. The health plans seek to maintain good relationships with practitioners.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

46. The health plans want to help me take better care of my patients.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

46. The health plans interfere with how I care for my patients.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

46. The health plans understand my situation and needs as a practitioner.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

46. The health plans have confidence in my ability to offer high quality care to my patients.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

46. The health plans encourage my questions and feedback about quality targets.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

46. I read the health plans’ performance reports thoroughly.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

• Do not get reports

46. The health plans’ performance reports are accurate.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

• Do not get reports

Section 4. Your Medical Practice Overall

If you practice in more than one location, please answer the following questions with regard to the medical practice where you received this questionnaire.

Your Medical Practice Overall

46. Overall, how satisfied are you with your current medical practice?

• Completely Dissatisfied

• Very Dissatisfied

• Dissatisfied

• Neither Satisfied nor Dissatisfied

• Satisfied

• Very Satisfied

• Completely Satisfied

46. How many years ago did you complete residency?

• Less than a year ago

• 1 to 2 years ago

• 3 to 5 years ago

• 6 to 10 years ago

• 11 to 20 years ago

• More than 20 years ago

46. Are you a/an (check one):

• Internist

• Family Medicine Physician

• Pediatrician

• Other _______________________

46. Do you have a faculty appointment at a university or medical school?

• Yes

• No

46. If you are a primary care provider, approximately how many active patients do you currently care for at the medical practice where you received this questionnaire?

• Less than 1000

• 1000 to 1500

• 1501 to 2000

• 2001 to 2500

• More than 2500

• Not a primary care provider

46. Please use the space below to tell us about any issues related to quality targets or financial incentives that you think it would be important for policy makers to consider.

Thank you for completing this survey!

Please return your completed questionnaire in the enclosed envelope to:

Quality Targets and Incentives Survey

Guideline Boston

109 State Street, Third Floor

Boston, MA 02109

Dear Colleague:

In cooperation with Dr. Richard Kalish and Dr. John Robinson at Boston Medical Center Health Plan, Boston University School of Public Health would like to invite you to participate in an interview for a U.S. Department of Health and Human Services study concerning the role of financial incentives in improving quality of care in safety net settings.

As you know, there is growing interest among public and private payers to link financial incentives to quality targets. Boston University School of Public Health is evaluating the pay-for-quality program sponsored by Boston Medical Center Health Plan and is conducting interviews with key stakeholders involved with the design or implementation of the program. By participating you will have a unique opportunity to influence both Boston Medical Center policy and to contribute to the knowledge base shaping national health policy on pay-for-quality strategies.

The in-person interview can be completed within one hour and will take place in a setting and time convenient for you. You will be paid $100 for your participation in the interview. Your responses will be treated with complete confidentiality and will never be shared with your employer.

A member of our research team will contact you shortly to schedule an interview if you wish to participate. We hope that you will choose to participate in the interview and contribute to what we believe to be a most important effort.

Thank you for your consideration.

Gary Young, Ph.D. Richard Kalish, M.D. John H. Robinson, M.D.

Principal Investigator Medical Director Chief Medical Officer

Boston University BMC Health Plan BMC Health Plan

PROTOCOL

Key Informant Interview

Boston Medical Center, Boston, Massachusetts

A MULTI-SITE COORDINATED EVALUATION OF THE IMPACT OF QUALITY-BASED PAYMENT STRATEGIES (IRB# H-25914)

Date/Time:
Name/Title of Interviewee(s):
Interviewed by:
Note-taker:

INFORMED CONSENT

Title: A MULTI-SITE COORDINATED EVALUATION OF THE IMPACT OF QUALITY-BASED PAYMENT STRATEGIES (IRB# H-25914)

Principal Investigator: Gary J. Young, J.D., Ph.D

Background

You are being asked to participate in an interview because you work in a setting that is involved in a project concerning improvement in patient care and the use of financial incentive arrangements. Boston University was selected by the Agency for Healthcare Research and Quality to conduct an evaluation of this project. Your responses to the study's questions will allow Boston University to identify factors that lead to successful patient care improvement efforts. The interview should take no more than 35 minutes.

Purpose

The purpose of this study is to describe how financial incentives contribute to patient care improvement efforts and how senior managers and other leaders perceive these incentives.

What Happens In This Research Study

You will be one of approximately 8 subjects to be asked to participate at this location. A total of 32 subjects at all institutions will be asked to participate in this study. The research will take place at Boston University Medical Center.

The interview asks you about your knowledge of the financial incentive program at your institution; perception of the impact of the financial incentive program, including the relevance of the quality measures, adequacy of the financial incentives, and the fairness of payout formulas; perception of the unintended consequences of financial incentive programs; and perception of the factors within your institution that affect the structure and implementation of the financial incentive program. We request that we be allowed to audio-tape the interview to help in the note-taking process.

Risks and Discomforts

There are no known risks for participating in this study. The study may inconvenience you in terms of the time or the effort that it takes to participate in the interview. At no time will the responses you give the interviewers be shared with your employer. If we wish to quote you anonymously in a publication, we will contact you beforehand to obtain your permission. We will never identify you as an individual in any report or publication. We would like to audio-tape the interview, but you may refuse to have the interview audiotaped for any reason without any risk. There may be unknown risks/discomforts involved. Study staff will update you in a timely way on any new information that may affect your health, welfare, or decision to stay in this study

Potential Benefits

You will receive no direct benefit from your participation in this study. However, your participation may help the investigators better understand financial incentive programs. We will provide you and leaders in your organization with anonymous feedback from all interviews that may be beneficial in planning quality improvement efforts. There are no costs to you or your organization for participating.

Alternatives

Your alternative is to not participate in the study.

Subject Costs and Payments

There are no costs to you for participating in this research study. You will be paid a one-time honorarium of $100 to participate in this interview.

Confidentiality

Information from this study and from your medical record may be reviewed and photocopied by the Food and Drug Administration (FDA) and/or state and federal regulatory agencies such as the Office of Human Research Protection as applicable, and the Institutional Review Board of Boston University Medical Center. Information from this study and from your medical record may be used for research purposes and may be published; however, your name will not be used in any publications. If we wish to quote you anonymously in a publication, we will contact you beforehand to obtain your permission. At no time will the responses you give the interviewers be shared with your employer.

Subject's Rights

By consenting to participate in this study you do not waive any of your legal rights. Giving consent means that you have heard or read the information about this study and that you agree to participate. You will be given a copy of this form to keep. If at any time you withdraw from this study you will not suffer any penalty or lose any benefits to which you are entitled. You may obtain further information about your rights as a research subject by calling the Office of the Institutional Review Board of Boston University Medical Center at 617-638-7207.

The investigator or a member of the research team will try to answer all of your questions. If you have questions or concerns at any time, or if you need to report an injury while participating in this research, contact Gary Young at (617) 414-1423 during the day and Karen Sautter at (781) 424-1477 after hours.

Compensation for Research Related Injury

If you think that you have been injured by being in this study, please let the investigator know right away. If your part in this study takes place at Boston Medical Center, you can get treatment for the injury at Boston Medical Center. If your part in the study is not at Boston Medical Center, ask the investigator where treatment for injury would be available locally. You and your insurance company will be billed for this treatment. Some research sponsors may offer a program to cover some of the treatment costs which are not covered by your insurance. You should ask the research team if such a program is available.

Right to Refuse or Withdraw

Taking part in this study is voluntary. You have the right to refuse to take part in this study. If you decide to be in the study and then change your mind, you can withdraw from the research. Your participation is completely up to you. Your decision will not affect your being able to get health care at this institution or payment for your health care. It will not affect your enrollment in any health plan or benefits you can get. If you choose to take part, you have the right to stop at any time. If there are any new findings during the study that may affect whether you want to continue to take part, you will be told about them as soon as possible. The investigator may decide to discontinue your participation without your permission because he/she may decide that staying in the study will be bad for you, or the sponsor may stop the study.

By signing below, I certify that I read or have been read all of the above. The study person named below has explained the study to me and answered all of my questions. I understand my rights as a research subject and I voluntarily consent to be in this study. I will receive a signed copy of this consent form.

___________________________________________

Signature of Participant Date

___________________________________________

Signature of Person Obtaining Consent Date

I consent to have this interview audio-taped.

___________________________________________

Signature of Participant Date

PROTOCOL & NOTES TEMPLATE

Thanks for agreeing to discuss financial incentives at Boston Medical Center and [INSERT NAME OF PRACTICE]. Do you have any questions before we begin?
QUESTION	RESPONSE
I. Background of Participant
What is role at [INSERT NAME OF PRACTICE] and how long have you been in that position?
Would you tell us a little about your background prior to joining the staff at [INSERT NAME OF PRACTICE]?
II. Practice Characteristics
How many physicians practice at [INSERT NAME OF PRACTICE]? Are they full-time, part-time? Are any of them residents?
How are your physicians paid?	Inquire if changes in payment structure have occurred.
What primary care specialties are represented among your physicians?	e.g., family practice, internal medicine, pediatrics
Do you think the physicians are happy with their work situations?	If not, why? What issues?
Would you describe the support staff you have at [INSERT NAME OF PRACTICE]?	e.g., nurses, nurse practitioners, other health care professionals, office staff, billing personnel, etc.
In addition to primary care, what other services does [INSERT NAME OF PRACTICE] provide on-site?	e.g., laboratory testing, dental, vision, OB/GYN, etc.
III. Patient Characteristics
Would you estimate the percentage of patients at [INSERT NAME OF PRACTICE] who are in the following racial/ethnic categories?	Latino African-American Asian Caucasian Other?
What percentage of patients at [INSERT NAME OF PRACTICE] have income below $25K?
What proportion of patients is insured by Medicaid? Uninsured?
What proportion of patients attend your practice that are covered by BMC’s HealthNet Plan?	This is a key question because the financial incentive is sponsored by HealthNet
IV. General Quality Improvement
Is [INSERT NAME OF PRACTICE] engaged in any quality improvement activities?
Do you have an electronic medical record system? If so, do you use the system to improve quality of care? How so?
Do physicians and other clinical staff meet regularly to discuss quality?
Do you use performance reports for physicians or other medical staff to improve quality?
V. Financial Incentive Program Implementation
Is your practice involved with BMC HealthNet’s financial incentive program?	If no, skip to section VI.
How did your practice become involved in the HealthNet program?	Did you have any role in getting your organization to participate?
Is [INSERT NAME OF PRACTICE] engaged in any other financial incentive programs or performance-based initiatives?	Are they health plan-sponsored?
How does HealthNet’s financial incentive payout structured?
How much money is at stake for your practice?
What are the performance targets? Who is targeted (physicians, organization, etc.)?
What do you do with the incentive reward?	Is any of the reward distributed to individual physicians?
Are physicians aware of the incentive program?
Are other clinical staff and support staff aware or involved in the incentive program?
VI. Impact of Financial Incentive Program
How would you describe the impact of the incentive program on your practice? Could you give us some examples of that impact?	Physicians Patients Organization
Has the program had any negative consequences for your practice?	e.g., diverted attention from other activities
How do you perceive physicians feel about the financial incentive program?
Has physician behavior changed as a result of the financial incentive program?
Do you have any recommendations to make the financial incentive program more effective?
Have I forgot to ask you anything that you think is important to the evaluation?
That concludes the interview. Thank you very much for your time and assistance.

2