Evaluation of the Electronic Health Records Demonstration (Discussion Guide for participating pratices)

EHRD EVALUATION SITE VISIT Discussion Guides

A. guide for participating treatment group practices

In the event that we cannot ask all questions in the following guides during a contact, question priority is reflected in the question numbering scheme. Numbered questions will always be asked during the discussion. Lower-case lettered questions are to be asked unless time is unusually short. Italicized questions are prompts to remind our staff of details to cover during discussion of the question.

A1. Background Obtained During Scheduling of Practice Contact—Treatment Group Practices

[All questions will be pre-filled from the application database where there is overlap (indicated with a “*”), to provide background information. The information will be verified during the contact.]

1. What year was the practice established?

2. *What is the organizational structure of the practice (e.g. is it part of a larger health care organization)?

3. *How many locations does the practice have?

4. *How many physicians are in the practice?

5. How many of them are participating in the demonstration?

6. How many and what types of other staff are part of the practice?

7. *How many Medicare FFS beneficiaries does the practice serve?

8. About what percentage of all the practice’s patients are Medicare fee-for-service?

9. Does the practice see Medicare Advantage (MA), that is, Medicare managed care patients?

10. [If sees MA patients:] What percentage of all patients in the practice are Medicare managed care?

11. Is the practice participating in CMS’s Physician Quality Reporting Initiative (PQRI)?

and pre-fill from OSS responses if available

A2. Administrative Staff Member Overseeing Adoption and Implementation of Health System—Treatment Group Practices

a) Practice Perspectives on the Demonstration and Early Response

1. What were the key reasons or factors that led you and your practice to decide to participate in this demonstration?

2. What if any initial concerns did the practice have, and how were those overcome?

3. [If practice is not independent and/or has more than 5 physicians:] Would you have been willing to participate in this demonstration if you were an independent smaller or solo practice?

4. Which features of the demonstration do you and the clinicians particularly like?

Do physicians feel differently about the demonstration from nurses or other clinicians in the practice?

5. Which if any features of the demonstration do you and the clinicians dislike?

a. More broadly, do you and clinicians in the practice think linking payment to quality of care through incentives is a good idea?

Do physicians feel differently about this from nurses or other clinicians in the practice?

Is there concern about patients being turned away who are noncompliant, because they might reflect negatively on the practice’s quality score?

[If concerned:] What could be done to allay these concerns?For example, would it help if the outcome measures were risk-adjusted to take into account characteristics of your patient population such as presence of comorbidities, income, educational attainment, and race/ethnicity?

In this demonstration, payment is linked to certain outcome and process of care measures. Is there concern that the quality of care for conditions that are not being measured under the demonstration will suffer, if the practice shifts its focus towards the outcome and process measures that are being evaluated?

What if any other concerns do you or others in the practice have about unintended consequences of the demonstration?

6. What are the practice’s expectations regarding the incentive payment from the demonstration—have you estimated how much the practice expects to receive as a result of participating over the next few years?

a. How much would that be in terms of a percent of revenue?

b. in the next year?

c. over the 5-year demonstration?

d. What do you think will be the key factors in whether these expectations are met?

4. What if anything, has the practice done differently thus far because of thinking about the incentives in the demonstration?

b) Adaptation of Practice Operations as HIT Is Implemented

1. Do you have an EHR?

2. [If no EHR:] Why not?

3. [If no EHR:] When do you plan to get one?

4. [If no EHR:] What would facilitate your acquiring an EHR?

5. [If no EHR:] Have you started any activities to prepare for an EHR, such as completing an office readiness assessment, or exploring vendor and product alternatives?

6. [If has EHR:] Please confirm if you still have [fill vendor and product from application]?

a. How long have you had that system?

7. Please give us an overview of the health IT you are using for each of the following functions. [Complete the table. “Using” column will be prefilled with Y/N from application.]

8. [If e-prescribing—yes to p or q:] Is e-prescribing accomplished through your electronic health record or through a stand-alone system?

9. a. [If stand-alone system:] Please briefly describe your e-prescribing system.

10. [If registry—yes to f:] Is your disease-specific registry through your EHR or is it a stand-along registry?

11. a. [If stand-alone registry:] Please briefly describe your registry.

12. [If EHR:] Are any of the other functions we discussed accomplished outside your EHR?

13. [If HIT:] On whom did you primarily rely for assistance in implementing the health IT that you use (QIO, vendor, consultant)?

14. [If HIT:] With implementation of HIT, what changes were made in how the practice operates day to day?

Were these changes the result of a specific effort to redesign office flow to meet the EHR process?

15. What other information sources or other factors influenced the practice’s thinking about what changes it should make with HIT implementation?

c) [If HIT:] Facilitators and Barriers to Adopting and Implementing HIT

1. Thinking about the health IT functions that you have started using in the past year, were there particular difficulties in selecting or acquiring the related product and/or getting it up and running?

2. Thinking about the health IT functions that you started using in the past year, what factors have been helpful in selecting or acquiring the product and/or getting it up and running?

3. Are there persistent problems in getting some of the functions to be used routinely in the practice—either the functions we just talked about or others? What are the issues you view as most important?

4. Would you describe anyone on the staff as a “champion” for HIT use? If yes, are hey having success at influencing others? Who is it?

Fill in this table for question b.7.

Function	Using (Y/N)	Year started (mo/yr in the past year)	Uses, Limited	Uses, Widespread	Timeframe, if plan to start using it
a. Electronic patient visit notes
b. Electronic patient-specific problem lists
c. Electronic patient-specific medication lists
d. Automated patient-specific alerts and reminders
e. Other clinical decision support/automated references to best practices Please describe:
f. Electronic disease-specific registries—that is, using the EHR or a stand-alone registry to identify patients with specific diagnoses, or to track information and prompt ordering of tests or communications for patients with those conditions
g. Patient e-mail
h. Patient-specific educational materials
i. On-line referrals to other providers
j. Clinical messaging with other physicians
k. Transmission of records to hospitals or other facilities
Laboratory Tests:
l. On-line order entry
m. On-line results viewing
Radiology tests:
n. On-line order entry
o. On-line results viewing; Specify reports or images or both:
E-Prescribing:
p. Printing and/or faxing Rx; Computerized faxing?
q. On-line Rx transmission to pharmacy
Other:
r. Receipt of electronic clinical information from hospitals, other facilities or doctors Which types of providers?

d) Relevant Context—Other Incentives, Reporting Programs, and HIT Initiatives

1. Is the practice participating in any other P4P initiatives/programs? If so, do they include incentives for adopting or using health IT?

Does the practice aim for its physicians to receive Medicare meaningful use incentive payments in 2011? Other years?

Is the practice eligible for and if so does it aim for its practitioners to receive Medicaid incentives related to adoption and meaningful use of EHRs?

2. Are there any other P4P activities that you know of going on in this area?

3. Have other incentives that the practice faces from other payers or other reporting programs such as CMS’s PQRI affected how the practice has responded to the incentives under the demonstration? If so, how?

4. Is the practice participating in any other HIT or EHR initiatives? What are they? Are there other HIT or EHR activities going on in this area?

e) Use of HIT for Care Management

Next, we have some specific questions about the extent to which the practice is using HIT to improve patient care for specific conditions or to ensure recommended services are provided—we are going to refer to this as “care management.”

1. E-prescribing [if applicable]

a. Do the practice’s e-prescribing activities include using the system to screen prescriptions for drug allergies, drug-drug interactions, or drug-disease interactions?

b. Is the system used to offer guidelines and evidence-based recommendations when prescribing medication?

c. To provide patient-friendly information about the medication to the patient?

d. Why does or doesn’t the practice use its system to do these things?

Is function available on the system and turned on?

Any technical issues that discourage use

2. Electronic disease-specific registries [if applicable]

a. Does the practice use its system to generate reminders to patients with certain diseases about needed or overdue visits or tests?

b. To prompt clinicians to order tests or services?

c. To create, prompt review of, or modify self-management plans for patients with chronic illness?

d. To print educational information to help patients understand their condition?

e. Why does or doesn’t the practice use its system to do these things?

Is function available on the system and turned on?

Any technical issues that discourage use

3. Does the practice use its EHR to review and act on reminders for care activities such as due or overdue health maintenance, that are not specifically focused on people with a particular disease?

a. Why does or doesn’t the practice use its system for this purpose?

Is function available on the system and turned on?

Any technical issues that discourage use

4 When you were shopping for HIT, how much did the practice care about whether or how well it could support these types of e-prescribing activities, and tracking and prompting for patients with specific diseases or more generally?

5. [If they cared about the system supporting e-prescribing, tracking and prompting during selection:] Is the system living up to your expectations?

6. What if any practice characteristics have influenced the practice’s view on using HIT for these types of care management activities? For example:

The characteristics of the practice’s patients, e.g. number of elderly with complex conditions, or that have many physicians?

Your views?

How busy the practice is at present?

How profitable the practice is at present?

Your comfort level with HIT?

The physicians’ comfort level with HIT?

f) Plans for Change

1. What if any specific plans does the practice have for changing how it uses HIT over the next few years?

Does the practice plan to begin using any functions that you are not currently using? If so, which ones?

Has the announcement of Medicare or Medicaid meaningful use incentives influenced the practice’s plans? (please explain)

Have any other new funding sources or health IT programs affected the practice’s plans? (please explain)

2. [If yes:] What will be the key factors that affect whether the practice is able to make these changes?

Financial

Knowledge/availability of technical assistance resources/tools

A3. Physician—Treatment Group Practices

a) Demonstration Participation and Operational Response

1. Who made the decision to participate in the demonstration?

2. What if anything, has the practice done differently thus far due to participating in the demonstration?

3. [If HIT:] Did participation in the EHR Demonstration influence the practice’s thinking about making changes when it implemented new HIT?

What other information sources or other factors influenced the practice’s thinking about what changes it should make with HIT implementation?[Be sure to achieve clarity about relationship if any to ARRA meaningful use incentives.]

b) HIT Experience and Effect on Practice Change

1. [If HIT:] Now that we’ve talked about the changes the practice made specifically in response to the demonstration, I’d like to ask a more general question—what HIT functions work best to support clinical care in the practice?

2. [If HIT:] Which if any HIT functions are problematic right now?

3. Have you observed any changes in specific aspects of the practice as a result of using [name of HIT type]? Such as changes in: [Repeat if multiple HIT types]

Time spent on each patient visit?

Physician time spent on administrative versus clinical functions?

Other clinical staff time spent on administrative versus clinical functions?

Completeness of the practice’s clinical documentation?

Usefulness of the information that is immediately in-hand at the start of patient appointments?

4. [If switched from paper to electronic in past year:] What have been the effects on the practice from switching from paper to electronic charts?

5. [If no HIT:] We understand that the practice does not have an EHR or other health IT in place at the present time. Can you tell us why not, and whether or when you plan to acquire an EHR and/or other health IT products such as an electronic registry?

6. [If no EHR:] What would facilitate your acquiring an EHR?

7. Would you describe anyone on the staff as a “champion” for HIT use? If yes, are they having success at influencing others? Who is it?

c) Care Management Views/Experience

1. Has participating in the EHR Demonstration affected the practice’s views on care management, or how easy it has been to implement it? By care management, we mean routines put in place in order to improve patient care for specific conditions, or to prompt clinicians or the patients about due or overdue services. This includes new ways to identify and remind patients needing preventive services or routine tests, new routines for educating patients about self-care, or new checks in place to better ensure clinical guidelines are being met for patients with certain chronic conditions.

2. What if any new care management activities has the practice implemented during the past year?

3. [If new care management:] What if any effects have you seen from these activities, thus far?

4. What, if any, factors outside the practice have influenced the practice’s view on care management, its decision to adopt care management processes, or the smoothness of implementation of the processes?

For example:

Did the announcement of federal meaningful EHR use incentives and future penalties particular sources of information on care management influence these perspectives or decisions?

A particular consultant or QIO staff member?

Pay-for-performance programs other than the demonstration?

5. What, if any, practice characteristics have influenced the practice’s view on care management, or how easy it was to implement it?

For example:

The characteristics of your patients, e.g. lot of complex conditions, tendency to visit many physicians, tendency to not seek care appropriately?

Your views vs. others in the practice?

How busy the practice is at present?

How profitable the practice is at present?

Your or the office manager’s comfort level with HIT?

6. Is anyone in the practice a “champion” for care management? Are they having success influencing others? Who?

d) Quality Measures & Improvement Activities

1. Are clinical measures currently produced for this practice?

[If yes:] At the practice or physician level?

2. [If any measures:] Which ones, and how are the measures used?

3. What benchmarks are available, and how useful are the benchmarks perceived to be? Why?

4. [If any measures:] Has the frequency with which the physicians review clinical measures for the practice, or the number of measures available for review changed since the practice decided to participate in the EHR Demonstration?

5. If data are being used more since the practice decided to participate in the EHR Demonstration, has this led to any changes in the care process?

6. Moving now to activities that could improve quality, what changes could be made at least in theory that could further improve the quality and/or safety for patients of the practice?

a. Using the/an EHR

b. Other types of changes

c. Are any of these changes actually planned?

7. Have we missed anything? Are there any changes the practice has made that we haven’t discussed yet to improve quality or safety for its patients?

a. [If yes:] What motivated these changes?

A4. Medical Director—Treatment Group Practices

[For practices with a Medical Director, we will meet with the Medical Director first using the physician protocol above, then the discussion would continue with this module.]

a) Physicians’ Use of HIT Functions¹

For each of the following functions of health IT, please tell us the extent to which physicians use them, and if they are using them to some degree, any problems and barriers encountered.

[Talk through a-b for those functions the practice is using.]

EHR and/or registry functions:

1. Recording visit and procedure notes in an EHR

a. Extent used

b. Problems/barriers

2. Clinical reminders of preventive services due/overdue

a. Extent used

b. Problems/barriers

3. Clinical reminders of routine tests for chronic illnesses due/overdue

a. Extent used

b. Problems/barriers

4. Ordering lab and/or radiology tests electronically

a. Extent used

b. Problems/barriers

5. Reviewing lab and/or radiology test results electronically

a. Extent used

b. Problems/barriers

6. Generating lists of patients requiring intervention (e.g. list of patients with diabetes who need an HbA1c monitoring test)

a. Extent used

b. [If uses:] What types of queries are made to generate these lists?

c. What type of follow-up occurs?

d. Problems/barriers

7. Generating educational materials for patients about their conditions and/or about their medications

a. Extent used

b. Problems/barriers

8. Using the EHR to create written self-management care plans for patients with chronic illnesses to have, prompt review of the plans, and modify them

a. Extent used

b. Problems/barriers

E-prescribing functions:

9. Screen prescriptions against the patient’s allergy information

a. Extent used

b. Problems/barriers

10. Screen new prescriptions for drug-drug or drug-disease interactions

a. Extent used

b. Problems/barriers

11. Identify generic alternatives at time of prescription entry

a. Extent used

b. Problems/barriers

12. Reference drug formulary of the patient’s health plan/PBM to recommend preferred drugs at the time of prescribing

a. Extent used, overall

b. Extent used for Medicare patients

c. Problems/barriers

13. Calculate appropriate dose and frequency based on patient parameters such as age and weight

a. Extent used

b. Problems/barriers

14. Reference guidelines and evidence-based recommendations when prescribing for a patient

a. Extent used

b. Problems/barriers

15. Reference patient’s medication history

a. Extent to which other providers’ prescriptions are included

Both inside and outside the practice

[If outside prescriptions included:] For essentially all patients in the practice?

[If outside prescriptions included:] What is your source for this information?

Now back to some more general questions…

1. For the functions that are used now, what if there were no incentive for them in the future through pay-for-performance, is use likely to drop?

[Select 4 electronic functions from a.1-a.15 that the practice does not do:]

2. What are the problems/barriers associated with

a. [name a function they do not do]?

b. [name a second function they do not do]?

c. [name a third function they do not do]?

d. [name a fourth function they do not do]?

Moving away now from the health IT functions,

b) Other HIT-Related

1. When you were shopping for HIT, how much did the practice care about whether or how well it could support these types of e-prescribing activities, and tracking and prompting for patients with specific diseases or more generally?

2. [If they cared about the system supporting e-prescribing, tracking and prompting during selection:] Is the system living up to your expectations?

3. About what percent of your time over say the past six months has been spent concerned with HIT-related planning/implementation issues?

a. How much does that grow if you add in time spent thinking about quality of care at the practice level and processes the office might use to improve quality?

4. Please tell us about the range of experience with and attitudes toward HIT among the physicians and other clinical staff in the practice.

a. How important is that in your thinking about next steps for the practice with HIT?

35. Did you or do you expect that adoption of health IT will have any effect on malpractice insurance premiums or related issues for the practice?

c) Changes in Job Responsibilities or Patient Interface

1. [If new HIT or new care management:] How if at all has implementing either new HIT or new care management affected staffing of the office?

Number of staff

Which staff

Staff responsibilities

2. Has the way the practice interacts with the patient changed in the past year, due to either HIT or care management-related changes?

d) Critical Factors for Success & Closing

1. In conclusion, what do you see as the most critical factors that will determine your success under the demonstration?

Has the announcement of meaningful use incentives under Medicare or Medicaid had any effect on your health IT planning and likely success under the demonstration?

2. Do you have any concerns about potential for unintended negative effects of the demonstration?

Any concern about the potential for patients to be turned away if they are noncompliant?

[If concerned:] What could be done to allay these concerns?For example, would it help if the outcome measures were risk-adjusted to take into account characteristics of your patient population such as presence of comorbidities, income, educational attainment, and race/ethnicity?

In this demonstration, payment is linked to certain outcome and process of care measures. Is there concern that the quality of care for conditions that are not being measured under the demonstration will suffer, if the practice shifts its focus towards the outcome and process measures that are being evaluated?

What if any other concerns do you or others in the practice have about unintended consequences of the demonstration?

3. Is there anything else you would like to convey at this time to CMS about the demonstration or pay-for-performance policy more generally?

A5. Nurse—Treatment Group Practices

[This protocol is for a nurse or other clinical staff member involved in care management.]

a) Effect of New Health IT or Changes in Use on Job Responsibilities

[If nurse is first respondent, ask 1-3. If not, start with #4]

1. Has the practice obtained any new health IT at this practice site in the past year? What kind?

2. [If yes:] Why was the decision made to obtain it?

3. [If new HIT:] How far along has the practice come in implementing it?

4. [If new HIT:] How if at all has [name of new HIT] affected your daily responsibilities?

5. [If new HIT:] How if at all has it affected the job responsibilities of others in the office?

6. Aside from new HIT, has the practice made any significant changes to the way HIT is used in the past year? What changes?

7. [If yes:] Why were the changes made?

8. [If changes:] How if at all have the changes affected your job responsibilities? The job responsibilities of others in the office?

9. Has it affected the way a patient experiences care here?

10. Would you describe anyone on the staff as a “champion” for HIT use? If yes, are they having success at influencing others? Who is it?

b) Adoption of Care Management

Next, we have some specific questions about the extent to which the practice has in place routines to improve patient care for specific conditions or to ensure recommended services are provided—we are going to refer to this as “care management.”

1. What care management processes does the practice use? These could include:

Ways to identify and remind patients who are due or overdue for preventive screenings

Ways to identify and remind patients with certain chronic conditions needing routine tests

Routines for educating patients about self-care

Ways of receiving and using information from a patient’s other providers

Ways to review medications for problems of polypharmacy

2. How long have these practices been in place?

3. [If more than a year:] Please summarize a few lessons you have learned about how best to do these things as you grew in your experience with them.

4. Would you describe anyone on the staff as a “champion” for care management? If yes, are they having success at influencing others? Who is it?

5. Have any new care management processes been established as a result of the demonstration?

6. What are the “next steps” in implementing [more] care management and what are the major factors affecting the timing of those steps?

Has the announcement of meaningful use incentives under Medicare and Medicaid influenced your planning for additional care management?

7. What do you and others in the practice perceive as the benefits and costs of adopting care management routines of the types we have been discussing?

What about the relative benefits and costs of adopting care management for different conditions?

8. [If implementation of care management for one or more condition:] How smoothly did implementation go? Why?

9. Has implementation of care management affected the functioning of the practice? For example, how has it changed the job responsibilities of those involved?

10. Is care management producing any results yet for the patients?

Can you think of any examples?

11. [If HIT] Does the HIT the practice has adopted provide good support for care management?

12. [If HIT] Are the care management capabilities of your current system being fully used? What if anything is constraining the practice from fully using them?

13. Has participating in the EHR Demonstration affected the practice’s views on:

a. Care management?

b. Its decision to adopt care management processes?

14 Has the announcement of meaningful use incentives under Medicare and Medicaid affected the practices views on care management, or the decision to adopt care management processes?

15. What if any factors outside the practice have influenced the practice’s view on:

a. Care management?

b. Its decision to adopt care management?

156. Did particular sources of information on care management influence these perspectives or decisions, such as a particular consultant or QIO staff member?

167. What if any practice characteristics have influenced the practice’s view on care management, or how easy it has been to implement care management?

For example:

The characteristics of your patients, e.g. number of elderly with complex conditions, or that have many physicians?

Your views?

How busy the practice is at present?

How profitable the practice is at present?

Your comfort level with HIT?

The physicians’ comfort level with HIT?

c) Greater Use of Data to Refine the Care Process

[If first respondent, ask 1 & 2, otherwise start with 3]

1. Are clinical measures currently produced at the practice or physician level for this practice?

[If yes:] [If no, skip to Section D.]

2. Who generates clinical measures for the practice (if used) and what conditions do they pertain to?

3. Do you routinely see any clinical quality measures for the practice?

4. Has the number of measures available for review changed since you decided to participate in the demonstration?

5. What benchmarks are available, and how useful are the benchmarks perceived to be? Why?

6. If data are being used more since the practice decided to participate in the EHR Demonstration, has this led to any changes in the care process?

7. Are quality data for the practice being reported to outside entities?

d) Enhanced Practice Orientation to Quality and Safety

1. How informed do you feel about the practice’s performance on quality measures?

a. [If well-informed:] Without referencing any documents, can you summarize what you recall about how well the practice is performing on the quality measures that are tracked?

b. [If well-informed:] Did you come to this understanding through reviewing tracking data, discussing this with others in the practice, or some other way?

2. Moving now to activities that could improve quality, what changes could be made at least in theory that could further improve the quality and/or safety for patients of the practice?

a. Using the/an EHR?

b. Other types of changes?

c. Are any of these changes actually planned?

3. Have we missed anything? Are there any changes the practice has made to improve quality or safety for its patients over the past two years that we have not already discussed?

4. [If increased focus on QI:] What has influenced the practice to increase the focus on quality improvement?

A6. Group Discussion with Administrative Personnel—Treatment Group Practices

[CEO, CFO, Marketing Director, as applicable for the practice.]

a) Demonstration’s Fit with Practice Goals

1. Does this practice have specific financial, market position, or clinical goals?

a. [If yes:] How does health IT fit in with those goals?

b. [If yes:] How does increased care management fit in with those goals?

b) Effects of HIT on the Practice

1. [If HIT:] How has the health IT that this practice has implemented thus far affected the practice?

1. Role of the nurse?

2. Ways information is provided to patients?

3. Communication links between physicians in the practice?

4. Connections with other parts of the health system (e.g. hospitals)?

5. Financial effect?

6. Other aspects of the practice?

c) Expectations/Thinking Regarding Incentive Payments

1. What are the practice’s expectations regarding the incentive payment from the demonstration—have you estimated how much the practice expects to receive as a result of participating over the next few years?

1. What would that be in terms of a percent of revenue?

2. How was that estimate made?

3. Is there an expectation within the practice’s budget that assumes payment of that amount?

2. Is there anything specific that is contingent on receiving that amount—such as you would buy X piece of hardware or software or support a specific activity that takes up staff time only if it is covered by the payment?

3. If you have calculated an expectation regarding Medicare and/or Medicaid meaningful use payment incentives for Stage 1, how does the expectation for the demonstration’s payment compare with that?

Could we talk a little about the competitive environment you operate in….

d) Market Factors

1. Is this practice on a par with, ahead, or behind other similar practices in the area in terms of using health IT? Why?

2. Are there any community-wide or provider-specific initiatives to promote health IT adoption or health information exchange in the market area?

3. [If yes:] How if at all is that affecting the thinking or actions by this practice?

4. Is there anything going on other than the demonstration with pay-for-performance in the market area?

5. [If yes:] How if at all is that affecting the thinking or actions by this practice regarding care process changes that might improve performance?

6. Is there anything else going on in the area that is affecting what this practice is doing or planning with health IT or care management?

7. Has participation in the demonstration affected whether or how the practice markets itself?

1 This list picks up on some of the same functions in the administrative staff guide, plus more care management functions; the timeframe does not allow us to probe with the medical director on every possible function.

K.1