Responses to public comments 040309

Contract No.: 500-2005-00025I (0006)

MPR Reference No.: 6479-232

Evaluation of the Electronic Health Records Demonstration (EHRD) and the

Medicare Care Management Performance (MCMP) Demonstration: Responses to Public Comments on the Supporting Statement for Paperwork Reduction Act Submission

April 1, 2009

Submitted to: US Department of Health and Human Services Centers for Medicare & Medicaid Services Office of Research, Development, and Information C3-23-04 Central Bldg. 7500 Security Blvd. Baltimore, MD 21244-1850 Project Officer: Lorraine Johnson

Submitted by: Mathematica Policy Research, Inc. P.O. Box 2393 Princeton, NJ 08543-2393 Telephone: (609) 799-3535 Facsimile: (609) 799-0005 Project Director: Jennifer Schore

Submitter #1

Name: Gary Lampman
Address: Hendersonville, TN 37075

Patient advocate

Electronic Health Records would be beneficial. However a back up needs to be
maintained due to corruption of Data Systems. I do believe that a strong oversight of
a third party would be prudent to curtail abuses and a legal recourse be in place.

As a Patient Advocate I think that much is needed for Patients to electronically
review their own records and make copies of these records. Further, I believe that
more transparency of record keeping between the provider and the consumer
needs to be available for review.

All record views should be recorded by provider and insurance company. Which
allows the patient to review entries. Give the Patient the ability to watch over their
own records and I assure you that auditing would be more effectual. We must
empower the Patient in order to provide the needed checks and Balances.

RESPONSE

We agree that patients should have access to their Electronic Health Records (EHRs) and that over time practices should be expected to implement functions that allow such access, with the proper security and confidentiality guidelines in place. Questions in Section 4 (Domain 4) address this.

While we also agree that procedures for EHR systems must be developed to protect them from abuse or data corruption, these procedures are outside the purview of the EHR demonstration evaluation.

Submitter #2

Name: Cindy Helstad
Address: Madison,  WI,  53715

Organization: Wisconsin Medical Society

Section 1 – General Information – Practice

1.2 – 1.11. The questions that are populated with data from the application form or last OSS is an improvement we support.

1.14

Could this be a MERGE FIELD question as well with the option to make corrections if anything has changed since the last survey?

RESPONSE

The “type of organization with which the practice is affiliated” (1.14) is not collected on the application. Therefore, at the first administration of the OSS, we will ask practices to respond without pre-filling the field. For subsequent years’ administrations, however, we will merge the previous year’s response into the survey and ask practices to verify or update the information.

1.15

Bold ‘currently’

Add ‘none’ as a response option if you must select an answer for this question.

Remove ‘Better Quality Information’ as this project was completed in 2008.

RESPONSE

We have made those three changes.

We believe the answers to this question will be unreliable, especially for practices that are part of large medical groups where the parent group may be participating in one or more programs. How will the data be used?

RESPONSE

The data will be part of a descriptive analysis of practice characteristics and will inform the evaluation of pay-for-performance initiatives external to the EHRD in which practices are participating. We have added “do not know” as a response category

Section 2 – Provider Profile

p. 5 – 2.0a.

Practices know FTEs (doctor that works 1 day/mo) not number of providers. To get more reliable information, may consider asking about FTEs, then the number of providers that are counted in the FTEs.

RESPONSE

Demonstration payments will be based on the number of providers in the practice who are participating in the demonstration; the proportion of all practice providers participating in the demonstration provides valuable information for the evaluation. Thus, both 2.0a-c are asked in terms of the number of providers, rather than FTEs.

It seems tricky to switch back and forth between the number of providers participating in the demonstration and all of the providers working at the practice location. Suggest moving question 2.0a to the end of section 2 and saying that the rest of the survey questions pertain to the total number of providers currently working in the practice location.

RESPONSE

We have made this change.

p. 6, 2.1

Great enhancement to the previous OSS. Capturing the clinician changes to a practice is good.

p.6, 2.4

Clarify if you are requesting the individual or group NPI

RESPONSE

We are requesting individual NPIs and have clarified the question.

Section 3 – Use or Planned Use of Electronic Health Records, an Electronic Patient Registry, or an Electronic Prescribing system

1. Electronic Health Record

p. 7, 3.1

What do you mean by ‘have’? Already purchased? Implemented? Installed? Selected? This is a really loaded question and it is vague about what is meant. You may want to add definitions in the instructions about what you mean by each of the terms that are used in the questions.

RESPONSE

We mean “ implemented” (that is, purchased, installed, tested, and being used). We have clarified the question.

3.3, 3.10, 3.16

The word ‘install’ is very confusing. It’s a techie word and the practices do not think about their system in those terms. Practices think this means implemented and using it. A more reliable question is ‘When did you purchase the ____ from the vendor?’ followed by the ‘current use’ question.

RESPONSE

We have substituted “purchase” for “acquire (that is, install).”

3.4, 3.11, 3.17

If have a drop down menu of possible EHRs, will need an ‘other’ category in case the practices selected EHR is not in the drop down menu.

RESPONSE

We do not plan on using a drop-down menu for these questions because the universe of vendors is always changing. We will request that practices fill in the actual names of vendors and products.

3.6

What do you mean by ‘using’? Many will say ‘yes’ if just using the billing function. Is that appropriate?

RESPONSE

We have clarified the questions to exclude systems used only for billing or practice management.

2. Electronic Patient Registry

Separation of 3.9 a, b, c is a good improvement from the previous OSS.

Section 4 – Electronic Health Record, Patient Registry, and Prescribing System Functions

Domain 1. Completeness of Information

4.1Xx –

Will the numbering in this section be modified? We found this very distracting.

RESPONSE

The survey has been renumbered sequentially. (The previous numbering was an artifact of the survey development process.)

4.1Ad -4.1Ag. The way the question is worded, it puts the emphasis on maintaining something. This begs the question, What is the standard for how often the maintenance functions should be done? At every physical? At every visit? The question from the instructions should be used rather than the ‘maintain’ phrases. For instance,

What is the proportion of patients for which providers use the EHR, electronic patient registry or e prescribing system for:

4.1Aa. Clinical notes

4.1Ad. Allergy lists

4.1Ae. Problem or diagnosis lists

4.1Af. Patient demographics

4.1Ag. Patient medical history

4.1Ca Ordering labs electronically or recording labs ordered by paper into the electronic system*

4.1cf1 Scanning paper lab results into the electronic system

4.1C1 Reviewing lab results electronically

4.1Cb. Ordering imaging tests electronically or recording imaging tests ordered by paper into the electronic system*

4.1Cf2 Scanning paper imaging results into the electronic system

4.1Cm Reviewing imaging results electronically

RESPONSE

We have revised the questions consistent with this suggestion.

Domain 2. Communication of Care Outside the Practice

Break the questions up by headers. E.g.

Before 4.1Cc1. insert ‘Lab Orders’

Before 4.1Cc2. insert ‘Imaging Orders’

Before 4.1Ch1. insert ‘Lab Results’

Before 4.1Cc2. insert ‘Imaging Results’

Before 4.1Dh. insert ‘Referrals’

Before 4.1Dj1. insert ‘Transmitting Results’

Before 4.2Ad. insert ‘Prescriptions’

RESPONSE

We have inserted headings consistent with this suggestion.

Group ‘Lab Results’ questions after Lab Orders’

Group ‘Imaging Results’ questions after ‘Imaging Orders’

RESPONSE

Grouping Orders (lab and then imaging) and then Results (lab and then imaging) is related to the demonstration’s approach to scoring OSS responses for payment.

Section 4 – General

Pretesting

We strongly suggest Section 4 should be pilot tested with small practices to see if they understand the instructions and can answer them reliably to earn their bonus.

RESPONSE

As per the “Evaluation of the EHRD and MCMP Demonstration: Supporting Statement for Paperwork Reduction Act” (11/24/08), the OSS was pretested with 8 practices during Fall 2008. The practices that completed the survey ranged in size from one to 26 physicians and included general and specialty practices. The responding practices were using EHRs daily to varying degrees in their practice. They were mailed a paper OSS survey to complete and fax back. Each respondent then participated in a fifteen to thirty minute telephone debrief to review their responses to the survey and to verify their understanding of the questions. The OSS under review incorporates changes made in response to their pretest experiences.

Domain 3. Clinical Decision Support

We think it will be difficult for anyone to know how all of the clinicians use the available clinical decision support.

RESPONSE

These questions did not pose a problem for pretest respondents. However, we have added clarifying language to the section introduction that says, “Please complete all questions in the survey unless directed within it to skip a section. If you are not aware of how all the providers in the practice are using the functions asked about in this section, please consult with them prior to answering the questions.” We have added similar language to the beginning of the survey as well.

Scoring

We were confused by the ‘proportion of’ scoring, especially for the hierarchy questions. It looks like you score the same whether you are using higher or lower level functioning (0-4 scale). It may help to remove those numbers beneath the description of the amount of time. Will practices be scored higher if they are using a function with higher technological sophistication?

We also think there will be confusion around whether a practice has a specific functionality and is not using it vs. not having the functionality. While that is not what the question asks, it’s what we thought about when we read the question. Would you please describe how the survey will be scored?

RESPONSE

We have removed the numbers beneath the description of the amount of time. This was done for instrument development only.

The OSS contains five sets of hierarchical items that are each scored as a set. A hierarchical item set consists of several consecutive questions in the OSS that represent progressively more advanced ways of using the EHR, so that as a practice advances in it use, it will indicate less use of the less advanced process and more use of the more advanced processes.

Practices that do not have a function available in its EHR will receive a score of zero, identical to practices that do have the function but do not use it. Although this is an analytic limitation, it does greatly facilitate the scoring of the OSS for payment purposes, which is a key goal of this instrument.

The OSS scoring plan relies on several principles: (1) be as simple as possible; (2) recognize that early in the demonstration some practices will be new to the use of EHRs, but that over the duration of the demonstration, use of EHRs could increase substantially; and (3) the overall summary score should be built up from domain scores on a relatively small number of domains that are conceptually distinct and would be perceived as relevant and meaningful to providers and CMS.

Fifty-three EHR functions are scored through response to questions on the OSS. Most questions are scored on a 0 to 4 (5-point) scale. The response choices for most items translate directly into their score, with 0 less desirable, representing no use of the function, and 4 indicating the function is used for “3/4 or more” patients. One question (proportion of paper charts pulled) requires scoring in reverse of the response choices, because a better score on this question is lower. For the items pertaining to report generation, we will recode the response on a 3-point scale such that 0 [Not used during last year]=0, 1 [As-needed basis at least once]=2, and 2 [Regularly for full practice]=4. Finally, as noted above, the OSS includes five sets of hierarchical questions (ordering laboratory tests, ordering radiology tests, receiving laboratory results, receiving radiology results, and prescription ordering). Each set of questions (representing a single function), will be scored together such that the result is a score between 0 and 4, just as with other items.

For the hierarchy questions, shouldn’t there be a written paper-based option for at least the first 2 years?

RESPONSE

As noted above, the hierarchical item set consists of several consecutive questions in the OSS that represent progressively more advanced ways of using the EHR, so that as a practice advances in it use, it will indicate less use of the less advanced process and more use of the more advanced processes. Among the less advanced processes is the use of paper-based records, which implies that the practice would be using a specific function for only a small fraction of patients (or records, depending on the question). CMS is not interested in giving credit to the practices for the use of paper-based records, even during a transition period, but recognizes that this transition may take place over several years. Thus, the proposed scoring of hierarchical questions addresses this policy.

Section 5 – Data Attestation & Section 6 – Attestation

Add a Warning that the respondents will be unable to make changes to the survey once they complete Sections 5 & 6

RESPONSE

We have added that warning to the instrument.

What is the difference between Section 5 and Section 6?

RESPONSE

Section 5 attests to the accuracy of survey responses and Section 6 is an acknowledgement that responses are subject to validation. We have combined these into one item in the instrument.

General Comments

We noticed many more definitions and explanations compared to the previous OSS and think these are good improvements. It’s still a very complex survey.

We are glad the ‘satisfaction section’ is removed in the current version of the OSS!

We recommend scheduling a group teleconference like the Open Door Forums to review how to complete the survey and answer questions about the survey. We also recommend focusing on the skip questions that will miss important information if answered incorrectly.

RESPONSE

CMS will be providing an OSS orientation to enrolled practices at the site kickoff meeting scheduled for May (2009 and 2010). CMS is also considering hosting a teleconference.

MPR will send an initial advance mailing to alert practices about the survey. It will include the email address and toll-free telephone number of MPR’s survey manager for the OSS, whom practices can call for assistance, as well as a fact sheet with answers to commonly asked questions. Roughly one week after the initial letter is mailed, an email from MPR will be sent to all enrolled practices inviting their participation in the survey and providing a secure login and password to use to access and complete the on-line web survey. The email will also include the email address and toll-free telephone number of MPR’s survey manager for the OSS, whom practices can email or call for assistance.

Make sure the practices know when and from whom they will receive the survey. With DOQ-IT, several states had an issue with the OSS not getting to the practices because it went in their junk mail.

RESPONSE

The initial advance letter will be printed on CMS letterhead, personally addressed, and signed by the CMS Privacy Officer. The letter will alert practices to expect an email in the coming week from MPR inviting their participation in the survey and providing a secure login and password to use to access and complete the on-line web survey. It will also include the email address and toll-free telephone number of MPR’s survey manager for the OSS, whom practices can call for assistance, as well as a fact sheet with answers to commonly asked questions.

Submitter #3

Name: Scott Eisenbeisz
Address: Sioux Falls, SD  57106

Organization: Sanford Health

Section 4 4.0b - Paper charts that are pulled for scheduled patient visits.

In this section it does not discuss a timeframe to evaluate this data. Sanford would suggest adding the following statement. When responding please refer to charts pulled over the past month by ALL providers in this practice locations or by other office staff acting on behalf of those providers.

RESPONSE

We have added the reference period to this question.

Section 4 4.1Cf1 – Receive laboratory results by fax or mail and scan paper version into electronic system.

Sanford would like a clarification on the above question on whether the intent is for those labs within the clinic, outside of the clinic or both.

RESPONSE

The question refers both to laboratory tests conducted within the practice and those that are ordered from providers external to the practice. We have clarified the question accordingly.

Section 4.1Dd – Record that instructions or educational information were given to patient.

Sanford Clinic would suggest adding the following statement. When responding please refer to instructions or educational information given over the past month by ALL providers in this practice locations or by other office staff acting on behalf of those providers.

RESPONSE

This instruction is already included in the general instruction box (second paragraph) and applies to the bulk of questions concerning the Completeness of Information (Domain 1).

Section 4.1Cc1 – Print and fax laboratory orders, Section 4.1Cd1 – Fax laboratory orders electronically from system, or order electronically through a portal maintained by the laboratory, Section 4.1Ce1 – Transmit laboratory orders electronically directly from system to facilities that have the capability to receive such transmissions.

Sanford would like a clarification on the above question on whether the intent is for those labs within the clinic, outside of the clinic or both.

RESPONSE

The questions in Domain 2 pertain to communication with providers outside the practice, as noted in the title and general instructions for Domain 2, first item. Thus, the questions noted are about laboratory tests ordered from providers external to the practice, including those that are part of a larger organization or network with which the practice is affiliated. We have clarified the questions.

Section 4.1Dj1 – Transmit laboratory results to other providers (for example, hospitals, home health agencies, or other physicians), 4.1Dj2 – Transmit imaging results to other providers (for example hospitals, home health agencies, or other physicians)

Sanford would like a clarification on the above questions on whether the intent is for those entities within the organization, outside of the organization or both.

RESPONSE

See response above.

Section 4.2Ad – Print prescriptions (new and refills) on a computer printer and fax to pharmacy or hand to patient, 4.2Ae – Fax prescription orders (new prescriptions and refills) electronically from electronic system, 4.2Af – Transmit prescriptions orders (new prescriptions and refills) electronically directly from system to pharmacies that have the capability to receive such transmissions.

Sanford feels there needs to be an exception added to the above questions for Schedule II-V drugs, since these drugs cannot be submitted via an electronic/fax means. This would also be immaterial data since those drugs at this time cannot be sent by electronic/or faxed means. Clinics should not be penalized if they have a higher number of Schedule II-V drugs.

RESPONSE

We agree and have added to the instruction box for Prescription Orders: “Note that these questions exclude Schedule II-V drugs.”

Section 4.3e - Electronic receipt of reports, such as discharge summaries, from hospitals that have the capability to send such transmissions.

Sanford feels this statement needs to be clarified. Is CMS looking for an electronic receipt or is the intent that a clinic can receive the information electronically?

RESPONSE

The latter: that a provider receives reports electronically. We have clarified the question.

16