Attachment A

2006

Health Information Technology
in the United States:
The Information Base for Progress

Robert Wood Johnson Foundation
www.rwjf.org
MGH Institute for Health Policy
http://mgh.harvard.edu/healthpolicy
George Washington University
School of Public Health and Health Services
The Health Law Information Project
www.healthinfolaw.org

About the Robert Wood Johnson Foundation

The Robert Wood Johnson Foundation focuses on the pressing health and
health care issues facing our country. As the nation’s largest philanthropy
devoted exclusively to improving the health and health care of all Americans,
the Foundation works with a diverse group of organizations and individuals to
identify solutions and achieve comprehensive, meaningful and timely change.
For more than 30 years the Foundation has brought experience, commitment,
and a rigorous, balanced approach to the problems that affect the health and
health care of those it serves. Helping Americans lead healthier lives and get the
care they need—the Foundation expects to make a difference in our lifetime. For
more information, visit www.rwjf.org.

About the George Washington University Medical Center

The George Washington University Medical Center is an internationally
recognized interdisciplinary academic health center that has consistently
provided high quality medical care in the Washington, D.C. metropolitan area
for 176 years. The Medical Center comprises the School of Medicine and Health
Sciences, the 11th oldest medical school in the country; the School of Public
Health and Health Services, the only such school in the nation’s capital; GW
Hospital, jointly owned and operated by a partnership between the George
Washington University and Universal Health Services, Inc.; and the GW Medical
Faculty Associates, an independent faculty practice plan. For more information
on GWUMC, visit www.gwumc.edu.

About the Institute for Health Policy

The Institute for Health Policy (IHP) at Massachusetts General Hospital
(MGH) and Partners Health System is dedicated to conducting world-class
research on the central health care issues of our time. The mission of the IHP is to
improve the health and health care of the American people through conducting
health policy and health services research, translating new healthcare knowledge
into practice, informing and influencing public policy, and training scholars and
practitioners of health policy.
This report was produced by a team of researchers at the Institute for Health
Policy at Massachusetts General Hospital and the School of Public Health
and Health Services at George Washington University: David Blumenthal,
M.D., M.P.P.; Catherine DesRoches, Dr.P.H.; Karen Donelan, Sc.D.; Timothy
Ferris, M.D., MPhil., M.P.H.; Ashish Jha, M.D., M.P.H.; Rainu Kaushal, M.D.,
M.P.H.; Sowmya Rao, Ph.D.; and Sara Rosenbaum J.D. 
The report was also informed by the discussions of an Expert Consensus
Panel. The authors gratefully acknowledge the support of the Robert Wood
Johnson Foundation and the efforts of the federal Office of the National
Coordinator for Health Information Technology on behalf of this report.
© 2006 Robert Wood Johnson Foundation

Table of Contents
2

Chapter 1: Preface

6

Chapter 2: Defining Critical Concepts Related to
the Adoption of EHRs

19

Chapter 3: Current Levels of EHR Adoption: What
Do We Know?

29

Chapter 4: Will Differential HIT Adoption
Exacerbate Health Care Disparities?

42

Chapter 5: Incentives and Barriers to HIT Adoption:
Requirements for Policy Relevant Measurement

56

Chapter 6: Survey Guidelines: Improving What We
Know About EHR Adoption

67

Chapter 7: Recommendations for Future Data
Collection

74

References

Health Information Technology in the United States: The Information Base for Progress  

Chapter 1: Preface
Health information technology (HIT) has the potential to advance health
care quality by helping patients with acute and chronic conditions receive
recommended care, diminishing disparities in treatment and reducing medical
errors. Nevertheless, HIT dissemination has not occurred rapidly, due in part
to the high costs of electronic health record (EHR) systems for providers of
care—including the upfront capital investment, ongoing maintenance and shortterm productivity loss. Also, many observers are concerned that, if HIT follows
patterns observed with other new medical technologies, HIT and EHRs may
diffuse in ways that systematically disadvantage vulnerable patient populations,
thus increasing or maintaining existing disparities in access to and quality of
care. These and other concerns have led to public and private efforts that aim
to increase the pace of and reduce disparities in HIT diffusion by formulating
national plans for dissemination, catalyzing the development of standards to
encourage interoperability and promoting public-private partnerships to develop
HIT infrastructures at the local and regional levels.

Estimating EHR Adoption and Use

An information base that includes data on variation in EHR adoption by provider
type and geography and reports on successful dissemination and implementation
strategies is thought to be critical for future policy development in this area.
Although numerous surveys have attempted to measure HIT adoption and use,
our current understanding is limited by inconsistencies in sampling techniques,
data collection instruments and terminology, as well as varying response rates. The
existing research allows some general inferences, but it cannot be used to generate
precise, valid and reliable estimates of rates and patterns of dissemination and use
at any point in time or longitudinally. This research also cannot systematically
identify areas where adoption and use are lagging, such as safety net institutions or
other facilities serving vulnerable populations.
This report aims to improve the collection of data on EHR adoption among
physicians, group practices and hospitals. The report first reviews existing data
sources, including ongoing national surveys and one-time or regional studies. It
then recommends a coordinated, systematic national approach to measuring EHR
dissemination and implementation that builds on past data collection activities.
Methodological guidelines, as outlined in this report, were developed to ensure
that new survey content is designed to answer the following critical questions:
■
■

■

What are the predictors of EHR adoption?
Where are there gaps in adoption and do these vary by location, organization
type, specialty, involvement with vulnerable populations and EHR
functionality?
How can precise, timely data on EHR adoption best be collected?

An important part of this work is to encourage efforts that focus on the
accessibility of EHRs among vulnerable populations and the best ways to measure
HIT adoption among providers serving these patients. This information should
help policy-makers understand barriers to adoption among these physicians
and hospitals and design policies to overcome them, laying the groundwork for
detecting and reducing disparities in HIT diffusion.

1:2  Health Information Technology in the United States: The Information Base for Progress

Major Content Areas

This report builds on a previous project, initiated by the Office of the National
Coordinator for Health Information Technology (ONC) last year, to design a
standardized approach to measure and interpret the adoption of information
technology within the American health care system. Our current work is
funded by the Robert Wood Johnson Foundation (RWJF). The Foundation has
a longstanding commitment to understanding and improving the quality of
American health care. This commitment includes a multitude of efforts designed
to help address all dimensions of the quality problem, including especially
the inequities of care. The report aims to share the lessons of the ONC work
more broadly and to provide a review of what is known about the state of HIT
adoption, with a special focus on EHRs and vulnerable populations. It develops
common terms and a definition of what constitutes “EHRs,” as well as suggestions
for the design and implementation of a multi-method approach to data collection.
Specifically, the second chapter, Defining Critical Concepts Related to the Adoption
of EHRs, defines key terms related to the measurement of EHR adoption among
doctors, medical groups and hospitals and recommends definitions for new and
existing surveys. Based on its review of existing data on adoption, it proposes that
an EHR definition based on multiple functionalities be used in future surveys.
Chapter 3, Current Level of EHR Adoption: What Do We Know?, assesses the
quality of existing surveys and their data and estimates current levels of adoption
based on those surveys found to be of high quality. It also lays the groundwork for
improving the information available to develop policies that promote EHR adoption
by identifying critical information gaps and optimal approaches to addressing those
gaps going forward. In Chapter 4, Will Differential HIT Adoption Exacerbate Health
Care Disparities?, we define vulnerable populations and approaches for ensuring that
the diffusion of EHRs among those providing care to these patients is monitored
going forward. Chapter 5, Incentives and Barriers to HIT Adoption: Requirements
for Policy Relevant Measurement, provides a framework for understanding barriers
and incentives for HIT adoption. It also describes possible policies to increase
adoption, including ways to evaluate their effectiveness. The next chapter, Survey
Guidelines: Improving What We Know About EHR Adoption, recommends general
methodological guidelines for applying best survey practices to the measurement
of EHR adoption in the United States. Finally, Chapter 7 makes recommendations
for improving existing, ongoing national surveys and for new survey efforts, where
needed. It includes specific recommendations for surveying providers who serve
vulnerable populations and for studying both the effect of EHRs on the provision of
care and the use of EHRs to efficiently capture quality data.

Surveying Adoption: Current
Findings and Future Directions

Based on existing, high quality survey data on EHR adoption, we estimate that 17
to 24 percent of physicians in ambulatory settings use EHRs to some extent. Our
best estimate, based on the most recent data, is that the proportion of physicians
with access to EHRs in 2005 was closer to 24 than to 17 percent. Also, 4 to 24
percent of hospitals have adopted computerized physician order entry (CPOE),
the best proxy in current studies for EHR adoption in the inpatient setting. Our
best estimate is that, as of 2005, the proportion of hospitals with functioning
CPOE systems was closer to 4 than to 21 percent, and was possibly as low as 5
percent. Our review of the 36 surveys conducted in the past decade on the state of
the science on EHR adoption in the United States also found that these surveys
vary widely in the functionalities they measure, the respondents they target,
the clinical settings they examine, the quality of their methodology, and, not
surprisingly, their estimates of EHR adoption.
Health Information Technology in the United States: The Information Base for Progress  1:3

As mentioned above, the existing data on EHR adoption suffers from numerous
technical problems. In addition, little information is available on stakeholders that
disproportionately serve vulnerable populations, such as community health centers
and public hospitals. Without reliable data on current EHR adoption levels, it will be
difficult, if not impossible, for policy-makers to develop relevant incentives for their
use, especially among providers serving vulnerable populations. Our review of existing
survey data, in consultation with experts in information technology and survey design,
led us to recommend a variety of methods to develop data that could be used for
policies promoting the adoption and use of EHRs. As part of this process, we:
■

Developed a common definition for EHRs and EHR adoption.

■

Identified data needs for assessing EHR diffusion and use.

■

Identified and evaluated existing data sources on EHR adoption and use.

■

Identified gaps in the existing survey data.

■

Previous Work

Designed a strategy for future data collection, including recommendations for
new and existing surveys.

Our team draws from several institutions with relevant expertise: the George
Washington University School of Public Health and Health Services, Department
of Health Policy, the Institute for Health Policy at Massachusetts General Hospital/
Partners Health System, the Division of Internal Medicine, Brigham and Women’s
Hospital and the Clinical and Quality Analysis Group of Partners Health System.
Previous projects of the groups involved include: a study defining and estimating
the costs of developing a national health information network (NHIN), published
in the Annals of Internal Medicine; a Robert Wood Johnson Foundation colloquium
on measuring the diffusion of health care technology structured to assist the
Office of the National Coordinator for Health Information Technology (ONC);
an Agency for Healthcare Research and Quality (AHRQ)-funded evaluation of the
Massachusetts’ E-Health Collaborative; and an RWJF analysis of legal barriers to
the widespread adoption of electronic health information reporting.
Also critical to our research process was the creation of an Expert Consensus
Panel (ECP), composed of a group of national experts in areas relevant to
developing definitive judgments and methodologies for measuring the adoption
of EHRs, including survey design and interpretation, statistics, meta-analysis,
EHR development and use, technology diffusion, qualitative research methods,
economics, sociology, psychology, physician and hospital behavior, health care
disparities and health care quality. These leaders represent agencies of the federal
government currently conducting surveys that could be used for the purposes
of this work; private sector consumers of the resulting data; and other potential
funders of efforts to measure diffusion and use of HIT. The ECP held three
meetings during the project, supplemented by smaller meetings of the technical
subgroups (see Table 1 for a complete list of these groups and their members).
These meetings addressed issues related to meta analysis, survey methods and
identifying providers who disproportionately serve vulnerable populations.
We are grateful to these individuals for their enormous contributions to this effort
and for their generosity in donating their time. We hope that their effort will be
rewarded by contributing to improved understanding of the pace and determinants
of HIT adoption and by the subsequent development of policies that optimize
adoption and employment of innovative electronic technologies in medicine.

1:4  Health Information Technology in the United States: The Information Base for Progress

Table 1. HIT Adoption Initiative Technical Working Groups
Meta Analysis Working Group
Barry I. Graubard, Ph.D.
Division of Cancer Epidemiology and Genetics
National Cancer Institute

Sally Morton, Ph.D.
RTI International

Joseph Lau, M.D.
Center for Clinical Evidence Synthesis
Division of Clinical Care Research
Tufts-New England Medical Center

Christopher H. Schmid, Ph.D.
Tufts Sackler School of Graduate Biomedical Sciences
Tufts-New England Medical Center
Institute for Clinical Research and Health Policy Studies

Thomas A. Louis, Ph.D.
Bloomberg School of Public Health
Johns Hopkins University

Alan Zaslavsky, Ph.D.
Department of Health Care Policy
Harvard Medical School

Disparities Working Group
Andrew Bindman, M.D.
The University of California,
San Francisco

Michael Painter, J.D., M.D.
Robert Wood Johnson Foundation

Steve Downs, S.M.
Robert Wood Johnson Foundation

Bruce Siegel, M.D., M.P.H.
George Washington University School of Public Health
and Health Services

Terry Hammons, M.D., S.M
Medical Group Management Association

Robin Weinick, Ph.D.
The Disparities Solutions Center, MGH/Harvard Medical School

Survey Methodology Expert Working Group
Robert J. Blendon, Sc.D.
Professor of Health Policy and Management
Department of Health Policy and Management
Harvard School of Public Health

Craig Hill, Ph.D.
Vice President, Survey Research Division

Martin R. Frankel, Ph.D.
Professor of Statistics and Computer Information Systems
Zicklin School of Business

Nancy Mathiowetz, Ph.D.
Professor of Sociology
Chair, AAPOR Standards Committee
Department of Sociology
University of Wisconsin–Milwaukee

Survey Content Working Group
Carmella Bocchino, R.N., M.B.A.*
Sr. V.P., Medical Affairs
America’s Health Insurance Plans

Sarah Hudson Scholle, M.P.H., Dr.P.H.*
National Committee for Quality Assurance

Terry Hammons, M.D., S.M.*
Sr. V.P., Research and Information
Medical Group Management Association

Paul Tang, M.D.*
Palo Alto Medical Foundation

Mark Leavitt, M.D., Ph.D.*
Chair, CCHIT
* ECP member

Health Information Technology in the United States: The Information Base for Progress  1:5

Chapter 2: Defining Critical Concepts Related to the Adoption of EHRs
Many groups and organizations have surveyed physicians and hospitals about
their use of electronic health records (EHRs). But, to date, these measurement
efforts have been of varying quality, used inconsistent terminology to describe
EHRs and targeted different respondents.1 Because of this methodological
diversity, the available survey data cannot be combined or compared among
different populations or among the same populations over time, and estimates of
EHR adoption in the United States remain tentative.
One purpose of this report is to provide guidance on the best way to determine
the level of EHR adoption nationally. Determining the level of EHR adoption
must begin with a clear definition of critical terms so that both collectors and
consumers of data on EHR adoption know what to measure and how to use the
resulting data. In this chapter, we define key terms related to the measurement
of EHR adoption among doctors and hospitals. We start by reviewing existing
definitions of EHRs and EHR adoption as found in previous surveys and the
health information technology literature, and then focus on how to further
develop the content of EHR adoption surveys by specifying what it is they
should attempt to measure. This includes recommended definitions for new and
existing surveys, as well as implications for their use in these surveys, such as the
appropriateness of various office personnel as respondents.
Guidance from the Expert Consensus Panel (ECP) was essential. The ECP helped
us, for instance, develop definitions of an EHR and of EHR adoption for survey
design. In order to generate better data than that currently available, we propose
that an EHR definition based on multiple functionalities be used in future EHR
adoption surveys. The steps the project team, along with the ECP, took to reach
this conclusion included a modified Delphi process. We discuss these steps further
in this chapter.

EHR Definitions: ISO and IOM

Many organizations have developed global definitions of EHRs. As an example,
the International Organization for Standards (ISO)2, a network of national
standards institutes from 156 countries, issued a technical report that defines both
a standard EHR and an EHR designed for an integrated health care system. The
definitions are as follows:
Standard EHR: A repository of information regarding the health of a subject of
care, in computer processable form.
Integrated Care EHR: A repository of information regarding the health of a
subject of care, in a form able to be processed by a computer that is stored and
transmitted securely and accessible by multiple authorized users using different
applications. It has a standardized information model which is independent of
an EHR system. Its primary purpose is the support of continuing, efficient and
quality integrated health care and it contains information that is retrospective,
concurrent and prospective.
Both ISO definitions emphasize that EHRs are not simply paper records viewable
in electronic form, but store information in a form that can be processed. The
integrated care version further specifies that information from individual records
be collected in a system that supports continuing, efficient and quality integrated

2:6  Health Information Technology in the United States: The Information Base for Progress

health care. This latter definition, however, focuses on the capacity of the EHR
rather than how it is actually used in practice. As another example, the American
Hospital Association (AHA), in a 2005 survey, defined EHRs as “electronically
originated and maintained critical health information, derived from multiple
sources, about an individual’s health status and health care. An EHR replaces the
paper medical record as the primary source of patient information.”3
As an alternative to a global definition of an EHR, researchers may ask survey
respondents about a series of functionalities that could be used to construct a
measure of EHR use. The Institute of Medicine (IOM), for example, has proposed
the following core EHR functionalities4:
Table 2: Basic EHR Functions Necessary to Promote Patient Safety, As Defined by the IOM
Core Functionalities

Key Elements

Health Information and Data: patient
information needed to make sound clinical
decisions

medical and nursing diagnoses, medication lists, allergies, demographics,
clinical narratives and test results

Results Management: ability to manage results
of all types electronically

computerized laboratory test results and radiology procedure result reports,
automated display of previous and current test results

Order Entry Management: entry of medication
and other care orders, as well as ancillary
services, directly into a computer

computerized physician order entry (CPOE); patient laboratory, microbiology,
pathology, radiology orders; electronic prescribing of medication orders;
nursing orders; ancillary service and consult referrals

Decision Support: computer reminders and
prompts to improve prevention, diagnosis and
management of patient disease

screening for correct drug selection, dosing and interactions with other
medications; preventive health reminders for vaccinations, breast cancer
screening, colorectal screening and cardiovascular risk detection; clinical
guidelines and pathways for patient treatment; management of chronic
diseases

Electronic Communication and Connectivity:
online communication between the health care
team, other care partners and patients

electronic communication tools—including integrated health records, e-mail
and Web messaging—for use among health care team members, between
physicians, laboratories, radiology and pharmacies and with patients;
telemedicine or electronic communications between providers and patients
who reside in remote areas; home telemonitoring for the elderly or others with
chronic diseases

Patient Support: education and self-testing

computer-based patient education; home telemonitoring for patients with
chronic diseases

Administrative Processes: electronic
scheduling systems and billing and claims
management

electronic scheduling systems for hospital admissions, inpatient and outpatient
procedures and visits; validation of insurance eligibility, claim authorization and
prior approvals; identification of patients eligible for clinical trials

Reporting and Population Health
Management: clinical data collection to meet
public, private and institutional requirements

clinical data represented with standardized terminology and in a machinereadable format to meet federal, state, local and public health reporting
requirements; also to meet organizational reporting requirements for key
quality indicators

Health Information Technology in the United States: The Information Base for Progress  2:7

A single set of functionality measures, however, is not sufficient for every health
care setting, or even for all respondents within a care setting (see following section
for details). Adoption issues can differ significantly between ambulatory and
inpatient settings, and separate content needs to be developed for each setting.
Important functionalities in an inpatient setting may include:
1. Radiology department systems (such as the Physics and Astronomy Classification
Scheme (PACS) and Radiology Information Management System (RIS))
2. Medication administration subsystems, possibly with extensions for automated
patient identification and bar coding
3. Laboratory information systems
4. Pharmacy department systems
5. Nursing notes
6. Operating room management systems
7. Critical care and cardiac monitoring systems
8. Emergency department systems
9. Clinical data repositories that integrate information from multiple
departmental systems
10. Clinical decision support systems that provide alerts, reminders and other care
guidance to the provider
11. Scheduling systems.

National Surveys: Defining EHRs

The need to develop a common, valid definition of an EHR is relevant both to
the revision of existing surveys and to the design of new surveys. Currently, the
only ongoing national surveys of physicians and physician groups that address
the use of EHRs are the National Ambulatory Medical Care Survey (NAMCS)
and the Medical Group Management Association (MGMA) Assessing Adoption
of Health Information Technology project; the only ongoing national survey of
hospitals is the National Hospital Ambulatory Medical Care Survey (NHAMCS).
However, the NHAMCS survey only measures EHR adoption in outpatient
hospital departments. (The AHA’s annual survey does not yet include measures of
EHR adoption).
First conducted in 1973, the NAMCS began asking about EHR use in 2001. The
EHR module was expanded in 2005 to include a number of questions about
EHR functionality. In 2005, the last year for which data is available, NAMCS
surveyed 1,281 office based physicians, asking them to provide information on
a random sample of patient visits during a one-week period.5 The 2006 NAMCS
survey asked: “Does your practice use electronic medical records (not including
billing records)?”6 If the answer is “yes, all electronic,” or “yes, part paper and

 In medical imaging, picture archiving and communication systems (PACS) are computers or networks dedicated to the storage, retrieval, distribution and presentation of images.
PACS replaces hard-copy based means of managing medical images, such as film archives. It expands on the possibilities of such conventional systems by providing capabilities of
off-site viewing and reporting (distance education, tele-diagnosis). Additionally, it enables practitioners at various physical locations to peruse the same information simultaneously,
(teleradiology). With the decreasing price of digital storage, PACS systems provide a growing cost and space advantage over film archives. A radiology information system (RIS)
is a computer system that assists radiology services in the storing, manipulation and retrieval of information. These systems electronically manage different aspects of radiology
workflow including exam ordering and scheduling, patient registration and worklist generation, transcription and management reporting and results distribution.

2:8  Health Information Technology in the United States: The Information Base for Progress

part electronic,” it then asks: “Does your practice’s electronic medical record
system include:
1. Patient demographic information?
2. Computerized orders for prescriptions?

If yes, ask— (a) Are there warnings of drug interactions or contraindications
provided?

(b) Are prescriptions sent electronically to the pharmacy?
3. Computerized orders for tests?
If yes, ask—Are orders sent electronically?
4. Lab results?
If yes, ask—Are electronic images returned?
5. Imaging results?
If yes, ask—Are electronic images returned?
6. Clinical notes?

If yes, ask— (a) Do they include medical history and follow-up notes?

(b) Do they include reminders for guideline-based interventions
and/or screening tests?
7. Public health reporting?
If yes, ask— Are notifiable diseases sent electronically?”
Respondents could answer “yes,” or “not known” for each question. Two questions
that follow ask: “Are there any of the above features of your system that you do not use or
have turned off?” and “Are there plans for installing a new EMR system or replacing the
current system within the next three years?” The 2006 NAMCS, however, lacks items
that address patient support or administrative functions.
The MGMA survey examines the use of information technology, including EHRs,
among medical group practices with three or more physicians. Last conducted in
2005, it contains data on 3,354 medical group practices that can be used to create
nationally representative estimates of adoption among medical group practices
(Flaws in the MGMA database, however, somewhat limit the survey’s findings.).
The MGMA survey defined an EHR as “accessible through a computer terminal
that stores patient medical and demographic information in a relational database.”
It then asked questions about a practice’s current level of information technology
adoption, including its patient appointment system, referral authorization
system, referral tracking system, clinical laboratory order entry system, clinical
laboratory results system, radiology or imaging order entry system, radiology or
imaging results system, prescription writing system, prescription refill system, drug
interaction warning system and medical records system.
Specific to EHRs, the MGMA survey asked: “As of today, what is your degree of
electronic health record implementation?” and offers the following responses:
1. Fully implemented for all physicians and all practice locations
2. Implementation in process or EHR is fully implemented for a portion of
practice physicians or locations
3. Implementation planned in next 12 months
Health Information Technology in the United States: The Information Base for Progress  2:9

4. Implementation planned in next 13 to 24 months
5. Not implemented
It then asked practices that have implemented an EHR about the cost of the
system. The next question looked at the EHR system’s functionality, asking: “If
your practice has implemented EHR, identify the specific functions currently
available from the system:”
a. Patient demographics
b. Presenting complaint
c. Past medical history
d. Physical exam/review of systems
e. Visit/encounter notes
f. Procedure/operative notes
g. Laboratory results
h. Radiology/imaging results
i. Patient medications/prescriptions
j. Problem lists
k. Referrals to specialists
l. Consult/reports from specialists
m. Clinical guidelines and protocols
n. Drug reference information
o. Drug formularies
p. Drug interaction warnings
q. Immunization tracking
r. Integration with practice billing system
The final questions assessed the potential benefits of various EHR features for
physician practices and barriers that have slowed, prevented or encouraged
EHR implementation.
NHAMCS is a nationally representative survey of hospital emergency and
ambulatory care departments. It includes questions on EHR use identical to
those in the NAMCS survey. NHAMCS data can be used to make representative
national estimates of EHR use in the outpatient hospital setting, but it does not
address inpatient EHR use.7
Physicians or practice managers can respond to the NAMCS and MGMA surveys.
As perceptions of EHR adoption may vary between these participants, the lack of
respondent specificity raises questions about the reliability of any survey findings.
Moreover, unless respondent physicians have received significant training or are

2:10  Health Information Technology in the United States: The Information Base for Progress

highly motivated to learn about EHR systems, they are unlikely to be aware of
functions they do not use. Practice managers and CIOs may be a more appropriate
respondent for questions that focus on EHR capabilities and physicians for questions
about their use in practice. Thus, to ensure appropriate interpretation of the data,
researchers should carefully track who the respondents are within each practice.

One-Time Surveys: Further EHR
In addition to the measures in these ongoing national surveys, other high
Definitions, Different Respondents quality survey items have been used to assess EHR adoption as part of one-time

investigations that are unlikely to be repeated in the future. These surveys also
developed their own explicit or implicit definitions of an EHR. The following
surveys asked respondents about specific EHR functionalities:
A survey conducted by researchers at the University of California, Berkeley, School
of Public Health in 2005, called the National Survey of Physician Organizations
and the Management of Chronic Illness8, asked:
1. Does your group make available an electronic medical record that includes any
of these components?:
a. Ambulatory care progress notes
b. Patient problem list
c. The patient’s medications
d. Alerts about important abnormal test results at the time they are received
e. Automatic alerts of potential drug interactions
f. Decision support in the form of prompts or reminders at the time the
physician is seeing the patient?
2. Do the majority of physicians in your group have electronic access to?:
a. Clinical information on the patient’s emergency room visits
b. Hospital discharge summaries
c. Outpatient reports from specialist physicians
d. Radiology results
3. If yes for each of the above—“Is this accessible within an individual patient’s
electronic medical record?”
4. Can a majority of your patients access any part of their electronic medical
record online?
5. Does your group access these electronic records to collect data for quality
measures?
The Commonwealth Fund National Survey of Physicians on Practice Experience
asked the following questions in 20039:
1. Are the following tasks currently performed in your office practice?
a. Physician receives an alert or prompt when special follow-up care is needed
Health Information Technology in the United States: The Information Base for Progress  2:11

b. Physician receives an alert or prompt about a potential problem with drug
dose or drug interaction
Responses:
Yes, using a computerized system
Yes, using a manual system
No, not done—plan to in the next year
No, not done—no plan to in the next year
2. With the patient medical records system you currently have, how easy would
it be for you (or staff in your practice) to generate the following information
about your practice?
a. List of patients by certain age groups
b. List of patients by diagnosis or health risk
c. List of patients by laboratory results
d. List of patients by medications they currently take
Responses:
Cannot generate
Very difficult
Somewhat difficult
Somewhat easy
Very easy
3. Do you currently use each of the following technology tools in your practice?
a. Electronic access to your patients’ test results
b. Electronic or computer-based decision support tools that provide realtime treatment recommendations or diagnostic support based on data
about your patients and practice guidelines
Responses:
Yes, used routinely
Yes, used occasionally
Not used, plan to use within the next year
Not used, no plan to use within the next year
The extensive capacity of emerging EHR systems poses a practical problem for
survey researchers trying to define EHRs, as guidelines and standards are likely
to become even more detailed and complex. Already, the Commission on the
Certification of Health Information Technology (CCHIT) has proposed draft
guidelines for the certification of ambulatory EHRs that include some 280
detailed functions. Even though inpatient and outpatient EHRs share many of
the same core functions, hospital EHRs are generally recognized to be different
and CCHIT is in the process of developing different certification standards for
these systems.10

2:12  Health Information Technology in the United States: The Information Base for Progress

ECP Defines EHRs by Selecting
Essential Functionalities From
IOM List

As this review of EHR measurement tools demonstrates, the lack of agreement
regarding the definition of an EHR remains a major challenge in developing
survey content about EHR adoption. Prior surveys, including those listed above,
address this issue by:
1. Telling respondents to use their own definition of an EHR.
2. Providing a definition and then asking a single global question.
3. Asking a series of questions regarding specific EHR functionalities and then
aggregating the results to determine whether EHRs have been adopted.
A simplified framework is critical to developing survey content that provides
reliable and valid measures of EHR adoption and is practical for designing and
conducting surveys. The project team used several information sources to develop
recommendations for a single set of survey domains. First, as part of our effort
to provide a definitive national estimate of EHR adoption, the project team
reviewed all known surveys of EHR adoption in the United States, including
those mentioned previously. (Details of this work are presented in Chapter 3).
Throughout this process, we received guidance from the ECP, the survey content
working group and the disparities working group.
At an ECP meeting, held January 24–25, 2006, in Washington, D.C., a subgroup
of the panel was given the responsibility of bringing suggestions to the full
committee on which measures identified by our research team should be used
to define an EHR. The ECP subgroup met three times to review the definitions
used in extant, high quality surveys. This led to a general agreement that the
core functions proposed by the IOM provided a useful framework for future
discussions. The ECP subgroup also proposed key questions that could be
developed to determine whether each core function is accomplished as part of an
EHR or as a paper or manual process.
Although the entire IOM list of functionalities could be included in new EHR
adoption surveys, existing surveys are likely to have space limitations. In order
to reduce the number of items that need to be included, the project team sought
guidance from the ECP on the EHR functionalities thought to be critical for
“EHR adoption.”
At the second in-person ECP meeting, held on April 5, 2006, in Washington,
D.C., ECP members participated in a modified Delphi process to rank the IOM
functionalities. The goal was to limit the functions to those that represent a basic
level of EHR use and that absolutely have to be present to call an electronic
data system an EHR. Votes were tallied and displayed to ECP members, who
discussed the results and then voted a second time. This led to the designation of
the following functions as essential to report that a practice or organization has a
functioning EHR:
■

health information and data;

■

results management;

■

order entry management; and

■

decision support.

Health Information Technology in the United States: The Information Base for Progress  2:13

Defining EHRs:
Further Considerations

ECP members who participated in the modified Delphi process were quick to
point out that many of the eliminated functions, specifically those addressing
interoperability, were key to integrated EHR systems. However, the percentage
of physicians and organizations with EHRs that currently are interoperable
with other providers and institutions is thought to be low. Thus, including
interoperability in the definition of a “basic EHR” might result in misleading
information about the extent of effective EHR adoption.
As EHR dissemination advances, connectivity is expected to become a core element
for inclusion in future EHR definitions. Also, the Department of Health and
Human Services (HHS) is interested in connectivity as a key functionality for health
information systems in the United States. Thus, the project team has considered, in
detail, what measuring connectivity as a dimension of EHRs would entail.
The American Health Information Community (AHIC), a private and public
sector collaboration was established by HHS to help develop standards for
electronic health information and advise the Secretary of HHS on HIT policy.
The AHIC has emphasized the following connectivity domains as priorities:
consumer-clinician messaging that includes online consultations, prescription
refills, scheduling and referrals and lab results available to patients; online
patient registration and medication history; and biosurveillance or public health
reporting.11 These topics have not been well covered in the ongoing, national
surveys, such as NAMCS, NHAMCS and MGMA.
As EHRs become more widespread, new survey question content should be
considered for the following connectivity domains:
■

exchange of information between hospitals and admitting physicians;

■

exchange of information among hospitals within a community;

■

■

■

■
■

A Second Challenge: Defining
EHR Adoption

exchange of information between physicians and physician groups within a
community;
exchange of information between patients and hospitals (such as availability of
patient portals that enable access to personal health records);
exchange of information between patients and physician offices beyond lab
results, e-mail and appointment scheduling;
exchange of information between health plans and patients; and
exchange of information between or among hospitals, physicians, pharmacies,
nursing homes and home health care providers.

Assessments of EHR adoption must go beyond measuring system functionalities
to consider the temporal element inherent to the implementation process. EHR
adoption is not one measurable event that occurs at a defined moment in time,
such as the purchase or acquisition of the technology. Instead, adoption measures
also must take into account the implementation and successful use of EHRs and
their component parts.
We define “adoption” as a process that, for measurement purposes, captures the
acquisition, installation and use of EHRs. The term “acquisition” can be further
understood as the process of obtaining the technology (through purchase or other
means), while “installation” consists of its deployment in working order within
a health care setting. The term “use” connotes the actual employment of EHRs

2:14  Health Information Technology in the United States: The Information Base for Progress

in providing patient care and related functions. Our final measures for EHR
adoption reflect these elements, as defined.
Acquisition, the first stage of the EHR adoption cycle, includes several measurable
steps: researching systems, budgeting to obtain a system and investing in a system.
The 2006 NAMCS survey addresses acquisition with the following question: “Are
there plans for installing a new EMR system or replacing the current system within
the next 3 years?” Others, such as the MGMA survey, do not address the issue of
acquisition, which suggests that it might be an area ripe for content development.
Subdomains that could be developed to gain a fuller understanding of the
acquisition process include:
■

Is the respondent planning to purchase, lease, rent or use or subscribe to a
system acquired by someone else?

■

Has the purchase, lease or rental occurred?

■

Has all the necessary hardware and software arrived?

Questions about whether a purchase, lease or rental has occurred are thought to
provide more reliable data on EHR acquisition than questions regarding plans to
purchase a system.
To measure physician acquisition accurately, it is critical that those most
knowledgeable about a practice’s IT purchasing decisions complete the survey.
Physicians in solo practice are the preferred respondent for questions targeting
acquisition, while practice managers, IT personnel, or CIOs are typically the
most appropriate choice in group practices. Surveys in the inpatient setting
need to measure the acquisition of EHR systems at the departmental level.
This is because hospitals often acquire systems for specific departments, such as
laboratories and radiology, first, and then move on to other departments. Thus,
an overall question about EHR acquisition in the hospital might not accurately
capture adoption data.
Prior surveys have included questions about the acquisition of EHR systems,
such as the Electronic Medical Records in Family Medicine survey used by
Glenn A. Loomis and colleagues to assess EHR adoption among family
physicians in Indiana.12 This survey asked:
1. Does your practice plan to implement an EMR?
2. When do you plan to implement the system?
The second stage of EHR adoption is system installation. The 2006 NAMCS
survey does not address installation; the MGMA survey asked about the degree
of electronic health record implementation and future plans for implementation
(see full question above in the National Surveys: Defining EHRs section). The
survey also asked group practices that have implemented an EHR system a second
question about the system’s cost.
Because installation is incremental, the following subdomains could be used to
clearly delineate where practices or organizations are in this process:
■

■

Has the deployment of hardware and software begun within the organization?
(By deployment, we mean that the equipment is available and working at
intended sites of use.)
Has deployment been completed?

Health Information Technology in the United States: The Information Base for Progress  2:15

■

If not, in what percentage of planned locations has it been deployed?

■

When is complete deployment anticipated?

Questions about the implementation process are best directed to practice
managers, IT personnel and CIOs at both the group practice and hospital level.
Again, in order to yield a complete picture of EHR penetration in hospital
settings, surveys should focus on the department level as well as the enterprisewide integration of data from departmental systems.
Questions about the third stage of the EHR adoption cycle, system use, are
included in both the 2006 NAMCS and the MGMA surveys. However, neither
survey contains items that address the entire list of functionalities recommended
by the IOM. To capture the dimension of use, the NAMCS survey first asks
respondents about their system’s functionality and then if any of those functions
are not in use or turned off. However, these questions may not adequately capture
EHR use for several reasons. The initial survey question asks if the practice’s
electronic medical record includes each of the functionalities but not if the
physician actually uses them. As physicians could choose not to use functions that
are turned on in their system or not use functions because they are unaware their
EHR has those particular capabilities, the “turned off ” response is not an adequate
proxy for use. Also, they may be reluctant to report that they do not use particular
functions, leading to an overestimation of use.
The MGMA survey, in contrast, asked about the following functionalities:
■

clinical laboratory order systems;

■

clinical laboratory results systems;

■

radiology/imaging order entry systems;

■

radiology/imaging results systems; and

■

prescription writing and refill systems.

For each functionality, respondents select from the following responses to indicate
how their EHR accomplishes the relevant task:
■

manual system using paper documents;

■

computerized system used by practice staff;

■

computerized system used by practice physicians; or

■

a combination of the responses above.

As with the NAMCS survey, it is not clear that the MGMA survey provides
reliable data on the use of EHRs within medical group practices. A practice
manager generally completes the MGMA survey; while they are likely to be aware
of the EHR system’s capabilities, practice managers may not be aware of the
functionalities that physicians in their practice are actually using.
Asking about the use of these functions at the point of care is likely to result
in the most reliable data. Physicians could be asked, for example, if they pull a
paper chart or turn to a computer for a patient’s health information during an
office visit. Physicians are more likely to provide reliable data when questions
are directly related to their behavior when providing care, such as whether or not
they use a particular EHR function, rather than asking if the EHR that they use
has a certain capability.

2:16  Health Information Technology in the United States: The Information Base for Progress

By asking doctors in solo or small group practice if they use a particular function,
researchers can infer that the practice has both acquired and installed the function.
This may not be the case for larger group practices and hospitals, where EHRs may
have been acquired, installed and used in some parts of an organization and not
even acquired in others. In order to ensure that researchers can infer acquisition
and installation from use in these settings, the study would require a sufficient
number of respondents distributed across all units of the organization.
CIOs also can provide information about EHR acquisition and installation and
are likely to be the most knowledgeable respondent for questions about EHR
functionalities. However, like practice managers, they may not be the most reliable
respondent for information about physicians’ actual use of the system, and survey
researchers may have to ask physicians to report on their own use of various
EHR components in hospitals where they admit patients. Physicians who admit
patients to more than one hospital could be asked to report on the hospital where
the majority of their patients are admitted. This would allow researchers to report
on the percentage of physicians using an EHR when they are seeing patients in a
hospital and not the percentage of hospitals where physicians use EHRs.
Like questions aimed at understanding the acquisition and installation process,
surveys of EHR use in the hospital setting need to be focused at the department
level. However, they also should include items that allow researchers to determine
the degree of hospital-wide data integration. If several different systems are in
place, questions should determine the extent of their interconnectivity and
physicians’ ability to access all relevant data from a single terminal proximate
to where they provide services in the hospital. For example, key indicators of
integration might be discovered by asking:
■

Can all data be accessed from a single workstation?

■

Is there a single log-on and password?

■

Can data be pulled together on one screen for viewing?

■

■

Other Surveys:
Assessments of Use

Does the clinical decision support system integrate knowledge from multiple
systems, or is it confined to only a partial view of the data?
Does the physician have to learn widely different user interfaces among
these systems?

One-time surveys also have attempted to measure the use of EHR systems.
Berkeley’s 2005 survey asked respondents, who reported that their practice
used a system with a particular capability, specific questions about its use.8 The
Community Tracking Survey asked if computers or other forms of information
technology were used for various care management functions and communication
about clinical issues.13 The Commonwealth Fund’s National Survey of Physicians
on Practice Experience asked whether specific “technology tools” were used
in a practice—electronic ordering, electronic patient records and electronic or
computer-based decision support tools—and, if so, how often they were used.9
Finally, the Electronic Medical Records in Family Medicine Questionnaire asked
physicians about their current EHR use as well as the year they began using
electronic records.12

Health Information Technology in the United States: The Information Base for Progress  2:17

Conclusion

The following summarizes our conclusions about the preferred approaches to
defining critical terms related to the adoption of EHRs and presents general
methodological guidelines for future research based on those conclusions. These
guidelines were developed for use by federal agencies, such as ONC and other
organizations that seek to apply principles of best survey practice to the complex
issue of measuring EHR adoption in the United States.
■

■

■

■

■

■

The minimal criteria for defining an EHR when measuring adoption include
four functionalities: collection of health information and data, results
management, order entry management and decision support.
EHR adoption surveys should ideally include the following domains: EHR
functionalities; acquisition, installation and use; barriers and incentives to EHR
adoption; and practice and market characteristics.
Both the NAMCS and MGMA surveys contain useful items on EHR
adoption; however, to meet the project’s goal of reliable adoption measures
for policy development and to assess the needs of providers serving vulnerable
populations, new content must be developed.
There will likely be a need to develop survey content specific to the inpatient
setting. The project team will seek guidance from the ECP and the survey
content subgroup on this issue.
Asking physicians about EHR use at the point of care is likely to result in
reliable data on their adoption of EHRs. Moreover, researchers can assume that,
if a physician reports using a function, then the solo or small practice has both
acquired and installed the function. This assumption may not be the case for
larger group practices and hospitals, where EHRs may be fully installed and
used in some parts of an organization, but not in others. In order to ensure that
researchers could infer installation from use, the study would require a sufficient
number of respondents distributed across all units of a hospital or large group.
Hospital EHR adoption surveys should focus at both the departmental level and
at the hospital-wide level.

2:18  Health Information Technology in the United States: The Information Base for Progress

Chapter 3: Current Levels of EHR Adoption: What Do We Know?
As interest in EHR adoption has grown, there has been a proliferation of
adoption surveys with varying methodologic rigor. Given that EHR adoption will
likely become an important component of quality measurement, performancerelated payments and population health assessments, understanding the level of
adoption will be critical to assessing these programs and to guiding private and
public policy interventions.
This chapter further describes and summarizes the recent available surveys and
their findings on EHR adoption. It also assesses the quality of existing surveys and
their data and lays the groundwork for improving the information available to
develop policies that promote EHR adoption. Specifically, this chapter provides:
(1) a current estimate of EHR adoption; (2) a method for arriving at a definitive
assessment of EHR adoption based on existing data; and (3) recommendations
regarding critical information gaps and optimal approaches to addressing those
gaps in assessing EHR adoption going forward. It also constitutes a building block
and precursor to a publicly accessible, searchable database of surveys assessing
EHR adoption and a summary assessment of those existing surveys.

Environmental Scan:
Choosing An Approach

The environmental scan undertaken for this report represents the most
comprehensive synthesis of available information on the status of EHR adoption
in the United States to date. Prior to settling on this approach, the project team
considered alternate methods of obtaining EHR adoption data from sources
other than surveys. Information generated in the process of using EHRs that
is recovered by the government or other agencies, for example, could provide
valuable data on whether EHRs have been adopted. The ECP agreed to consider
data collection options, such as those related to automatic reporting devices, as
they become available, with careful consideration of any privacy concerns.
The Expert Consensus Panel (ECP) members also highlighted several other sources
of non-survey data. Researchers could examine financial trends in the information
technology sector to gain information on both the sales of EHR systems and the
number of companies producing them. This data could serve as a proxy for the
demand for these systems. Trended, it also could indicate adoption rates and changes
in the functionalities sold reflect, indirectly, changes in the way EHRs are used.
Further, the number of companies applying for system certification from the Office
of National Coordinator for Health Information Technology (ONC) could be used
to estimate demand for EHRs with the minimal functionalities defined by the ECP.
There are, however, several challenges inherent to using financial data for
estimating EHR adoption: vendors may have an incentive to overstate their sales
data; and hospital and large group practices may purchase a number of different
systems that are then integrated to form an EHR. In addition, though sales figures
could provide an estimate of the number of systems purchased in a particular
year, they would not be as useful for estimating the total proportion of providers
and hospitals using EHRs. Thus, for all these reasons, financial data are unlikely
to constitute a valid and reliable source of information for assessing the rate of
EHR adoption. Such data should be evaluated as part of a thorough assessment
of trends in EHR adoption, but they are unlikely to serve as a substitute for data
developed through sample surveys that meet the guidelines laid out in this report.

Health Information Technology in the United States: The Information Base for Progress  3:19

Environmental Scan: Process

This section discusses the process we used to identify and then evaluate existing
EHR adoption surveys of physicians, group practices and hospitals—including
regional as well as national and international surveys—for the environmental scan.
Specifically, the project team identified and collected all extant surveys addressing
EHR adoption and use; developed a comprehensive data abstraction protocol to
use in the environmental scan; identified key domains and variables for the creation
of a searchable database; developed an objective scoring system for assessing the
quality of existing surveys, including explicit criteria for the quality of data sources
and studies (e.g. methodological rigor, relevance to priority populations and policy
issues); scored each study or data source in terms of performance against those
explicit criteria; and provided a summary assessment of existing data on critical
questions related to EHR adoption. This section also identifies critical information
gaps concerning EHR adoption that remain to be addressed.
The environmental scan carried out by our project team included published and
unpublished data and reports completed between 1995 and 2005. Published
data were initially obtained from the peer-reviewed medical literature, based on
PubMed searches. Standard search techniques—Ovid, Google, Google Scholar
and other search engines—were then used to obtain reports and data from nonpeer-reviewed sources. Local and national EHR adoption experts reviewed the
identified surveys and pointed out surveys that had been overlooked. With
the survey reports in hand, the project team attempted to collect actual survey
instruments. In many cases, these were proprietary and could not be accessed
easily. But, requests for cooperation with this effort enabled us to obtain 22 of 36
survey instruments.
The next step was to identify key survey characteristics, or variables, useful for
searching and assessing available surveys. Using an iterative process, we created an
initial set of survey variables, which were subsequently reviewed by the ECP and
its subcommittees. The final variables include:
1. Survey Demographics: survey name, sponsor, periodicity, year fielded, target
respondent
2. Clinical Functionalities Measured: clinical notes, access to clinical evidence,
patient registries, clinical reminders, e-prescribing, electronic order entry
(non-medication), results viewing/tracking, critical results viewing, electronic
referrals, links to regional data exchange network, personal health records—
access to electronic data, personal health records—access to scheduling, personal
health records—communication with electronic billing
3. Survey Population Characteristics: national versus state level, rural versus urban
providers, unit of analysis (provider), safety net providers data, practice size
and composition
4. Survey Design: survey methods, survey quality
5. Clinical Setting Studied: ambulatory versus inpatient, solo versus group, rural
versus urban, safety net providers versus others
EHR functionalities were defined from a clinical operations perspective, according
to the EHR features that providers or patients would recognize as aids in the
clinical or administrative tasks needed to provide patient care.

3:20  Health Information Technology in the United States: The Information Base for Progress

Survey Quality:
Measures Established

The project team rated the quality of a survey’s methodology independently from
its content. This is because surveys of high methodological quality may contain
content measures that are not useful for measuring the adoption and use of EHRs
and surveys of low or medium quality may contain very useful questions.
Survey quality was judged according to benchmarks set by the quality assessment
literature and practice, which we adapted to meet the challenge of assessing surveys
of EHR adoption. These efforts drew on the consensus in the field on the standards,
best practices and guidelines for the conduct of survey and polling research developed
by professional organizations including: American Association for Public Opinion
Research (AAPOR), American Statistical Association (ASA), National Council on
Public Polls (NCPP) and Council of American Survey Research Organizations
(CASRO). The federal Office of Management and Budget (OMB) survey clearance
process and several survey research texts also provided useful guidance.
Among these criteria sets, the AAPOR’s standards for assessing survey quality are
highly regarded and widely used. They include:
■

have specific goals for the survey;

■

consider alternatives to using a survey to collect information;

■

select samples that well represent the population to be studied;

■

■

■
■

■
■

■

■

use designs that carefully balance the costs of the survey with the need for data
that is as accurate as possible.
take great care in matching question wording to the concepts being measured
and the population studied;
pretest questionnaires and procedures to identify problems prior to the survey;
train interviewers carefully on interviewing techniques and the subject matter of
the survey;
construct quality checks for each stage of the survey;
maximize cooperation or response rates within the limits of ethical treatment of
human subjects;
use statistical analytic and reporting techniques appropriate to the data
collected; and
carefully develop and fulfill pledges of confidentiality given to respondents.

As mentioned above, the environmental scan only rated surveys of physicians, group
practices or hospitals (and not the general public or consumers), including regional
as well as national and international surveys. Also, in cases where a publication
reported the results of more than one survey, the individual data sets underlying the
reported surveys were separated out and rated independently when possible.

Assessing Quality of
Survey Methods

Survey methodology was evaluated through a two-part assessment protocol:
(1) availability of key methodologic issues, and (2) performance on critical
methodologic indicators. The project team referred to publications, Web sites and
other sources to evaluate whether information about the following survey variables
was available: source of sample; sample size attempted and completed; sample design,
response rate and method used to calculate it; dates of fieldwork; full questionnaire;
disclosure of sponsor; and professional survey or research organization.

Health Information Technology in the United States: The Information Base for Progress  3:21

The second step involved rating the quality of survey administration on four
critical dimensions:
1. Representativeness: Was the survey designed and conducted in such a way that
the collected data well represents the stated population of interest?
■

■

■

High quality surveys had a well-defined population and sample source that
avoided bias; they used a scientific method of drawing a random sample subset
and data collection methods that minimized selection and response bias.
Medium quality surveys exceeded the standards of low quality surveys but
fell short of the highest quality.
Low quality surveys used convenient or volunteer samples and modes of data
collection that created unacceptable bias.

2. Response rate effort: Were diligent efforts made to enhance response rate and
reduce response bias?
■

■

■

High quality surveys were diligent in achieving high response rates (50 percent
or greater)—used multiple and varied respondent contacts, a sufficient field
period, efforts to convert non-responders and refusals, and incentives where
appropriate. They disclosed sufficient sample disposition elements to allow
examination of cases (individuals in the original sample) from sample selection
to completion.
Medium quality surveys (response rates of 30 to 50 percent exceeded the
standards of low quality surveys but fell short of the highest quality.
Low quality surveys (response rates of less than 30 percent) used minimal
contacts.

3. Questionnaire development: Was the survey pre-tested? Was the reliability and
validity of key measures assessed in prior survey efforts or in the present one?
Did the researchers attempt to minimize response bias and other sources of
bias in question wording and context?
■

■

■

High quality surveys documented the questionnaire development processes,
include pre-testing and provided information on the reliability and validity
of key measures.
Medium quality surveys exceeded the standards of low quality surveys but
fell short of the highest quality.
Low quality surveys had poorly designed or biased questions, were not pretested and provided inadequate information on the validity and reliability of
key measures.

4. Sample size: Was the sample size sufficient to minimize sampling error and to
achieve analytical objectives?
■

■

■

High quality surveys had sample sizes sufficient to minimize sampling error
and assure sufficient statistical power for the analysis of critical variables and
outcome measures.
Medium quality surveys exceeded the standards of low quality surveys but
fell short of the highest quality.
Low quality surveys had sample sizes that provided inadequate statistical
power to have confidence in survey results or to conduct key analyses.

3:22  Health Information Technology in the United States: The Information Base for Progress

Surveys that scored high on at least three of the four areas were given an overall
“high” methods score, those that rated low on at least three of the four items were
given a “low” score and all other combinations received a “medium” score.

Assessing Quality of
Survey Content

Unlike the methods assessment, an overall indicator for content quality was
not created because each content issue stands alone as an important and useful
contributor to our understanding of EHR adoption. Also, numerous pieces of
information about a survey, including the survey itself, must be available in
order to rate a survey successfully and critical elements for assessing quality were
missing or had to be inferred or calculated for several surveys identified by the
environmental scan. A lack of data limited our ability to assess the quality of
some surveys.
Content quality was rated according to a survey’s relevance to six key areas
approved by the ECP as the critical core elements for measuring EHR adoption.
(One domain, “distinctions between acquisition, installation and use,” was
subsequently dropped as the project team found this content was rarely included
in identified surveys). These areas include:
1. Whether the practice or organization has an EHR
2. Nature of EHR functionalities
3. Whether the survey distinguishes between EHR acquisition, installation and use
4. Measures of incentives for EHR adoption
5. Measures of barriers to EHR adoption
6. Ability to identify disparities in adoption among different at-risk populations.
Content quality in these areas was rated through a two-stage process. First, the
project team determined (yes/no) whether the content area was addressed. Second,
in areas that were addressed, the team then assessed whether the survey questions
were well-designed and likely to result in valid and unbiased content estimates.
Surveys with questions that were judged to both adequately cover one of the five
content areas and result in valid, unbiased estimates were given a high quality
rating in that content area; those that adequately covered a content area, but
with questions in which the project team had only modest confidence, received a
medium content quality rating; and surveys that covered a content area but with
questions unlikely to provide valid, unbiased estimates received a low content
quality rating.

Survey Quality Assessed

Four members of the project team, all experienced researchers, examined each
survey, discussed their assessments, reconciled discrepant judgments and arrived
at consensus quality assessments for both its methods and content. The team
was able to obtain both the survey instrument and complete results for 22 of the
36 identified surveys and definitively rated the quality of these 22 surveys—17 of
which were physician or physician group surveys and five of which were hospital
surveys. (Sufficient information was collected from the remaining 14 surveys to
determine that they were unlikely to contribute meaningfully to our estimates of
EHR adoption.)

Health Information Technology in the United States: The Information Base for Progress  3:23

Only ten surveys received a high quality methodology rating. All surveys were
given a content quality rating, regardless of their methodology rating, in the five
content areas (see Table 3 for details).
Table 3. Quality Assessments of Available Surveys
Physicians or groups

Hospitals

High quality methodology

8

2

Medium quality methodology

3

1

Low quality methodology

6

2

High quality content rating

7

2

Medium quality content rating

3

2

Low quality content rating

4

1

High quality content rating

7

2

Medium quality content rating

5

3

Low quality content rating

1

0

High quality content rating

4

1

Medium quality content rating

2

2

Low quality content rating

1

0

High quality content rating

4

0

Medium quality content rating

0

1

Low quality content rating

0

1

High quality content rating

1

1

Medium quality content rating

3

2

Low quality content rating

0

0

Methodology Quality

Content Quality
Practice/organization has an EHR

EHR Functionalities

Barriers to adoption

Incentives for adoption

Disparities

No survey was rated high in all five content areas. Only three physician/physician
group surveys and one hospital survey were rated as having high quality content
in at least three of the five content areas. Further, only two surveys achieved a
high quality rating for both methodology and at least three of the five contents
areas, leading us to conclude that the quality of available surveys is variable and
generally inadequate to form the basis for national policy development.

3:24  Health Information Technology in the United States: The Information Base for Progress

Environmental Scan: Results

This section summarizes the results of the surveys identified by the environmental
scan. In drawing conclusions, we emphasized surveys with high ratings for
methodology or content quality.
The first of the 36 surveys was fielded in 1997 and the latest in 2005, with all but
four fielded or published in the last five years. The scan also included ten surveys
that used a nationally representative sampling frame. About half of the rated
surveys assessed outpatient use of the EHR exclusively, an additional 25 percent
assessed both outpatient and inpatient EHR use and the remainder assessed
inpatient EHR use only. Most outpatient studies focused on EHR use, but EHRs
usually were not well defined. Two surveys were judged to be high quality in both
methodology and content:
The 2005 National Ambulatory Medical Care Survey found that 24 percent of
physicians had a full (11 percent) or partial (13 percent) EHR in their office-based
practice.1 This represents a significant increase from prior NAMCS surveys which
estimated EHR use at 17 percent (NAMCS 2001–2003).2 The earlier NAMCS
surveys lacked definitional precision, asking only one global EHR question. The
2005 NAMCS survey1 asked physicians about a series of functionalities, including
minimal set of functionalities that had to be present for a functional EHR as
defined by the ECP: health information and data; results management; order
entry management; and decision support. This does not track exactly to the items
used in the 2005 NAMCS. In order to estimate the ECP definition as precisely
as possible within the NAMCS survey items, the analysis included computerized
orders for prescriptions, computerized orders for tests, reporting of test results and
physician notes. Using this minimal definition, the 2005 NAMCS found that only
9 percent of physicians had an EHR with the minimal functionalities identified by
the ECP.
Another high quality survey, completed by the Commonwealth Fund in 2003,
found that 18 percent of physicians routinely use EHRs.3 A study conducted by
the Center for Studying Health System Change was rated high in methodological
and content quality. It reported EHR use by specific functionalities and found
that about 25 percent of respondents used IT for at least one EHR function, while
10 percent reported using at least four EHR functions.4
Differences in the results of these high quality surveys are likely due to variations
in the questions that were asked, the specificity with which EHRs were defined
and the time at which they were fielded. Expanding the environmental scan’s
scope to include surveys of high or medium quality in both methodology and
content led to the inclusion of two additional studies: the 2005 MGMA survey
of medical groups, which used a strict EHR definition and found that 15 percent
of group practices had EHRs,5 and another survey by the University of Kentucky,
which, while high in quality, focused only on a small number of physicians in
Kentucky and found that 21 percent use EHRs.6
Some surveys suggest adoption levels are higher, such as the 2005 American
Academy of Family Physicians survey, which found that 46 percent of 2,569
respondents had some form of undefined EHR.7 However, surveys that defined
EHRs very carefully tended to have lower estimates of adoption and are
believed to be more accurate. For example the 2004 Medical Economics survey of

 Please note that in determining the number of surveys with high quality content ratings, we considered all four domains of content quality. This differs from the number
reported in the related paper by Jha et al. in which high content quality was only attributed to those surveys rated as high for ‘presence of an EHR’.

Health Information Technology in the United States: The Information Base for Progress  3:25

10,000 physicians that found 15 percent of physicians had adopted “electronic
documentation of clinical information.” The data from these studies also suggests
that solo practitioners were less likely to have EHRs than practitioners who
worked in groups, especially those who worked in large groups. This includes
the 2003 Commonwealth survey, which found 13 percent of solo practitioners
routinely or occasionally use EHRs compared to 57 percent of physicians in
groups of 50 or more.9 The MGMA survey found that while 15 percent of
all groups used EHRs, this number varied from 13 to 20 percent based on
practice size.5 Similarly, the NAMCS survey found that only 13 percent of solo
practitioners used EHRs, but nearly 39 percent of physicians in practices with 20
or more physicians used EHRs.10

Results: Inpatient EHR Use

There were very few high quality surveys of inpatient EHR use. The following
section details the findings of surveys that were rated as either high or medium
quality on methodology and content. The 2005 NHAMCS survey is not included
here. Although it is rated high on methodology, the survey only measures EHR
use in hospital outpatient departments and emergency rooms.
The 2005 CMS/Mathematica Hospital survey,11 rated high on methodology and
medium on content, asked senior hospital executives about the use of several EHR
functionalities. It reported that 83 percent of hospitals used electronic lab results
with decision support; 59 percent used electronic clinical notes, although the
functionalities weren’t specified and could include patient demographics, medical
history, physician or nurse notes, or follow-up orders; 50 percent had electronic
images available throughout the hospital and used electronic lab orders; and
smaller percentages used electronic reminders for guideline based interventions (24
percent) and e-prescribing (21 percent).
Many inpatient surveys have focused on CPOE. A nationally representative survey
of randomly selected hospitals by Ash, Gorman, Seshadri, & Hersh (2004) found
that 16 percent had CPOE in 2002.12 A more recent survey by the American
Hospital Association (AHA) found that 21 percent of hospitals had CPOE in 2005.13
The 2003 Leapfrog survey, which explicitly scored adoption, indicated that only
5 percent of hospitals had fully implemented CPOE.14 Although it is difficult
to reconcile these estimates, the discrepancies are likely due to differences in
survey question wording and their success in eliciting responses from small and
large hospitals.

Table 4. EHR Adoption Based on Best estimates Data (through 2005)
Range from Medium or
High Quality Surveys

Best Estimates Based on
High Quality Surveys

EHRs in physician offices

17 to 25%

24%

Solo practitioners

13 to 16%

16%

Large physician offices*

19 to 57%

39%

EHRs in hospitals

N/A

None

CPOE in hospitals

4 to 21%

5%**

* Large is defined as ≥ 20 physicians by one study (with an estimate of 39 percent) and ≥ 50 physicians (with an estimate of 57 percent) in another.

3:26  Health Information Technology in the United States: The Information Base for Progress

Limitations: Safety Net Providers

A striking finding from this review is that there is a dearth of data on the
adoption and use of EHRs among those who care for vulnerable populations.
An example is the lack of data about EHR adoption among safety net providers.
Though safety net providers do not deliver the majority of care to vulnerable
populations, they are an important component of the health care safety net.
A recent Institute of Medicine (IOM) report defined safety net providers as
those “that organize and deliver a significant level of health care and other
health-related services to uninsured, Medicaid and other vulnerable patients.”
Public hospitals, community health centers, and, to some extent, rural providers
in some areas have a mandated responsibility to provide care to all patients
regardless of ability to pay.15 The 2006 NAMCS survey will include a subsample
of 100 community health centers, however this data will not be available until
the summer of 2008.
The environmental scan only identified one survey expressly devoted to assessing
HIT adoption among safety net providers. Sponsored by the Community
Clinics Initiative (CCI), it surveyed community clinics in California, which had
applied for funding from the Tides Foundation to improve their clinic’s HIT,
about their level of EHR adoption. The CCI survey found that two-thirds of 112
clinics surveyed in 2000 and 2001 had implemented basic IT systems to support
their business operations, but fewer than 10 percent were using these systems
to support individual patient care such as appointment scheduling and patient
tracking and recall.16
To the extent that EHRs enhance quality of care, ensuring safety net providers have
access to EHRs is a critical component of reducing disparities in the care received
by low-income, uninsured or minority Americans. Tracking the adoption and use of
EHRs among safety net providers relative to other provider groups, understanding
unique barriers to adoption that contribute to the HIT “adoption gap,” and
identifying appropriate policy levers to close this gap remain important challenges.

More Limitations and
Future Challenges

Other limitations of current survey data make it difficult to reliably assess
EHR adoption levels. There are insufficient numbers of national surveys with
an adequate sample size, adequate response rates and high quality content to
allow valid, generalizable estimates of EHR adoption in the American health
care system. Nearly all-available surveys are based on self-reported data, with no
auditing mechanism to confirm reported levels of adoption and use. Even surveys
that are well formulated, have high response rates and use national samples,
rarely provide adequate information about adoption rates among specific types of
providers (e.g., solo practitioners versus group-based providers).
Further, the majority of identified surveys are proprietary, making efforts to
obtain the full set of survey results and the survey instruments challenging,
and there is a lack of uniformity in definitions and measures among those
that provide detailed information. Most surveys do not clearly define terms
such as “electronic health record,” leaving the meaning of these terms open
to interpretation by survey respondents and making comparisons of results
across surveys and over time extremely difficult. Even when surveys use precise
definitions, they are usually novel and lack consistency with other surveys.
Therefore, although existing surveys might provide insights into rates of
adoption or barriers to adoption of specific functionalities, they do not provide
information that is generalizable to other functionalities.
Health Information Technology in the United States: The Information Base for Progress  3:27

Another limitation is inpatient EHR surveys’ frequent use of physician
respondents. Their analyses do not account for the clustered nature of the sample
and, thus, their estimates of EHR use within hospitals may be biased. Also, some
surveys asked physicians about inpatient and outpatient use while others asked
exclusively about inpatient use. Finally, adoption of certain functionalities, such
as computerized physician order entry, may be subjective. These systems typically
take years to fully adopt and use and, therefore, whether a hospital “has” CPOE
will depend to a large extent on how the question is asked.
Future work must address these definitional issues. It may be helpful to include a
legal definition of medical records—the systemic parent of EHRs—in this process.
All states, through statutes, regulations and judicial decisions, define and regulate
the content, structure, maintenance, ownership and preservation of medical
records. We anticipate that a body of law related to model medical records exists,
given the central importance of the medical record to the legal environment
in which health care practice takes place. The concept of a medical record
undoubtedly appears in numerous places in federal law and we expect federal
legislation addressing HIT and EHRs will address this basic definitional issue.
EHRs inevitably will operate as a technological “overlay,” transforming the current
medical system over time but inevitably linked into and harmonized with the legal
underpinnings of care. We believe that gaining a greater understanding of medical
record legal policy will help ensure that EHR adoption research is structured to
measure the adoption of systems that are consistent with existing expectations and
patient safeguards.
Both the definition of key terms related to EHR adoption and the measured rates
of adoption will likely change over time. The definitions and rates discussed in this
report represent our current, best judgment for defining those terms and assessing
the extent of adoption. EHR adoption is a dynamic phenomenon that will require
continued evaluation and tracking over time to lay the best foundation for policy
formation in the future.

3:28  Health Information Technology in the United States: The Information Base for Progress

Chapter 4: Will Differential HIT Adoption Exacerbate Health Care Disparities?
Eliminating health disparities has emerged as a national priority in recent
years. Healthy People 2010,1 a set of health objectives developed by the federal
government through a broad public-private consultative process, named the
elimination of health disparities as one of its two primary goals for improving the
health of Americans. The Crossing the Quality Chasm report, issued by the Institute
of Medicine (IOM) in 2001, further highlighted the importance of eliminating
health disparities, including “the provision of equitable health care that does
not vary in quality because of personal characteristics such as gender, ethnicity,
geographical location and socioeconomic status,”2 as one of its six priority goals.
A subsequent IOM report, Unequal Treatment (2002), provided further impetus for
public efforts focused on reducing disparities by comprehensively documenting
racial and ethnic disparities across a wide range of health care settings.
There has been a concomitant focus on the potential for health information
technology (HIT) and electronic health records (EHRs) in particular, to
significantly improve health care by enhancing the clinical quality and effective
management of care.4 While empirical evidence documenting the impact of HIT in
practice remains limited, its potential to improve the care received by the general
population—and possibly reducing health disparities—is widely affirmed.5-25 Crossing
the Quality Chasm, for example, drew attention to EHRs’ potential to produce
care that is more equitable,2 and the American Medical Informatics Association
(AMIA) 2003 Congress (Bridging the Digital Divide: Informatics and Vulnerable
Populations) proceedings concluded that members of underserved and vulnerable
populations are “particularly in need of health information support” due to their
increased risk for adverse outcomes.26
To the extent that EHRs prove to be a powerful means for improving care,
monitoring EHR diffusion among providers who serve vulnerable populations will
be essential to eliminating health disparities. Enhanced EHR capacity may provide
new leverage points for addressing health disparities, but only if underserved
patients have access to the clinical benefits associated with HIT. Conversely,
slower adoption of EHR-enhanced health care among providers serving these
patients could exacerbate existing health disparities. Concerns that underserved
populations may have reduced or delayed access to the benefits of EHRs are
underscored not only by existing disparities in care, but also by studies that
document a lag in access to beneficial developments in clinical care among these
communities.27, 28 A recent study, for example, found that minority children were
the last to receive new asthma medications,29 perhaps partially explaining racial
differences in asthma outcomes.
EHR adoption, if uneven, may further exacerbate existing health disparities. Thus,
monitoring the diffusion of EHRs among providers who serve populations most
likely to experience poorer quality care is an important public policy goal and
should be part of any comprehensive approach to reducing health disparities in
the United States. This chapter assesses the current state of knowledge regarding
EHR adoption among providers of underserved populations and discusses
strategies for ensuring that the diffusion of EHRs among these providers is
monitored going forward. First, we define vulnerable populations and review
conceptual frameworks and analytic approaches for studying EHR adoption
among providers serving these populations. We then review existing data that
might be useful in assessing current rates of EHR adoption among providers
Health Information Technology in the United States: The Information Base for Progress  4:29

of care to populations more likely to experience disparities. Finally, we address
concrete methods and strategies for building the capacity to assess whether there is
differential adoption of EHRs among providers who serve vulnerable populations
and, if so, what impact this might have on quality of care and health disparities in
the future.

Defining Vulnerable Populations

Developing the capacity to monitor HIT diffusion among providers who serve
vulnerable populations requires an operational definition of vulnerable populations
and an effective way of identifying providers who care for them. Ultimately, our
policy concern is the health care experience of vulnerable patients and the ways
in which variable access to the benefits of EHRs might affect their quality of
care or health outcomes. These populations, which experience diminished access
to health services and lesser quality of care once they do access the health care
system, have been variously defined. Available data documents health disparities
according to a patient’s race,30-38 socioeconomic status,39-41 and insurance status.42-44
Other relevant populations include patients with a primary language other than
English,33, 45-47 those living in rural or other underserved areas,48, 49 and those with
special health care needs.48, 50 At an Expert Consensus Panel (ECP) meeting in April
2006, there was agreement that racial and ethnic minorities and low-income patient
populations were the highest priority groups with respect to tracking access to
EHRs and their potential implications for health disparities.51
While race and ethnicity continue to be a primary criterion for assessing health
disparities in the United States, there have been longstanding definitional and
procedural issues surrounding the collection of race data to track them. Nearly
all race and ethnicity data is collected according to categories specified in the
1997 Office of Management and Budget (OMB) revised minimum standards
for presenting data on race and ethnicity.52 OMB Directive No. 15 specifies
five minimum race categories for data collection: American Indian or Alaska
Native, Asian, Black or African-American, Native Hawaiian or other Pacific
Islander and White and two ethnicity categories: Hispanic or Latino and nonHispanic or non-Latino. These categories are used in the U.S. Census and are
the most often used categories for health data collection. In October 2001, the
National Institutes of Health (NIH) amended its Policy on Inclusion of Women
and Minorities in Clinical Research52 to formally adopt the OMB categories for
monitoring underrepresented groups in all agency research. This further solidified
these categories as the favored approach for capturing racial and ethnic variation
in health and health care. However, even when the OMB racial and ethnic
categories are used, data collection can vary and includes self-identified report,
birth certificate data and assignment by others. Changes to census protocol in
2000, which allow individuals to select more than one racial group to identify
themselves, have further complicated analyses of racial and ethnic disparities.53
Medical, epidemiological and health services research lack a consistent definition
of race.54-56 And while thousands of publications address racial health care
disparities, the underlying variables used in these analyses are not consistent
or clearly defined.57 The OMB racial and ethnic categories are by far the most
consistent and widely used categories. Hence, it makes sense to use these categories
in efforts to track HIT diffusion and the potential impact on health outcomes and
racial and ethnic disparities.

4:30  Health Information Technology in the United States: The Information Base for Progress

Identifying Safety Net Providers

Researchers seeking to understand the impact of health system change on
vulnerable populations often have focused on providers whose patient base
is overwhelmingly drawn from uninsured, minority or other underserved or
vulnerable populations. Community health centers (CHCs), for example, serve
about one in eight uninsured patients nationally. This strategy has been driven by
two main factors. First, data is available on “core safety net providers” (e.g., CHCs
and public hospitals) that have a legal obligation to serve all patients regardless
of their ability to pay. Second, it is difficult to identify and collect data from the
many providers, such as not for profit hospitals, that individually make a modest
contribution to care for vulnerable populations but in aggregate account for the
majority of all such care provided.
The most exhaustive assessment of the U.S. health care safety net to date, the
recent IOM study titled “The Health Care Safety Net: Intact but Endangered,”48
did not provide an operational definition of safety net providers. Instead it
offered a broad definition of the “health care safety net” as “those providers that
organize and deliver a significant level of health care and other related services to
uninsured, Medicaid and other vulnerable patients.”48 Core safety net providers
were defined by the IOM study as follows: “(1) either by legal mandate or
explicitly adopted mission they maintain an ‘open door,’ offering access to services
for patients regardless of their ability to pay; or (2) a substantial share of their
patient mix is uninsured, Medicaid and other vulnerable populations.”
Safety net providers have a unique perspective on the needs of underserved
populations, as these populations make up the bulk of such providers’ patient
panels (e.g., more than three-fourths of CHC patients are uninsured or on
Medicaid).58 But focusing only on this subgroup of providers to study EHR
adoption has important limitations. Most notably, measuring EHR penetration
among providers with a large proportion of vulnerable patients does not capture
the experiences of vulnerable patients in a nationally representative way. Private,
not-for-profit hospitals, for example, account for the vast majority (about 56
percent) of all free care provided,59 even though the proportion of uninsured
patients at public hospitals is far higher.
Another approach is to focus on providers who serve large numbers of patients
from designated subpopulations, or “high volume” providers. Due to their large
size, these providers may account for the majority of services provided to minority
or uninsured patients in their service area—even though these patient subgroups
only account for 10 percent or less of their total patient panel.
In the end, it is likely that a multi-pronged approach will be necessary to capture
patterns of EHR use in the care of poor, minority, uninsured or other vulnerable
populations. This combination of data from traditional safety net providers and
those who serve many such patients will allow us to understand whether and to
what extent patterns of EHR adoption differ among providers serving distinct
patient subsets and what impact this variation has on quality of care, clinical
outcomes and health disparities.

Health Information Technology in the United States: The Information Base for Progress  4:31

Current EHR Adoption Levels

As discussed in Chapter 3, our environmental scan found that the data currently
available on EHR adoption among physicians, physician groups and hospitals
is limited and variable.60 Even less information is available on providers serving
a large proportion or a large number of minority or low-income patients. Most
of the available data have focused on community health centers and clinics in
California. CHCs are key ambulatory providers for poor, uninsured, minority and
other underserved populations. By statute, CHCs receiving federal 330 grants are
mandated to serve all patients regardless of their ability to pay. Nationwide, about
76 percent of CHC patients are covered either by Medicaid or are uninsured,
64 percent are minorities and 29 percent have a primary language other than
English.58, 61 Overall, they care for more than 10 percent of uninsured patients.62
The California Community Clinics Initiative (CCI), a partnership between
the Tides Foundation and the California Endowment, has provided about $41
million to help 163 clinics and 15 regional associations strengthen their health
information management capacity. Over the last six years, these clinics have
upgraded their practice management systems, hired HIT staff, improved their
communications networks and developed more comprehensive plans for using
HIT to support their missions. Each year, CCI and its external evaluator, Blueprint
Research and Design Inc., have administered a written survey to these clinics, titled
“The Clinic Information Management Assessment Survey.” Surveys collected
in the fall of 2002 yielded an 80 percent response rate from executive directors
and 84 percent from medical directors. CCI’s 2003 Information Technology Fact
Book reported that 5 percent of medical clinics had EHRs and 3 percent of dental
clinics; in addition, about 23 percent of medical clinics and 9 percent of dental
clinics had established EHR implementation planning committees.
EHR use in Hospital Emergency Outpatient Departments

Hospital emergency rooms and outpatient departments play an important role
in providing care to the uninsured, those on Medicaid and those with no other
regular source of care. The National Hospital Ambulatory Medical Care Survey
(NHAMCS) collects annual data on the state of EHR adoption in both of
these settings. In between 2001 and 2003, the last year for which data is publicly
available, finds that 31 percent of emergency departments and 29 percent of
hospital outpatient departments have an EHR.

New Analyses

We have sought to address the gap in information on EHR adoption among
providers who serve vulnerable populations by analyzing unpublished data,
including new findings on CHCs as well as analyses of 2004 National Ambulatory
Medical Care Survey (NAMCS) visit data and 2005 NAMCS physician survey
data. With respect to NAMCS, we are indebted to the National Center for Health
Statistics (NCHS), the agency that fields the survey, for their help in making these
data and analyses available to us.
Community Health Centers

The most recent CCI survey, conducted in March 2005 (76 percent response
rate for executive directors and 78 percent for medical directors), showed marked
improvements in key aspects of EHR adoption including: connectivity and
communications capabilities; practice management systems; clinical technology
such as disease registries and immunization registries; staff technical skills such as
 Burt C, Hing E. Use of Computerized Clinical Support Settings in Medical Settings: 2001–2003. Advance Data from Vital and Health Statistics. No 353. March 15, 2005

4:32  Health Information Technology in the United States: The Information Base for Progress

application support and systems administration; and quality of data available to
support business and clinical administration decisions. In 2000, for example, twothirds of multi-site clinics had remote sites that were unable to access the clinic’s
practice management system. But, by 2005, this decreased to only 7 percent.
Also in 2005, 73 percent of clinics had a diabetes registry, up from 55 percent
in 2002, and 61 percent had an immunization registry, up from 55 percent in
2002.64 The California experience suggests that substantial gains in connectivity
across practice sites is possible within a short period of time among providers who
serve a disproportionate number of vulnerable patient populations, but only with
substantial capital investment, such as the $41 million invested in community
clinics in California.
Alexandra Shields and colleagues, in collaboration with the National Association
of Community Health Centers, fielded a survey in March 2006 to generate the
first national estimates of EHR capacity and adoption barriers among federally
funded health centers.65 In order to facilitate comparisons between CHCs and
physicians, this survey (N=912, response rate=79.5 percent) included items from
the 2005 NAMCS survey of physicians. One of these items asked providers if their
practice had an electronic medical record, with the following possible responses:
“yes, all electronic; yes, part paper and part electronic; no; and don’t know.”
Virtually all CHCs were able to answer this question without resorting to the
“don’t know” response.
Figure 1 summarizes data from the 2005 NAMCS survey of physicians and 2006
data on EHR adoption among CHCs. Close to 9 percent of CHCs report having
a fully electronic medical record and an additional 15.9 percent report having
a partial EHR, compared to more than 11 percent and 13 percent respectively
of physicians in the NAMCS data. Thus, a greater proportion of CHCs have at
least some electronic capability, most notably in maintaining patient registries,
than does a representative sample of American physicians. Some portion of this
difference is likely due to the Health Disparities Collaborative, sponsored by
the Health Services Resource Administration, which requires that participating
CHCs maintain disease registries.66 CHCs also stand as an example of the kinds
of substantial improvements in HIT capacity that can be realized with targeted
resources and programmatic support. Indeed, among CHCs nationally who
currently do not have an EHR, nearly 90 percent cited lack of capital to invest in
an EHR as the primary barrier .65 Overall, about one fourth of both physicians
generally and CHCs reported using a full or partial EHRs and the remaining three
fourths had no electronic health information capability.

Health Information Technology in the United States: The Information Base for Progress  4:33

Proportion of Responses in Percent

Figure 1: EHR Adoption Among U.S. Physicians and Health
Centers (2005-2006)
76.1 75.5

80
70
60
50
40
30
20
10
0

11.2

8.6

Full EMR

12.7

15.9

Partial EMR

None

EMR Adoption Status
Physicians

CHCs

Note: Physicians percentages are based on preliminary data from the 2005 National Ambulatory Care Survey
(N=1,281 eligible physicians; 66.2 percent response rate). CHC percentages are based on preliminary data
from the 2006 Survey of Health Center Use of Electronic Health Information (N=725 health center CEOs or
Executive Directors; 79.5 percent response rate)

County-Level Population Characteristics

Researchers from the National Center for Health Statistics analyzed 2005
NAMCS data to investigate whether physicians’ self-reported adoption of EHRs
differed according to key characteristics of their practice location. Burt, Hing and
Woodell67 used zip codes to link county-level data from the Area Resource File
(ARF)68 (i.e., percent county population that is non-Hispanic white and per capita
income for the county in which the practice is located) to physician practices.
They then determined if EHR adoption varied according to a county’s per capita
income or racial composition; they also assessed geographic variation. The study
found EHR use was not significantly associated with these county-level variables
(Table 5). This may be due to the segregation of vulnerable populations within
county-level markets, which is not captured in geographic analyses, or there may
be little actual variation in EHR adoption among providers who serve vulnerable
populations compared to those serving the general population. Thus, further
research is needed to understand patterns of adoption among policy relevant
subsets of providers. Adoption was found to vary with a practice’s geographic
region: physicians in the Midwest and West were more likely than those in the
Northeast to use EHRs and physicians in metropolitan statistical areas (MSAs)
were more likely than non-MSA physicians to use EHRs (Table 5).

4:34  Health Information Technology in the United States: The Information Base for Progress

Table 5. Location Characteristics of Office-Based Physicians: United States, 2005.
Percent distribution of
all physicians (based
on weighted responses
from 1,281 sample
physicians)
All physiciansc

Percent of physicians
reporting full or partial
use of electronic medical
recordsa (standard error)

Percent of physicians
reporting minimum set
of required features
for electronic medical
recordsb (standard error)

100.0

23.9 (1.5)

9.3 (1.1)

Northeast

20.9

14.4 (2.3)

3.4 (1.2)

Midwest

21.4

26.9 (3.6)

7.5 (1.5)

South

34.9

21.7 (2.7)

9.3 (2.2)

22.7

33.4 (3.5)

16.7 (2.8)

89.4

24.8 (1.6)

10.1 (1.1)

10.6

16.9 (3.1)

3.1 (1.7)

Under $25,000

11.6

20.3 (5.3)

10.0 (3.2)

$25,000-$45,000

77.2

24.0 (1.8)

8.8 (1.2)

Over $45,000

11.3

27.2 (4.2)

12.3 (3.4)

Geographic region

d

West
Metropolitan status

d

Metropolitan statistical area
Non-metropolitan statistical area
Per capita income for county

e

Percent of county population that is non-Hispanic white

e

a

b
c
d
e

Over 75%

40.5

24.8 (2.4)

8.3 (1.6)

50–75%

34.1

21.6 (2.6)

10.6 (1.9)

Under 50%

25.4

25.8 (3.2)

9.3 (2.2)

Electronic health record (EHR) refers to physicians reporting that their medical records are either fully or partially electronic. Percentages may underestimate because electronic
health record use is unknown for 4.5 percent of physicians and they are assumed not to use EHRs.
Required minimum features include computerized prescription ordering, computerized test ordering, electronic results and electronic physician clinical notes.
Includes nonfederal, office-based physicians who see patients in an office setting and excludes radiologists, anesthesiologists and pathologists.
Significant relationships between electronic health record (EHR) use and characteristic location.
Based on data from the Area Resource File

EHR Adoption and Percent Medicaid Revenue

The proportion of a practice’s revenues derived from Medicaid is often used as
a proxy for the proportion of low-income patients served. Burt and colleagues67
analyzed EHR adoption among physicians with varying levels of Medicaid
revenues, along with other covariates, using the 2005 NAMCS physician data.
They compared rates of self-reported EHR adoption and rates of EHR adoption,
using a variable that represents a minimal set of EHR functionalities, among
providers with varying Medicaid revenues. As determined by the ECP, the
minimum set of EHR functionalities51 include: health information and data,
results management, order entry management and decision support. This does not
track exactly to the items used in the 2005 NAMCS. In order to estimate the ECP
definition as precisely as possible within the NAMCS survey items, the analysis
included computerized orders for prescriptions, computerized orders for tests,
reporting of test results and physician notes.
As shown in Table 6, the impact of patient-mix is seen only when analyzing EHR
adoption among providers with EHRs that have a minimal set of functionalities.
There is early evidence of slower diffusion of EHRs among physicians who serve
a greater proportion of low-income patients. Specifically, only about 5 percent of
physicians in practices that receive 20 percent or more of their practice revenue
Health Information Technology in the United States: The Information Base for Progress  4:35

from Medicaid have an EHR with a minimal set of functionalities, compared
to about 10 percent of physicians whose practice revenues are less dependent
on Medicaid. Tracking access to EHR-enhanced medical care among Medicaid
beneficiaries and other low-income patients deserves particular attention in future
research. EHR use was significantly related to other practice characteristics as well,
including the number of physicians in the practice, scope of services as measured
by single- or multi-specialty practices, ownership and number of managed care
contracts (see Table 6). The primary driver of EHR adoption was practice size, with
a clear linear relationship between practice size and EHR use (see Figure 2).

Figure 2: Percent of physicians using electronic medical records
and percent of physicians using electronic medical
record system by practice size: United States, 2005

Percent of physicians

50

46.1

40

33.8

30
20
10
0

25.3

20.8

20.2

16.0

16.5
10.2

4.4
Solo

6.0
Partner

3–5

6–10

11 or more

Number of physicians
General EMR

EMR system

NOTES: Both trends are significant (p<.05). EMR is electronic medical record. General EMR is positive
response to single question on full or partial EMR use. EMR system is a positive response to four minimal
features: computerized orders for prescriptions, computerized orders for tests, test results and physician
notes. Includes nonfederal, office-based physicians who see patients in an office setting. Excludes radiologists,
anesthesiologists and pathologists
SOURCE: National Ambulatory Medical Care Survey.

4:36  Health Information Technology in the United States: The Information Base for Progress

Table 6. Use of electronic medical records by characteristics of office-based physicians: United States, 2005

Physician or practice characteristic
All physiciansc
Age of physician
Under 35 years
35–44 years
45–54 years
55–64 years
65 years and over
Physician specialty typee
Primary care
Surgical
Medical
Physician gender
Male
Female
Practice characteristic
Practice sized
Solo
Partner
3–5
6–10
11 or more
Unknown
Scope of servicesd
Solo and single-specialty
Multi-specialty
Unknown
Practice ownershipd
Physician/physician group
Health maintenance organization (HMO)
Other
Number of managed care contractsd
None
1–2
3–10
More than 10
Unknown
Percentage revenue from Medicaidd
Under 5%
5–19%
20% or more
Unknown

Percent distribution of
all physicians (based on
weighted responses from
1,281 sample physicians)

Percent of physicians
reporting full/partial use
of EMRsa (standard error)

Percent of physicians
reporting minimum set
of required features for
EMRb (standard error)

100.0

23.9 (1.5)

9.3 (1.1)

3.92
27.2
34.5
24.5
9.9

44.0 (8.5)
26.8 (3.0)
25.1 (2.5)
18.1 (2.4)
18.4 (4.2)

15.2 (8.1)
10.4 (2.0)
10.5 (1.8)
7.0 (1.6)
5.7 (2.0)

50.5
21.7
27.8

22.4 (2.3)
22.3 (2.6)
28.1 (2.8)

9.2 (1.6)
8.5 (1.8)
10.3 (2.0)

76.4
23.6

24.1 (1.8)
23.5 (2.7)

9.4 (1.2)
9.2 (2.0)

38.5
11.3
25.4
12.9
9.7
2.2

16.0 (2.2)
20.2 (4.2)
25.3 (2.9)
33.8 (4.7)
46.1 (5.8)
12.0 (7.9)

4.4 (1.3)
6.0 (2.6)
10.2 (1.8)
16.5 (3.2)
20.8 (4.9)
11.2 (7.8)

78.6
20.0
1.4

21.8 (1.5)
34.2 (4.1)
—

7.5 (1.0)
17.1 (3.4)
—

83.3
2.9
13.9

20.3 (1.6)
66.5 (10.1)
37.1 (5.1)

7.3 (1.0)
49.6 (11.5)
13.2 (3.6)

9.7
9.8
35.6
39.3
5.5

15.4 (3.5)
38.2 (5.4)
23.0 (2.2)
23.7 (2.4)
22.0 (6.0)

6.8 (2.6)
22.6 (4.4)
5.4 (1.2)
9.8 (1.7)
11.8 (4.7)

30.9
32.1
25.11
11.81

21.9 (2.4)
25.6 (3.0)
21.5 (2.8)
30.2 (4.3)

11.1 (2.0)
10.1 (1.9)
5.5 (1.6)
10.8 (2.6)

a Electronic medical record (EMR) refers to physicians reporting that their medical records are either fully or partially electronic. Percentages may be underestimates because
electronic medical record use is unknown for 4.5 percent of physicians and are assumed to not use EMR.
b Required minimum features include computerized prescription ordering, computerized test ordering, electronic results and electronic physician clinical notes.
c Includes nonfederal, office-based physicians who see patients in an office setting. Excludes radiologists, anesthesiologists and pathologists.
d Significant relationship between electronic medical record (EMR) use and physician or practice characteristic based on chi-square test ( p<.01)
e Specialty type based on categorization of physician subspecialties obtained from the American Medical Association.
* Figure does not meet standards for reliability and precision.

Health Information Technology in the United States: The Information Base for Progress  4:37

Patient Reports of EHR Use According to Zip Code

Finally, we report on some exploratory analyses conducted on the 2003 and 2004
NAMCS visit file by staff at the NCHS, which reflects patient visits documented
over a one-week period for each physician included in the nationally representative
sample of practicing physicians. Using ARF, EHR adoption was analyzed
according to characteristics of the zip code in which patients lived to determine
whether these area-level characteristics were associated with differential access
to EHR-enhanced health care. This research found no evidence that physicians
in zip codes with a higher proportion of families living in poverty, with higher
concentrations of minority or Hispanic populations, with a higher proportion
of individuals with lower levels of education, or with a higher proportion of
households with limited English language proficiency were less likely to adopt
EHRs. Nor did the analysis find significant associations between EHR use and
patients’ self-reported age, sex, race, ethnicity or source of insurance coverage.67

Looking Ahead: Building Capacity

While these data reveal no dramatic, early indications of major disparities in EHR
adoption by physicians or clinics that are likely to see disproportionate numbers
of underserved populations, preliminary data showing that physicians serving
a greater proportion of Medicaid patients are significantly less likely to have an
EHRs relative to physician practices with fewer Medicaid patients warrants further
study. The critical role of targeted resources to facilitate capital investments
needed to develop a functional EHRs system is also evident in the experience of
community health centers and clinics in California. Overall, existing studies are
far from adequate and further work is needed to identify subsets of providers who
serve vulnerable patients populations, to consistently track the diffusion of HIT
among these providers groups and to assess the ultimate impact on the patients
they serve.
As we move forward, developing the capacity to monitor the impact of EHR
adoption on the quality of care for vulnerable populations and on health
disparities will require additional data collection efforts. There are two broad sets
of strategies that should be employed—those related to sampling frames and those
related to survey content. To a large extent, the existing information gap can be
effectively addressed through sampling strategies that ensure future data collection
efforts include a representative sample of providers who serve vulnerable
populations. Measures of EHR adoption, prevalence of key EHR functionalities,
patterns of use of these functionalities and prevalence of barriers to EHR adoption
can then be compared among providers who serve vulnerable populations versus
other providers. With respect to survey content, there may be a need for additional
survey items or other data collection efforts that relate specifically to providers
who serve vulnerable populations, including funding streams that support EHR
adoption, infrastructure pertinent to a particular provider group, such as CHCs
and barriers and incentives to EHR adoption. The 2006 NAMCS, currently in the
field, includes a subsample of 100 CHCs and will provide excellent baseline data
for measuring adoption among this group.
Developing data that allow EHR use among providers who serve vulnerable
populations to be confidently determined will require complex sampling designs,
including the use of disproportionate sampling techniques. A simple random
sample will not produce a sample sufficient to provide reliable estimates of EHR
adoption among these providers, unless that random sample is prohibitively
large. Before implementing a complex sampling design, a clear, operational

4:38  Health Information Technology in the United States: The Information Base for Progress

definition of the variable or construct to be measured is necessary to drive the
sampling strategy.
As previously mentioned, the ECP has identified racial and ethnic minority
patients and low-income or publicly insured patients as the two highest priority
patient populations. Strategies to identify providers who disproportionately
serve these patients can be developed by using provider self-report of patient
panel composition, linking provider IDs to Medicare claims to empirically
assess the racial and ethnic composition of providers’ elderly patient panel,
or, where available, using discharge or payer-mix data to determine patient
panel characteristics along key dimensions. These data can then be used to
disproportionately sample providers of certain patient groups, adequately
powering future EHR adoption surveys. Analytical variables such as sociodemographic characteristics of physicians, organizational characteristics of their
practices and hospitals and market identifiers can also be linked to databases that
contain market characteristics. This may enable richly textured descriptions of
which providers serving vulnerable patients groups are achieving early and later
stages of adoption.

Identifying Physicians

There are no optimal strategies for identifying physicians who play an important
role in caring for vulnerable populations. One approach is to directly query
providers in EHR adoption surveys. The AMA Physician Masterfile, the best
source of data on physicians practicing in the United States, does not contain
complete data on the race or ethnicity of physicians, case-mix or payer-mix of
physician practices, or demographic profiles of physicians’ patient populations.
These variables could be elicited from physicians by asking them to estimate
the percentage of their patients who come from different racial or ethnic groups
and the payer-mix and insurance status of their patients, allowing them to be
categorized according to rough proportions (e.g., low, medium or high) of
vulnerable patients served. The NAMCS induction interview, for example, asks
physicians to self-report their payer mix—asking them “Roughly, what percent of
your patient care revenue comes from: Medicare, Medicaid, private insurance,
patient payments, or other sources (including charity, research, CHAMPUS, VA).”
Based on these data, physician EHR use could be profiled according to payer-mix
(e.g., high Medicaid volume versus low).
While possible to query physicians about their patient mix and stratify them
according to this information, the reliability of such estimates is questionable,
particularly among small groups or solo practices. Large physician practices,
however, may have information systems in place that enable reliable estimates
of the number or proportion of minority or low-income patients served. Such
information could also be enhanced through audit or validation. Studies that
first validated this metric would ensure that providers did not systematically
overestimate, underestimate or misclassify the proportion of their patients falling
into selected categories. Systematic overestimation or underestimation would
lead to erroneous assessments of EHR adoption rates among these providers
and misclassification (where providers simply estimated incorrectly, sometimes
overestimating and other times underestimating) would severely reduce the power
of the survey to accurately estimate EHR adoption rates.
It may be possible to enhance the usefulness of data collection efforts addressing
patient mix. While the NAMCS induction interview does not collect information
Health Information Technology in the United States: The Information Base for Progress  4:39

on the racial and ethnic composition of physicians’ patient panels, it does gather
empirical information about physicians’ patient mix through their completion of
a Patient Record Form. This part of the NAMCS survey requires physicians and
their office staff to keep a listing of all patients seen over the course of an assigned
week and collects data on Hispanic ethnicity, race of the patients and expected
source of payment. These data may provide more accurate methods for profiling
physicians’ patient panels but are limited in that they are based on a one-week
snapshot of patient cases. To support meaningful estimates, several years of data
would need to be aggregated or the sample for this part of the NAMCS survey
would need to be increased.
More robust approaches that could be used to identify providers with a high
volume or high proportion of minority or publicly insured populations include
linking provider information to claims data, whereby provider IDs are used to
profile physicians according to volume or proportion of patients who fall into a
policy-relevant category (e.g., minority, Medicaid-enrolled). Bach and colleagues,69
for example, analyzed Medicare data on 87,803 primary care physicians and 58
million patient visits (54 million white patients and 4 million black patients)
and found that 22 percent of physicians account for 80 percent of visits by black
patients and 22 percent by white patients; the other 78 percent of physicians see
78 percent of white patients and 20 percent of black patients.69
Although not focused on EHR use, Bach’s work provides a possible method for
using claims data to empirically assess patient mix. With collaboration across
institutions, connecting survey responses to patient data may provide insight
into EHR use at the patient level. But there are two important limitations to this
approach. First, Medicare claims only identify patients who are elderly or disabled
and are eligible for Medicare. Whether providers who care for a high volume or
proportion of elderly minorities also provide care to a high volume or proportion
of non-elderly minorities is not known. It is possible that the racial and ethnic
composition of the elder population in a given service area differs markedly from
the racial and ethnic composition of the non-elderly population in the same
service area. Second, the reliability of Medicare data on Hispanics is problematic.
Medicare data are limited in their designation of Hispanic ethnicity. While more
than 4 percent of America’s elderly (≥ 65 years old) are Hispanic, only about
2.5 percent of Medicare enrollees were listed as Hispanic in the year 2002. The
sensitivity of the Hispanic designation is about 50 percent,70 and thus it does not
identify all Hispanic-focused providers. However, the designation of someone as
Hispanic in the Medicare data is highly specific70 and should ensure that providers
designated as caring for Hispanics truly do care for Hispanic populations.
Identifying physicians and physician practices that disproportionately care for
Medicaid or uninsured patients using claims data would be challenging. In theory,
Medicaid claims could be used in a similar fashion as in Bach’s Medicare study
to empirically assess the distribution of Medicaid patients throughout systems of
care. But Medicaid databases are maintained at the state level and have varying
capabilities for linkage with providers. While states contribute data to the State
Medicaid Research Files (SMRF), federally maintained Medicaid data files that
have standardized data elements across states, these data typically lag by several
years and are less useful for monitoring a timely issue such as EHR adoption.

4:40  Health Information Technology in the United States: The Information Base for Progress

Identifying Hospitals

The American Hospital Association (AHA) annual survey can be used to identify
hospitals that serve a large number of Medicaid patients or for whom Medicaid
patients comprise a large proportion of their total patient revenues. This high
quality survey has a very high response rate and includes data on the proportion
of hospital discharges accounted for by Medicaid patients, as well as on the total
number of Medicaid patients seen in a given year. These two variables can identify
hospitals that care for either a high volume or a high proportion of Medicaid
patients. The Medicare claims approach, described above, could also be used
to identify hospitals that serve a high volume or high proportion of minority
patients. Identifying hospitals that care for the uninsured is more difficult,
however, as these data are not available in the AHA dataset. Several states have
data on the level of uncompensated care provided by each hospital, but these
data are not easily tied to assumptions about the number of uninsured patients
they serve. Thus, it would be challenging to create reliable national estimates of
the proportion of providers who serve a large number or proportion of uninsured
patients using these data.

Conclusion

Identifying providers that disproportionately care for members of racial and
ethnic minority communities or Medicaid enrollees will likely require a multipronged approach. While directly querying providers may be the easiest tactic,
the reliability of such data remains unclear. Linking providers to claims data
might identify providers that disproportionately care for minority patients, but
this method would not readily identify providers who disproportionately care
for Medicaid, uninsured or other vulnerable patient subpopulations. While it
is relatively easy to identify hospitals that disproportionately care for Medicaid
patients using links to other data sources, it will likely be more difficult to identify
ambulatory providers that care for this population. Further work is needed to
develop reliable data that enables policy-makers to monitor the diffusion of EHRs
and HIT more generally, among providers who serve vulnerable populations
compared to other providers. Ultimately, these efforts need to utilize the strength
of provider-level data in combination with patient-level survey and claims data
to form a comprehensive picture of EHR adoption. They also need to assess the
impact of differential rates of adoption among certain providers on quality of care
received, clinical outcomes and health disparities. Developing reliable and timely
methods for tracking the access of vulnerable populations to EHR-enabled care
should be a high priority for policy-makers and researchers at this time.

Health Information Technology in the United States: The Information Base for Progress  4:41

Chapter 5: Incentives and Barriers to HIT Adoption:

Requirements for Policy Relevant Measurement
The ultimate purpose of measuring electronic health record (EHR) adoption rates
is to inform policy development and, by so doing, ensure this potentially vital
technology becomes readily available. Although the analysis of policy barriers to
adoption is not a key goal of this report, we nevertheless think it is useful to lay
out a conceptual framework upon which such policies may be based and against
which they may be evaluated and assessed. Indeed, having such a framework is
essential when designing data collection efforts to inform and evaluate federal
efforts promoting EHR adoption.
Among the tools for addressing barriers to adoption are policy changes that
have the potential to influence HIT adoption. Policy-setting bodies encompass
policy-makers from both the public and private sectors. Policy-making can occur
at the local, state and federal governmental levels; much policy-making occurs
through private bodies such as professional societies, industry trade associations
and accreditation bodies. These entities formulate and apply policies through
a variety of mechanisms: financial; legal and regulatory; technological; and
organizational. It is rare to find policy-making on any subject of significance in
any single venue; for example, accreditation standards, licensure requirements
and conditions of participation in insurance programs are quite frequently
interactive. Furthermore, policy-making is dynamic, seldom involving only a
single intervention, but instead relying on a combination of incentives and
standards to achieve a particular outcome For example, the development of
common technical standards may reduce the cost of EHR adoption, thereby
easing significant barriers to interoperability by increasing the probability that
purchasers will choose the correct software. These standards might be combined
with a provider compensation arrangement that includes incentives for the
purchase of a standard technology.
It is also important to understand that policies typically do not exert a uniform
effect on the groups that are the subject of policy-making. Policies that influence
adoption by small groups may not have the same implications for large academic
medical centers or health care safety net providers. Effective policies should
recognize differing baseline levels of capacity and infrastructure among provider
subsets (e.g. community health centers, rural providers and public hospitals)
and address any special needs for disseminating policies widely and effectively
throughout the provider community.
To clarify the challenges that arise in efforts to evaluate EHR adoption, this
chapter sets out a policy framework to address barriers and incentives for HIT
adoption and briefly reviews what is known about these barriers and incentives.
The chapter also describes policy options for encouraging further HIT adoption.
Then, building on our examination of the policy landscape, the chapter ends
with a discussion of the quantitative and qualitative methods that might be used
to evaluate these policies and their success at overcoming barriers and providing
incentives for HIT adoption.

5:42  Health Information Technology in the United States: The Information Base for Progress

Conceptual Model of HIT Adoption

A complex interplay of barriers and incentives influence HIT adoption.
The framework presented in Figure 3 outlines one possible approach to
conceptualizing these barriers. Based on prior work by Institute for Health Policy
staff and the HIT adoption literature,1, 2, 5-14 this model refers to HIT generally, not
just EHRs. This broader approach is justified by literature on determinants of HIT
adoption, which does not single out EHRs for special consideration. We expect
that the influences on EHR adoption will generally resemble those observed for
other types of HIT.
As shown in Figure 3, four factors affect potential HIT adopters: (1) financial
incentives; (2) legal issues; (3) the state of technology; and (4) organizational
influences. These four factors are embedded in and affected by trends in the U.S.
health care system generally and large socioeconomic forces both within the
United States and globally.

Figure 3. A Conceptual Model of Factors Influencing HIT Adoption
U.S. Socio-economic and Health System Trends

Financial Incentives

Technology

MDs & Hospitals

Legal & Regulatory

Organizational
HIT Adoption & Use

What is Known about Barriers
and Incentives?

Financial Barriers

Financial barriers have a significant influence on HIT adoption. These barriers
can be best understood as “twins:” the high cost of HIT systems; and provider
uncertainty regarding the value they will derive from adoption in the form of
return on investment.2, 14 Stated another way, many providers do not perceive that
there is a business case for HIT acquisition and use. They argue that the absence of
a business case stems from a form of market failure within the HIT sector: current
dysfunctional market dynamics and incentive structures do not work efficiently
and effectively to realize the societal benefits of HIT.
There are several reasons for this market failure. The first is that economic
incentives in the health care industry generally do not reward good performance,
reducing the motivation of self-interested health care actors to acquire HIT and
compete more effectively. Often, health care compensation arrangements reward
poor performance. Inefficient and sub-optimal care, for example, can generate

Health Information Technology in the United States: The Information Base for Progress  5:43

more visits, tests and procedures and thus more revenue for providers. At a
minimum, this reduces incentives for physicians and others to invest in systems
to improve performance. Making matters worse, the purchasers of HIT—mostly
doctors and hospitals—would capture only a small fraction of HIT’s potential
economic benefits. It has been estimated that as much as 80 percent of the
potential savings generated through HIT inure to insurers and health care group
purchasers, including the federal government, in the form of lower premiums and
enhanced worker productivity.
As part of our original project for the Office of the National Coordinator for
Health Information (ONC), we collected information on barriers and incentives
using focus groups composed of hospital administrators and physicians in Chicago,
Boston and Denver. Participants were specifically asked about financial incentives
to adopt EHRs. Those who had adopted the technology noted that the EHR
simplified billing procedures and supported faster reimbursement cycles. However,
any gains in efficiency were offset by reduced productivity as the technology
was implemented, as well as the need to increase information technology staff
to maintain the system. Overall, physicians in the focus groups did not see any
financial incentives for adopting an EHR. Physicians and hospitals who had
not adopted the technology reported financial disincentives, such as a decrease
in productivity during the first phase of adoption and the cost of maintaining a
system. Many believed that financial incentives alone would not be enough to spur
adoption, and other issues such as standardization and interoperability would need
to be resolved before widespread adoption would occur.
Legal Barriers

The health care system operates within a complex legal environment whose
standards and requirements can both speed and impede HIT adoption. Mandates
imposed through regulations, accreditation standards, or as the result of judicial
liability rulings linked to the failure to incorporate new technologies into practice
can push the health care industry to adopt new treatment modalities and patient
safety techniques.15 But until health care leaders recognize that the legal benefits of
HIT adoption outweigh the risks, the industry’s response may remain slow—even
in the face of legal incentives that aim to spur adoption. The lack of response
may be more notable in situations in which the business case for adoption
of technology is weak, as dynamic interactions between legal and business
considerations affect all industries. This is particularly true for industries where
costs are high and pressure to avoid unnecessary expenditures is great.
The numerous legal issues that arise in the context of HIT fall into three categories:
■

Concern about newly created legal exposures. Health care providers may be concerned
that decisions conducive to HIT adoption may violate certain legal standards. For
example, a hospital’s decision to provide staff physicians with the systems necessary
to prescribe electronically, in order to maintain and improve patients’ quality of
care, may expose both the institution and its physicians to potential liability under
federal and state fraud laws. To diminish these concerns, the Centers for Medicare
and Medicaid Services (CMS) and the United States Department of Health and
Human Services Office of the Inspector General (OIG) in 2005 issued regulations
establishing legal “exceptions” and “safe harbors” for market conduct related to the
use of financial incentives to spur adoption of e-prescribing and electronic health
records technology.16 Determining the extent of the protections afforded under
these exceptions and safe harbors, once interpreted and applied by the medical
community and legal counsel, may influence HIT adoption rates.

5:44  Health Information Technology in the United States: The Information Base for Progress

■

■

Concern over the actual or perceived legal burden of compliance. The use of certain
technologies is associated with an actual or perceived increase in legal burdens.
For example, a health professional or organization may be unwilling to adopt HIT
systems in the belief that issues associated with compliance stemming from laws
such as the Health Insurance Portability and Accountability Act of 1996 (HIPAA)
would add to the burden and cost of practice. Non-electronic personal health
information is protected by extensive federal, state and common law concepts of
privacy and confidentiality; indeed, the legal duty to protect the confidentiality of
patient medical records is a basic aspect of health care providers’ legal obligations
toward their patients. Yet because of the attention given to electronic records, many
health professionals may mistakenly believe that they are insulated from possible
legal ramifications by remaining a “paper” practice.
Concerns regarding actual or perceived legal exposures associated with the disclosure
of information. HIT expands the accessibility of health information for
governmental agencies and private parties, as well as for health professionals
and health care institutions using the technology. Compelling information
disclosures is not unique to electronic information but, as a practical matter,
electronic information can be transmitted and reviewed more easily than
information stored in paper files. Considerable attention has been focused
on HIT’s role in detecting health care fraud, substandard care or patterns of
care and services that may violate applicable laws. While the management and
integrity of health care payment systems are essential, health professionals
are concerned about heightened exposure to legal scrutiny due to software
systems designed to detect and measure the validity of claims on a “real time”
basis. Similarly, civil liability litigation arising from allegations that involve
legal violations (e.g., medical malpractice lawsuits alleging discriminatory care,
where there is no privilege against having to produce certain information) may
result in court-ordered production and disclosure of information ranging from
medical errors to discriminatory conduct against persons with disabilities. To
the extent that electronic technology makes the meaning of a medical record
ambiguous, the scope of discovery could extend beyond the limits now imposed
in paper medical record cases. Possible policy solutions to these barriers include
immunity against liability under certain circumstances,ii as well as privileges
against disclosure. The law also could establish safe harbors against liability
if certain forms of prescribed conduct are followed.iii However, the privilege,
immunity, or safe harbor created in these cases is likely to be narrow and apply
only to select circumstances; it will not be a blanket protection against all
forms of legal liability following HIT adoption. Currently, privilege rules that
protect against disclosure typically are drafted narrowly to limit their reach and
immunity from suit is extremely rare. The extent to which electronic health
information will produce new sources of liability or demonstrative burden on
health care providers remains to be seen.

 See, e.g., Doe v Medlantic Health Care Group, Inc. 814 A. 2d 939 (D.C.Ct. App., 2003). See generally,
ii For example, the Health Care Quality Improvement Act, 42 U.S.C. § immunizes health care peer review procedures against most forms of civil liability.
iii For example, federal guidance implementing language access standards governing health care providers considered federally assisted entities under Title VI of the 1964 Civil
Rights Act also provide for a finding of compliance if providers can demonstrate adherence to standards set forth in the guidance.

Health Information Technology in the United States: The Information Base for Progress  5:45

Technology Related Barriers

As is the case with many other technological advances, HIT is in a constant state
of evolution. Findings from our focus groups suggest that concerns about ease of
use and obsolescence were second only to financial barriers as a primary reason
for not adopting this new technology. Participants reported that current systems
are unwieldy and difficult to use. Those who had not adopted expressed concerns
about investing in a system, only to have it become obsolete through mergers on
the supplier side or technology upgrades.
Aspects of technology that affect ease and value of use can facilitate or impede
adoption. Reliable, transparent and simple hardware and software lowers the cost of
adoption, maintenance and use while simultaneously increasing the likelihood that
providers will purchase and employ them. Widespread adoption of sophisticated
handheld devices and the development of technologies and systems for making
interoperability simple, straightforward and reliable will increase the attractiveness
of HIT among physicians and hospitals. Finally, federal efforts currently under
way to promote standardization and certification should accelerate adoption. An
example of such efforts is the Certification Commission for Health Information
Technology (CCHIT) charged with developing criteria and evaluation processes for
certifying EHRs and interoperability components. These criteria include the ability
of EHRs to protect health information, standards by which EHRs must share health
information and clinical features that improve patient outcomes.
As previously noted, these technological changes operate to some extent
through financial mechanisms, reducing the effective cost of a given level of
technology or increasing its quality and thus its benefits. But this does not lessen
the psychological barriers to adoption. As we found in our focus groups, some
physicians in solo and small practices, for example, fear being stuck with a failing
EHR system, no access to a paper record and a full waiting room. Overcoming
this anxiety through technological improvements and consensus around the best
practices for HIT adoption would hasten implementation.
Organizational Barriers

Organizational factors, attributes of the organizations in which care is provided,
also influence HIT adoption. Existing evidence, for example, suggests that large
physicians practices are more likely to adopt HIT than small practices.6, 9 This
finding also holds true for hospital adoption of computerized physician order entry
(CPOE) and likely is a reflection of their greater capital and human resources.5
Payer-mix, including the proportion of uninsured or Medicaid patients, directly
affects financial resources and thus providers’ ability to acquire HIT. Health care
safety net providers often labor under strained financial circumstances owing
to their uncompensated care missions, their disproportionate dependence on
Medicaid and their comparatively limited ability to generate other sources of
patient revenues. Research is needed to further define the barriers to HIT adoption
among safety net providers and the best way to ensure that they do not lag behind.
Other organizational factors affecting HIT acquisition include the size of provider
organizations and whether they are part of an integrated care system. For lack of a
better term, we call this latter attribute the provider’s “system attributes.” Solo or
small groups of physicians and small hospitals are less likely to have the resources
required to adopt HIT. However, even small providers have better adoption levels
when they are part of systems of care. Those systems provide financial support,
technical assistance and legal protection to small physician groups.
5:46  Health Information Technology in the United States: The Information Base for Progress

Finally, many studies, including our focus group research, have shown that
organizational leadership—or lack thereof—has a major effect on HIT adoption.
Organizations such as Intermountain Health System in Salt Lake City, Utah, are
pioneers in HIT acquisition because their leaders decided it was the right thing to
do. Surveys of hospital executives suggest that hospitals with CPOE frequently
were driven to adopt HIT by leaders who placed the organization’s mission above
its financial considerations. Although this is not a realistic approach for all health
care organizations, clearly the values, vision and capacity of organizational leaders
influence decisions about whether to invest substantial resources in HIT systems.

Policies that Affect HIT Adoption

The barriers discussed above serve as a guide to potential policies for spurring HIT
adoption. To be effective, these policies must address the critical barriers currently
faced by providers. In Table 7, we list possible policy options related to each of the
major barriers identified in Figure 3.

Health Information Technology in the United States: The Information Base for Progress  5:47

Table 7. Policies Influencing HIT Adoption and Applicable Market Sectors
Domain
Financial incentives

Barrier
Lack of a business case for
performance

Policy
■

■

Lack of a business case for HIT

■

■
■

■
■

■

Legal/ Regulatory

Fraud and abuse related to HIT
adoption, including “Stark Law”

Grants to providers, including AHRQ implementation grants
Loans to providers to cover the costs of acquisition, training
and/or maintenance
In kind assistance
Performance standards/certification (reduces the risk of wasting
funds on substandard equipment/software)
Interoperability standards (reduces risk of lost investment due
to poor choice of IT solution)

■

Clarification

■

Greater liability protection

Lack of interoperability

■

Standards for interoperability

Lack of interconnectedness

■

Support for regional health information organizations (RHIOs)

Lack of accountability for quality

■

Internal reporting requirements

Lack of workforce skills

■

Workforce training/certification

Leadership

■

Training of health care leaders

System attributes

■

Size

■

Assistance to small providers

■

Assistance to safety net providers

Organizational Surplus/Capital
Availability

Financial Incentives

Pay for use of HIT: practices and hospital that adopt EHRs
would receive a higher reimbursement rate from third-party
payers

Modifications and exemptions

Liability exposure from more
and more accessible, health
information

Organizational

Public reporting of performance: information on the level of
EHR adoption among providers and hospitals would be made
publicly available

■

Privacy and security obligations
under HIPPA electronic health
information standards

State of Technology

Pay for performance: practices and hospitals that adopt EHRs
to improve quality would receive a higher reimbursement rate
from third-party payers

Providing incentives for practices to form networks and
negotiate contracts

Although policies that offer financial incentives for HIT adoption take several
forms, all aim to reduce HIT’s costs and increase the return to providers on their
technology investments. Changing payment policies to reward good health care
performance, for example, creates a non-specific incentive for providers to improve
their quality and reduce costs of care. If HIT delivers on its promise, this should lead
many providers to invest in information technology to improve their performance.
Other policies focus on reducing the costs of HIT acquisition or increasing
revenues associated with the system’s use, such as: reimbursing providers for
using HIT in clinical care, providing grants or low-interest loans to providers

5:48  Health Information Technology in the United States: The Information Base for Progress

who purchase HIT systems, providing equipment or software free of charge and
reducing uncertainty about software performance. This last intervention involves
certification and standardization of HIT products. It assures buyers that they
will acquire products to meet their needs and expectations, thus lowering HIT’s
effective price (or improving its effective quality). The Wired for Health Care
Quality Act of 2005 (Senate Bill 1418), which passed the Senate unanimously
on November 18, 2005, included a number of these policy options. Policies
addressing the unique infrastructure and financing challenges faced by safety net
providers will likely be necessary to facilitate widespread HIT acquisition and
use—and consequently its benefits—among this group of institutions. Safety net
providers disproportionately serve poor, minority, publicly insured and uninsured
patients. They lack the margins necessary to allow large capital investments
without support from grants, loans or augmented reimbursement policies. Thus,
HIT adoption initiatives may have the unintended consequence of widening
the health disparities gap in the U.S. health care system unless targeted policies
addressing the needs of these provider groups are developed.

Technological Policies

There are a variety of approaches that could be used to reduce technological
obstacles to HIT adoption. CCHIT is pursuing one important option: providing
the equivalent of a “Good Housekeeping” seal of approval to particular HIT
applications by certifying them as compliant with certain federal or other
standards. This approval assures potential purchasers that the products in question
are state of the art and provide the essential functionalities required of HIT
applications. Another approach, also part of HHS’ current portfolio of activities,
is to create standards that, once incorporated into HIT systems, will enable them
to communicate or inter-operate effectively. Finally, government and private
sector organizations at multiple levels are trying to stimulate the creation of local
health care stakeholder networks that use HIT to develop local mechanisms for
health information exchange. The networks, called regional health information
organizations (RHIOs), aim to help providers overcome technical barriers, thus
enabling them to communicate about common patients.

Organizational Policies

It is difficult to develop policies that directly affect how providers of care organize
themselves, even when it is clear that larger and more integrated groups of
providers may facilitate HIT adoption. Policies that explicitly favor one type of
organizational form over another are likely to be fiercely opposed by doctors and
hospitals disadvantaged by such interventions. In some cases and settings, such
as rural areas, the promotion of larger groups of doctors and the development of
integrated systems may be inherently more difficult than in other areas.
Nevertheless, pertinent options exist. Many observers believe that forcing
providers to be more transparent about their performance, through public
reporting of efficiency and quality data, may stimulate doctors and hospitals
to join larger groups and integrate into systems. This is because small and
disorganized providers will have more trouble meeting these requirements.
Another approach is to provide centralized training opportunities for doctors
and hospitals that lack the resources to do such training themselves. Finally, for
organizations that face systematic barriers to HIT adoption, such as safety net
facilities, direct subsidies may be required.

Health Information Technology in the United States: The Information Base for Progress  5:49

Critical, Policy-Relevant Questions The next major consideration in evaluating EHR interventions is: What do

policy-makers want to know about the interventions? While we can only speculate
about policy-maker’s information needs as they seek to spur the adoption of this
technology, several critical questions are likely to arise.
These questions come in two basic forms. The first asks: Did the policy work?
That is, did it affect the adoption or use of EHRs and, if so, how much? The
second asks: Why or/how did it work or not work? While the first question addresses
outcome, the second question addresses the process or mechanism of change—
critical information for improving policy development in the future. Tables 8 and
9 summarize other questions that are likely to arise in these two areas.
Table 8: Questions Related to Policy Intervention Outcomes
1. Did the policy affect
adoption of EHRs in
physician offices and
in hospitals as
indicated by:

■

The number of physicians with access to EHRs in their office practice?

■

The number of physicians that use EHRs in their office practice?

■

The number/type of EHR functionalities used by physicians in their office practice?

■

The number/type of office-based decisions in which computerized decision support plays a role?

■

The number of hospitals with access to EHRs?

■

The number/proportion of physician staff at these hospitals that use EHRs for inpatient care?

■

The number/type of EHR functionalities used by physicians for inpatient care?

■

2. Did the policy
affect the level of
interoperability in the
health care system as
indicated by:

■

■

■

3. Did the policy affect
the cost and/or quality
of care in inpatient or
office-based settings
as indicated by:

The number/proportion of physicians’ inpatient decisions in which computerized decision
support plays a role?
The number of physicians who report that their practices’ EHRs communicate with EHRs at
the hospitals where they admit patients?
The number of physicians reporting that their practices’ EHRs communicate with: other
physicians to which they refer, pharmacies, imaging facilities, laboratories, home care
services and insurance companies?
The number of hospitals reporting that their EHRs communicate with: referring physicians,
pharmacies, independent imaging facilities and laboratories, other hospitals and insurance
companies?

■

The number of patients who communicate electronically with health care providers?

■

Physicians’ perceptions of cost and quality of care?

■

Nurses’ perceptions of cost and quality of care?

■

Patients’ perceptions of cost and quality of care?

■

Objective indicators of cost and quality of care as demonstrated by:
●

●

Administrative data obtained from payers such as Medicare and private insurers?
Quality measures reported to CMS, Joint Commission on Accreditation of Healthcare
Organizations (JCAHO) and state data repositories on various procedures and rates of
safety problems?

5:50  Health Information Technology in the United States: The Information Base for Progress

Table 9: Questions Related to Mechanisms/Processes of Policy Effect
1. Did the policy affect providers’ decision to acquire an EHR?
If so, how?

■

By reducing the cost of the product?

■

By improving the reliability of the product?

■

By increasing availability of technical support?

If not, why?
2. Did the policy affect the EHR purchase process?
If so, how?

■

By reducing the price of the product?

■

By increasing the understandability of the product?

■

By making comparative data on the performance of alternative products available?

■

By providing technical support?

If not, why?
3. Did the policy affect providers’ use of an existing EHR?
If so, how?

■

By improving reliability of the product?

■

By increasing understandability of the product?

■

By providing technical support?

■

By increasing financial rewards for product use?

■

By increasing financial rewards for meeting quality/cost targets?

If not, why?

In thinking about questions related to process of change, it is useful to consider the
mechanisms by which policies affect the behavior of doctors and hospitals towards
EHRs. As noted above, policy interventions typically affect EHR adoption by
working through a specific type of influence and/or reducing a particular barrier to
acquisition and use. Figure 4 makes the manner in which policies affect barriers and
influences more explicit and thereby influence the process of adoption. By breaking
down the process of adoption, it shows that barriers and influences may exert their
effect at several points in the acquisition process. Thus, this model lays out the “life
cycle” of EHR adoption and the points at which different policies come to bear
on it. While the adoption process is not always linear and sequential, this model
approximates the steps providers take from contemplating HIT adoption to its
effective use and integration in clinical care.

Health Information Technology in the United States: The Information Base for Progress  5:51

Figure 4. Sequence of HIT Adoption
Contemplation
Barriers/
Influences
Decision
Barriers/
Influences
Purchase

Policy interventions
Barriers/
Influences

Effective Use
Barriers/
Influences
Higher quality,
Lower costs

Assessing the Impact of
Policy Changes on HIT Adoption

Methods for evaluating policy interventions must be customized to the
experimental context, the intervention and the questions under study. While it is
not feasible to provide a detailed description of methods for approaching every
possible question and policy intervention related to HIT adoption, certain generic
considerations can be summarized. In the next section, we describe the types of
data that are likely to be useful and alternative methodologies for collecting data.
Then, using two examples, we illustrate how these can be brought to bear for
evaluating policy interventions.
Generic Considerations

The first generic consideration is the recognition that policy interventions of
interest will rarely be the subject of well-designed, controlled studies. Evaluations
typically are natural experiments, with control groups constructed post-hoc or
completely lacking. Thus, these evaluations rely heavily on statistical methods to
control for confounding variables.
Second, the natural experiments take place in complex and changing environments
where many potential influences on EHR adoption are operating at the same
time. It is likely that multiple policy interventions will occur simultaneously. This
complexity reduces evaluators’ ability to arrive at clear and convincing conclusions
about the effects of any one policy.
Third, methodologies are likely to vary with the unit of analysis and questions of
interest. ONC, for example, has expressed an interest in measuring dissemination
and use of EHRs among physicians in solo, small and large groups and among
hospitals. Each of these providers constitutes a different unit of analysis, which may
vary depending on the perspective taken to evaluate their adoption of EHRs. Levels
of adoption and use among specific providers, for example, can be approached
from a national or local perspective (e.g., regions, states or local markets) or among
policy-relevant groups of providers (e.g., safety net providers or rural providers),
depending on the question at hand. The desired outcome also needs to be specified;
investigating policy effects on levels of use (an outcome variable) will differ
significantly from understanding mechanisms of policy effect.
5:52  Health Information Technology in the United States: The Information Base for Progress

Data Necessary to Evaluate Policy Interventions

Primary or secondary data collected de novo by evaluators or derived from preexisting data sources and quantitative or qualitative data can be used for evaluating
policy interventions. Primary, quantitative data can be used to evaluate EHR
policy interventions and include the following:
1. Surveys of relevant adopter populations (physicians, hospitals administrators,
pharmacists, home health workers, etc.);
2. Surveys of patients who are affected by or witness the use of EHRs, which
is relevant for understanding the extent to which EHRs facilitate patient
communication with physicians and the use of EHRs to build personal
health records;
3. Chart reviews to assess the quality of care provided with and without EHRs;
4. Direct observation of the care process in health care institutions/physician
offices with and without EHRs (including time motion studies, error
documentation and communication measures);
5. Focus groups of providers and other relevant populations;
6. Case studies of particular provider institutions or markets; and
7. One-on-one interviews with providers and other relevant individuals.
Similarly, a variety of secondary data also may be useful for evaluating policy
interventions. These include:
1. Administrative data from public and private third parties;
2. Survey data collected by public and private vendors, including:
a. National Ambulatory Medical Care Survey (NAMCS)
b. National Hospital Ambulatory Medical Care Survey (NHAMCS)
c. National Health Interview Survey (NHIS)
d. The Medical Expenditure Panel Survey (MEPS)
e. U.S. Census data
f. Data collected by provider associations: American Medical Association
(AMA), American Hospital Association (AHA) and Medical Group
Management Association (MGMA)
g. Data collected by private foundations: Robert Wood Johnson
Foundation, Commonwealth Fund and Kaiser Family Foundation
(among the most interesting data sources on EHR adoption among
physicians are surveys by Stephen M. Shortell and colleagues at the
University of California, Berkeley and the Center for the Study of Health
System Change—both of which were supported by the Robert Wood
Johnson Foundation)
3. Reports on quality of care from CMS, JCAHO and state governments; and
4. Other standard government data sets including the Area Resources File (ARF).
Health Information Technology in the United States: The Information Base for Progress  5:53

Surveys are likely to be a key data collection methodology, and can be conducted
cross-sectionally or longitudinally depending on the questions to be addressed.
Longitudinal surveys, which can use both cross-sectional and time-dependent
controls, are most useful for before- and after-study designs. Cohorts also can
be followed over time, in combination with multiple cross-sectional surveys.
Alternatively, a multiple cross-sectional design can be used. Which approach is
used depends on the resources available, questions of interest, timing and nature
of interventions, among other considerations.
Administrative data and quality reports made to public and private authorities
are useful when they can be linked to institutions or physicians variably affected
by policy interventions. In such cases, an intervention’s effect on HIT adoption
can be associated with the ultimate outcomes of interest: cost and quality of
care. However, establishing links between data on EHR use and administrative
and quality data may be difficult as there are no publicly available data on EHR
acquisition and use that permit identification of specific providers. Without
knowing their identities, and obtaining their permission, such linkages might come
only with increased public reporting (through Leapfrog and other initiatives).
Qualitative approaches to data collection can provide texture and validation to
quantitative findings, filling in cases where quantitative findings are hampered by a
lack of available data. Focus groups and interviews with groups affected by policies
promoting EHR adoption may generate new hypotheses and help to confirm or
refute these hypotheses.
Study Design

Several methodological approaches can be used to evaluate the policy
interventions of interest. Given the complexity of natural experiments, the ideal
study designs are likely to be multi-method or use different methodologies to
collect several types of data.
Study design involves the following critical steps:
1. Choice of policy intervention to be studied
2. Choice of questions of interest
3. Choice of unit of analysis
4. Identification of experimental situation
5. Characterization of the policy intervention and its timing
6. Choice of key dependent and independent variables
7. Identification of possible data sources
8. Design of data collection methods
9. Design of analytic methods
The policy intervention, questions of interest, and unit of analysis will, to some
degree, be defined by policy-makers. In an ideal experimental situation, possible
control groups include unaffected institutions or practices within the same
geographic area or in a different geographic area (for example, different states
or regions), or groups that are affected at different points in time. This way the
5:54  Health Information Technology in the United States: The Information Base for Progress

behavior of cohorts experiencing the intervention early can be compared to those
that have not yet experienced it (but will later on). The affected population can
also serve as the control group, if baseline data is available from the same cohort
or a comparable group.
Once experimental and control groups (if any) are identified, the next step is to
characterize the intervention and its timing: what precisely happened and when?
This can be challenging as policy interventions often do not occur at a precise
moment in time. Instead, they are telegraphed ahead of time, leading providers
to change their behavior in anticipation of a new policy. They also are phased in
gradually, triggering change over an extended period.
Choosing the key dependent and independent variables is critical to translating the
policies and questions of interest into measurable variables that accurately indicate
behaviors of interest and a policy’s influence on these behaviors. Confounding
variables need to be controlled for in the analysis, and understanding them is
critical to choosing the independent variables. This is especially true in cases where
the control groups are not comparable to experimental groups. Descriptive and
multivariate approaches that are designed to control for confounding variables
will be used in the analysis. Advanced statistical methods, such has hierarchical
modeling and instrumental variables, may be necessary to account for nested data
sources and a lack of adequate controls.
Measuring the policy-relevant effects of efforts to influence HIT adoption is an
extremely challenging task. However, measurement will be enhanced by a sound
understanding of the influences on HIT adoption, the barriers to its spread, and
the basic principles of research design discussed above.

Health Information Technology in the United States: The Information Base for Progress  5:55

Chapter 6: Survey Guidelines: Improving What We Know About EHR Adoption
This chapter recommends general methodological guidelines for use by the Office
of the National Coordinator for Health Information Technology (ONC) and
by other agencies and organizations seeking to apply principles of best survey
practice to the measurement of electronic health record (EHR) adoption in the
United States. It is not intended to be a comprehensive guide to designing and
conducting surveys or to provide detailed and comprehensive standards for federal
surveys. This information is available from other sources, such as the recently
released version of the Office of Management and Budget (OMB) guidance on
the conduct of surveys for the federal government,1-3 which summarizes issues in
survey development, design, testing, conduct and analysis.
The measurement of EHR adoption is a complex and occasionally vexing
methodological problem, and high quality surveys are needed to improve the data
available to inform policy-making. As stakeholders’ priorities for the cost, quality
and timeliness of quantitative data about EHR adoption vary, we have presented
guidelines for data collection of the highest quality while acknowledging that
practical issues may lead some stakeholders to choose different survey methods.

Approaching Methodological
Guidelines

Several investigative and analytic approaches were considered for measuring the
adoption of EHRs, each of which has strengths and weaknesses. Not all of these
approaches involved surveys. The methods we assessed included:
■

analyzing measures in existing surveys using meta-analytic techniques;

■

evaluating non-survey data from vendors, certification processes or registries;

■

expanding existing surveys to include new or improved measures; and

■

designing and conducting surveys to meet specified objectives.

The project team chose to focus on improving survey methods and measurement,
which is at the center of three of these four approaches.
An expert panel was convened to evaluate the use of meta-analysis or summary
descriptive analysis to estimate current levels of EHR adoption. However, as
discussed in Chapter 3, prior surveys were found to vary widely with respect to
the functionalities they measure, respondents they target, and clinical settings
they examine—as well as the quality of their methodology. This heterogeneity
led the Expert Consensus Panel (ECP) to conclude that meta-analysis is not
an appropriate approach, as it may not be possible to obtain reliable and valid
estimates of critical parameters by combining data from existing sources.
As further input into the guideline development process, we commissioned RTI
International, an independent scientific research and technology development
institute, to conduct focus groups and case study interviews. Hospital and group
practice health care providers, health information technology professionals,
and key trade association and e-health collaborative leaders were included in
focus groups and interviews in three market areas: Denver, Colo., Chicago, Ill.,
and Framingham/Natick, Mass. This research gave us insight into the best way
to measure EHR adoption rates and learn about the factors that positively or
negatively influence adoption. These findings are discussed in detail in Chapter 5.

6:56  Health Information Technology in the United States: The Information Base for Progress

Survey Design

Specify Objectives

Developing annual national estimates (and standard errors for those estimates)
of EHR adoption for three populations—solo and small physician groups, large
physician group practices and hospitals—is essential to informing policies that aim
to increase HIT adoption. As part of this process, it is critical to detect variations
in adoption rates among health care providers in different geographic locations
and specialties, and among those serving populations of diverse race, ethnicity,
insurance status, and socioeconomic status. Further, the ECP recommended that
margin of error estimates be approximately the same (+/- 3 percent) for data
collected on the experience of vulnerable populations and for data collected on
the population as a whole.
Specify Populations and Respondents of Interest

Determining the appropriate unit or units of analysis is essential to valid, reliable
measurement. Because our research focused on measuring adoption in three
groups—solo and small physician practices, large groups and hospitals—our
recommendations largely pertain to measuring adoption among these providers.
In future reports, we are likely to extend our recommendations to other groups,
including consumers. While it is relatively straightforward to define the population
of interest for individual physicians and hospitals, physician groups and ambulatory
practices are highly variable in organizational structure and are not licensed or
accredited in a way that facilitates this process. This variability makes identifying
and locating respondents for physician groups somewhat more problematic.
Once the populations of interest are identified and defined, the next challenge is
to select the appropriate respondent within each population. Individual physicians
are likely to be the preferred respondent for solo physician practices and small
groups. The respondent (or respondents) most able to provide valid and reliable
information about the adoption of EHRs and their component functionalities
within larger physician group practices and hospitals must be identified, and
could include a medical or nursing director, practice administrator, technology
officer, chief executive, or financial officer. As respondents’ perspectives—and
their reliability—are likely to vary, it may be desirable to gather data from multiple
respondents within an organization. Survey procedures should allow flexibility
to identify the most knowledgeable respondents, and survey pre-testing and
development should focus on the best way to get valid and reliable estimates.
Although our current focus is on a selected group of providers, an important and
likely future population of interest is patients or individual health care consumers.
The ultimate goal of EHRs is to improve the quality and effectiveness of patient
care. Thus, research designs that collect patient data eventually will provide a
valuable perspective on EHR use. Data collection from patients and consumers
also should help policy-makers anticipate the effect of EHR adoption on diverse
patient populations.

Survey and Sampling Design

Many factors shape the selection of survey and sample design, including the target
population, available sample frame, key measures, desired response rate, mode of
data collection, budget, and time until data are needed.
Survey Design

Surveys can be cross-sectional, conducted at one point in time, or longitudinal,
allowing for the same questions to be asked at two or more points in time. Many
Health Information Technology in the United States: The Information Base for Progress  6:57

EHR adoption studies are one-time, cross-sectional surveys. While valuable for
measuring use at a point in time, they do not provide repeated, comparable
measures of EHR adoption.
Longitudinal designs—trend, cohort and panel studies—are intended to collect
repeat measures. Repeat uses of the same survey over time can provide trend
data, insofar as the samples are randomly selected and comparable in terms of
their demographic characteristics. Cohort studies can be used to measure changes
in specific groups (people who belong to a given profession, organization or
location) over time. They are used when the aggregate group characteristics are of
interest, as the individuals surveyed might change. Panel studies enroll a sample of
respondents and ask the same question(s) to the same individuals or organizations
over time. These studies have become increasingly expensive, due to the challenge
of maintaining panels and response rates, and trend or cohort designs are likely to
be the favored approach.
Sampling

Sampling design describes the procedures used to select a survey’s sample
population, which consists of the individuals or organizations selected randomly
from a sample frame to represent the target populations of interest. Ideally, the
sample frame is the same as the target population (e.g., if the target population is
all U.S. physicians, the ideal sample frame is a complete and accurate listing of
all physicians). In practice, however, there are often differences between sample
frames and target populations, resulting in both insufficient sample coverage and
excess sample coverage. Sample coverage, a term used to describe the extent to
which the sampling frame resembles the target population, is complete when the
sampling frame is identical to the target population.
There are several major types of probability samples—simple random, systematic
random, stratified (proportionate and disproportionate), and area or cluster
probability—and multiple sampling stages are often required to identify the
respondent(s) of interest. For example, surveys of a target population might begin
by identifying geographical areas to conduct survey fieldwork and, in further stages,
hospitals selected from these geographical areas, followed by physicians from those
hospitals, and then randomly selected patients from the physician’s practice. At
each stage, there is a selection procedure and an associated probability of selection.

Disproportionate Sampling

There are simple and complex approaches to sampling. For example, in the case
of sampling physicians nationally, approaches range from selecting a random
sample from a list of all U.S. physicians and collecting data from those providers,
to selecting physicians and physician practices within an area probability sample
of the U.S. population and collecting data on randomly selected patient visits
to those physicians (as done by the National Ambulatory Medical Care Survey
(NAMCS)).4 Complex sampling designs are often necessary to answer policy
questions about subsets of providers. In our case, we want to know whether
providers serving vulnerable populations are developing HIT capacity at the same
rate as other providers and, if not, what impact this is having on quality of care
and health disparities.
We identified racial and ethnic minority patients and low-income or publicly
insured patients as high priority patient populations. A simple random sample
is not sufficient to provide reliable estimates of HIT adoption among those

6:58  Health Information Technology in the United States: The Information Base for Progress

providing care to these populations. As discussed in greater detail in Chapter
4, provider subgroups can be identified through self-report of patient panel
composition, linking provider IDs to Medicare claims to assess the racial and
ethnic composition of elderly patient panels, and using discharge data or payermix data to determine patient panel characteristics along key dimensions.
Disproportionate sampling based on these data should adequately power HIT
adoption estimates for these subgroups.
Special attention must be given to understanding design effects and survey weights
that adjust estimates by the probability of subject selection, as disproportionate
sampling impacts estimation and standard errors. Complex sampling designs
that rely on public use data, and data being interpreted in organizations where
appropriate methodological staff or consultant help is not available, may pose
challenges to getting accurate estimates and standard error calculations.

Sampling Frames

The sampling frames that have the best sample coverage and quality for drawing
probability samples of solo physicians and small groups, large physician groups,
and hospitals are listed below. In all surveys, it is important to understand sample
coverage and its determination; the source of the data; the age of the data; the
proportion of incomplete, missing, or inaccurate data elements; and the methods
of updating and validating data elements.
Solo and Small Group Physicians

The preferred source of data on physicians in solo practice and small groups is the
American Medical Association (AMA) Masterfile. The AMA Masterfile includes
current and historical data on physicians residing in the United States who
have met the educational and credentialing requirements to practice medicine.
Records include the physician’s name, medical school and year of graduation,
gender, birthplace, and birth date, as well as residency training, state licensure,
board certification, geographical location and address, type of practice, present
employment, and practice specialty. There is one caveat: the time between a
physician’s change in practice status and its notation in the Masterfile record
can be substantial. Also, it does not contain complete data on physician race
or ethnicity, practice case-mix, or payer-mix, or on the patient population’s
demographic profile.5, 6
Large Physician Group Practices

The Medical Group Management Association currently has the most
comprehensive information on medical groups composed of three or more
physicians in the United States, with a database that includes about 35,000 groups.
It was created by a merger of MGMA’s database of members, past members, and
customers, with the FIRSTMARK commercial database, AMA medical group data
file, and Veterans Health Administration medical group file. A current MGMA
effort to identify physician practices in Colorado that serve vulnerable populations
could suggest novel strategies for identifying such groups and also inform
sampling frame development to ensure their representation in national samples.
There’s also one caveat with respect to the MGMA database: the individual,
merged databases used to create it contain errors. MGMA has taken steps improve
the reliability of the database.7 However, in light of problems with the MGMA
database, the best strategy for understanding EHR adoption in physician group
practices may be to sample physicians and then elicit information about group
Health Information Technology in the United States: The Information Base for Progress  6:59

size and organization in order to assure adequate data on groups. This would
enable surveys to generate conclusions about EHR adoption by physicians
practicing in groups. If desired, this strategy could be expanded to sample the
groups identified by physicians and to survey relevant respondents in those
groups. Such a strategy would produce data that represented adoption of EHRs
by groups in which physicians practice in the United States. Though not ideal,
this approach is recommended in part due to concerns about the accuracy and
currency of the available lists of group practices and the lack of a definitive source
for all medical groups in the United States. Short of this revised approach, the
MGMA database is the preferred database.
Inpatient Hospitals

The American Hospital Association (AHA) maintains a database of more than
6,000 U.S. hospitals with information on approximately 700 characteristics,
including bed size, location, teaching status, and region. Although declining
response rates have affected the database’s quality, it is updated annually for
85 percent of hospitals nationwide and considered the best available source of
hospital sample data in the United States. AHA data is also linked to Centers for
Medicare and Medicaid Services (CMS) and financial data.2 This, in combination
with geographical data, might enable sample selection to target hospitals that
serve a disproportionate share of vulnerable patients.8 The AHA database is the
preferred sampling frame for acute care hospitals in the U.S.

Sample Size, Survey Weights,
and Modes and Methods

Sample Size

The desired precision of estimates for key analytic variables and the desired
analytic power for statistical comparisons should drive sample size calculations.
Working with sampling and survey methodology experts, these estimates should
take into account the specific measures of interest, estimates of the population
proportion reporting on this measure, information on anticipated survey response
rates, and effects of complex sample designs. The margin of sampling error, which
will vary both with the size of the total sample, or sample subgroup, as well as the
response frequency for any analytic variable, can be measured and controlled in
probability sample surveys. Surveys measuring EHR adoption should have sample
sizes sufficient to provide estimates with margins of error no greater than +/- 3
percent, as defined by the ECP.
Survey Weights

Probability samples might under- or over-represent some elements of the
population. Thus, weights should be assigned to each respondent case and used
to adjust the overall results to more closely conform to the known characteristics
of the total population. (Failing to weight data can result in biased or imprecise
estimates and should be avoided.)
Modes and Methods in Data Collection

The recent OMB advisory summarizes the advantages and disadvantages of
different data collection modes, including personal interviews, telephone (voice,
interactive voice response, or facsimile), mail, and Web-based surveys. Factors
that should be considered when choosing a sample mode include: availability of
contact information, cost of contacting respondents, time to administer the survey,
length of the questionnaire, speed with which data are needed, respondent burden
to perform survey tasks, complexity of the survey instrument, question type (open
6:60  Health Information Technology in the United States: The Information Base for Progress

or closed ended and need for visual aids), question sensitivity, need for computerassisted interviewing, desired response rate, possibility of interviewer bias, and need
to control survey completion by respondent or other proxy or substitute.
Facsimile numbers and e-mail addresses are not widely available for physician,
medical group, and hospital samples. As a result, reliance on these modes will
have a negative impact on sample coverage and increase selection bias. Although
researchers may be tempted to collect data for information technology surveys
online, initial contacts should be made in a mode common to all potential
subjects, and the survey should allow for multiple modes of response. Web
administration might not be an option for some respondents, and surveys that rely
on this approach may result in biased data as those with Web access may be more
likely to have EHRs in their practice.
Physician surveys initiated by mail, with an honorarium check enclosed, have
the highest response rates, according to recent studies.9 Group practice and
hospital data quality and response rate rely on reaching the most knowledgeable
respondent—specific physicians, practice administrators, executives, and
information technology professionals. Finally, survey response rates have declined
in recent years, making repeated attempts to contact eligible respondents and the
ability to conduct surveys in multiple modes necessary.

Practice and Market

To understand how adoption rates may vary across provider groups, it is essential
to collect information about those providers that will allow detection of such
variation and its potential causes.
One of our policy concerns is to learn the ways in which EHRs affect the quality
of care and health outcomes of vulnerable populations including the poor,
uninsured, racial and ethnic minorities, and Medicaid beneficiaries. Accurate
measurement of practice and market characteristics nationally will help researchers
to examine differential adoption rates among providers who serve vulnerable
populations relative to other providers. These analyses can highlight the need for
policy development in specific areas and motivate the development and adoption
of relevant interventions.
Physician Characteristics

The NAMCS survey includes data on payer-mix, provider demographics
(including age, gender, ethnicity and race), year of graduation from medical school,
specialty, number and type of locations where the physician sees patients, number
of office visits per week, group membership states, and number of providers in
the physician’s medical group. NAMCS also collects extensive information about
patients at the visit level including, race, ethnicity, expected source of payment,
reason for visit, diagnosis, and services provided. These physician and practice
level characteristics would allow for the analysis of EHR adoption by subgroups.
Analyses of NAMCS data will assist in determining if different subgroups of the
provider population are adopting the technology at differential rates.4
As discussed in Chapter 4, the ECP has recommended that racial and ethnic
minorities and low-income patient populations be the highest priority groups
with respect to tracking access to EHRs and their potential implications for
health disparities. This suggests that the following subdomains are essential to
capture important information about a physician’s practice environment and

Health Information Technology in the United States: The Information Base for Progress  6:61

could be added to the NAMCS survey instrument to supplement the data that is
currently collected:
■

urban/rural status (MSA/non-MSA) and

■

socio-economic status of the local population.

Concerns about survey length and respondent burden might lead to the use of
other analytic techniques to capture data on market characteristics. Geocoding,
for example, can link respondents’ zip codes to demographic information.
Medical Group Practices

The MGMA survey also collects information on the type of practice, number of
full-time equivalent physicians, number of full-time equivalent non-physician
providers, and majority owner of the practice. In order to fully understand the
diffusion of EHRs among physician group practices, the ECP suggested adding
the following sub-domains:
■

payer mix;

■

zip code;

■

years of operation;

■

form of physician compensation;

■

membership in a network;

■

rural/urban location of practice;

■

socio-economic profile of the local market;

■

level of hospital/medical competition in the market area; and

■

ethnic/racial composition of the patient panel.

As with physician characteristics, these items could be added to the MGMA
survey, although researchers must be mindful of the survey length. Geocoding
could again be used to capture information about a practice’s geographic area.
Hospital Characteristics

There are no ongoing hospital surveys that examine EHR use in the inpatient
environment. As discussed above in relation to solo physicians and group
practices, understanding the characteristics of hospitals that do and do not adopt
EHRs will further our understanding of the adoption process and the diffusion
of the technology among different hospital subgroups. In order to ensure an
adequate understanding of these subgroups, the following sub-domains could be
included in a hospital survey:
■
■

■
■

■

racial and ethnic composition of the patient population;
hospital characteristics: size, payer-mix, location, membership in multihospital system;
financial status, public vs. private;
market characteristics: rural/urban, SES profile of the market area, level of
hospital/medical competition; and
safety net vs. non-safety net.

6:62  Health Information Technology in the United States: The Information Base for Progress

The annual AHA hospital survey provides a useful model for the type of
information that can be collected from hospitals. While the AHA survey does not
include measures of EHR adoption, barriers, or incentives, it does collect extensive
data on hospital financing, beds, utilization, and staffing. Creating new survey
content to capture EHR adoption data that could be linked to existing AHA data
might be an efficient way to collect the necessary information without burdening
respondents with a lengthy list of hospital characteristics.

Fielding the Survey

Survey errors, including both non-response and response errors, can threaten
researchers’ ability to obtain valid and reliable estimates and to accurately
calculate their precision. Non-response errors occur when only some of those
sampled respond to a survey. In order to measure their impact on data quality, it
is important to understand differences in survey response between those sampled
and those who complete the survey. (AMA and AHA samples describe respondent
characteristics; other sample sources make it difficult to evaluate non-response
error). Response errors include problems with question wording, questionnaire
design flaws, item non-response, interviewer effects, respondent selection, and
mode of interview effects. These errors can be minimized by carefully developing
and testing survey instruments, but response errors that occur during the field
period can also have impacts.
Survey errors arise in part from the logistical challenges of administration—training
interviewers and research staff, constructing quality checks at each stage, and
maximizing response rates. To overcome these challenges and minimize errors,
using a professional survey research organization with established fieldwork
procedures is desirable. Achieving acceptable response rates in busy professional
and organizational settings can require months of data collection, and efforts to
maximize cooperation are critical to survey quality. Many organizations faced
with the time and cost demands of careful data collection choose to compromise
their methods, but this decision results in biased estimates where the bias is not
measurable or even detectable.
Statistical Analysis and Reporting

The use of appropriate, valid statistical methods and consultation is essential
for analyzing survey data about EHR adoption, and these methods should be
reported along with the results of such surveys. Analyses should take into account
the sample size of the relevant subgroup, and the adequacy of power and sampling
design needed to support those analyses. Data analyses should include design or
response weights appropriate to the survey and sampling design; estimates should
include standard errors. All analytic reports should include disclosure information
in accordance with the guidelines presented in this chapter.

Confidentiality and Disclosure

Survey researchers should adhere to the full code of professional ethics and
practices for survey research, including protecting the confidentiality of
respondents and disclosing the full details about the survey’s methodology.
Maintaining confidentiality is essential to ensure that respondents are willing to
participate in future survey research. Transparency of methods will allow end users
of the data to have confidence in the validity and reliability of the results.

Health Information Technology in the United States: The Information Base for Progress  6:63

Confidentiality

An essential element of best survey practice is protecting research subject
confidentiality. Data should be reported in the aggregate and great care should be
taken to limit information about respondent identifiers to those with a need to
know, and Institutional Review Boards (IRBs) should review procedures to ensure
compliance with human subject protection guidelines. The following steps should
be taken in order to ensure respondent confidentiality:
■
■

■

data should be reported in the aggregate form;
respondent identifiers should be removed from any data that is publicly
available; and
researchers should consider limiting the data that is publicly available when it
may be possible for analysts to determine a given respondent’s identity based
on demographic information (for example, when there is only one hospital in a
particular zip code with a given demographic profile).

Disclosure

Surveys reported in the public domain should make methodology information
available for review, including survey questionnaires or other relevant sections of
the survey. Researchers and organizations also must be prepared to report response
rate information in accordance with professional standards and to conduct quality
checks at every phase. Meta-analysis will require incorporating survey data that
rates the quality of existing surveys. The American Association of Public Opinion
Research (APPOR) recommends disclosure of the following elements:
■

who sponsored the survey, and who conducted it;

■

exact wording of questions asked, including explanatory text;

■

■

■

■

■

■

a definition of the population under study, and a description of the sampling
frame used to identify this population;
a description of the sample design that indicates the method used to select
respondents;
sample sizes and, where appropriate, eligibility criteria, screening procedures,
and response rates computed according to AAPOR standard definitions;
a discussion of the findings’ precision, including estimates of sampling error,
and a description of any weighting or estimating procedures used;
which results are based on parts of the sample, rather than on the total sample,
and the size of such parts; and
method, location and dates of data collection.

Modifying existing national surveys As discussed in previous chapters, both the NAMCS and MGMA survey contain

useful items on EHR adoption. However, based on our research, we believe that
new content—either for these surveys or for new surveys—must be developed to
accurately measure adoption among all provider groups.
As discussed in Chapter 2, the NAMCS contains a list of EHR functionalities.
However, the survey does not allow for a full understanding of EHR adoption
as the “use” measures lack precision (see Chapter 2 for a full discussion of this
issue). Moreover, the survey does not include items that adequately cover EHR
acquisition and installation, and barriers and incentives to adoption; also, the
validity of the patient race and ethnicity data has not been assessed.
6:64  Health Information Technology in the United States: The Information Base for Progress

Similarly, the MGMA survey does not include items on all of the domains
necessary to measure adoption. In addition, as discussed earlier in this chapter,
the MGMA sampling frame has significant drawbacks that may cast doubt on the
reliability of any findings. Thus, it may be necessary to supplement the MGMA
sample in order to ensure that it is representative of physician groups nationwide.
In both surveys, respondents may either be physicians or practice managers. For
solo or small groups, physicians are the appropriate respondents. However, for
large groups, both individual physicians and practice managers have limitations
as respondents. The surveys assume that respondent physicians will be completely
familiar with all of the functions of their EHR system. However, unless they have
received significant training or are highly motivated to learn the system, physicians
are not likely to be aware of functions that they do not use but may be available in
an organization’s EHR. Practice managers and CIOs may be a more appropriate
respondent for questions that focus on EHR capabilities, and researchers should
carefully track who the respondent is within each practice. However, these
respondents may not know what functionalities physicians actually employ.
When developing any new survey content, survey researchers should be mindful
of the differences, not only between respondents within a care setting, but also
differences across care settings. At this time, the AHA’s annual survey does not
include measures of EHR adoption. As NHAMCS only focuses on hospital
emergency and ambulatory care department, there will likely be a need to develop
survey content specific to the inpatient setting. Adoption issues may differ
significantly between inpatient and outpatient care settings, and surveys need
to reflect the need for separate content. Also, it is important that hospital EHR
adoption surveys focus at both the departmental and the hospital-wide level.

Next Steps and Summary

Our focus has been on the best approaches to generating annual national measures
of EHR adoption by physicians and hospitals in a timely way. As meta-analysis
of existing data from multiple surveys was not found to be a viable approach, our
attention has turned to the expansion of existing surveys to include new content
or samples and the development of entirely new surveys.
Discussions about expanding surveys and conducting new probability sample
surveys of physicians, physician group practices, and hospitals raised several
critical issues. In consultation with the ECP and in accordance with best survey
practices, we have concluded the following:
■

■

■

Defining units of analysis and appropriate respondents is likely to be a
challenge. Our core interest is in whether electronic health records have been
adopted—acquired, installed and used—in the patient-provider encounter.
Although our current objective is to develop measures of adoption among
physicians and hospitals, patient-level visit data is expected to be of value too.
Surveys of physicians and hospitals are best conducted using samples drawn
from AMA and AHA databases.
Group practice and hospital EHR adoption surveys may have to be
approached in more than one phase, with surveys of a first respondent to elicit
organizational information followed by a second phase of surveys to gather
data from knowledgeable respondents. It is also likely that critical information
will need to be elicited from more than one knowledgeable respondent within
a practice or organization. Necessary adjustments for probabilities of selection

Health Information Technology in the United States: The Information Base for Progress  6:65

can be made between phase one and two to allow additional stratification
or oversampling for organizational size, understanding of vulnerable patient
populations, or other factors. Finally, survey measure and method testing will be
key to obtaining valid and reliable EHR adoption estimates.
■

■

■

■

■

■

EHR acquisition, installation, and use, and key functionalities will be difficult
to assess solely through the use of existing surveys. Thus it is likely that new
content and new data collection efforts will be necessary.
NAMCS could serve as a useful framework for additional data collection efforts
on physician practices and groups, both through content or sample expansions.
An annual survey, it provides both physician and physician practice level
data and is linked to patient-encounter information, thus meeting many of
our defined objectives. NAMCS already includes content on EHR adoption,
and a preliminary meeting with National Center for Health Statistics (NCHS)
representatives was held April 26, 2006, to discuss the opportunities for
collaboration.
While it might be possible to add new content to the 2007 NAMCS survey,
major content or method changes are unlikely until 2008, given the need for
testing and development, OMB clearance, and the time to publish preliminary
and final estimates.
Further analyses will be necessary to determine whether current NAMCS
data is sufficient to understand access to EHRs in practices serving vulnerable
populations and whether, working in collaboration with NCHS, needed
estimates will be available in a suitable time frame for the ONC and ECP.
AHA’s annual survey has a high response rate and is currently the best source of
hospital data. Although the survey is already extensive, preliminary discussions
suggest that AHA is willing to include a limited number of new data elements to
estimate hospital EHR adoption. More extensive content would likely need to
go in stand-alone surveys of a similar population.
Adding content to existing surveys raises many challenges that need to be
addressed in order to ensure the timeliness of collaborative estimates and their
validity and reliability as specified by the ECP, especially for providers serving
vulnerable populations. Our content guidelines, for example, established
specific data collection objectives, many of which would not be achievable
given the cost and respondent burdens of existing questionnaires. Current
surveys serve many important public purposes other than ascertaining rates
of EHR adoption, and incorporating additional content could threaten those
purposes. Further, it is unlikely that existing national surveys will have the
space or scope to explore broader questions, such as the incentives and barriers
to adoption and other policy-relevant issues. Thus, it is possible that new
surveys that collect and disseminate policy-making data and meet ECP and
ONC objectives might still be necessary.

6:66  Health Information Technology in the United States: The Information Base for Progress

Chapter 7: Recommendations for Future Data Collection
In the prior chapter, we discuss general recommendations for improving what
is known about EHR adoption through surveys of physicians, group practices
and hospitals. This chapter focuses on specific recommendations for improving
existing, ongoing national surveys, and for new survey efforts where these are
required to meet the needs of policy-makers concerned with EHR adoption. In
addition, we make further recommendations for surveys of providers and hospitals
serving vulnerable populations. Last, we lay out a research agenda for studying
both the effect of EHRs on the provision of care and the use of EHRs to capture
quality data efficiently.
Our recommendations have several central goals. The first is to ensure that
policymakers and managers have the information they need to maximize the
speed and effectiveness with which EHRs become available to U.S. health care
providers. The second goal of our recommendations is to make certain that the
information developed, and the policies and decisions undertaken, recognize and
address inequities that may arise as EHRs diffuse through our health care system.
If the past is a prelude, the risk that the spread of vital new health information
technologies, such as EHRs, will sustain or even increase disparities in health and
health care is substantial. The advent of health information technology creates an
opportunity to avoid perpetuating the history of unequal treatment for vulnerable
groups of U.S. citizens. To seize this opportunity requires a careful but relentless
search for data documenting the existence of disparities and suggesting policies to
overcome them.
1. Improving estimates of EHR adoption among physicians using existing,
ongoing national surveys
1.a. The National Ambulatory Medical Care Survey (NAMCS)
NAMCS, an annual survey of patient visits to physicians in the United States,
is a high quality national survey that meets many critical needs for information
on health care services in the United States. It offers the potential to use a welldesigned federal survey to measure EHR adoption. Moreover, its patient-level
data could provide some information about the affect of EHR availability on
patterns of care among sampled physicians. However, the survey’s sample size is
currently not adequate to track adoption on an annual basis within the accuracy
levels specified by the Expert Consensus Panel (ECP). In the past, most NAMCS
EHR adoption estimates were made by combining several years of survey data,
which somewhat reduced the survey’s ability to meet the need for annual estimates
of EHR adoption rates. However, in 2005, researchers at the National Center for
Health Statistics (NCHS) made significant efforts to increase NAMCS’ response
rate and effectively increased their sample. However, it is too early to assess
whether the sample size is adequate to meet the ECP’s standards for accuracy in
examining differential patterns of adoption among providers disproportionately
serving vulnerable populations.
Should the federal government provide additional funding, there are several
possible scenarios for expanding NAMCS to track EHR adoption among
physicians and meet the central goals outlined above.

Health Information Technology in the United States: The Information Base for Progress  7:67

1.a.1. Supplement the NAMCS sample to ensure an adequate number of providers serving
vulnerable populations are surveyed
The NAMCS sample could be supplemented so that it includes both a national,
random sample of physicians (as it now does) and an adequate oversample of
physicians serving vulnerable populations. According to ECP recommendations,
surveys should be designed and sufficiently powered to detect variations of
approximately +/- 3 percent in adoption rates among health care providers who
serve vulnerable populations, the same margin of error that the ECP recommends
for estimates of adoption rates among all health care providers. In the case of
NAMCS, the sample could supplement its current, nationally representative
sample of 1,200 physicians with an additional oversample of 1,000 physicians
serving vulnerable populations. This would yield a margin of error of +/- 3
percentage points for the national sample and approximately +/- 3 percentage
points for the subgroup of providers serving vulnerable populations.
However, there are two drawbacks to supplementing the NAMCS sample. First,
supplementing the sample without additional content will not meet several
data collection needs. NAMCS contains only a limited number of questions
about EHR functionalities. It does not contain any questions on other domains
of interest: rates of EHR acquisition and installation (as opposed to use), and
incentives and barriers to adoption. Second, the physician induction interview,
which contains questions about EHR use, is fielded throughout the year. Thus,
survey data is generally not available until the middle of the following data
collection year—limiting its usefulness for making timely, annual estimates of EHR
adoption. We should note, however, that NAMCS is working hard to make data
available in a more timely fashion.
1.a.2. Create a new survey module for EHR adoption that could be added to the current
NAMCS survey
This option offers several of the same advantages as the prior one, including the
NAMCS survey’s high response rate and an inherent link to patient-level data. In
addition, creating a new survey module would allow researchers to gain a much
fuller understanding of EHR adoption by allowing them to ask multiple questions
in each survey domain. This data could be combined with NAMCS patient-level
data to determine if subgroups of patients have differential access to EHRs.
However, given the limited sample size, researchers would not be able to conduct
additional analyses on providers serving vulnerable populations. Moreover, the
timeliness issue would also apply to this option and may be exacerbated by adding
a large amount of new content to the survey that would have to be approved
by the Office of Management and Budget (OMB), a process generally lasting
nine months. This would significantly delay data the first time the new module
was fielded and have serious implications for timely annual estimates of EHR
adoption in the short term.
1.a.3. Supplement the NAMCS sample and create a separate survey module on EHR adoption
Supplementing the sample by including an oversample of physicians serving
vulnerable populations and creating a new module focused on EHR adoption
would give researchers a full understanding of adoption among the general provider
population as well as provider subgroups. While costly, this new survey effort
would build on an established, high quality survey and thus give researchers a high

7:68  Health Information Technology in the United States: The Information Base for Progress

level of confidence in the data’s quality. As with the other options, data timeliness
would remain an issue as modifications of this magnitude require OMB approval.
As discussed above, using the NAMCS survey offers several advantages, among
them its high response rate. However, the timeliness of the data remains a
significant drawback. The ECP has recommended yearly data collection. In
order to be useful to policy-makers, data on EHR adoption, a rapidly evolving
phenomenon, should be collected and disseminated as quickly as possible without
jeopardizing its quality. This need may preclude the use of NAMCS, at least in the
first and second years of the data collection effort. A major modification of the
NAMCS survey, such as those discussed in the first and second options, would
require OMB submission in January 2007, and generally takes nine months to
receive clearance. The survey would then be fielded in January 2008, with publicly
available data released in the spring of 2009.
2. Improving estimates of EHR adoption among physician group practices
As discussed in the prior chapter, there currently is no reliable sample of physician
group practices in the United States. In light of this limitation, the preferred way
to obtain data on physician group practices would be to start with a national
random sample of physicians or build off an existing physician survey and use this
sample to create a sample of groups. Researchers could design a survey module
for practice managers that includes questions on size of practice, region, multi or
single specialty, multi- or single- site location, and market integration. In addition,
practice managers could be surveyed about any assistance their practice might
need in order to adopt EHRs.
One possible design would be to obtain the names of practice heads. These could
then be matched against the AMA database and other commercially available
lists, such as the MGMA database, that include the names of practice managers.
For practices missing this information, researchers could follow up the physician
survey by telephone to obtain practice manager names and confirm addresses.
With weighting based on the number of physicians in the group, this design would
allow researchers to generalize to physicians that practice in groups of a particular
size and with specific characteristics.
It is possible to use NAMCS to generate estimates based on group practices.
However, the sample size is not adequate to obtain estimates for a given year with
desired accuracy, and all NAMCS group practice estimates must be combined
over several years of data. To obtain an adequate sample size, the NAMCS sample
would have to be supplemented.
3. Improving estimates of EHR adoption among hospitals
3.a. National Hospital Ambulatory Medical Care Survey (NHAMCS)
NHAMCS is a nationally representative survey of hospital emergency and
ambulatory care departments that provides high quality data on a range of vital
health care matters. Should the federal government wish to provide additional
funding for NHAMCS, there are several options for supplementing it to further
our understanding of EHR adoption among hospitals. The current NHAMCS
design only includes hospital outpatient departments and emergency rooms.
Any changes to the NHAMCS design should expand the sample to include the
inpatient environment, as EHR adoption issues may vary between inpatient and
outpatient settings.
Health Information Technology in the United States: The Information Base for Progress  7:69

3.a.1. Supplement the NHAMCS sample to ensure an adequate number of providers
serving vulnerable populations are surveyed
Similar to the options for expanding NAMCS, the NHAMCS sample could
be supplemented to ensure that it includes an adequate number of hospitals
disproportionately serving vulnerable populations. The oversample should be
sufficient to allow for estimates of hospitals serving vulnerable populations with
the same or similar margin of error as estimates based on the national sample.
Building on NHAMCS offers several advantages. It would allow researchers to have
a high level of confidence in the reliability of the data, as NHAMCS questions
have been cognitively tested, the sampling method is rigorous, and the response
rate is excellent. However, as with the NAMCS, the timeliness of the data—which
would only be available to the public in the spring of 2009—is an issue. In addition,
interviewing in the inpatient setting would require a major modification of this
federal survey. It is likely that these changes would require lengthy development
work to identify correct respondents and obtain OMB approval, a process that could
add approximately one year to the project timeline, with data available in 2010.
3.a.2. Create a new survey module for inpatient hospital EHR adoption that could be
added to the current NHAMCS survey
The current NHAMCS survey contains a limited number of survey items related
to EHR adoption. A new survey module would allow researchers to gain a fuller
understanding of EHR use in the inpatient setting, as it could ask about a number
of items in each of the survey domains. This data could be combined with patient
level NHAMCS data to understand differences in EHR adoption among different
patient groups.
However, NHAMCS’ sample size may not be adequate for comparisons between
hospitals that disproportionately serve vulnerable populations and those that do
not. (NHAMCS estimates are generally made using data combined over several
years to increase their precision).
3.b. The American Hospital Association (AHA) Annual Survey
The AHA conducts an ongoing, annual survey of inpatient hospitals. The survey
currently does not include any questions on EHR adoption, but it provides a
very useful potential platform for administering such questions. Adding a limited
set of items to this survey may be possible and would offer the advantages of a
high response rate survey. However, it is unlikely that the AHA would consider
adding more than a very small number of items. The survey currently serves
many important purposes for the nation’s hospitals and policy-makers, providing
information about staffing, utilization and other critical matters. Adding a
substantial module about EHRs would greatly add to the respondent burden and
could jeopardize the traditional purposes of the survey.
3.c. Conducting a new hospital survey in cooperation with the AHA
Creating a new hospital survey in cooperation with the AHA may be the best way
to fully understand EHR adoption in the inpatient environment. The survey could
include a nationally representative sample of hospitals, with an oversample of
facilities disproportionately serving vulnerable populations. It may be possible to
build on other existing work supported by the Robert Wood Johnson Foundation
to identify these hospitals.
7:70  Health Information Technology in the United States: The Information Base for Progress

The survey would require a considerable amount of development work to:
■

identify hospitals serving vulnerable populations;

■

identify the correct respondents;

■

develop effective questions; and

■

assure an adequate response rate.

As discussed in the prior chapter, this design may need to include multiple
respondents within each hospital in order for researchers to understand the process
of adoption, as well as the functionalities that are actually used by health care
providers in the hospital.
Creating a new survey instrument in cooperation with the AHA offers several
advantages. First, partnering with the AHA may help to improve the survey’s
response rate. While the AHA annual survey has a response rate in the 80 percent
to 85 percent range, the majority of other hospital surveys only achieve response
rates in the 30 percent range. In addition, creating a new survey would ensure
that the data needs of policy-makers and managers are met, as it would be able
to include questions in all domains of interest. In addition, creating a new survey
may produce data more quickly than either adding to the AHA annual survey or
modifying the NHAMCS survey.
Summary Recommendations
■

■

■

■

■

■

■

Building on NAMCS and NHAMCS to study the rate of EHR adoption
offers the possibility of using high quality surveys with excellent response rates.
However, the timeliness of these data may limit somewhat the value of relying
on these existing, ongoing national surveys, particularly in the first and second
years of the monitoring effort.
NHAMCS would require significant design changes in order to survey the
inpatient environment.
Policy-makers should explore the possibility of conducting independent surveys
of physicians and group practices in order to produce more timely data.
Researchers surveying physicians and physician group practices could field
their own data collection efforts and, at the same time, work with the National
Center for Health Statistics to supplement the NAMCS sample and create
additional survey modules. Ideally, the two data collection efforts would use
identical survey modules to allow for trending the data over time.
New survey efforts should have sample sizes that are sufficient to detect
variations within subgroups at approximately +/- 3 percentage points.
New surveys of physician group practices should start with a national random
sample of physicians, or build off an existing physician survey, and use this
sample to create a sample of groups. Researchers could design a survey module
for practice managers that includes questions on practice size, region, multi or
single specialty, multi- or single-site location and market integration.
Researchers designing new hospital survey efforts should consider partnering
with the AHA.

Health Information Technology in the United States: The Information Base for Progress  7:71

4. Improving estimates of EHR adoption among providers serving
vulnerable populations
As discussed in the preceding chapters, the HIT health disparities workgroup
identified a range of policy-relevant patient subgroups (e.g., defined by racial,
ethnic, socioeconomic, geographical, insurance states, and English proficiency)
who stand to benefit from information on the use of EHRs to direct their medical
care. The ECP has identified racial and ethnic minority patients and patients who
have low-incomes or are publicly insured as high priority patient populations.
Strategies to identify providers who disproportionately serve these patients could
be developed in a number of ways. Physicians could be asked to report on the
composition of their patient panels, for example. However, the reliability of these
estimates is unknown. Surveys relying on self report should conduct an audit on
a randomly selected subsample of respondents to verify the reliability of their
estimates. Linking provider identification to Medicare claims data would provide
more reliable data than self-report for empirically assessing the racial and ethnic
composition of providers’ patient panels. The primary drawback of this approach
is that it would provide data on the elderly population only. Investigators relying
on Medicare claims data should conduct supplementary research to determine
whether the distribution of elderly patients from racial and ethnic subgroups
among health care providers is similar to that of non-elderly populations.
The two approaches discussed above could also be used to determine the racial
and ethnic characteristics of a hospital’s patient population. In addition, discharge
data could be used to the extent that individual hospitals accurately collect and
record patient racial/ethnic data. Surveyors choosing to rely on discharge data
should conduct an audit on a randomly selected sub-sample of hospitals in order
to verify the data’s reliability. Payer-mix also could be determined using self-report
for individual providers and group and discharge data for hospitals. Researchers
using self report on payer-mix will need to determine the correct respondent for
these questions at the level of the individual provider, practice and hospital.

Recommendations

Identifying providers that disproportionately care for vulnerable populations will
likely require a multi-pronged approach.
■

■

■

■

Directly querying providers may be the easiest choice; however, the reliability of
such data is unclear.
Linking providers to Medicare claims data would provide reliable information
on the elderly population but not the non-elderly.
Payer-mix would be a good choice for hospitals; however, it may be of limited
usefulness for individual providers.
In order to clearly understand if differential rates of EHR adoption are
contributing to disparities in health care, further research is needed to reliably
identify providers serving vulnerable populations.

7:72  Health Information Technology in the United States: The Information Base for Progress

Conclusions

If, as many predict, the advent of HIT generally, and EHRs in particular,
constitutes a revolutionary change in the organization of our health care system
and the practice of medicine, tracking the adoption of these technologies and
understanding their impact on the health care system are vital to effective policy
development. Interested stakeholders have a valuable foundation to build on
in this regard, consisting of public and private surveys of providers conducted
by various agencies and groups. These existing data collection efforts should be
supported and continued. But to provide the information that policy-makers
need and, especially, to avoid recapitulating past inequities associated with the
introduction of new technologies, new data collection initiatives will be required.
In future reports, we hope to be able to summarize what has been learned from a
new wave of investigations that will keep stakeholders fully informed of what the
HIT revolution means for our changing health care system.

Health Information Technology in the United States: The Information Base for Progress  7:73

References

Chapter 2

1. Jha AK, Ferris T, Donelan K, et al. How common are Electronic Health
Records in the US? A summary of the evidence. Health Affairs. in press.
2. ISO/TC. ISO/TC 215 Technical Report: Electronic Health record definition,
scope, and context (2nd draft). August 2003.
3. American Hospital Association. Annual Hospital Survey. 2005.
4. Institute of Medicine. Key Capabilities of an Electronic Health Record system: Letter
report. Washington, D.C. 2003.
5. Burt C, Sisk J. Which physicians and practices are using electronic medical
records? Survey data show limited use of these information tools. Health
Affairs. 2005;24(5):1334–1343.
6. National Center for Health Statistics. National Ambulatory Care Survey.
Available at: http://www.cdc.gov/nchs/data/ahcd/namcs1-2006.pdf. Accessed
20 September, 2006.
7. National Center for Health Statistics. National Ambulatory Health Care
Data. Description. Available at: http://www.cdc.gov/nchs/about/major/ahcd/
nhamcsds.htm. Accessed 20 September, 2006.
8. Simon JS, Rundall T, Shortell SM. Drivers of electronic medical record
adoption among medical groups. Journal of Quality and Patient Safety.
2005;31(11):631–639.
9. The Commonwealth Fund. 2003 National Survey of Physicians and Quality
of Care. Available at: http://www.cmwf.org/usr_doc/2003_nationalsurvey_
physicians&qualitycare.pdf. Accessed 20 September, 2006.
10. Certificate Commission for Health Information Technology. The Official site
for CCHIT. Available at: http://www.cchit.org/. Accessed 20 September, 2006.
11. Office of the National Coordinator for Health Information Technology.
American Health Information Community. Available at: http://www.hhs.gov/
healthit/ahic.html. Accessed 20 September, 2006.
12. Loomis GA, Ries JS, Saywell RMJ, Thakker NR. If electronic medical records
are so great, why aren’t family physicians using them? The Journal of Family
Practice. 2002;51(7):636–641.
13. Center for Studying Health Systems Change. Community Tracking Study
2000–01 Physician Survey Public Use File: User’s Guide. Vol No. 47.
Washington, D.C.; Sept 2003.

74  Health Information Technology in the United States: The Information Base for Progress

Chapter 3

1. National Center for Health Statistics. National Ambulatory Care Survey.
Available at: http://www.cdc.gov/nchs/data/ahcd/namcs1-2006.pdf. Accessed
20 September, 2006.
2. Burt CW, Sisk JE. Which physicians and practices are using Electronic
Medical Records. Health Affairs. 2005; 24(5):1334–1343.
3. Audet AJ, Doty MM, Shamasdin J, Schoenbaum SC. Physicians’ views
on quality of care: Findings from the Commonwealth Fund National Survey of
Physicians and Quality of Care. pub. no. 823 2005.
4. Reed MC, Grossman JM. Limited Information Technology for Patient Care in
Physician Offices. Issue Brief from Center for Studying Health System Change. 2004;89.
5. Gans DN, Kralewski JE, Hammons T, Dowd B. Medical groups’ adoption
of electronic health records and information systems. Health Affairs. 2005;
24(5):1323–1333.
6. Andrews JE, Pearce KA, Sydney C, Ireson C, Love M. Current state of
information technology use in a US primary care practice-based research
network. Informatics in Primary Care. 2004; 12:11–18.
7. American Academy of Family Physicians. AAFP 2005 EHR Survey. Available
at: http://www.centerforhit.org/PreBuilt/chit_2005ehrsurvey.pdf#search=%22
american%20academy%20of%20family%20physicians%2C%202005%2C%20
survey%22.
8. Terry K. Exclusive survey: Doctors and EHRs. Medical Economics; 2005:
Available at: http://www.memag.com/memag/article/articleDetail.jsp?id=143
144&pageID=143141&sk=&date=. Accessed 14 June, 2005.
9. Audet AM, Doty M, Peugh J, Shamasdin J., Zapert K, Schoenbaum S.
Information technologies: When will they make it into physicians’ black bags?
Medscape General Medicine; 2004.
10. National Center for Health Statistics. Ambulatory Health Care Data. Centers
for Disease Control and Prevention. Available at: http://www.cdc.gov/nchs/
about/major/ahcd/ahcd1.htm. Accessed April, 2006.
11. Laschober M, Maxfield M, Lee M, Kovac M, Potter F, Felt-Lisk S. Hospital
Responses to Public Reporting of Quality Data to CMS: 2005 Survey of Hospitals.
Washington DC 2005.
12. Ash JS, Gorman PN, Seshadri V, Hersh WR. Computerized Physician Order
Entry in U.S. Hospitals: Results of a 2002 Survey. Journal of American Medical
Informatics Association. 2004; 11(2):95–99.
13. American Hospital Association. Hospitals & Health Networks’ 2005 Most
Wired Survey and Benchmarking Study.
14. Cutler DM, Feldman NE, Horwitz JR. U.S. adoption of computerized
physician order entry systems. Health Affairs. 2005; 24(6):1654–1663.
15. Institute of Medicine. America’s Health Care Safety Net: Intact but Endangered.
Washington, D.C.: National Academy Press; 2000.
16. Blueprint Research & Design. Wiring California’s health care safety net: How
community health centers are using information technology to improve their
services. Paper presented at: Grantmakers in Health Conference, 2003.

Health Information Technology in the United States: The Information Base for Progress  75

Chapter 4

1. U.S. Department of Health and Human Services. Healthy People 2010. Available
at: http://www.healthypeople.gov/Publications/. Accessed July 6, 2006.
2. Institute of Medicine. Crossing the quality chasm: A new health system for the
21st century. Washington, DC 2001.
3. Institute of Medicine. Unequal treatment: confronting racial and ethnic disparities in
health care. Washington, DC: The National Academies Press; 2003.
4. Office of the National Coordinator for Health Information Technology (ONC).
Mission. Office of the National Coordinator for Health Information Technology (ONC).
Available at: http://www.hhs.gov/healthit/mission.html. Accessed July 6, 2006.
5. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order
entry and clinical decision support systems on medication safety. Archives of
Internal Medicine. 2003; 163:1409–1416.
6. McConnell T. Computerized physician order entry promises to streamline and
improve healthcare delivery. Health Management Technology. 2001; 22(3):16–18.
7. Bates DW, Gawande Aa. Improving safety with information technology. New
England Journal of Medicine. 2003; 348(25):2526–2534.
8. Chertow GM, Lee J, Kuperman GJ, et al. Guided medication dosing for
inpatients with renal insufficiency. JAMA. 2001; 286(22):2839–2844.
9. Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical
decision support systems on physician performance and patient outcomes: a
systematic review. JAMA. 1998; 280(15):1339–1346.
10. Raschke RA, Gollihare B, Wunderlich TA, et al. A computer alert system to
prevent injury from adverse drug events: development and evaluation in a
community teaching hospital. JAMA. 1998; 280(15):1317–1320.
11. Lehmann ED. Electronic medical records in clinical diabetes care. Diabetes
Technology & Therapeutics. 1999; 1(4):555–557.
12. Abookire SA, Teich JM, Sandige H, et al. Improving allergy alerting in a
computerized physician order entry system. Proceedings of the AMIA Annual
Symposium. 2000:2–6.
13. Bates DW, Kuperman GJ, Teich JM. Computerized physician order entry and
quality of care. Quality Management in Health Care. 1994; 2(4):18–27.
14. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician
order entry on medication error prevention. Journal of the American Medical
Informatics Association. 1999; 6(4):313–321.
15. California HealthCare Foundation. The Quality Initiative: Computerized
Physician Order Entry. Avaliable at http://admin.chcf.org/documents/quality/
CPOEfactsheet.pdf. 2002.
16. Gandhi TK, Weingart SN, Seger A, et al. Does computerized prescribing
help reduce med errors and ADRs in the outpatient setting? Formulary. 2001;
36(8):613–614.
17. Gandhi TK, Weingart SN, Seger A, et al. Impact of Basic Computerized
Prescribing on Outpatient Medication Errors and Adverse Drug Events.
Journal of General Internal Medicine. 2001; 16(195).
18. Overhage JM, Dexter PR, Perkins SM, et al. A randomized, controlled trial
of clinical information shared from another institution. Annals of Emergency
Medicine. 2002; 39(1):14–23.

76  Health Information Technology in the United States: The Information Base for Progress

19. iHealthBeat. North Carolina practice saves with EMRs. iHealthBeat - Business
and Finance. 2003: Available at http://ihealthbeat.org/members/basecontent.
asp?oldcoll=547&contentid=25164&collectionid=25546&program=25161 .
20. Overhage JM, Suico J, McDonald CJ. Electronic laboratory reporting:
barriers, solutions and findings. Journal of Public Health Management & Practice.
2001; 7(6):60–66.
21. iHealthBeat. California Practices Increase Efficiency with EMRs. iHealthBeat
- Care Delivery. October 27, 2003 2003: Available at http://www.ihealthbeat.
org/index.cfm?Action=dspItem&itemID=100142.
22. Brailer D. Connection tops collection. Peer-to-peer technology lets caregivers
access necessary data, upon request, without using a repository. Health
Management Technology. 2001; 22(8):28–29.
23. Siegrist Jr. RB, Kane NM. Exploring the relationship between inpatient hospital
costs and quality of care. American Journal of Managed Care. 2003; 9, Spec-9.
24. Bates DW. Using information technology to improve surgical safety. British
Journal of Surgery. 2004; 91(939–40).
25. Schmittdiel J, McMenamin SB, Halpin HA, et al. The use of patient and
physician reminders for preventive services: results from a National Study of
Physician Organizations. Preventative Medicine. 2004; 39(5):1000–1006.
26. Chang BL, Bakken S, Brown SS, et al. Bridging the digital divide: reaching
vulnerable populations. Journal of American Medical Informatics Association.
2004; 11(6):448–457.
27. Sambamoorthi U, Moynihan PJ, McSpiritt E, Crystal S. Use of protease
inhibitors and non-nucleoside reverse transcriptase inhibitors among Medicaid
beneficiaries with AIDS. American Journal of Public Health. 2001; 91(9):1474–1481.
28. Groeneveld PW, Laufer SB, Garber AM. Technology diffusion, hospital
variation, and racial disparities among elderly Medicare beneficiaries 1989–
2000. Medical Care. 2005; 43(4):320–329.
29. Ferris TG, Kuhlthau K, Ausiello J, Perrin J, Kahn R. Are minority children
the last to benefit from a new technology? Technology diffusion and inhaled
corticosteriods for asthma. Medical Care. 2006; 44(1):81–86.
30. Stevens GD, Shi L. Racial and ethnic disparities in the quality of primary care
for children. Journal of Family Practice. 2002; 51(6):573.
31. Lanting LC, Joung IM, Mackenbach JP, Lamberts SW, Bootsma AH. Ethnic
differences in mortality, end-stage complications, and quality of care among
diabetic patients: a review. Diabetes Care. 2005; 28(9):2280–2288.
32. Schraufnagel TJ, Wagner AW, Miranda J, Roy-Byrne PP. Treating minority
patients with depression and anxiety: what does the evidence tell us? General
Hospital Psychiatry. 2006; 28(1):27–36.
33. Weech-Maldonado R, Elliott MN, Morales LS, Spritzer K, Marshall GN,
Hays RD. Health plan effects on patient assessments of Medicaid managed
care among racial/ethnic minorities. Journal of General Internal Medicine. 2004;
19(2):136–145.
34. Saha S, Arbelaez JJ, Cooper LA. Patient-physician relationships and racial
disparities in the quality of health care. American Journal of Public Health. 2003;
93(1713–1719).

Health Information Technology in the United States: The Information Base for Progress  77

35. Fiscella K, Franks P, Doescher MP. Disparities in health care by race, ethnicity,
and language among the insured: findings from a national sample. Medical
Care. 2002; 40(52–59).
36. Mayberry RM, Mill F, Ofilil E. Racial and ethnic differences in access to
medical care. Medical Care Research & Review. 2000; 57(Suppl 1):108–145.
37. Schneider EC, Zaslavsky AM, Epstein AM. Racial disparities in the quality of
care for enrollees in Medicare managed care. Journal of the American Medical
Association. 2002; 287:1288–1294.
38. Franks P, Fiscella K, Meldrum S. Racial disparities in the content of primary
care office visits. Journal of General Internal Medicine. 2005; 20(7):599–603.
39. Fiscella K, Franks P, Doescher MP, Saver BG. Do HMOs affect educational
disparities in health care? Annals of Family Medicine. 2003; 1(2):90–96.
40. Lantz PM, Lynch JW, House JS, et al. Socioeconomic disparities in health
change in a longitudinal study of US adults: the role of health-risk behaviors.
Social Science & Medicine. 2001; 53(1):29–40.
41. Simpson L, Owens PL, Zodet MW, et al. Health care for children and youth
in the United States: annual report on patterns of coverage, utilization,
quality, and expenditures by income. Ambulatory Pediatric. 2005; 5(1):6–44.
42. Asplin BR, Rhodes KV, Levy H, et al. Insurance status and access to urgent
ambulatory care follow-up appointments. Journal of the American Medical
Association. 2005; 294(10):1248–1254.
43. Mort EA, Edwards JN, Emmons DW, Convery K, Blumenthal D. Physician
response to patient insurance status in ambulatory care clinical decisionmaking. Implications for quality of care. Medical Care. 1996; 34(8):783–797.
44. Shi L. Type of health insurance and the quality of primary care experience.
American Journal of Public Health. 2000; 90(12):1848–1855.
45. Ponce NA, Hays RD, Cunningham WE. Linguistic disparities in health care
access and health status among older adults. Journal of General Internal Medicine.
2006; 21(7):786–791.
46. Sarver J, Baker DW. Effect of language barriers on follow-up appointments
after an emergency department visit. Journal of General Internal Medicine. 2000;
15(4):256–264.
47. Cohen AL, Christakis DA. Primary language of parent is associated with
disparities in pediatric preventive care. Journal of Pediatrics. 2006; 148(2):254–258.
48. Institute of Medicine. America’s Health Care Safety Net: Intact but Endangered.
Washington, D.C.: National Academy Press; 2000.
49. Kindig DA. Medical Underservice in Rural America in a Changing Health Care
System. Washington, DC: Kaiser Family Foundation; 1994.
50. Bodenheimer T, Wagner EH, Grumbach K. Improving primary care for
patients with chronic illness: the chronic care model, Part 2. JAMA. 2002;
288(15):1909–1914.
51. The GWU School of Public Health and Health Services, The Institute for
Health Policy at MGH/Partners HealthCare System, Division of Internal
Medicine at the Brigham & Women’s Hospital, Clinical and Quality Analysis
Group of Partners HealthCare System. HIT Adoption Initiative Expert
Consensus Panel Meeting 2; April 5, 2006; Washington, DC.

78  Health Information Technology in the United States: The Information Base for Progress

52. National Institutes of Health. NIH Policy and Guidelines on the Inclusion of
Women and Minorities as subjects in clinical research—Amended, October,
2001. Vol 2003: NIH; 2001.
53. U.S. Census Bureau. Racial and Ethnic Classifications Used in Census 2000
and Beyond. Available at: http://www.census.gov/population/www/socdemo/
race/racefactcb.html, 2006.
54. Jones C, LaVeist T, Lillie-Blanton M. “Race” in epidemiologic literature: an
examination of the American Journal of Epidemiology, 1921–1990. American
Journal of Epidemiology. Vol 134; 1991:1079–1084.
55. Williams DR. The concept of race in health services research. Health Services
Research. Vol 29; 1994:261.
56. Oppenheimer G. Paradigm lost: race, ethnicity, and the search for a new
population taxonomy. American Journal of Public Health. Vol 91; 2001:1049–1055.
57. Osborne NG, Feit MD. The use of race in medical research. Journal of the
American Medical Association. Vol 267; 1992:275–279.
58. National Association of Community Health Centers. Health Center Fact Sheet.
Washington, DC 2005.
59. Mann JM, Melnick GA, Bamezai A, Zwanziger J. A profile of uncompensated
hospital care, 1983–1995. Health Affairs. 1997; 16(4):223–232.
60. The GWU School of Public Health and Health Services, The Institute for
Health Policy at MGH/Partners HealthCare System, Division of Internal
Medicine at the Brigham & Women’s Hospital, Clinical and Quality Analysis
Group of Partners HealthCare System. Report to the Office of the National
Coordinator: An Environmental Scan of the Current State of EHR Adoption
Measurement in the United States 2005 (draft).
61. Bureau of Primary Health Care. Section 330 Grantees Uniform Data System 2004.
62. Taylor J. The fundamentals of community health centers. National Health
Policy Forum Issue Brief. 2004; August 31.
63. Public Health Service Act. 42 USC 330.
64. Guthrie K, Louie J, Luckey A, Paley J, Blueprint Research & Design Inc.
A Drive for Data: The Evolution of Information Management at California’s
Community Clinics 2000 to 2005. San Francisco: Tides Foundation; In press.
65. Shields AE, Shin P, Betancourt RM, et al. 2006 Survey of Health Center Use
of Electronic Health Information: Unpublished data. 2006, Boston.
66. Health Resources and Service Administration. Health Disparities
Collaborative. Available at: www.healthdisparties.net, 2006.
67. Burt CW, Hing E, Woodwell D. Electronic medical record use by office-based
physicians: United States, 2005: Unpublished data. 2006.
68. Health Resources and Service Administration. User Documentation for the
area resource file (ARF). 2005.
69. Bach PB, Pham HH, Schrag D, Tate RC, Hargraves JL. Primary care
physicians who treat blacks and whites. New England Journal of Medicine. 2004;
351(6):575–584.
70. Waldo DR. Accuracy and Bias of Race/Ethnicity Codes in the Medicare
Enrollment Database. Health Care Finance Review. 2004; 26(2):61–72.

Health Information Technology in the United States: The Information Base for Progress  79

Chapter 5

1. Kaushal R, Blumenthal D, Poon EG, et al. The costs of a national health
information network. Annals of Internal Medicine. 2005; 143:165–173.
2. Kaushal R, Bates DW, Poon EG, Jha AK, Blumenthal D. Functional gaps
in attaining a national health information network. Health Affairs. 2005;
24(5):1281–1289.
3. Walker JD, Pan E, Johnston D, Adler-Milstein J, Bates DW, Middleton B.
The Value of Health Care Information Exchange and Interoperability. Health
Affairs. 2005; 24(Suppl):10–18.
4. Hillestad R, Bigelow J, Bower A, et al. Can Electronic Medical Record systems
transform health care? Potential health benefits, savings, and costs. Health
Affairs. 2005; 24(5):1103–1117.
5. Ash JS, Gorman PN, Seshadri V, Hersh WR. Computerized Physician Order
Entry in U.S. Hospitals: Results of a 2002 Survey. Journal of American Medical
Informatics Association. 2004; 11(2):95–99.
6. Audet AM, Doty M, Peugh J, Shamasdin J., Zapert K, Schoenbaum S.
Information technologies: When will they make it into physicians’ black bags?
Medscape General Medicine; 2004.
7. Bates DW, Gawande Aa. Improving safety with information technology. New
England Journal of Medicine. 2003; 348(25):2526–2534.
8. Burt CW, Hing E. Use of computerized clinical support systems in medical
settings: United States, 2001–03. Advance data from vital and health statistics. Vol
no 353. Hyattsville, Maryland: National Center for Health Statistics; 2005.
9. Burt CW, Sisk JE. Which physicians and practices are using Electronic
Medical Records? Health Affairs. 2005; 24(5):1334–1343.
10. Cutler DM, Feldman NE, Horwitz JR. U.S. adoption of computerized
physician order entry systems. Health Affairs. 2005; 24(6):1654–1663.
11. Gans DN, Kralewski JE, Hammons T, Dowd B. Medical groups’ adoption
of electronic health records and information systems. Health Affairs. 2005;
24(5):1323–1333.
12. Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical
decision support systems on physician performance and patient outcomes: a
systematic review. JAMA. 1998; 280(15):1339–1346.
13. Overhage JM, Suico J, McDonald CJ. Electronic laboratory reporting:
barriers, solutions and findings. Journal of Public Health Management & Practice.
2001; 7(6):60–66.
14. Poon EG, Blumenthal D, Jaggi T, Honour MM, Bates DW, Kaushal R.
Overcoming barriers to adopting and implementing computerized physician
order entry systems in U.S. hospitals. Health Affairs. 2004; 23(4):184–190.
15. Rosenblatt R, Law S, Rosenbaum S. Law and the American Health Care System.
New York: Foundation Press; 1997.
16. U.S. Department of Health and Human Services. E-Prescribing. U.S.
Department of Health & Human Services. Accessed Dec 6, 2005.
17. Foundation of Research and Education of AHIMA. Report on the use of Health
Information Technology to Enhance and Expand Health Care Anti-Fraud Activities.
Chicago September 30, 2005.

80  Health Information Technology in the United States: The Information Base for Progress

Chapter 6

1. National Center for Education Statistics. NCES Statistical Standards: U.S.
Department of Education; 2002.
2. Centers for Medicare and Medicaid Services. Research, Statistics, Data
and Systems: Cost Reports Overview. http://www.cms.hhs.gov/CostReports/.
Accessed April, 2006.
3. Office of Management and Budget. Questions and answers when designing surveys
for information collections: Office of Information and Regulatory Affairs; 2006.
4. National Center for Health Statistics. Ambulatory Health Care Data. Centers
for Disease Control and Prevention. Available at: http://www.cdc.gov/nchs/
about/major/ahcd/ahcd1.htm. Accessed April, 2006.
5. Kletke PR, Polsky D, Wozniak GD, Escarce JJ. The Effect of HMO Penetration
on Physician Retirement. Health Services Research. 2000; 35(5):17–31.
6. Rittenhouse DR, Mertz E, Keane D, Grumbach K. No Exit: An Evaluation of
Measures of Physician Attrition. Health Services Research. 2004; 39(5):1572–1588.
7. Hammons T. In: Donelan K, ed; 2006.
8. American Hospital Association. Annual Hospital Survey. 2005.
9. Olmsted M, Murphy J, McFarlane E, Hill C. Evaluating Methods for
Increasing Physician Survey Cooperation. Paper presented at: American
Association for Public Opinion Research Annual Meeting; May 14, 2005;
Miami Beach, FL.

Health Information Technology in the United States: The Information Base for Progress  81

About the Robert Wood Johnson Foundation

The Robert Wood Johnson Foundation focuses on the pressing health and
health care issues facing our country. As the nation’s largest philanthropy
devoted exclusively to improving the health and health care of all Americans,
the Foundation works with a diverse group of organizations and individuals to
identify solutions and achieve comprehensive, meaningful and timely change.
For more than 30 years the Foundation has brought experience, commitment,
and a rigorous, balanced approach to the problems that affect the health and
health care of those it serves. Helping Americans lead healthier lives and get the
care they need—the Foundation expects to make a difference in our lifetime. For
more information, visit www.rwjf.org.

About the George Washington University Medical Center

The George Washington University Medical Center is an internationally
recognized interdisciplinary academic health center that has consistently
provided high quality medical care in the Washington, D.C. metropolitan area
for 176 years. The Medical Center comprises the School of Medicine and Health
Sciences, the 11th oldest medical school in the country; the School of Public
Health and Health Services, the only such school in the nation’s capital; GW
Hospital, jointly owned and operated by a partnership between the George
Washington University and Universal Health Services, Inc.; and the GW Medical
Faculty Associates, an independent faculty practice plan. For more information
on GWUMC, visit www.gwumc.edu.

About the Institute for Health Policy

The Institute for Health Policy (IHP) at Massachusetts General Hospital
(MGH) and Partners Health System is dedicated to conducting world-class
research on the central health care issues of our time. The mission of the IHP is to
improve the health and health care of the American people through conducting
health policy and health services research, translating new healthcare knowledge
into practice, informing and influencing public policy, and training scholars and
practitioners of health policy.
This report was produced by a team of researchers at the Institute for Health Policy
at Massachusetts General Hospital and the School of Public Health and Health
Services at George Washington University: David Blumenthal, M.D., M.P.P.;
Catherine DesRoches, Dr.P.H.; Karen Donelan, Sc.D.; Timothy Ferris, M.D.,
MPhil., M.P.H.; Ashish Jha, M.D., M.P.H.; Rainu Kaushal, M.D., M.P.H.; Sowmya
Rao, Ph.D.; Sara Rosenbaum J.D.; and Alexandra Shield, Ph.D.
The report was also informed by the discussions of an Expert Consensus
Panel. The authors gratefully acknowledge the support of the Robert Wood
Johnson Foundation and the efforts of the federal Office of the National
Coordinator for Health Information Technology on behalf of this report.
© 2006 Robert Wood Johnson Foundation

2006

Health Information Technology
in the United States:
The Information Base for Progress

Robert Wood Johnson Foundation
www.rwjf.org
MGH Institute for Health Policy
http://mgh.harvard.edu/healthpolicy
George Washington University
School of Public Health and Health Services
The Health Law Information Project
www.healthinfolaw.org