Jencks_q2003

ORIGINAL CONTRIBUTION

Change in the Quality of Care
Delivered to Medicare Beneficiaries,
1998-1999 to 2000-2001
Stephen F. Jencks, MD, MPH
Edwin D. Huff, PhD
Timothy Cuerdon, PhD

H

EALTH CARE IN THE UNITED
States can be improved substantially, and even people
with apparently good access
to care receive care that falls far short of
what it could be. In the area of public
health and prevention, Healthy People
20101 showed wide gaps between public health performance and actual
achievements on many quality indicators, including some delivered by the feefor-service health care system. Two years
ago, a report from the Institute of Medicine showed serious problems of harm
to patients from medical errors2; last year
another Institute of Medicine report,
Crossing the Quality Chasm,3 identified
major system problems as the principal source of many errors. In 2000, Congress instructed the Agency for Health
Care Research and Quality to prepare an
annual report on quality of health care
in the United States, and the first of these
reports is scheduled to be made public
next year.
In 2000, the Health Care Financing
Administration (now the Centers
for Medicare & Medicaid Services)
reported on 24 indicators of the quality of care delivered to Medicare beneficiaries (primarily in fee-for-service)
in 1998-1999. 4 These indicators
measure delivery of services that evi-

For editorial comment see p 354.

Context Despite widespread concern regarding the quality and safety of health care,
and a Medicare Quality Improvement Organization (QIO) program intended to improve that care in the United States, there is only limited information on whether quality is improving.
Objective To track national and state-level changes in performance on 22 quality
indicators for care of Medicare beneficiaries.
Design, Patients, and Setting National observational cross-sectional studies of
national and state-level fee-for-service data for Medicare beneficiaries during 19981999 (baseline) and 2000-2001 (follow-up).
Main Outcome Measures Twenty-two QIO quality indicators abstracted from statewide random samples of medical records for inpatient fee-for-service care and from
Medicare beneficiary surveys or Medicare claims for outpatient care. Absolute improvement is defined as the change in performance from baseline to follow-up (measured in percentage points for all indicators except those measured in minutes); relative improvement is defined as the absolute improvement divided by the difference
between the baseline performance and perfect performance (100%).
Results The median state’s performance improved from baseline to follow-up on
20 of the 22 indicators. In the median state, the percentage of patients receiving appropriate care on the median indicator increased from 69.5% to 73.4%, a 12.8% relative improvement. The average relative improvement was 19.9% for outpatient indicators combined and 11.9% for inpatient indicators combined (P⬍.001). For all but
one indicator, absolute improvement was greater in states in which performance was
low at baseline than those in which it was high at baseline (median r=−0.43; range:
0.12 to −0.93). When states were ranked on each indicator, the state’s average rank
was highly stable over time (r=0.93 for 1998-1999 vs 2000-2001).
Conclusions Care for Medicare fee-for-service plan beneficiaries improved substantially between 1998-1999 and 2000-2001, but a much larger opportunity remains for
further improvement. Relative rankings among states changed little. The improved care
is consistent with QIO activities over this period, but these cross-sectional data do not
provide conclusive information about the degree to which the improvement can be
attributed to the QIOs’ quality improvement efforts.
www.jama.com

JAMA. 2003;289:305-312

dence shows to be effective in preventing or treating breast cancer, diabetes, myocardial infarction, heart
failure, pneumonia, and stroke.4 This
report provides follow-up data on care
given in 2000-2001 and makes
comparisons with the 1998-1999
baseline data.

©2003 American Medical Association. All rights reserved.

Author Affiliations: Office of Clinical Standards and
Quality, Centers for Medicare & Medicaid Services, Baltimore, Md (Dr Jencks); Division of Clinical Standards
and Quality, Centers for Medicare & Medicaid Services, John F. Kennedy Building, Boston, Mass (Dr Huff );
Health and Behavioral Science Research Branch, National
Institute of Mental Health, Bethesda, Md (Dr Cuerdon).
Corresponding Author: Stephen F. Jencks, MD, MPH,
Centers for Medicare & Medicaid Services, 7500 Security Blvd, Mail Stop S3-02-01, Baltimore, MD 21244
(e-mail: [email protected]).

(Reprinted) JAMA, January 15, 2003—Vol 289, No. 3 305

QUALITY OF MEDICARE HEALTH CARE DELIVERY

METHODS
The tracking system used for the 19981999 data that was first reported in 2000
is used again for the 2000-2001 data in
this report. This system is used in evaluation of the Medicare Quality Improvement Organizations (QIOs) and is
independent of them.
TABLE 1 summarizes the clinical topics, quality indicators, sampling frame,

and data sources that were used for the
baseline article and are used again
herein. The quality indicators and their
rationale have been described in the
2000 report.4 The Medicare Quality Improvement Organization program
tracks 24 quality indicators through
contracted data abstraction centers, surveys, and analysis of claims data. Two
of these (time to thrombolysis and time

to angioplasty) are shown in TABLE 2
but are not analyzed herein (they were
not in the 2000 report) because the
number of cases observed in most states
was quite small.
We followed the same fee-forservice sampling strategy and data collection procedures as were first reported for the baseline data with 2
exceptions. Information on influenza

Table 1. Quality Indicators for Care of Medicare Fee-for-Service Beneficiaries
Topic
Inpatient setting
Acute myocardial
infarction

Indicator

Sampling Frame for
Denominator

Short Name

All Medicare patients with
principal discharge
diagnosis of acute
myocardial infarction and
no contraindications

Systematic random sample
of up to 750 inpatient
records per state

All Medicare patients with
principal discharge
diagnosis of heart failure

Systematic random sample
of up to 800 inpatient
records per state

Afibrillation

All Medicare patients with
any discharge diagnosis
of atrial fibrillation

Systematic random sample
of up to 750 inpatient
records per state

Antithrombotic prescribed at discharge for
patients with acute stroke or transient
ischemic attack

Antithrombotic

Systematic random sample
of up to 750 inpatient
records per state

Avoidance of sublingual nifedipine for patients
with acute stroke

Nifedipine

All Medicare patients with
principal discharge
diagnosis of stroke
(nifedipine and
antithrombotic) or
transient ischemic attack
(antithrombotic)

Antibiotic within 8 h of arrival at hospital

Antibiotic time

Antibiotic consistent with current
recommendations
Blood culture drawn (if done) before antibiotic
given
Patient screened for or given influenza
vaccine
Patient screened for or given pneumococcal
vaccine

Antibiotic Rx

All Medicare patients with a
discharge diagnosis of
pneumonia

Systematic random sample
of up to 750 inpatient
records per state

Influenza immunization every year

Flu immun

Pneumococcal immunization at least once
ever

Pneu immun

All noninstitutionalized
persons aged ⱖ65 y
(includes managed care)

Breast cancer

Mammogram at least every 2 y

Mammography

Behavioral Risk Factor
Surveillance System for
1998-1999; similar CMS
survey for 2000-2001
All Medicare claims

Diabetes

Hemoglobin A1c at least every y

HbA1c

Eye exam at least every 2 y

Eye exam

Lipid profile at least every 2 y

Lipid profile

Heart failure

Stroke

Pneumonia

Any setting
Pneumonia

Administration of aspirin within 24 h of
admission

Aspirin 24 h

Aspirin prescribed at discharge 24 h
Administration of ␤-blocker within 24 h of
admission
␤-Blocker prescribed at discharge
ACE Inhibitor prescribed at discharge for
patients with left ventricular ejection
fraction ⬍0.40
Smoking cessation counseling given during
hospitalization
Time to angioplasty, min
Time to thrombolytic therapy, min
Evaluation of ejection fraction

Aspirin disch
BB 24 h

Data Source

ACE Inhibitor prescribed at discharge for
patients with left ventricular ejection
fraction ⬍0.40
Warfarin prescribed for patients with atrial
fibrillation

BB disch
ACEI in AMI

Smoking
PTCA, min
Thrombolytic, min
LVEF
ACEI in HF

Blood culture
Flu screen
Pneu screen

All female Medicare
beneficiaries aged
52-69 y
All Medicare patients with 2
ambulatory diagnoses or
1 inpatient diagnosis of
diabetes

All Medicare claims

Abbreviations: BB, ␤-blocker; ACE, angiotensin-converting enzyme; PTCA, percutaneous coronary intervention; CMS, Centers for Medicare & Medicaid Services.

306

JAMA, January 15, 2003—Vol 289, No. 3 (Reprinted)

©2003 American Medical Association. All rights reserved.

QUALITY OF MEDICARE HEALTH CARE DELIVERY

Table 2. Quality Indicator Averages (Absolute Change From Baseline) by State, 2000-2001*
Quality Indicators
Congestive
Heart Failure

58 (3)

78 (18)

69 (6)

38 (4)

35 (−18)

86 (−10)

78 (−1)

67 (−6)

100 (18)

61 (24)

57 (3)

Arizona

25

29

84 (−3)

84 (−2)

65 (2)

84 (16)

69 (1)

32 (−21)

55 (−3)

Arkansas

50

48

78 (3)

84 (6)

50 (−5)

57 (−6)

69 (12)

39 (15)

California

39

44

87 (2)

86 (2)

66 (6)

66 (−2)

70 (4)

Colorado

9

7

92 (6)

93 (3)

75 (10)

92 (16)

82 (8)

Connecticut

Antithrombotic

87 (1)

82 (−5)

53 (3)

84 (4)

58 (−33)

52 (−1)

84 (−2)

103 (−4)

81 (11)

61 (−4)

55 (−2)

89 (7)

99 (8)

37 (−5)

109 (15)

60 (8)

43 (−22)

51 (0)

80 (3)

100 (8)

31 (−10)

77 (41)

146 (39)

66 (4)

70 (5)

52 (8)

77 (2)

96 (9)

53 (6)

51 (13)

121 (42)

69 (4)

65 (−7)

65 (8)

85 (1)

100 (6)

101 (−7)

79 (2)

69 (−4)

64 (7)

90 (0)

100 (2)

77 (3)

78 (5)

55 (5)

89 (3)

99 (1)

177 (127)

75 (4)

72 (−4)

55 (1)

82 (3)

99 (0)

95 (−36)

76 (6)

70 (4)

61 (5)

80 (1)

97 (6)

68 (5)

64 (−3)

51 (1)

80 (1)

100 (9)

100 (1)
90 (−13)

Nifedipine

70 (8)

70 (19)

PTCA, min

69 (4)

Smoking

Afibrillation

80 (2)

33

Stroke

ACEI in HF

Aspirin Disch

42

22

ACEI in AMI

Aspirin 24 h

46

Alaska

State

BB Disch

2000-2001

Alabama

BB 24 h

1998-1999

Thrombolysis,
min

Acute Myocardial Infarction

LVEF

Average
State
Ranks

100 (4)
97 (4)

6

9

89 (−3)

86 (−5)

78 (10)

83 (8)

80 (6)

39 (−2)

39 (2)

Delaware

12

14

88 (2)

96 (10)

69 (7)

84 (12)

82 (10)

33 (−37)

28 (−22)

District of Columbia

31

37

87 (−10)

91 (8)

69 (−5)

85 (−7)

78 (5)

39 (12)

Florida

40

41

80 (3)

79 (1)

65 (4)

79 (10)

60 (−10)

33 (3)

45 (9)

Georgia

48

47

73 (−6)

84 (3)

58 (−4)

74 (6)

71 (3)

38 (4)

38 (4)

Hawaii

23

16

90 (6)

83 (2)

62 (6)

84 (33)

79 (4)

50 (14)

79 (35)

96 (21)

82 (7)

74 (3)

47 (2)

90 (1)

100 (3)

Idaho

19

22

90 (3)

87 (2)

66 (−4)

84 (11)

78 (19)

44 (−12)

30 (−8)

107 (−32)

58 (4)

73 (−15)

56 (−1)

83 (3)

99 (1)

Illinois

47

46

83 (7)

80 (4)

67 (0)

75 (20)

73 (−1)

35 (6)

51 (30)

110 (−51)

67 (2)

68 (7)

57 (2)

83 (3)

99 (7)

Indiana

29

27

83 (−1)

89 (2)

69 (8)

83 (12)

79 (12)

55 (2)

35 (2)

165 (45)

71 (6)

60 (−5)

62 (7)

84 (3)

8

6

85 (1)

88 (2)

78 (14)

89 (10)

77 (2)

41 (4)

51 (8)

104 (−29)

66 (13)

70 (−1)

60 (3)

83 (−1)

Kansas

34

30

84 (5)

84 (0)

68 (14)

74 (15)

67 (9)

51 (8)

49 (4)

134 (49)

59 (1)

60 (−10)

53 (2)

86 (10)

97 (8)

Kentucky

37

40

85 (5)

81 (−2)

65 (2)

80 (7)

66 (−4)

52 (16)

32 (2)

63 (1)

52 (−10)

54 (3)

84 (1)

100 (9)

Louisiana

49

51

85 (4)

81 (2)

65 (7)

71 (−2)

65 (1)

25 (−15)

44 (11)

66 (6)

58 (12)

49 (2)

74 (−1)

100 (6)

3

3

89 (4)

91 (4)

82 (1)

91 (7)

82 (14)

49 (−12)

38 (13)

73 (6)

71 (−1)

60 (−1)

89 (2)

99 (1)

24

25

85 (−1)

87 (3)

71 (2)

77 (1)

79 (0)

27 (−13)

30 (−24)

83 (−86)

75 (2)

61 (−4)

55 (2)

84 (3)

99 (1)

4

15

87 (0)

86 (−2)

82 (9)

88 (−4)

72 (−7)

45 (1)

45 (4)

135

80 (4)

65 (3)

66 (2)

91 (5)

99 (3)

Michigan

28

26

85 (1)

90 (4)

70 (3)

93 (20)

82 (8)

43 (1)

49 (11)

110 (−29)

70 (1)

68 (6)

57 (6)

86 (7)

99 (3)

Minnesota

7

10

88 (−2)

83 (−6)

80 (14)

87 (3)

69 (−11)

51 (13)

42 (2)

117 (21)

64 (3)

69 (−1)

62 (3)

89 (1)

100 (3)

Missouri

35

28

81 (5)

88 (10)

67 (8)

78 (7)

74 (0)

54 (16)

80 (36)

124 (−255)

73 (7)

71 (12)

58 (6)

83 (−1)

Montana

17

13

88 (2)

89 (−1)

70 (17)

71 (−1)

71 (13)

46 (−17)

44 (−3)

58 (11)

77 (7)

62 (3)

86 (1)

99 (3)

Mississippi

51

50

80 (0)

84 (7)

60 (16)

66 (19)

66 (5)

43 (9)

38 (11)

141 (−51)

61 (2)

55 (−6)

54 (7)

80 (6)

100 (2)

Nebraska

27

12

85 (1)

89 (4)

74 (8)

74 (−9)

81 (13)

51 (14)

41 (2)

107 (−311)

74 (3)

69 (−7)

67 (9)

90 (6)

95 (7)

Nevada

36

35

88 (5)

84 (4)

59 (1)

69 (−1)

73 (−4)

45 (1)

45 (−7)

178 (70)

82 (0)

62 (−13)

56 (14)

81 (3)

96 (9)

1

1

92 (4)

93 (2)

86 (11)

89 (−1)

87 (6)

36 (−13)

35 (−14)

260 (159)

82 (1)

77 (2)

70 (8)

86 (1)

100 (1)

New Jersey

41

43

76 (−1)

65 (−10)

61 (−4)

68 (−1)

64 (4)

31 (−7)

47 (3)

128 (10)

72 (6)

59 (6)

55 (0)

73 (0)

99 (3)

New Mexico

32

36

89 (4)

89 (2)

65 (12)

74 (12)

67 (−10)

53 (3)

43 (6)

58 (3)

70 (−14)

58 (1)

77 (−1)

97 (6)

New York

30

24

84 (1)

84 (3)

81 (14)

85 (12)

76 (1)

36 (−13)

44 (−9)

81 (4)

76 (−8)

62 (7)

84 (2)

100 (2)

North Carolina

18

23

84 (3)

92 (3)

69 (4)

81 (2)

76 (−1)

47 (13)

58 (27)

95 (−29)

74 (15)

65 (3)

53 (−7)

87 (0)

98 (1)

North Dakota

5

4

92 (7)

92 (5)

75 (6)

84 (−3)

65 (−16)

42 (13)

103 (60)

73 (−122)

45 (5)

68 (−10)

64 (−1)

90 (4)

100 (5)

Ohio

33

38

83 (−5)

83 (−3)

72 (10)

79 (6)

64 (−7)

25 (−2)

42 (−11)

82 (10)

75 (5)

57 (−7)

62 (10)

84 (4)

98 (6)

Oklahoma

44

45

83 (5)

82 (3)

57 (12)

76 (13)

76 (6)

39 (14)

46 (8)

59 (−25)

59 (7)

56 (−10)

50 (−2)

76 (4)

97 (7)

Oregon

20

11

87 (1)

87 (3)

79 (10)

80 (3)

78 (8)

45 (−8)

38 (−2)

77 (−25)

70 (11)

76 (7)

56 (−1)

82 (4)

100 (6)

Pennsylvania

16

31

85 (3)

85 (4)

68 (−3)

79 (−9)

63 (−21)

26 (−15)

50 (11)

103 (−88)

77 (3)

61 (−12)

63 (2)

86 (1)

99 (0)

Puerto Rico

52

52

70 (4)

69 (10)

53 (20)

67 (14)

65 (6)

33 (3)

78 (12)

230

56 (12)

61 (2)

39 (8)

76 (3)

98 (0)

Rhode Island

15

17

88 (6)

88 (1)

81 (6)

95 (16)

84 (1)

31 (5)

38 (−2)

106 (−158)

79 (2)

74 (−6)

65 (5)

86 (−2)

98 (3)

South Carolina

38

32

83 (3)

85 (5)

70 (12)

76 (6)

79 (20)

34 (9)

45 (−10)

78 (−481)

73 (6)

60 (−6)

59 (7)

90 (7)

100 (1)

South Dakota

26

20

89 (6)

92 (4)

70 (1)

82 (11)

75 (8)

52 (15)

121 (57)

163 (−117)

61 (10)

56 (−10)

69 (8)

90 (6)

97 (7)

Tennessee

42

39

78 (−5)

84 (0)

63 (7)

75 (9)

71 (4)

39 (−5)

41 (15)

170 (64)

67 (0)

66 (15)

46 (−15)

81 (5)

100 (6)

Texas

45

49

83 (5)

75 (−9)

65 (14)

72 (14)

64 (1)

37 (18)

54 (16)

111 (26)

65 (1)

58 (−4)

48 (4)

80 (8)

98 (8)

Utah

14

5

87 (3)

93 (3)

75 (17)

88 (20)

78 (−1)

61 (10)

30 (−20)

71 (14)

73 (−6)

65 (8)

88 (2)

98 (7)

Vermont

2

2

92 (6)

90 (1)

82 (4)

86 (7)

70 (−2)

56 (−3)

48 (−1)

230 (45)

81 (10)

81 (4)

66 (8)

88 (2)

99 (1)

Virginia

21

18

89 (4)

90 (6)

67 (2)

88 (11)

78 (11)

53 (10)

51 (6)

141 (−23)

80 (3)

73 (−1)

56 (−4)

88 (−2)

98 (0)

Washington

13

19

92 (6)

87 (−1)

71 (4)

78 (12)

71 (−5)

51 (−9)

50 (4)

98 (−23)

69 (6)

61 (−19)

59 (9)

83 (−1)

100 (6)

West Virginia

43

34

85 (1)

86 (1)

65 (13)

62 (−3)

65 (1)

51 (8)

40 (−21)

122 (−15)

58 (−4)

53 (−5)

56 (11)

87 (1)

100 (7)

Wisconsin

11

8

86 (1)

87 (−1)

74 (3)

79 (−6)

75 (10)

58 (16)

49 (16)

108 (−104)

72 (5)

72 (−3)

65 (5)

86 (2)

100 (5)

Wyoming

10

21

95 (4)

91 (−4)

71 (1)

62 (0)

89 (−1)

44 (−22)

33 (−3)

116 (−41)

42 (8)

82 (3)

65 (7)

82 (2)

85 (3)

86 (2)

69 (6)

79 (7)

74 (4)

43 (3)

45 (4)

107 (−19)

70 (4)

68 (−4)

57 (3)

84 (2)

Iowa

Maine
Maryland
Massachusetts

New Hampshire

Median

67

110 (6)

71 (−48)
105 (11)

87 (0)

94 (−46)
100 (6)

89 (−64)

98 (5)
100 (1)

99 (7)

98 (−2)
99 (4)

(continued)

©2003 American Medical Association. All rights reserved.

(Reprinted) JAMA, January 15, 2003—Vol 289, No. 3 307

QUALITY OF MEDICARE HEALTH CARE DELIVERY

Table 2. Quality Indicator Averages (Absolute Change From Baseline) by State, 2000-2001* (cont)
Breast
Cancer

Lipid Profile

Eye Exam

Diabetes

HgbA1c

Mammography

Pneu Imm

Flu Immun

Pneu Screen

Immunizations

Flu Screen

Blood Culture

Antibiotic Rx

State

Antibiotic
Time

Pneumonia

Alabama

87 (0)

84 (7)

74 (−5)

26 (13)

30 (20)

65 (1)

54 (0)

60 (4)

70 (12)

64 (1)

65 (18)

Alaska

91 (4)

83 (8)

89 (−5)

30 (6)

27 (9)

67 (7)

66 (22)

55 (3)

71 (2)

57 (1)

69 (13)

Arizona

86 (4)

90 (12)

80 (−8)

37 (10)

40 (18)

Arkansas

87 (−1)

83 (6)

83 (1)

12 (6)

8 (4)

69 (−2)

68 (15)

60 (3)

71 (4)

64 (−1)

72 (16)

74 (6)

63 (12)

55 (5)

69 (12)

70 (3)

64 (21)

California

88 (4)

83 (16)

82 (−5)

12 (2)

12 (6)

73 (1)

66 (9)

56 (3)

73 (8)

70 (0)

77 (16)

Colorado

88 (2)

85 (1)

82 (−3)

30 (10)

31 (12)

78 (3)

69 (6)

60 (5)

84 (7)

69 (2)

74 (23)

Connecticut

90 (5)

84 (9)

83 (−2)

35 (12)

27 (9)

72 (8)

66 (17)

63 (3)

80 (7)

77 (0)

76 (14)

Delaware

87 (5)

89 (8)

79 (−8)

22 (9)

19 (8)

72 (4)

69 (3)

64 (5)

80 (10)

76 (1)

82 (26)

District of Columbia

82 (5)

82 (10)

65 (−2)

27 (5)

25 (9)

61 (5)

48 (13)

52 (1)

65 (5)

69 (0)

68 (16)

Florida

80 (4)

84 (10)

79 (−3)

14 (8)

18 (15)

66 (3)

61 (8)

65 (3)

79 (10)

75 (0)

83 (14)

Georgia

83 (0)

84 (7)

81 (1)

14 (3)

17 (9)

71 (14)

63 (13)

57 (5)

74 (11)

63 (1)

67 (15)

Hawaii

89 (−1)

80 (1)

87 (2)

23 (12)

22 (11)

78 (4)

75 (19)

57 (5)

82 (9)

72 (4)

82 (11)

Idaho

89 (0)

88 (10)

84 (0)

24 (12)

29 (19)

72 (3)

59 (4)

60 (5)

82 (5)

69 (1)

75 (16)

Illinois

86 (1)

85 (8)

77 (−1)

21 (8)

17 (8)

69 (1)

59 (11)

58 (4)

74 (11)

66 (3)

69 (20)

Indiana

84 (3)

84 (5)

83 (5)

36 (8)

37 (11)

73 (7)

61 (10)

60 (6)

77 (10)

65 (1)

73 (15)

Iowa

90 (3)

88 (10)

90 (4)

33 (12)

31 (12)

78 (8)

68 (7)

65 (5)

85 (7)

78 (1)

74 (14)

Kansas

89 (0)

85 (8)

82 (−4)

30 (11)

24 (13)

72 (5)

67 (12)

64 (6)

83 (8)

77 (2)

70 (20)

Kentucky

85 (2)

85 (5)

82 (1)

22 (5)

25 (11)

66 (−2)

58 (6)

59 (6)

75 (12)

65 (1)

74 (15)

Louisiana

84 (3)

81 (9)

79 (−3)

Maine

90 (1)

82 (4)

84 (−2)

7 (−1)
39 (0)

7 (3)

65 (4)

59 (18)

55 (6)

69 (12)

63 (1)

71 (17)

28 (9)

75 (2)

65 (7)

71 (6)

85 (9)

81 (3)

76 (16)

Maryland

83 (3)

88 (6)

77 (−4)

31 (17)

28 (18)

72 (9)

67 (13)

61 (3)

77 (7)

69 (1)

79 (17)

Massachusetts

88 (2)

85 (4)

75 (−10)

19 (6)

16 (7)

79 (9)

71 (14)

66 (2)

83 (6)

76 (0)

69 (13)

Michigan

82 (−2)

84 (13)

76 (−4)

29 (10)

28 (14)

72 (2)

63 (5)

67 (3)

78 (6)

66 (1)

70 (15)

Minnesota

88 (1)

86 (11)

81 (−4)

33 (−5)

26 (4)

78 (14)

68 (16)

67 (5)

87 (5)

79 (4)

76 (17)

Missouri

83 (−2)

85 (6)

79 (−1)

14 (3)

11 (5)

70 (7)

60 (10)

52 (5)

65 (14)

65 (4)

59 (20)

Montana

84 (0)

84 (4)

79 (2)

28 (12)

25 (10)

77 (9)

64 (11)

59 (5)

78 (6)

68 (1)

71 (14)

Mississippi

93 (0)

88 (9)

89 (1)

30 (15)

24 (13)

74 (1)

68 (7)

65 (6)

81 (11)

74 (2)

74 (24)

Nebraska

91 (3)

87 (5)

83 (2)

28 (13)

33 (21)

79 (10)

70 (15)

63 (7)

77 (6)

77 (1)

69 (15)

Nevada

82 (−4)

87 (7)

72 (−4)

22 (11)

46 (35)

62 (−1)

65 (3)

56 (6)

78 (8)

65 (1)

77 (15)

New Hampshire

90 (1)

83 (8)

87 (−2)

32 (−5)

23 (4)

76 (11)

66 (6)

68 (5)

87 (5)

78 (2)

77 (17)

New Jersey

85 (6)

85 (11)

82 (3)

35 (23)

33 (25)

68 (3)

60 (6)

54 (4)

73 (11)

72 (0)

79 (14)

New Mexico

87 (0)

85 (15)

83 (−3)

26 (2)

24 (6)

72 (3)

64 (11)

56 (5)

71 (6)

63 (0)

66 (16)

New York

81 (1)

83 (13)

77 (−1)

37 (23)

32 (20)

70 (6)

64 (14)

58 (2)

76 (11)

73 (2)

76 (20)

North Carolina

83 (−1)

85 (4)

84 (5)

20 (1)

19 (7)

72 (9)

70 (11)

62 (6)

81 (11)

72 (2)

75 (21)

North Dakota

90 (−1)

86 (1)

83 (6)

33 (6)

30 (11)

78 (10)

68 (13)

71 (6)

87 (6)

80 (3)

80 (16)

Ohio

83 (2)

82 (10)

80 (−2)

30 (7)

20 (7)

69 (0)

63 (8)

61 (5)

78 (13)

69 (2)

75 (23)

Oklahoma

86 (3)

83 (3)

83 (−3)

27 (11)

33 (19)

78 (6)

69 (15)

55 (6)

72 (9)

61 (0)

68 (14)

Oregon

92 (2)

85 (7)

89 (1)

33 (19)

20 (7)

77 (12)

67 (11)

64 (5)

85 (6)

73 (2)

74 (22)

Pennsylvania

86 (1)

79 (0)

87 (−1)

22 (10)

17 (8)

74 (11)

66 (13)

60 (4)

80 (10)

71 (1)

78 (18)

Puerto Rico

47 (9)

69 (15)

63 (−2)

37 (30)

37 (32)

40 (−1)

31 (9)

52 (6)

55 (15)

53 (0)

64 (22)

Rhode Island

89 (8)

92 (8)

76 (−5)

22 (12)

23 (16)

73 (−3)

62 (5)

62 (4)

80 (10)

79 (1)

76 (21)

South Carolina

81 (1)

84 (5)

83 (−3)

15 (7)

17 (11)

76 (6)

66 (10)

61 (5)

77 (10)

68 (3)

72 (16)

South Dakota

90 (−1)

85 (0)

85 (1)

22 (8)

22 (8)

78 (4)

56 (6)

65 (8)

80 (8)

71 (1)

69 (13)

Tennessee

87 (8)

84 (4)

79 (0)

24 (14)

24 (16)

74 (9)

68 (14)

58 (6)

78 (12)

62 (1)

71 (23)

Texas

82 (2)

84 (5)

81 (−3)

17 (5)

17 (9)

69 (−1)

62 (6)

56 (5)

75 (6)

65 (−1)

77 (14)

Utah

90 (1)

86 (1)

84 (2)

52 (33)

49 (32)

74 (−1)

64 (3)

60 (6)

85 (6)

72 (3)

77 (22)

Vermont

88 (−1)

81 (0)

87 (−2)

33 (0)

17 (0)

77 (4)

69 (12)

68 (5)

87 (4)

76 (1)

75 (19)

Virginia

87 (2)

88 (4)

84 (2)

21 (11)

26 (17)

74 (8)

63 (8)

60 (5)

80 (9)

70 (1)

74 (16)

Washington

91 (3)

80 (7)

83 (−4)

24 (2)

24 (8)

72 (4)

67 (11)

64 (5)

86 (5)

74 (2)

78 (18)

West Virginia

86 (2)

81 (1)

80 (−2)

35 (27)

29 (23)

70 (7)

64 (10)

61 (6)

77 (15)

65 (3)

75 (23)

Wisconsin

88 (1)

88 (10)

88 (6)

29 (6)

25 (8)

77 (12)

68 (14)

67 (7)

86 (6)

74 (1)

76 (15)

Wyoming

90 (−2)

86 (−1)

87 (−1)

18 (5)

20 (11)

74 (1)

68 (7)

62 (7)

76 (13)

71 (3)

59 (18)

Median

87 (2)

85 (7)

82 (−2)

27 (9)

24 (11)

72 (5)

65 (10)

60 (5)

78 (8)

70 (1)

74 (16)

*For an explanation of abbreviations, see Table 1. For details of indicators, see Table 1. Values are percentage of patients receiving appropriate care except for Thrombolysis and
PTCA, which are reported in minutes. A blank indicates that there were no cases in the sample that met the selection criteria. Typeface indicates number of cases on which value
is based: italic, 1-30 cases; regular, 31-100 cases; bold, 101-200 cases; bold italic, 301 or more cases.

308

JAMA, January 15, 2003—Vol 289, No. 3 (Reprinted)

©2003 American Medical Association. All rights reserved.

QUALITY OF MEDICARE HEALTH CARE DELIVERY

and pneumococcal vaccination rates
came from a specially contracted survey using the influenza and pneumococcal vaccination items from the Behavioral Risk Factor Surveillance
System (BRFSS) and designed to emulate the BRFSS sampling strategy as
closely as possible. This was done because appropriately timed data from the
regularly scheduled BRFSS were not
available.5 We also substituted the 1999
BRFSS data for the earlier 1997 BRFSS
data in our baseline rates because these
later data represent state rates during
the 1998-1999 baseline period better
than the 1997 data. In addition, we
made minor corrections in the claims
processing algorithms used to construct the diabetes indicators for the
1998-1999 period. These changes resulted in small, nonmaterial, changes
in the baseline rates first reported in the
2000 report. The corrected baseline
rates for the immunization and diabetes indicators are used to make comparisons with the follow-up performance from the 2000-2001 period.
Reliability was calculated as the percentage agreement on all abstraction data
elements between 2 blinded, independent abstractors at different abstraction
centers. Each abstraction center also performed internal reliability assessments on
a monthly random sample of 30 cases
taken from abstracts completed during
the previous month.6
Absolute improvement is defined as the
change in performance from baseline
to follow-up (measured in percentage
points for all indicators except those
measured in minutes); relative improvement is defined as the absolute improvement divided by the difference between the baseline performance and
perfect performance (100%); relative
improvement can also be called the decrease in the error or failure rate. The
definition of relative improvement differs from the usual method of using the
baseline rate as the denominator. We
used this definition because dividing by
the baseline rate exaggerates small
changes for poorly performing states
while minimizing changes in states that
already perform well.

Performance was calculated at the
state level for each of the quality indicators. For the 22 quality indicators discussed herein, results were calculated
as the percentage of patients who had
no contraindications and who received the indicated treatment. We direct our attention both to variation
among states (including the District of
Columbia and Puerto Rico) and to national trends. Therefore, we calculated for each indicator both performance of the median state and the
national average (weighted by the number of aged Medicare beneficiaries in
each state). We calculated the SD of
each indicator rate across the set of
states. To summarize the overall
changes we observed on each indicator, we calculated the absolute and relative improvement on the indicator in
the median state. To summarize the
overall changes that we observed within
each state, we calculated a median
amount of absolute and relative improvement across the set of indicators
in the state. Finally, we characterized
the median absolute and relative national improvement as the median of
these state medians.
We also calculated the rank of each
state on each quality indicator based on
performance rates during the 20002001 follow-up period and the rank on
each quality indicator based on the
amount of relative improvement observed. We then calculated the average
rank for each state across the 22 quality indicators and arrayed the states according to their average rank, again
based on their performance rates during the 2000-2001 follow-up period. We
ranked states in a similar way on the
amount of relative improvement. The
changes in data described above and
changes in our algorithm for breaking
ties on ranking resulted in slight changes
of ranking for 1998-1999 from those reported in the earlier article.
We tested the equality of the relative improvement for the inpatient indicators (the first 16 indicators in Table
1) and outpatient indicators (the last 6
indicators in Table 1) using a t test without assumption of equal variances and

©2003 American Medical Association. All rights reserved.

treating each indicator rate in each state
as an observation.
RESULTS
The reliability of data elements used to
construct quality indicators based on
medical record abstraction ranged from
80% to 95% with a median interrater
reliability of 90%.
Table 2 shows the 2000-2001 performance and change from baseline for
each indicator in each state. Across the
1144 pairs of baseline vs re-measurement comparisons (ie, 52 states and territories ⫻ 22 indicators), absolute increases in performance occurred in 81%
(925/1144) of the observations (␹21 =
240.8; P⬍.001). For all 22 indicators,
state performance at baseline predicted performance at follow-up, generally quite powerfully (median r=0.74;
range: 0.29-0.98). A state’s average rank
on the 22 indicators was highly stable
over time (r = 0.93 for 1998-1999 vs
2000-2001). For all but one indicator,
absolute improvement was greater
when performance was low at baseline than when it was high at baseline
(median r=−0.43; range: 0.12 to −0.93);
a similar pattern occurred for state performance as measured by performance on the median indicator in the
state (r, −0.30) and for indicator performance as measured by the median
state’s performance (r, −0.43).
TABLE 3 shows summary statistics for
each indicator for the country as a
whole. The performance of the median
state as well as the weighted national
average improved on 20 of the 22 indicators (all but use of angiotensinconverting enzyme inhibitors in heart
failure and performance of blood culture prior to starting antibiotics in
pneumonia). Performance in the
median state on the median indicator
was 69.5% appropriate care in 19981999 and 73.4% in 2000-2001; the
median absolute improvement was
3.9%, and the median relative
improvement was 12.8%. The average relative improvement was 19.9%
for outpatient indicators combined
and 11.9% for inpatient indicators
combined (P⬍.001).

(Reprinted) JAMA, January 15, 2003—Vol 289, No. 3 309

QUALITY OF MEDICARE HEALTH CARE DELIVERY

Table 3. National Summary of Quality Indicators and Changes, 1998-1999 to 2000-2001*
Inpatient Setting

Any Setting

Congestive
Heart Failure

Pneumonia

Breast
Cancer

Diabetes

55

70

68

65

60

78

70

60

Weighted average
2000-2001

84

84

68

78

71

38

50

114

71

66

57

83

99

85

84

81

24

23

71

64

77

70

74

16

3

2

6

7

4

3

4

−19

5

−4

3

2

4

2

7

−2

9

11

5

10

5

8

1

16

2

1

6

7

0

0

9

−17

8

−2

4

3

5

2

8

−2

9

12

5

10

4

9

1

17

Median relative†

15

14

17

28

10

5

NA

NA

14

−10

7

12

77

10

32

−9

10

12

16

22

11

29

4

38

Weighted average
relative

10

6

17

23

1

0

NA

NA

22

−6

8

13

78

12

34

−9

11

13

14

22

10

28

3

40

Improvement
Median
Weighted

Nifedipine

Lipid Profile

55

72

Eye Exam

67

24

HgbA1c

11

27

Mammography

14

82

Pneu Imm

82

85

Flu Immun

79

87

Pneu Screen

85

99

Flu Screen

95

84

Blood Culture

83

57

Antibiotic Rx

55

68

Antibiotic Time

69

70

Antithrombotic

65

107

Afibrillation

120

45

ACEI in HF

41

43

LVEF

40

74

Smoking

71

79

ACEI in AMI

72

69

BB Disch

64

86

BB 24 h

85

85

Aspirin Disch

84

2001-2000

Variables
Median state rate
1998-1999

Aspirin 24 h

PTCA, min

Stroke

Thrombolysis, min

Acute Myocardial Infarction

Adult
Immunization

74

SD
1998-1999

5.0

5.7

9.1 10.2

7.7 11.6

9.7

93.0

10.0

8.4

6.3

4.7

3.5

7.6

5.4

5.1

8.2

5.2

5.9

7.0

4.6

8.4

5.9

2000-2001

4.9

5.6

7.9

8.7

7.7

9.6 17.8

41.4

9.2

8.2

6.5

4.4

1.3

6.3

3.4

5.3

8.5

8.4

6.3

6.6

4.7

6.6

6.1

5.3

.73

.71

.74

.54

.41

.31

−.07

.90

.41

.71

.83

.58

.94

.64

.78

.56

.48

.75

.77

.96

.94

.98

.84

−.41

−.56

−.35 −.39 −.93

−.27

−.42

−.07

−.71 .12 −.58

Correlation of 1998-1999
with 2000-2001

.29

Correlation of improvement −.39 −.41 −.52 −.60 −.53 −.68 −.27 −.93
with baseline

−.60 −.79 −.26 −.44 −.16

6.5

*See Table 1 for explanation of abbreviations.
†Relative improvement is calculated as improvement/(100 − baseline improvement).

Figure 1. State Ranking on Provision of Appropriate Care, 2000-2001

DC

Quartile Rank
1
2
3
4
States are ranked according to their average performance across indicators in 2000-2001.

FIGURE 1 shows the national pattern of performance in 2000-2001
(follow-up). As in the previous report
on 1998-1999, better performance is
concentrated in northern states and less
populous states. FIGURE 2 shows the
pattern of relative improvement. Geographic trends are similar but less
marked than for follow-up performance.
310

COMMENT
We believe this is the first national study
to show improvement in quality of care
over time for multiple conditions in inpatient and outpatient settings. However,
these quality indicators give a somewhat unbalanced picture of Medicare services. They overrepresent inpatient and
preventive services, underrepresent
ambulatory care, and represent very few

JAMA, January 15, 2003—Vol 289, No. 3 (Reprinted)

interventional procedures. This study is
also generally limited to care delivered
in fee-for-service Medicare. Nationally,
about 85% of Medicare beneficiaries are
cared for under fee-for-service care and
about 15% under managed care, but in
Arizona, California, Florida, and Pennsylvania more than 25% of beneficiaries are enrolled in managed care. Comparing Health Employer Data and
Information Set (HEDIS) data from managed care with this fee-for-service Medicare data presents technical problems
that we have not yet solved for these measures, but HEDIS data for managed care
demonstrate similar trends.7 Furthermore, because of technical challenges
such as risk adjustment, we focused on
measuring processes of care critical to
outcomes rather than on measuring outcomes themselves.
Growing national alarm over unrealized opportunities to improve care has
been accompanied by a significant improvement in care, although far more
remains to be done than has been accomplished. The improvement reported herein is consistent with the
goals of the Medicare QIO program,
which has performance-based con-

©2003 American Medical Association. All rights reserved.

QUALITY OF MEDICARE HEALTH CARE DELIVERY

tracts with QIOs to achieve precisely
these kinds of improvement.8 The QIO
program has created the performance
measurement system that tracks
progress on these topics and has dramatically heightened national awareness of the opportunity for improvement. However, these cross-sectional
data do not provide conclusive information about the degree to which the
improvement can be attributed to the
QIOs’ quality improvement efforts.
There is evidence that QIO interventions can cause improvement,9 but the
effort during the period of this study
was national, with no control group,
and the strong emphasis on partnerships for improvement makes isolating the contribution of the QIO program almost impossible. Indeed, using
a clinical model to conduct research that
will prove linkages between interventions (such as fail-safe systems) and improved quality faces many of the same
difficulties as using a clinical research
model to study many aspects of patient safety.10 Nor does current evidence allow us to estimate how much
of the improvement reported herein
may be attributed to heightened awareness of specific clinical treatments and
how much may be attributed to changes
in health care systems.
Ten years ago, Rogers et al11 and Kahn
et al12 reported an improvement in quality of inpatient care for Medicare beneficiaries with 5 conditions during the
mid 1980s. Our study suggests that this
trend continues and is broader. However, despite this evidence, a wide gap
remains between the care that could be
delivered and the care that is delivered
to Medicare beneficiaries. In part the explanation for this discrepancy is that the
diffusion of standards of care is relatively slow, that new standards are developed continually, and that the performance gap is very wide compared
with progress. The greatest improvements in inpatient care were (1) prescription of ␤-blockers for patients with
acute myocardial infarction at discharge, (2) delivering antibiotics within
8 hours of reaching the hospital for patients with pneumonia, and (3) avoid-

Figure 2. Median Relative Improvement in the Provision of Appropriate Care

DC

Quartile Rank (Range)
1 (17.1% to 22.4%)
2 (12.3% to 16.9%)
3 (10.7% to 12.0%)
4 (5.6% to 9.8%)
Relative improvement is calculated as improvement/(100 − baseline improvement).

ing the administration of sublingual nifedipine to patients with acute stroke.
Yet, in 2000-2001, 21% of patients with
myocardial infarction and without contraindication to ␤-blockers were still discharged without a prescription and 13%
of patients with pneumonia still waited
more than 8 hours for antibiotics. By
contrast, the number of patients receiving sublingual nifedipine dropped by
77% to about 1%, and the measure has
been dropped from QIO contracts because so little opportunity for improvement remains. Growing evidence suggests that improvement and adoption of
best practices is limited or promoted by
the systems within which care is delivered and that we cannot close those gaps
unless we change the systems.3 Although it is risky to generalize from these
few examples, it seems intuitive that
changing the system to prevent doing
something risky would be easier than
changing it to do something of potential benefit both reliably and promptly.
Centers for Medicare & Medicaid Services is dropping stroke from the QIO
contracts because there seems to be little
further systemic improvement to be
achieved on use of sublingual nifedipine and because clinically valid abstraction of eligibility for warfarin use in
patients with atrial fibrillation is very
difficult.
Centers for Medicare & Medicaid Services will be adding 3 indicators related

©2003 American Medical Association. All rights reserved.

to patient safety in the inpatient setting: use of appropriate antibiotics for
prophylaxis against surgical infection,
appropriate timing of the administration of those antibiotics, and appropriate discontinuation after surgery.13,14
Centers for Medicare & Medicaid Services and the Joint Commission on
Accreditation of Healthcare Organizations have modified their performance
indicators to make them virtually identical for areas that both organizations
cover. Quality Improvement Organizations will also extend their work to
improving performance on quality indicators for both nursing homes and home
health agencies. The National Quality
Forum endorsed a group of indicators
for hospitals in 200215 and is scheduled to endorse additional hospital measures, as well as nursing home measures, in 2003. Quality Improvement
Organizations will also be working to
help hospitals collect their own data,
with the hope that those hospitals will
soon decide to publish their performance data.16 The health care system
still urgently needs systems that will
help it to keep up with change and needs
partnerships among those who support quality improvement to move it
forward more rapidly.17
The findings of this study are encouraging in showing that improvement is possible and is taking place.
They should not lead to complacency:

(Reprinted) JAMA, January 15, 2003—Vol 289, No. 3 311

QUALITY OF MEDICARE HEALTH CARE DELIVERY

there is still a very long way to go, and
medicine is changing at least as fast as
our progress in implementing what was
the standard of care just a few years ago.
Author Contributions: Study concept and design:
Jencks, Huff, Cuerdon.
Acquisition of data: Jencks, Huff, Cuerdon.
Analysis and interpretation of data: Jencks, Huff,
Cuerdon.

Drafting of the manuscript: Jencks, Cuerdon.
Critical revision of the manuscript for important intellectual content: Jencks, Huff, Cuerdon.
Statistical expertise: Huff, Cuerdon.
Obtained funding: Jencks.
Administrative, technical, or material support: Huff,
Cuerdon.
Study supervision: Jencks, Cuerdon.
Funding/Support: All funding for this work was provided by the Centers for Medicare & Medicaid Services.
Disclaimer: The opinions herein are the authors’ and

not necessarily those of the Centers for Medicare &
Medicaid Services.
Acknowledgment: We especially thank Joyce V. Kelly,
PhD, who coordinated the national PRO quality improvement efforts and Jeffrey Kang, MD, MPH, without whom this work would not have been possible.
We also thank Dale Burwen, MD, Barbara Fleming,
MD, Peter Houck, MD, Annette Kussmaul, MD, David
Nilasena, MD, and Diane Ordin, MD, for their leadership on the individual clinical topics and Susan Arday, PhD, for support of the immunization survey.

ance. Available at: http://www.ncqa.org/somc2001
/KEY/SOMC_2001_key.html#patterns. Accessed
December 12, 2002.
8. Quality Improvement Organization resource page.
Centers for Medicare and Medicaid Web site. Available at: http://www.cms.hhs.gov/qio/default.asp. Accessed December 12, 2002.
9. Marciniak T, Ellerbeck EF, Krumholz HM, Radford
MJ, Vogel RA, Jencks SF. Improving the quality of care
for Medicare patients with acute myocardial infarction: results from the cooperative cardiovascular project.
JAMA. 1998;279:1351-1357.
10. Leape LL, Berwick DM, Bates DW. What practices will most improve safety? JAMA. 2002;288:501507.
11. Rogers WH, Draper D, Kahn KJ, et al. Quality of
care before and after implementation of the DRGbased prospective payment system: a summary of effects. JAMA. 1990;264:1989-1994.
12. Kahn KJ, Keeler EB, Sherwood MJ, et al. Comparing outcomes of care before and after implementation of the DRG-based prospective payment system. JAMA. 1990;264:1984-1988.

13. Mangram AJ, Horan TC, Pearson ML, Silver LC,
Jarvis WR, for the Hospital Infection Control Practices Advisory Committee. Guideline for prevention of
surgical site infection, 1999. Infect Control Hosp Epidemiol. 1999;20:250-278.
14. Burke JP. Maximizing appropriate antibiotic prophylaxis for surgical patients: an update for LDS Hospital, Salt Lake City. Clin Infect Dis. 2001;33(suppl 2):
S78-S83.
15. Hospital Care National Performance Measures
(Group 1) - Interim report, 2002, Washington, DC: the
National Forum for Healthcare Quality Measurement
and Reporting, National Quality Forum. Available at:
http://www.qualityforum.org/txhospGrp1publicweb
.pdf. Accessed December 18, 2002.
16. Brown D. Hospitals will be rated on their
performance. Washington Post. December 13,
2002:A1.
17. Sprague L. Contracting for Quality: Medicare’s
Quality Improvement Organizations NHPF Issue Brief.
Washington DC: George Washington University; June
3, 2002. National Health Policy Forum. No. 774. Available at: http://nhpf.org/cfm2/results.cfm.

REFERENCES
1. US Department of Health and Human Services.
Healthy People 2010, Understanding and Improving
Health. Washington DC: US Government Printing Office; 2000.
2. Kohn LT, Corrigan JM, Donaldson MS, ed. To Err
Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999.
3. Committee on Quality of Health Care in America.
Crossing the Quality Chasm. Washington, DC: National Academy Press; 2001.
4. Jencks SF, Cuerdon T, Burwen DR, et al. Quality
of medical care delivered to Medicare beneficiaries: a
profile at state and national levels. JAMA. 2000;284:
1670-1676.
5. Behavioral Risk Factor Surveillance System resources page. Centers for Disease Control and Prevention Web site. Available at: http://www.cdc.gov
/brfss. Accessed December 12, 2002.
6. Huff ED. Comprehensive reliability assessment and
comparison of quality indicators and their components. J Clin Epidemiol. 1997;50:1395-1404.
7. The State of Managed Care Quality, 2001. Washington, DC: National Committee on Quality Assur-

True science teaches, above all, to doubt and to
be ignorant.
—Miguel de Unamuno (1864-1936)

312

JAMA, January 15, 2003—Vol 289, No. 3 (Reprinted)

©2003 American Medical Association. All rights reserved.