Attachment 3 - Quantitative Evaluation Protocol_012808

Health Research and Educational Trust (HRET)/The Alliance Health Centers

IMPROVING QUALITY THROUGH HEALTH IT: TESTING THE

FEASIBILITY AND ASSESSING THE IMPACT OF USING EXISTING HEALTH IT INFRASTRUCTURE FOR

BETTER CARE DELIVERY

Contract No. HHSA290200600022, Task Order No.3

Quantitative Evaluation Protocol

Using quantitative methods we will measure the impact of health IT tools on compliance with evidence-based guidelines for lab tests for HIV patients and women screened for cervical cancer. In addition, we will measure the impact of health IT on the numbers of duplicate lab tests and results lacking follow-up

To assess the impact of the health IT tools we will gather lab data from three points in time at two Alliance Health Centers. Data taken from six months prior to the implementation of the electronic health record system (EHRS) will be compared to data six and 12 months post implementation. For HIV patients, we also will assess the impact of health IT by comparing patients for whom providers used the chronic disease management form and those whose providers did not. The approximate number of patients meeting our inclusion criteria for each population are described in Table 1.

Table 1. Sample Sizes

Alliance Partner	HIV	Women 21-65
Heartland Health Outreach	1,600	3,484
Howard Brown Health Center	2,055	1,092
Total N	3,655	4,576

To assess guideline compliance, we will measure the percentage of patients compliant with the lab protocols described in Table 2. To examine redundant lab tests we will measure the frequency of Viral Load tests for the same patient within a seven day period. This test was selected both because of its clinical importance in the management of HIV and because of its high cost. On average the viral load test costs $160 with a range of $100-250. To assess problems with lab test follow-up we will measure the number of patients with abnormal Pap smears without a subsequent Pap smear or colposcopy within a 3-5 month period. These additional measures are listed in Table 3.

Table 2. Lab Test Guideline Compliance Measures

	Patients with HIV		Frequency
Viral Load CD4 count Lipid Profile (HDL, LDL, Triglyceride, and Cholesterol) Hep B antibodies Hep A antibodies Hep C antibodies RPR (syphilis screen) Liver function tests (basic metabolic panel) Toxoplasmosis titer		3 months 3 months 1 year Once or until immune Once or until immune Once Once 3 months–1 year (depending on medication regimen) At diagnosis
Women
Pap smear		1 year

Table 3. Additional Measures

Patients with HIV	Measurement Area
2 or more Viral Load tests within a 7 day period	Duplicate lab tests
Women
Abnormal Pap smear without a follow-up Pap smear or colposcopy within 3-5 months.	Lab results lacking of follow up

In analyzing the data we also will investigate whether disparities exist in lab screening rates and abnormal labs without follow up in vulnerable populations. We are specifically interested in whether the use of health IT tools reduces disparities in these areas.

Data Extraction Specification

The following describes the specific criteria and data fields required for the quantitative analysis. For both the HIV and Pap smear measures identical datasets from three separate time periods are needed. The first time period is six months prior to implementation of the EHRS in each site followed by six and 12 months post implementation. Howard Brown implemented in October 2006 and Heartland in November 2006.

Inclusion and Exclusion Criteria

HIV positive patients

Include:

• All patients with HIV (ICD9 of 042.XX or V08)

• 12 years of age at the start of the measurement period

• At least one office visit within the measurement year

Exclude:

• Patients incarcerated for 60 days or more during measurement period (use Patient Status obs term is “inactive”).

• Patient has expired during measurement period (use Patient Status obs term is “expired”).

• Patient has relocated or left the clinic during measurement period (use Patient Status obs term is “inactive”).

Women in need of cervical cancer screening

Include

• All women over 20 years of age at the start of the measurement period

• At least one office visit within the measurement year

Exclude:

• Patients incarcerated for 60 days or more during measurement period (use Patient Status obs term is “inactive”).

• Patient has expired during measurement period (use Patient Status obs term is “expired”).

• Patient has relocated or left the clinic during measurement period (use Patient Status obs term is “inactive”).

Data Elements

For each patient meeting the inclusion criteria, the data elements described in Table 4 will be extracted.

Table 4. Data Fields to be Extracted

Measurement Specification

Table 5 provides specification for the measures that will be used in the quantitative analysis.

Table 5. Measure specifications

HIV Metrics	Measure Details	Notes/Issues
1. Viral load every 6 months	Percentage of active HIV patients/clients who have a viral load test at least every 3 months Numerator: Number of active clients who had viral loads measured at least twice in the measurement year, >30 days apart and < 3 months apart. There is no requirement on visit type (e.g., does not have to be a medical visit), but visit does have to occur. Denominator: Number of active HIV patients/clients who were seen at least once in the measurement year.
2. CD4+ test every 3 months	Percentage of active HIV patients/clients who have a CD4+ test done at least every 3 months Numerator: Number of active clients who had CD4+ counts measured at least twice in the measurement year, >30 days apart and < 3 months apart. There is no requirement on visit type (e.g., does not have to be a medical visit), but visit does have to occur. Denominator: Number of active clients who were seen at least twice within the measurement year, > 30 days apart and < 3 months apart.	Use obs term CD4 COUNT.
3. Lipid Profile (HDL, LDL, Triglyceride, and Cholesterol)	Percentage of active HIV patients/clients who have a Lipid profile done at least every year. Numerator: Number of active clients who had Lipid profile (HDL, LDL, Triglyceride, and Cholesterol) measured at least once in the measurement year. There is no requirement on visit type (e.g., does not have to be a medical visit), but visit does have to occur. Denominator: Number of active clients meeting HIV inclusion criteria.
4. Hep B antibodies	Percentage of active HIV patients/clients who have been screened for Hep B. Numerator: Number of active patients who had a lab test for Hep B antibodies at any point in time or has a record of a vaccination. Denominator: Number of active clients meeting HIV inclusion criteria.
5. Hep A antibodies	Percentage of active HIV patients/clients who have been screened for Hep A. Numerator: Number of active patients who had a lab test for Hep A antibodies at any point in time or has a record of a vaccination. Denominator: Number of active clients meeting HIV inclusion criteria.
6. Hep C antibodies	Percentage of active HIV patients/clients who have been screened for Hep C. Numerator: Number of active patients who had a lab test for Hep C antibodies at any point in time. Denominator: Number of active clients meeting HIV inclusion criteria.
7. RPR screen	Percentage of active HIV patients/clients who have been screened for syphilis. Numerator: Number of active clients who have at least one RPR screen. Denominator: Number of active clients meeting HIV inclusion criteria.
8. Liver function tests (basic metabolic panel)	Percentage of active HIV patients/clients who have had a liver function test every 3 months Numerator: Number of active patients who had a liver function test every 3 months. Denominator: Number of active clients meeting HIV inclusion criteria.	Some contextual, population level data on patient medication regimens will be provided.
Toxoplasmosis titer	Percentage of active HIV patients/clients who had a toxoplasmosis titer at time of diagnosis. Numerator: Number of active patients with a toxoplasmosis titer at any point in time. Denominator: Number of active clients meeting HIV inclusion criteria.
Duplicate Viral load tests	Number of instances where 2 or more viral load tests were ordered for the same active HIV patient/client within 7 days. Numerator: Number of instances where 2 or more viral load tests were ordered for the same active HIV patient/client within 7 days. Denominator: Number of Viral load tests ordered for all active HIV patients/clients during the measurement year.
Women Metrics	Measure Details	Notes/Issues
Pap smear	Percentage of women who have a Pap smear done every year. Numerator: Number of active patients/clients who had at least one Pap smear during the measurement year. Denominator: Number of active clients meeting the inclusion criteria.
Abnormal Pap smears lacking follow up	Percentage of abnormal Pap smears without a follow-up within 5 months. Numerator: Number of active female clients who had an abnormal Pap smears within the measurement year without a subsequent office visit related to the abnormal Pap within 5 months. Denominator: Number of active female clients who had an abnormal Pap smear within the measurement year.

These measure specifications are already programmed into the CDW and are used to calculate aggregate rates. We will use these aggregate rates to define the population for each study time period. The Alliance will provide us with patient-level data for patients included in these measures for further analysis.

Data Analysis

Guideline compliance

Data will be analyzed using SAS, version 9.1 (Sas Institute, Cary, NC) and SPSS, version 13.0 for Windows. Initial data exploration will include examination of variable ranges, means, medians and distributions and assessment of bivariate correlations between variables.

To evaluate the impact of HIT on lab guideline compliance, for each of the ten measures listed in Table 3 we will calculate the number and percentage of patients that are not compliant with the guideline at each of the three time intervals. Subsequently we will compare the levels of guideline compliance for each measure using several statistical tests. First, we will use difference of means tests (independent and paired t tests) to see if there are statistically significant differences (p<0.05) before and after implementation of the EHRS. We will also examine differences in levels of compliance between Centers and between providers using and not using the HIV disease management form. Finally, multivariate logistic regression will be used to predict the effect of the various organizational and provider characteristics, including Center, presence of the EHRS, provider type, and provider use of HIV form, on guideline compliance for each measure. A model which also includes patient characteristics, such as race/ethnicity, age, gender, primary language and socioeconomic status, will be developed.

To assess whether disparities exist in compliance levels in vulnerable populations we will identify two groups of patients at each time interval: those compliant on all appropriate measures and those not compliant on all measures. We will compare these two groups along various patient characteristics including race/ethnicity, age, gender, primary language and socioeconomic status indicators at each of the three time intervals. Students t-tests (independent and paired) will be used for the continuous variables and Chi-square tests for the categorical. Multivariate logistic regression will be used to predict the effects of these patient characteristics and the presence of the EHRS on lab guideline compliance. An additional model which predicts the effects of both organizational and patient characteristics and presence of the EHRS on lab guideline will be created.

Tables of results reported will include descriptive statistics, correlation matrices, t tests, Chi-square results and odds ratios and confidence intervals.

Duplicate Labs

A similar set of analyses will be conducted to assess the impact of HIT on level of duplicate labs. At each time interval we will calculate the number of instances when a viral load test for the same patient was conducted within a seven day period. We will also calculate the percentage the duplicate labs represent of total viral load tests during the time interval. To assess the impact of the EHRS, we will again use t tests (independent and paired) to assess whether there is a statistically significant difference (p<0.05) in the percentage of duplicate labs before and after implementation of the EHRS. We will also examine differences between Centers and between providers using and not using the HIV disease management form. Finally, multivariate logistic regression will be used to predict the effect of the various organizational and provider characteristics, including Center, presence of the EHRS, provider type, and provider use of HIV form, on the number of duplicate viral load tests. A model which also includes patient characteristics, such as race/ethnicity, age, gender, primary language and socioeconomic status, will be developed.

Tables of results reported will include descriptive statistics, correlation matrices, t test results, odds ratios and confidence intervals.

Labs Lacking Follow-Up

To measure the impact of HIT on lab results lacking follow-up we will calculate the number of abnormal Pap smears without a follow-up Pap smear or colposcopy within 3-5 months for each time interval. We will also calculate the percentage of all abnormal Pap smears that lacked follow-up. To assess the impact of the EHRS, we will again use t tests (independent and paired) to assess whether there is a statistically significant difference (p<0.05) in the percentage of abnormal Pap smears lacking follow-up before and after implementation of the EHRS. We will also examine differences between Centers and by provider type. Multivariate logistic regression will be used to predict the effect of the various organizational and provider characteristics, including Center, presence of the EHRS, and provider type, on abnormal labs lacking follow-up. A model which also includes patient characteristics, such as race/ethnicity, age, gender, primary language and socioeconomic status, will be developed.

To assess whether disparities exist in the frequency of results lacking follow-up in vulnerable populations we will look at comparisons between those patients who had appropriate follow-up and those that did not along various patient characteristics including race/ethnicity, age, primary language and socioeconomic status indicators at each of the three time intervals. Students t-tests (independent and paired) will be used for the continuous variables and Chi-square tests for the categorical. Multivariate logistic regression will be used to predict the effects of these patient characteristics and the presence of EHRS on an appropriate follow up. An additional model which predicts the effects of organizational and patient characteristics and presence of the EHRS on appropriate follow-up will be created.

Tables of results reported will include descriptive statistics, correlation matrices, t test, Chi-square results, odds ratios and confidence intervals.

3