CDC 50.42A Adult HIV/AIDS Confidential Case Report

Attachment M. Statistical Consultation Summary

HIV Incidence Estimation Consultation
June 15–16, 2006
Corporate Square, Building 8, Conference Room 1 B/C
Meeting Notes
June 15
Welcome
Introductions
Charge to the Group

Tim Green
Tim Green

Main Objectives
1. Examine the validity of the HIV incidence estimation method proposed by Karon, Song,
Kaplan, and Brookmeyer, if the necessary information can be gathered, with respect to
a. Stochastic uncertainty
b. Bias (violation of assumptions)
c. Performance characteristics of the assay
2. Identify other methods of HIV incidence estimation.
3. Determine what sort of an estimate can be produced by December 2006 for calendar year
2005.
Overview of Estimation Procedure
1. Background
2. Sample survey approach
3. Potential problems and points for discussion
4. Assumptions about procedure

John Karon

Discussion of Karon model
1. Stochastic uncertainty
a. Variances have been worked out for several of the terms that involve uncertainty
(e.g., Io, I1, IB).
b. One big source of variability is the uncertainty in the multiplier in equation 5 of the
Statistics in Medicine submission (i.e., the coefficient of variation of φ). The estimate
is very sensitive to the estimate of the mean window period.
2. Bias and assumptions
a. Incidence and testing patterns (i.e., hazard, or instantaneous risk, of being tested)
remain constant during AIDS incubation period (steady state).
Problem: These assumptions are quite strong.
Question: On the basis of these assumptions, can we use simpler models to arrive
at a similar estimate?

HIV Incidence Estimation Consultation—Page 2

b. All data are complete and accurate (including results of the assay and all responses to
supplemental questions about testing history).
Problem: Realistically, we can only expect about 50% of assay results and an even
smaller proportion of data on testing history. Missing data could be a
big problem and needs further evaluation.
c. The window period distribution is valid.
Problem: The window period distribution for the BED assay hasn’t been fully
validated.
d. Testing characteristics of persons who avoid testing (i.e., test-resistant) differ from all
others at risk for testing and must be stratified accordingly.
Problem: Conscious delay of testing may not be an accurate reflection of
infection, especially if motivation for testing is not related to infection
(i.e., we must assume a rational motive in both directions: early [soon
after person becomes infected] and delayed testing).
3. Testing history information
a. We do not currently use motivation to classify individuals. Supporting information
and justification is available from HICSB on request. We will test the sensitivity of
the estimate to this assumption.
b. We do not need individual-level information on testing frequency.
c. We can obtain subgroup information from NHBS to determine the intertest
distribution.
Problem: NHBS surveys only populations at high risk.
Question: Is it possible to survey HIV-negative persons who receive HIV
counseling and testing services (CTS)?
d. We will consider incorporating into simulations the increased risk of testing soon
after infection.
e. We have not resolved whether it is necessary to stratify according to whether
individuals consciously delay testing.
i. The proportion of persons with AIDS who have not had a previous HIV test
seems to be stable. A proportion of this population might be test-resistant.
ii. It may be possible to evaluate test-resistance by using testing frequency
information from various populations (e.g., incidence data, NHBS, HITS,
SHAS, BRFSS, NHANES, NHIS). We need to investigate whether these data
sets are useful for us based on the available data variables and sample sizes.
4. General criticism
a. The group seemed to consider the mathematics valid but to believe that some of the
steady state assumptions may be too strong.
b. Some participants felt that because of the steady state assumptions, this approach uses
a sophisticated weighting scheme to essentially produce an estimate of the total
number of new diagnoses—information that could be obtained more directly from
surveillance data.

HIV Incidence Estimation Consultation—Page 3

Status of Implementation
1. Data collection
Maria Rangel
2. Data completeness
Rick Song
a. Dr. Rangel highlighted the collection of data on testing history and the collection of
blood specimens for BED testing.
b. Dr. Song presented preliminary surveillance data, with emphasis on completeness of
data collected during 2005. Of the roughly 29,000 new cases diagnosed during 2005,
i. More than 80% of reports of cases of HIV infection (not AIDS) were missing
data on the most recent negative test result.
ii. Only 13% of cases of HIV infection were BED tested.
iii. 24% of cases of HIV infection received a diagnosis of AIDS within 1 month
after receiving a diagnosis of HIV infection.
c. In response to a request, Dr. Rangel presented the testing frequency distribution
among cases diagnosed during 2005.
Review of Estimators
1. Estimators for persons with a previous negative test result of known date
2. Estimators for persons not previously tested
3. Estimators for persons whose BED test was delayed
4. Extension of estimators to persons for whom data were missing
a. Stratification
b. Propensity scores
c. Incorporation of patterns of missing data into simulations

John Karon

HIV Incidence Estimation Consultation—Page 4

June 16
Other Estimators and Supplemental Approaches
1. Naïve estimator: Number of newly diagnosed cases scaled to the national population
(accounting for persons who will never get tested)
Problem: Assumes that new diagnoses = new infections.
2. Back-calculation model developed by Rhodes and Glynn
3. Simplified STARHS estimator:
The following proposal, although less mathematically sophisticated than the Karon et al.
proposal, is much simpler. It is based on, but different from, I0 + IB.
Is = (# STARHS-recent) ÷ [P(1)*P(2)*P(3)], where
P(1) = P(test HIV+ within 1 year | newly HIV infected)
I.e., the probability that a person will be tested within 1 year after infection (stratified
by subgroup).
P(2) = P(STARHS administered | test result HIV+)
I.e., the probability that this person receives a STARHS test. This estimate can be
obtained from incidence surveillance data (13% for 2005). The number varies,
depending on geographic location as well as testing location.
P(3) = P(detected during window period | STARHS administered within 1 year after
HIV infection)
I.e., the probability the infection will be detected during the window period if
STARHS is administered within 1 year after the person becomes infected.
Discussion:
a. Local areas may be able to use this estimator to compute local estimates.
b. Like IB, the information on testing behavior does not have to be linked to individual
test results. Thus, several estimates can be obtained for each component of the
estimator and combined to obtain a range of credible estimates for HIV incidence.
c. Example.
Number STARHS-recent = 757. This is the current number reported to HIV
incidence surveillance for 2005.
P(1) = 0.48 = 252/527. This is the proportion of newly diagnosed cases with at
least 2 HIV tests during the 2 years before diagnosis, based on the data from the
pretest questionnaire for incidence surveillance. Because this proportion does not
include persons who had never tested before or who did not respond, it is likely to
be inflated.
P(2) = 0.13. This is the proportion of incidence surveillance cases that were
STARHS tested in 2005.
P(3) = 0.42. This is based on the mean window period = 5/12.

HIV Incidence Estimation Consultation—Page 5

d. Ways to obtain a better estimate of P(1).
i. Use incidence data to estimate P(1) among HIV+ persons.
ii. Use NHBS data to estimate P(1) in a population at high risk for infection.
iii. Use BRFSS or other sources (see list above) to estimate P(1) in a general
population.
e. Seattle data on intertest intervals are linked with results from an incidence assay
(probably Abbott).
f. Try to verify the assumption that P(1) does not actually depend on infection status or
how recently one was infected once membership in a population (e.g., MSM, IDU, or
heterosexual adults or adolescents at high risk) is accounted for.
4. Direct (back of the envelope) estimator
a. Use incidence rate and population size data on MSM in NYC.
b. Scale to all transmission categories and national population.
5. Alternative approaches and issues
a. To detect trends in incidence, monitor the number of new infections as a proportion
of the total number of infections diagnosed over time.
b. Obtain information on the total number of persons tested during a given period. This
would require information on the number of negative test results and an adjustment
for repeat testing among both positive and negative individuals—information is
generally available only on the number of test kits distributed or the number of tests
performed rather than on the number of persons tested.
c. Obtain the testing frequency in a general population by surveying persons with a
negative test result at CTS sites.
d. Test the independence between the proportion recently tested and the proportion BED
tested by site or facility (there should be no association).
e. Evaluate uncertainty, consistency, and plausibility. Possibly convene an expert
working group.
f. Produce plausible ranges and lower and upper bounds for N and for large subgroups.
g. Compare 2005 estimates with historical estimates for the mid-1990s.
h. Evaluate window period estimates.
Questions: Can better estimates of the window period be obtained? Very few data
are available on people who have been infected more than 3 years.
What about those who remain STARHS-recent even after a long time?
Are incidence trends robust to STARHS results that falsely indicate
recent infection (i.e., false-recents)?
i. Investigate whether the probability of being tested within a year after becoming HIV
infected is higher than the probability of being tested before infection.
j. Investigate whether our sample of HIV+ persons whose specimens have been
subjected to STARHS is biased because the early implementation of STARHS has
been mostly at public testing sites.
k. Estimate the proportion/number of HIV+ persons determined only by AIDS
diagnosis.

HIV Incidence Estimation Consultation—Page 6

l. Investigate what might happen if testing behavior changes.
m. Produce subgroup estimates.
i. age (young adults 18–25, …)
ii. sex
iii. race/ethnicity
iv. transmission category (male-to-male sexual contact, injection drug use, highrisk heterosexual contact)
Estimates of Window Period Distribution
Bob Byers
Need to Adjust for Persons with Very Long Window Periods
Meade Morgan
No recommendations were made to adjust the window period or formally incorporate
adjustments for false-recents into the BED results.
Next Steps
1. Have draft report of 2005 estimates ready for internal review in 3 months (30 Sep 2006);
have final report by 31 Dec 2006.
2. Convene groups to work on each of the estimation methods suggested. Each approach will
incorporate data from multiple sources and will account for bias as well as variability.
DHAP Principals
Timothy A. Green, Chief, Quantitative Sciences and Data Management Branch (QSDMB)
Irene Hall, Lead (Acting), HIV Incidence and Viral Resistance Team (IVRT), HIV Incidence
and Case Surveillance Branch (HICSB)
Susan Hariri, Epidemiologist, IVRT
John Karon, Emergint Corporation (contractor to DHAP/QSDMB)
Lillian Lin, Lead, Statistical Science Team (SST), QSDMB
Matthew McKenna, Chief, HICSB
Maria Rangel, Epidemiologist, IVRT
Philip Rhodes, Mathematical Statistician, QSDMB
Ruiguang Song, Mathematical Statistician, SST, QSDMB
External Consultants
Ron Brookmeyer, Johns Hopkins University Bloomfield School of Public Health
Stephanie Broyles, Louisiana State University Health Sciences Center
Bob Byers, CDC (retired)
Jason Hsia, Division of Reproductive Health, NCCDPHP, CCHP, CDC
Ronaldo Iachan, ORC Macro
Ed Kaplan, Yale University School of Management
Meade Morgan, Global AIDS Program, NCHHSTP, CCID, CDC
Sally Morton, RTI International
Phil Rosenberg, National Cancer Institute
Glen Satten, Division of Reproductive Health, NCCDPHP, CCHP, CDC
Ping Yan, Public Health Agency of Canada