Part B-ADAM II 2009 Statistical Methods OMB Final

Cambridge, MA

Lexington, MA

Hadley, MA

Bethesda, MD

Chicago, IL

Durham, NC

Abt Associates Inc.

55 Wheeler Street

Cambridge, MA 02138

Arrestee Drug Abuse Monitoring (ADAM II)

Request for

OMB Review

Part B – Collection of

Information Employing

Statistical Methods

Date

December 30, 2009

Prepared for

Robert L. Cohen

Executive Office of the President

Office of National Drug Control Policy (Planning and Budget)

Washington, DC 20503

Prepared by

Sarah Kuck Jalbert

Dana Hunt, Ph.D.

Contents

Part B: Collection of Information Employing Statistical Methods 1

1. Respondent Universe and Sampling Methods 1

Sample Size Determination 1

Confidence Intervals for Point Estimates 2

Estimating Trends Beyond 2007 2

Annualizing Point Prevalence Estimates Beyond 2007 4

Sampling Sites and Facilities 5

Sampling Within A County 7

Sampling Method for Respondent Selection 7

2. Information Collection Procedures 9

Selecting Study Subjects 9

The Role of Census Data 10

3. Methods to Maximize Response Rates 13

4. Tests of Procedures 16

5. Individuals Consulted on Statistical Aspects of the Design 18

Part B: Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The Arrestee Drug Abuse Monitoring (ADAM) Program II is the continuation of ONDCP’s ADAM II program, a three year project designed to collect critical information about drug use among arrestees. The respondent universe will be booked male arrestees in the 10 “sentinel” counties designated as sites in the ADAM II program, which began in 2006. Data collection takes place in 10 counties from an annual, county-based representative sample of 250 arrestees (within 48 hours of arrest) per quarter, for an approximate total of 5000 arrestees. Collection will occur in two cycles at each site to provide estimates for two back-to-back calendar quarters in the time period from April 1, 2010 to September 31, 2011. If additional data collection periods are optioned by ONDCP, subsequent quarters of collection will occur annually between April 1 and September 31 ending in the fall of 2013. Data collected will include a voluntary and anonymous personal interview and urine specimen.

Sample Size Determination

The ADAM II program implemented a quarterly target for each site (250), which will be continued in the ADAM II program. This target yielded 4334 total interviews in 2007, 4592 in 2008, and 4746 interviews in 2009.

Because the trend estimates are based on a regression model that uses data from the four ADAM years and the three ADAM II years, the parameter estimates for the yearly trends are not independent, a fact that changed the method for development of confidence intervals. We describe the rationale below.

Let V represent the parameter covariance matrix for the year dummy variables in the logistic regression with dependent variable “testing positive for drug D”. If there were four years of data, the covariance matrix would be a symmetric 4x4 matrix:

The terms in this matrix represent the variances and covariances for the four parameter estimates pertaining to year dummy variables, and one of the four year dummy variables is the omitted category. The terms below are not shown for below the diagonal, because the matrix is symmetric.

Let B represent a row vector with the parameter estimates for the four year dummy variables:

Let P represent a second row vector that records the average probability of testing positive for each of the four years.

The difference between the parameter estimates for year i and year j has an approximately normal distribution with a sampling variance of:

This sampling variance can be used to test the null hypothesis that the probability of testing positive for drug D has changed between year i and year j.

Confidence Intervals for Point Estimates

In ADAM II the study team anticipates replicating the methodology used to develop confidence intervals in ADAM II. For ADAM II data, the delta method was used to estimate a confidence interval for a point estimate. This approach requires no new notation, because the discussion surrounding equations [3], [4] and [5] presented earlier already introduced suitable notation when explaining how to estimate the standard error for the confidence interval for testing positive for drug use. See equation [5].

Estimating the confidence interval for the estimate of testing positive for drug use conditional on a fixed set of covariates is actually a simplification. First, in equation [3], the probability of testing positive during a year for the average arrestee is estimated. The average arrestee is a hypothetical arrestee who has an average value on all variables that enter into the logistic regression used to estimate the probability of testing positive for a specified drug, except that the year dummy variable is not averaged. Second, the estimate for each year is computed for the average arrestee, allowing only the year variable to change from year to year. This approach is a simplification compared with equations [3], [4] and [5] because calculations require no summations. All calculations are specific to the average offender.

Estimating Trends Beyond 2007

From 2000 through 2003, ADAM used poststratification to estimate sampling probabilities and to calculate weights. Data were stratified by jail, stock and flow, and day of the week. Within each stratum, the sampling probability was estimated as the number sampled per number booked. Although conceptually simple, the approach was operationally difficult. The principal operational difficulty was that strata sometimes had no or few members of the sample. This meant that strata had to be merged, and it often resulted in heterogeneous strata being combined.

To avoid these complications, ADAM II adopted propensity scores as an alternative device for estimating sampling probabilities and computing weights. The propensity score approach does not require stratification, because the sampling probability can be modeled as a continuous function of factors that affect the sampling rate. As mentioned earlier, because 2000 and 2001 ADAM data provided the necessary census data, the survey team replaced the original weights for the 2000 and 2001 ADAM data with new weights based on propensity scores. That is, the survey team replicated the ADAM II weighting procedure using the 2000 and 2001 ADAM data.

This replication was not possible for the 2002 and 2003 ADAM data because the ADAM contractor did not retain the census data for those years. Thus, for purposes of reporting trend statistics, the ADAM II survey team:

• Uses the reweighted ADAM data for 2000 and 2001;

• Uses the ADAM data for 2002 and 2003 without changing the weights; and

• Uses the propensity score weights for the ADAM II data.

It is important to note that there was nothing wrong with the original ADAM weights. They simply led to sampling variances that were larger than necessary, so the ADAM II study team improved the weights when possible. Because there was nothing wrong with the original sampling weights, there is nothing misleading about mixing the reweighted data for 2000-2001, the 2002-2003 data with their original weights, and the new ADAM II data in producing trend estimates.

However, this reweighting has two consequences. The first is that the 2000- 2001 estimates changed slightly from those reported earlier. The second is that estimates from year-to-year in reweighted years are no longer independent. Consequently, to test for trends, an analyst requires an estimate of the parameter covariance matrix.

As anticipated, this has the result of potentially slightly changing the prior years’ estimates that appeared first in the 2007 report. Although this approach improves the efficiency of the estimates, there is concern that yearly revisions going forward, regardless how slight, would be confusing. Consequently, 2008 and 2009 estimates are developed holding earlier estimates at their previously reported levels.

In this procedure, the 2007 and earlier estimates for parameters and standard errors are treated as fixed for subsequent estimation. There are five steps for estimation procedures for 2008 and beyond data.

1. The first step uses the regression results that are part of the 2007 report. Recall from equation [2] that these are a function of the offense, the Fourier transformations, and the year dummy variables. All the θ parameters and the covariance matrix are retained for those θ parameters V_θ. For convenience, equation [2] is repeated as equation [7].

2. Following Bayesian logic, analysts sample the covariance matrix V_θ from an inverted Wishert distribution. Conditional on that sampled V_θ, analysts sample θ from a multivariate normal distribution.

3. Conditional on the θ in the previous step, a new regression is estimated with the specification described by equation [8]. Note that this regression has a single free parameter α.

The β could be estimated using just the 2008 ADAM II data, but using the entire set of ADAM II data is a programming convenience.

4. Steps 3 and 4 are cycled through 100 iterations. Each iteration provides a somewhat different estimate of β (100 β estimates β₁ through β₁₀₀) and somewhat different estimates of (100 estimates through ). The final estimate of β and are:

[9]

4. The above steps provide everything needed for trend estimation except the covariances between β and the θ7t. To estimate the covariance estimates for β and any selected θ, the formula is:

[10]

Given estimates of the variance and covariance for the parameters associated with the year dummy variables, the statistical significance of any pair of years can be tested.

Annualizing Point Prevalence Estimates Beyond 2007

Most of the statistics appearing in the ADAM II reports are point prevalence estimates. A point prevalence estimate is straightforward, because it only requires weighting the desired variable by the propensity score weights. The statistics reported in the 2007 ADAM II report use this estimator.

As mentioned above, in preparation for the 2007 ADAM II report, it was determined that the prevalence estimates should be annualized to account for the fact that the ADAM sample was collected at different times during the year (3 or 4 quarters versus 2 quarters in ADAM II). This complicates the estimation explained in the previous subsection.

Annualizing the prevalence estimates requires applying the same five steps as above, except that equation [7] is replaced with equation [11]:

F(…) is a link function that depends on the context. When the variable of interest is binary, for example, F(…) represents the logit. Also Z is simplified:

The Q are dummy variables representing quarters of the year. The λ parameters are estimated using data from before 2008. The β parameter is estimated using 2008 data. The parameter covariance matrix is estimated by following steps 1 through 5 from the previous subsection.

When making the estimates, each Q is set equal to 0.25 and each year before 2008 is set equal to 0. This gives an annualized estimate of every variable reported in the 2008 and beyond ADAM II reports except for estimates based purely on drug test results. For estimates based purely on drug test results, the formulation described in the previous subsection is used.

Because the sample sizes remain the same when testing for other drugs, the confidence intervals are not much different for other drugs. Using the comparison for testing positive during 2000/2001 and 2002/2003 for illustration, the confidence intervals are for cocaine, heroin, and for marijuana. We expect that the confidence intervals from Portland will mimic confidence intervals from other ADAM II sites that have collected comparable sample sizes. We note, however, that some of the ADAM II sites (Washington, D.C. for example) have fewer sampling periods and smaller samples, so comparing estimates for 2007 with earlier years will be less precise.

Sampling Sites and Facilities

Like the original ADAM sites, sites in ADAM II do not constitute a probability sample of counties. ADAM sites were originally selected by NIJ through a competitive grant process rather than in a sampling plan, and ten former ADAM sites have been selected for ADAM II data collectionseven east and three west of the Mississippi. The ten sites selected for this study will provide a sentinel system: If these ten sites reveal trends in drug use, those trends are likely to appear in other sites that are not part of the sentinel system. Consequently, ADAM II sites were selected to provide:

• Geographic spread, as trends in drug use are often regional;

• A focus on counties east of the Mississippi to monitor the emergence of methamphetamine use; and

• Consistent, biannual data collection points for three years prior to support statistical trend analysis.

ADAM II will continue to provide a basis for drawing probability-based estimates of prevalence and trends in drug use within the sentinel counties. Because, as in the original ADAM system, these counties are not a probability sample of all counties in the United States, it’s not possible to easily derive a probability-based estimate of trends in the U.S., or even of trends east of the Mississippi. The ten ADAM II sites will continue as ADAM II sites:

Manhattan Borough, NY (New York City)

District of Columbia (Washington, DC)

Mecklenburg County, NC (Charlotte)

Fulton County, GA (Atlanta)

Marion County, IN (Indianapolis)

Cook County, IL (Chicago)

Hennepin County, MN (Minneapolis)

Multnomah County, OR (Portland)

Santa Clara County, CA (Sacramento)

Denver County, CO (Denver)

Table 1.1 shows selection criteria used for determining ADAM II sites.

Table 1.1: ADAM II Site Selection Criteria
Site Name	Annual Arrests per 1,000 Residents1	Number of Male Booking Facilities	Number of Booking Facilities in Sampling Plan	Sampling Design	Number of quarters of ADAM Data Collection (2000-2003)	Census Data Format
Charlotte	40.8	1	1	Single	10	Electronic
Indianapolis	65.8	1	1	Single	15	Electronic
Chicago	463.3	12	1	Single	9	Electronic
Minneapolis	24.8	17	1	Stratified	14	Electronic
New York	183.8	2	1	Stratified	15	Electronic
Atlanta	74.6	2	2	Stratified	9	Electronic
Washington, DC	Not Reported	8	8	Stratified	6	Electronic
Denver	171.9	1	1	Single	15	Electronic (beginning 2008)
Sacramento	61.3	1	1	Single	15	Electronic
Portland	44.0	1	1	Single	15	Electronic

Sampling Within A County

Within each county sampling will be done from the total number of booking facilities in the county to create a representation of all persons arrested in that county in the two, two-week data collection periods for this project. The two week data collection periods is consecutive for all sites, starting in April 2010 and ending on September 30, 2010 for the first period and continuing during the same time frame annually for two additional years should the research options be exercised by ONDCP. The process of creating that sample involves developing a frame of all facilities in each county, tracking any movement of arrestees that may occur (transport prior to booking) and assembling data on the numbers of arrestees booked into each facility. Facilities are stratified by size of arrestee volume and random selection made within the strata.

Many counties have one central intake facility for booking all county arrestees; others have only a few and others have many facilities with a complex relationship to each other in terms of arrestee transfers. In identifying sites appropriate for the ADAM II data collection, the ease of implementing sampling methodology was taken into consideration. Jurisdictions with a complicated feeder jail system of booking and holding arrestees were considered less desirable, as were jurisdictions with numerous booking facilities. Two of the three sampling models developed for the original ADAM project will be implemented for ADAM II:

• Single booking facility

Arrestee cases will be sampled in the one central intake facility as described below (applies to all sites except those two noted below)

• Multiple booking facilities (fewer than 6)

Arrestee cases will be sampled in all facilities as described below (applies to the District of Columbia and Fulton County, GA.

.

Sampling Method for Respondent Selection

There are important practical challenges inherent in surveying the ADAM target population and creating a representative sample of all persons arrested over the course of a typical booking day. First, jails are chaotic, and law enforcement officials may not allow interviewers to be stationed within the jail during certain hours, particularly during hours when the booking process is most intense, due to security concerns and disruptions caused by our need to access booking records and arrestees. In contrast, there are also certain shifts during which so few arrestees are booked into the jail that interviewers stationed in the jail during those hours of relative quiescence could interview just one or two arrestees in an eight hour shift, compared to high-volume periods when many arrestees are available to be interviewed.

The sampling design in each facility divides the data collection day (and the interview cases) into periods of stock and flow. Interviewers arrive at the jail at a fixed time during the day. Call this H. They work a shift of length S. The stock comprises all arrestees booked between H-24+S and H, and the flow comprises all arrestees booked between H and H+S. For example, if interviewers start working at 4 PM and work for 8 hours, then the stock period runs from 12PM to 4PM, and the flow period runs from 4PM to 12PM. Cases are sampled from the stock and flow strata.

In the stock period, sampling is done from arrestees who have been arrested between H-24+S and H. This sampling begins at time H, and while arrestees identified as having been brought in during that time remain in the sample frame, interviewers can only interview those arrestees who remain in jail as of time H. In the flow period, sampling is done continuously for arrestees as they are booked between H and H+S.

To determine sampling rate, supervisors estimate the number of bookings that occur during the stock and flow periods based on data for each facility reflecting the two-week period prior to the quarter’s collection. Call the daily total N; call the number booked during the stock period N_S; and call the number booked during the flow period N_F. Then . Supervisors set quotas from the stock and flow for each site equal to n_S and n_F, respectively, such that:

The actual sample size (n=n_S+n_F) depends on the number of interviewers and sometimes (for small jails) the number of bookings (N=N_S+N_F), since n cannot exceed N.

The supervisor sorts arrestees based on booking time during the stock period and forms n_s equal sized strata based on that ordering. Sampling is systematic within each stratum: 1, n_S+1, n_S+2, etc. If the sampled arrestee is unavailable or unwilling to participate, the supervisor selects the nearest temporal neighbor—meaning the arrestee whose booking time occurs immediately after the arrestee who is unavailable or who declined. Replacement continues until the already established stock quota is filled. Because of administrative practices of jails and courts, arrestees are frequently unavailable to interviewers, i.e., they have been transferred to another facility, have already been released or are in court. The selection of the nearest neighbor is intended to reduce or eliminate any bias that otherwise would occur from apparently low response rates.

During the flow period, the supervisor selects the arrestee booked most recently and assigns an interviewer. If the arrestee is unavailable or unwilling to participate, the supervisor selects the next most recently booked arrestee as a substitute. This process continues until the workday ends at time H+S.

This procedure produces a sample that is reasonably well balanced, meaning that arrestees have about the same probability of being included in the sample. If the sample were perfectly balanced, weighting would be unnecessary for unbiased estimates; and, in fact, estimates based on weighted and unweighted ADAM data are similar. The sample is not perfectly balanced, however, for several reasons.

First, while supervisors attempt to sample proportional to volume during the stock and flow periods based on recent data from the facility, achieving this proportionality requires information that is not available at the time that supervisors set quotas. A supervisor can only estimate N_S and N_F based on recent historical experience; furthermore, the supervisor can not know the length of time required to complete interviews because the length of the ADAM II interview depends on the extent of the arrestee’s comprehension and cooperation level, as well as the extent of his reported drug use and market activity. So the achieved value of nF is variable.

Second, the number of bookings varies from day-to-day, but the number of interviewers arriving each day is constant. Days with a high number of bookings result in lower sampling probabilities than days with a low number of bookings. Furthermore, the number of bookings varies over the flow period, so that arrestees who are booked during periods with the most intensive booking activity have lower sampling rates than do arrestees who are booked during periods with the least intensive booking activity. Sampling rates do not vary as much across the stock period because of the way that the period is partitioned.

Third, as noted above, arrestees can exit the jail during the stock period. The probability that an arrestee has been released prior to being sampled depends on both the time during the stock period when he is booked and his charge. The earlier that booking occurred during the stock period, the greater the opportunity he has had to be released. The more serious the charge, the lower the probability of being released, because serious offenders are more likely to be detained pending trial or require time-consuming checks for outstanding warrants. Neither factor plays an important role during the flow period because of the way that the sample is selected.

The Illustration Box below shows site-specific modifications.

2. Information Collection Procedures

Data collection protocols are described in detail in the ADAM II 2007 Annual Report and the ADAM II 2008 Annual Report available through ONDCP’s website. The protocols are briefly summarized here to provide some context for the discussion of weighting and estimation methodologies.

Selecting Study Subjects

Interviewers work in teams in each jail. The supervising interviewer samples from the stock and flow. Sampling from the stock requires a list of all individuals who were booked since the interviewer’s last work period. Not all arrestees are still in the facility, but the supervising interviewer does not know that. He or she seeks the sampled arrestee, and, if that arrestee is unavailable or unwilling to be interviewed, the supervising interviewer records the reason and seeks a replacement. Sampling from the flow requires a list of individuals as they are booked into the jail. The supervising interviewer continuously compiles a list of incoming arrestees and seeks the most recently booked arrestee. If that arrestee is unavailable or unwilling to be interviewed, the supervising interviewer records the reason and seeks the closest temporal replacement.

When any arrestee is sampled (regardless of their availability), the supervising interviewer completes a facesheet. The facesheet contains sufficient identifying information that the arrestee can be matched with census data (that is, a census or records representing all bookings into the jail in each of the fourteen data collection days) that are collected long after sampling. The supervising interviewers use the facesheet to record that an interview occurred, and if it did not, the reasons why it did not. Analysts use the facesheet to compute response rates. Bar-coded labels are attached to the facesheet, the interview form and the urine specimen bottle, tying all data together. All arrestees sampled have a facesheet, but not all have the other components of the collection (interview, urine specimen). To be eligible for interview an arrestee must be: male, arrested no longer than 48 hours prior, coherent enough to answer questions and not held on a federal charge (for example an ICE or Federal Marshalls’ hold).

Arrestees who consent to an interview are administered the ADAM series questionnaire. The request for a urine sample is made at the beginning of the interview and repeated at its completion. If the arrestee consents, he is given a specimen bottle which he takes to a nearby lavatory to produce a sample. The bottle is returned to the interviewer, bagged and sent at the end of the shift to a national laboratory for urinalysis. In most sites over 80% of arrestees consent to provide a urine specimen. The urine specimen is linked to the facesheet and the interview through common bar-coded labels.

The Role of Census Data

Developing propensity scores for case weighting requires complete data on all bookings (a census) that occurred in each ADAM II facility during the two-week period of data collection. These data are provided by each law enforcement agency participating in ADAM II and sent to the Abt Data Center for processing. Site law enforcement partners submit census data in a variety of forms: electronic files listing each case, PDF or other text files of cases and paper format listing all cases. The Abt Data Center staff transforms each into site and facility specific data sets containing the following data elements for each arrestee:

• Date of Birth and or Age

• ID (computer generated number)

• Most serious charge

• Time of arrest

• Time of booking

• Day of arrest

• Race

Whether the census data are transmitted electronically, as a PDF file, or a paper file, the data are transformed into a SAS dataset. The census data become the sampling frame. As noted, ADAM interviewers complete a facesheet that includes the above variables for every arrestee sampled for the study, records whether the arrestee answered the interview and whether he provided a urine specimen.

Figure 2.1 represents the steps included in the manipulation of the raw census data done in preparation for matching with the ADAM facesheet data. The raw census data received from booking facilities are cleaned to correct invalid data and reformatted for compatibility with the other data components. The census data typically have one row of data per charge and must be converted to single records identifying arrestees with multiple charges. First, arrestees are excluded in the census data who are ineligible for the ADAM survey: juveniles, women and people booked on days other than those when ADAM surveys were conducted. Second, charges recorded in the census data are converted into a set of standardized ADAM charges. Additionally, the top severity, top charge and top charge category (violent, property, drug, other) are determined for each individual.

Figure 2.1: First Step in Matching Process

Figure 2.2 shows the process of matching the census records to the ADAM facesheet records. The variables common to both the facesheet and the census data that are used to match the records are: booking date/booking time, date of birth, arrest date/arrest time, charges and race. Potential matches are outputted if records match on any single key variables; they are then ranked into tiers based on the goodness of the fit. For example, a facesheet record that matches a census record on just booking date/booking time and charges will be superseded in rank by a facesheet-census match that links on booking date/booking time, charges and date of birth. Out of all the potential matches the best census match is selected for each facesheet. If, in fact, multiple census records match the same facesheet, and these duplicate matches have equivalent rankings, booking date/time is used as a tiebreaker. The output dataset from this process is a one-to-one match between each facesheet record and census records.

Rarely, a facesheet fails to match any booking record. When this happens, a pseudo-booking sheet is created and inserted into the booking data. This process is represented by the right-hand flow in Figure 2.

Figure 2.2: Matching Census with Facesheet Data

Figure 2.3 demonstrates the last step in the construction of the analysis file for each site and each data collection quarter. The linked census-facesheet data are merged with the appropriate urinalysis and survey record using unique identification numbers recorded in barcoded labels on the facesheet, interview and urine specimen. The result is the final analysis dataset for each quarter for each particular ADAM site.

Figure 2.3: Creation of Final Analysis File

3. Methods to Maximize Response Rates

A key to conducting a successful study is ensuring that response rates remain at acceptable levels throughout the study. Abt Associates’ rigorous procedures for interviewer selection, training, and quality control have resulted in steadily improving response rates for ADAM II sites since 2006. In 2008 the ADAM II program continued to achieve the high response rates of the original ADAM (see Table 3.1). In 2007, three ADAM II sites had a conditional response rate² of over 80%, for an overall conditional response rate of 75.2%. In 2008, seven sites had a response rate of 80% or higher, for an overall conditional response rate of 81.8%. The study team anticipates an even higher conditional response rate in 2009.

Table 3.1: 2008 Sample Sizes and Response Rates for Interview and Urine Specimen
	Atlanta	Charlotte	Chicago	Denver	Indianapolis	Minneapolis	New York	Portland	Sacramento	Washington DC	Overall
Sample Sizes
Provided Urine Specimen	354	396	426	460	524	383	365	453	508	55	3,924
Completed Interviews	419	468	485	511	578	433	515	526	562	95	4,592
Eligible and Available to be Interviewed	482	568	558	621	646	554	777	627	623	161	5,617
Eligible to be Interviewed	711	742	593	722	1,083	854	1,094	932	808	177	7,716
Interview Response Rates
Conditional Response Rate	0.869	0.824	0.869	0.823	0.895	0.782	0.663	0.839	0.902	0.590	0.818
Overall Response Rate	0.589	0.631	0.818	0.708	0.534	0.507	0.471	0.564	0.696	0.537	0.595
Urine Response Rates
Urine Agreement Rate	0.845	0.846	0.878	0.900	0.907	0.885	0.709	0.861	0.904	0.579	0.855
Conditional Response Rate	0.734	0.697	0.763	0.741	0.811	0.691	0.470	0.722	0.815	0.342	0.699
Overall Response Rate	0.498	0.534	0.718	0.637	0.484	0.448	0.334	0.486	0.629	0.311	0.509

Table 3.2 shows response rates by various characteristics of the sample in 2008. This analysis is repeated for each data collection period, and used by the ADAM II study team to identify problems in agreement rates and target areas for improvement. This strategy has resulted in improvements in response rates since the 2007 data collection

Table 3.2:
	Atlanta	Charlotte	Chicago	Denver	Indianapolis	Minneapolis	NewYork	Portland	Sacramento	Washington DC
Day of Week
Monday	69%	52%	78%	73%	67%	36%	39%	50%	67%	53%
Tuesday	63%	54%	96%	66%	51%	55%	47%	58%	75%	61%
Wednesday	64%	82%	86%	77%	61%	54%	50%	52%	74%	53%
Thursday	45%	56%	85%	72%	61%	55%	56%	64%	75%	56%
Friday	67%	69%	75%	78%	57%	48%	46%	59%	74%	55%
Saturday	55%	79%	75%	70%	45%	61%	47%	60%	58%	50%
Sunday	57%	60%	81%	61%	42%	47%	45%	54%	68%	44%
Total N (non-missing)	700	742	593	722	1083	854	1094	932	808	172
Chi-Square	17.7	37.5	17.2	11.2	32.0	19.3	9.4	8.0	14.5	1.8
P Value	0.007	0.000	0.008	0.083	0.000	0.004	0.155	0.236	0.024	0.937
Booking Time
12:00am-8:59am	45%	47%	100%	66%	43%	43%	27%	43%	53%	38%
9:00am-3:59pm	63%	65%	82%	77%	61%	63%	40%	56%	75%	38%
4:00pm-11:59pm	73%	73%	80%	71%	59%	56%	67%	72%	78%	55%
Total N (non-missing)	710	741	526	722	1065	849	1093	930	805	166
Chi-Square	41.7	33.1	1.4	7.1	31.5	24.8	111.1	56.8	47.8	2.5
P Value	0.000	0.000	0.507	0.029	0.000	0.000	0.000	0.000	0.000	0.282
Sample Type
Stock	51%	58%		67%	45%	46%	33%	47%	65%	60%
Flow	79%	73%	82%	77%	72%	67%	66%	75%	78%	53%
Total N (non-missing)	710	738	589	713	1077	853	1092	929	803	177
Chi-Square	47.5	14.8	14.8	8.4	67.9	27.2	111.5	66.0	14.8	0.3
P Value	0.000	0.000	0.000	0.004	0.000	0.000	0.000	0.000	0.000	0.578
Age
18-23	57%	62%	84%	71%	57%	50%	59%	55%	69%	57%
24-29	50%	58%	78%	65%	49%	53%	58%	65%	70%	64%
30-35	53%	55%	80%	67%	55%	46%	54%	55%	64%	59%
36-44	66%	66%	83%	74%	55%	53%	60%	60%	72%	46%
45+	64%	71%	83%	74%	51%	51%	59%	51%	72%	47%
Total N (non-missing)	695	740	590	720	1083	849	878	922	807	169
Chi-Square	11.8	9.1	2.3	4.8	4.4	1.7	1.0	9.2	2.3	4.8
P Value	0.019	0.059	0.675	0.310	0.356	0.790	0.903	0.057	0.674	0.305
Race
Black	60%	62%	83%	74%	53%	52%	59%	61%	70%	54%
Hispanic	71%	65%	78%	71%	42%	100%	59%	53%	69%	38%
White	52%	63%	81%	67%	55%	46%	54%	55%	70%	50%
Other	40%	89%	50%	85%	100%	62%	8%	58%	62%	80%
Total N (non-missing)	711	741	593	722	1083	854	1091	932	808	177
Chi-Square	4.2	2.7	4.5	4.1	3.1	6.9	195.7	3.1	1.5	2.1
P Value	0.237	0.436	0.215	0.248	0.371	0.074	0.000	0.376	0.686	0.556

4. Tests of Procedures

The utility of the core ADAM instrument for analysis and ease of administration has been demonstrated in the nearly 100,000 interviews collected since 2000. This instrument was developed beginning with de novo question creation in a series of focus groups of drug users and sellers in sites throughout the country. Final sets of questions underwent cognitive testing with heavy drug users in Boston. Additional validity testing of specific sections³ was undertaken during development (the calendar portion and the dependency screener). The entire instrument and sampling protocols underwent beta testing and revisions at two large ADAM sites (New York and San Antonio) in the Fall of 1999. The modifications (see below) are minimal.

ADAM also has the advantage of being able to assess the validity of one of its most important pieces of data, self-report of drug use, through a separate source, the urine test results. While there is variation in the reliability of time frames for particular drugs in urine tests, the rate of truth telling can be approximated. Specifically, ignoring a small rate of false positive tests, arrestees who test positive for a drug have used that drug within the last two or three days (longer for marijuana). In 2008, the rate of congruence for the major drugs used across all sites was over 80% for marijuana and cocaine, and over 90% for heroin and methamphetamine (Figure 4.1). This proportion is driven, in large part, by the high number of arrestees who do not report any drug use and do not test positive for any drugs. Among those who test positive, the percent admitting to use when testing positive congruence rate is lower for cocaine (45%), heroin (48%), and methamphetamine (55%)(Figure 4.2). This lower congruence rate among drug users highlights the critical importance of continuing the ADAM series, as no other major drug use series include a biometric sample for each respondent.

Figure 4.1. Rate of Congruence Between Self-reports and Urine Tests for Selected Drug Use, 2008

83%

84%

96%

97%

Figure 4.2: Percent Admitting to use When Testing Positive, 2008

82%

45%

48%

55%

Protocols for sampling in ADAM II are identical to those used and approved in ADAM. For ADAM II, revisions to the original ADAM instrument and protocols are minimal. The study team has determined that the methamphetamine manufacture section of the questionnaire adds little to our knowledge of methamphetamine markets in the ADAM series sentinel sites, and have consequently been removed from the data collection. The only remaining change to the interview protocol is the removal of a major skip pattern in the questionnaire. While we believe this change may add several minutes to the average administration time for the interview, we also believe the benefits outweigh this added burden.

No additional tests of procedures will be necessary for these changes.

5. Individuals Consulted on Statistical Aspects of the Design

Abt Associates Inc. Dana Hunt, Ph. D. William Rhodes, Ph.D. Ryan Kling, M.A. Richard Kulka, Ph.D.

National Institute of Justice Christine R. Crossland

1 Based on male arrest figures in 2003 UCR, except in Chicago (2001) and New York (2001).

2 The conditional response rate is the number of completed interviews divided by the number of sampled arrestees available to be interviewed.

3 Hoffman, N., Hunt, D., Rhodes, W. and Riley, J. (2003). UNCOPE: A brief substance abuse screener for use with arrestees, Journal of Drug Issues, Winter, 29–44.

Abt Associates Inc. Table of Contents 2