1117-0034_SupportingStatementPartB_2021FINAL

Supporting Statement for Paperwork Reduction Act Submissions

The National Forensic Laboratory Information System

Collection of Drug Analysis Data

OMB Approval #1117-0034

Part B. Statistical Methods

1. Universe and Respondent Selection

The universe for the current National Sample of Forensic Laboratories and future Survey of Forensic Laboratories for the “NFLIS-Drug” collection consists of all 168 state and local drug chemistry laboratories and laboratory systems in the United States. NFLIS-Drug National Sample laboratories were selected in 1999 by a stratified probability proportional to size (PPS) sample drawn based on annual cases analyzed per laboratory. See Table 1.

Table 1. NFLIS 2017 Drug National Sample. Strata, unit counts, number of sampled units and current response rates.

Stratum or Group	Units	Sampled	Responding	Current response rate
Stratum 1: Certainty state system	14	14	14	100.0
Stratum 2: Non-certainty state system	15	14	14	100.0
Stratum 3: Certainty municipal or local laboratory	9	9	8	88.9
Stratum 4: Non-certainty municipal or local laboratory	44	20	20	100
Post-1999 sample Volunteer laboratories	83		83	100
Identified post-2017	3		0	0

All laboratories on the NFLIS-Drug frame will be selected as respondents for the future NFLIS-Drug Survey, which is conducted about every 4-5 years. Response rates for the future Survey of Forensic Laboratories are expected to be 85% or higher based on response rates from the 2004, 2008, and 2013 Surveys of Crime Laboratory Drug Chemistry Sections.

The universe for the NFLIS-Tox future National Sample of Toxicological Laboratories (NFLIS-Tox) and future Survey of Toxicological Laboratories consists of all 325 toxicological labs and lab-systems (386 individual toxicological labs) identified which analyze antemortem and postmortem toxicological samples in the United States. The NFLIS-Tox collection includes all toxicological laboratories performing the following types of toxicological tests: postmortem, clinical, human performance, workplace, probation/parole, and performance enhancing (e.g. sports testing). The 131 respondents for the NFLIS-Tox National Sample were selected by stratified probability sampling (see Table 2). After recruitment of the NFLIS-Tox National Sample is complete, additional lab-systems will be recruited to the full 325 lab-systems on the frame.

All NFLIS-Tox laboratories on the frame will be contacted to participate in the future NFLIS-Tox Survey, which is expected to field for 5 months in 2020. Response rates for the future Survey of Toxicological Laboratories are expected to be greater than 68% as based on a 2017 Survey of Toxicological Laboratories and later relationship development with staff at the lab-systems.

Table 2. NFLIS-Drug National Sample. Strata, unit counts, number of sampled units and current response rates.

Stratum	Units	Sampled	Expected response rate
Certainty lab-system	18	18	Recruitment in progress. Expect 85% responding based on NFLIS-Drug
Respondent to 2017 Survey	190	54
Non-respondent to 2017 Survey	87	44
Identified post-2017 Survey	30	15

The universe for the future National Sample of Medical Examiners and Coroners (MECs; NFLIS-MEC) and future NFLIS-MEC Survey of MECs consists of all 2,069 MECs and MEC-systems (2093 individual MECs) identified in the United States. Using a systematic probability proportional to size (PPS) sample design, MECs were randomly selected from the frame with probabilities roughly proportional to a size measure based on estimated or stated caseload: After removing 4 certainty selection MECs, the steps were:

1. A relative size measure for each remaining MEC was calculated; that is, the MEC’s size measure divided by the sum of all the remainder size measures. Samples will be recruited annually for four years. A sampled MEC recruited in a year remains in the sample for collection in subsequent years. To select at least m MECs for the sample of the remainders over four years, multiply each remaining MEC’s relative size measure by m. That product is the MEC’s tentative four-year selection probability.

2. If any probability is greater than 1, make that MEC an additional certainty selection and start again.

3. After removing all certainties, sort the remaining MECs by region, state within region, and ZIP code within state. Compute the cumulative selection probability for each remaining MEC on the sorted list. Draw a random number u between 0 and 1. Select for the sample those MECs for which the cumulative selection probability first exceeds u, u + 1, …, and u + (m-1) (assuming there will be m non-certainty selections).

The resulting sample design assures that the non-certainty sample of MECs and the population of non-certainty MECs have similar distributions across regions and states. Using the same ordering as in selection, we will attempt to recruit the first sampled MEC and then every fourth sampled MEC afterwards in year 1. In year 2, we will attempt to recruit the second sampled MEC and then every fourth sampled MEC afterward. In year 3, we would start with the third sampled MEC, and in year 4 the fourth. We would then sort the additional certainties the same way and break them into four recruitment classes like the non-certainties.

The four-year selection probability of all certainties, a priori and additional, is 1. The selection probability of an additional certainty and a non-certainty recruit in an estimation made in year 1 is 1/4^th its four-year selection probability. The selection probability of a non-certainty recruit in an estimation made in year 2 is one half its four-year selection probability. The selection probability of a non-certainty recruit in an estimation made in year 3 is 3/4^th its four-year selection probability. The selection probability of an a priori certainty is 1.

Because the number of certainty selections may be large, a target final sample size of 400 MECs was chosen to control the overall sample size. Assuming an expected 50% MEC response rate, a sample of size 400 will lead to 200 total responding MECs, with 50 responding MECs each year. With 4 a priori certainties we simulated to identify m=246 which produced 158 additional certainties leading to an overall sample size of 408 (4+246+158.)

All MECs on the frame will be contacted to participate in the 2022 NFLIS-MEC Survey, which is expected to field for 6 months between 2022 and 2023. Response rates for the future Survey of MECs are expected to be greater than 60% based on a 2018 Survey of Medical Examiners and Coroners and later relationship development with MEC offices.

2. Procedures for Collecting Information

For the NFLIS-Drug National Sample, strata (Table 1) were defined as a cross of state or municipal/local lab with certainty sampling status. For these strata, labs were sampled using PPS sampling with the number of drug cases analyzed by the laboratory each year (i.e., its caseload) chosen as the measure of size. Spatial serpentine selection of units within strata provided implicit stratification. The initial 1999 sample provided estimates of desired accuracy at the national and regional level. Later addition of volunteer labs increased sample representation to census levels, allowing identification of trends at state and county levels. Weighted estimation methods were used for point estimates and trends at regional and national levels.

For the NFLIS-Tox National Sample, strata (Table 2) were defined according to response status on the 2017 NFLIS-Tox Survey. Within strata, samples were collect by systematic maximal PPS for stratum 2 and systematic sampling for strata 3 and 4. Systematic maximal PPS sampling in stratum 2 used information obtained from the 2017 NFLIS-Tox Survey to achieve sampling targets for labs testing each of the six test types (see B.1). The MOS for systematic maximal designs was based on lab test totals as revealed by the 2017 survey. For strata 2, 3, and 4, spatial serpentine sorting of units within strata provided implicit stratification. Weighted estimation methods will be used to produce point and trend estimates at regional and national level like those of the NFLIS-Drug National Sample. Additional recruitment is expected to bring sample sizes to census levels allowing small-scale state and county-level estimation.

For the future NFLIS-MEC National Sample, spatial serpentine sorting of cases by MEC and then MECs by state and region will provide implicit stratification after systematic sampling. Weighted estimation methods will be used to produce point and trend estimates at regional and national level similar to those of the NFLIS-Drug and NFLIS-Tox National Samples. After collection of the National Sample, additional recruitment is expected to bring sample sizes to census levels allowing small-scale state and county-level estimation.

For the future surveys of Forensic labs, Toxicology labs, and MECs these are censuses of all units on their respective frames and thus have no stratification nor require special statistical methodology. Estimation procedures are unweighted, and they should provide high degree of accuracy for estimation population parameters, given expected response rates (see B.1). A nonresponse bias analysis will be conducted if the unit response rate falls below 80 percent to see if participants differ from those who choose not to participate in the NFLIS surveys. Administrative data—such as the type of lab, type of MEC office, state in which the lab or office is located, and population served—will be used in the nonresponse bias analysis.

3. Methods to Maximize Response Rates

For the National Samples for NFLIS-Drug, NFLIS-Tox, and NFLIS MECs and their monthly data collections, our general approach to maximize interest in participating in NFLIS across all three NFLIS components is to request a modest number of data elements that are already being collected as a matter of course through their standard business practices (see Table 3) and can thus, be easily reported. In cases where NFLIS participants or potential participants are unable or unwilling to share particular data elements, the team will exclude those data elements in the memorandum of understanding that is negotiated with each participant to facilitate their participation and reporting. With respect to reporting in particular, we use standardized data automation where possible, explore reporting alternatives (e.g., data entry) as stop-gap solutions where appropriate, and maintain positive working relationships with software companies to facilitate recruitment and reporting for NFLIS participants.

In particular, and as we have done for the NFLIS-Drug data collection (and will do for the NFLIS-Tox and NFLIS-MEC collections), we maximize monthly data collection response by accepting the data in the format most convenient for the lab to ensure that the burden is minimal to zero, which in turn results in greater coverage and a more robust surveillance system for DEA. In some cases, we have worked with software companies to develop automated reporting data abstraction and reporting routines. In other laboratories, we have developed software solutions that help them cull the NFLIS data elements in a report that can be easily transmitted to DEA. Once the data are received from the laboratories, our team uses a phased automated data processing and exception handling approach (e.g., data de-duplication, identifying new substances) that includes 1,800+ data reporting and integration procedures developed to validate, standardize, and perform data accuracy and other quality checks on monthly data sent in various formats (e.g., open text, XML, XLS, MDB). This time-tested step-wise approach, which will be applied to the NFLIS-MEC and NFLIS-Tox collections, has enabled us to process an annual average of 1.6M drug items for the NFLIS-Drug program, representing ~2,750+ substances across 33 drug categories.

Participation in NFLIS is entirely voluntary for the forensic laboratories. Reporting for a specific laboratory or laboratory system may be interrupted due to various reasons (e.g., staff reduction, lack of resources, problems with data transmissions). These interruptions may lead to delayed or loss of data for these labs for a certain period of time. Imputation procedures are included in the estimation process to account for any missing lab data at the monthly level. In addition, weighting procedures are used to account for both laboratories that have never participated in NFLIS-Drug, as well as labs that are temporarily unable to provide data but are expected to do so in the future in order to continue to generate nationally representative estimates. Similar imputation and weighting procedures will be developed for the National Samples for NFLIS-Tox and NFLIS-MEC.

Table 3. Data Elements Collected Across the NFLIS Components

NFLIS Component	Data Elements Collected
NFLIS-Drug	Laboratory case number. Unique identifier assigned to a submission by the laboratory. Laboratory item number. Unique identifying number assigned by the laboratory to each integral component of physical evidence within a submission or case that is examined and individually specified in a laboratory report. Submission date. Date physical evidence submitted to the laboratory for analysis. If this date is unknown, date physical evidence received by the laboratory for analysis. Location of the submitting agency. County, city, zip code of the submitting agency. Form of material. Description of the physical form (i.e., crystal, powder, liquid, tablets, capsules, caplets) of each item within a submission or case received for analysis. Quantity. For each item within a submission or case received for analysis, quantity of material in grams (weight), milliliters (liquid), or units (tablets, capsules). Completion date. Approximate date analysis completed. Controlled substance(s) identified. For each item within a submission or case received for analysis, all controlled substances that are identified by laboratory analysis.
NFLIS-MEC	Reporting Entity Name. Name of data transmitting entity. Either an ME/C or a toxicological laboratory. MEC Case Number. Unique identifier assigned to a death case Name and location of the MEC assigned the death case: city, county, state, ZIP code. Date of decedent’s death: Date of decedent death as listed on the official death record. Date of final death record: The date signifying that the case is closed (e.g., toxicology results received and cause and manner determined). Cause of death: The official medical cause(s) of death for those deaths determined by the MEC to have the presence of drug(s) identified, regardless of whether the drug(s) caused or contributed to the cause of death. Manner of death: The official manner of death (e.g., natural, homicide, suicide, accidental and undetermined) as determined by the MEC for all accepted cases in which a drug or drugs were present. Location of Incident: County, city, zip code of the incident location. Incident Date and Location: city, county, state, ZIP code. Age of decedent: The age of the decedent. Gender of decedent: The gender of the decedent. Autopsy performed: Indication that an autopsy was performed for the accepted case. Performing Toxicology Laboratory Name: Name of the toxicology laboratory that performed the analysis. Can be internal or external, including a reference laboratory. MEC Sample identification number and Lab identification number Identifier of samples within a case as assigned by the MEC and/or the toxicological laboratory. Substance(s) tested: Drug, metabolite, or other substance tested for. Concentrations of confirmed drug(s) and metabolite(s) including units; If available, include concentrations with units for each confirmed drug or metabolite. Result Type: Type of analytical result. Date of toxicology analysis: Date the postmortem toxicology analysis was completed. Sample matrix (e.g., blood (peripheral or cardiac), urine, oral fluid, hair, or tissue) used for testing. Notes: A general field that can map to a comment or note field in the Case Management System (CMS) database. May provide additional information regarding the result or nature of the test.
NFLIS-Tox	Case ID/Unique identifier: Unique identifier assigned to a toxicology case by the laboratory. Requesting office type or agency: The type of office or agency submitting the request for toxicology analysis (pain management, substance abuse treatment clinic, primary care, medical examiner/coroner office or law enforcement agency). Case Type: Identify the type of case submitted for toxicology analysis (e.g., driving, postmortem, major crime, drug facilitated sexual assault, pain management and primary care). Requesting office/agency location: County, city, zip code of the office/agency requesting toxicology analysis. Date of case submission: The date the toxicology request was submitted to the laboratory. Date of case completion: The date the toxicology analysis was completed by the laboratory. Drug(s) and metabolite(s) confirmed: List of all confirmed drug(s) and metabolite(s) found in the toxicology sample of each individual in a toxicology case. Concentrations of confirmed drug(s) and metabolite(s) including units: If available, include concentrations with units for each confirmed drug or metabolite. Sample matrix used for confirmation results: List the sample matrix (e.g., blood (peripheral or cardiac), urine, oral fluid, hair, or tissue) used for testing the confirmed results.

As with the prior surveys for NFLIS-Drug (2013), NFLIS-Tox (2017), and NFLIS-MEC (2017), verification calls will be used when necessary to establish or confirm appropriate contact information (i.e., name, address, telephone number, and e-mail address) for each future survey respondent to ensure survey efficiency. Initial mailings will be sent to the primary contacts identified during the verification call effort. This mailing will include lead letters explaining the NFLIS program, directions for survey completion, whom to contact with questions, encouragement to complete the survey, and a non-monetary token of appreciation. A second reminder letter mailing will be sent approximately two weeks after mailing the lead materials to encourage survey response and emphasize the importance of the survey. Prompting calls would take place approximately 2 weeks after the second mailing to encourage nonresponders to complete a survey via mail or the web, as well as provide information on how to complete the survey, and address any challenges regarding survey completion. Nonresponse calls for critical items would begin about month before the end of the data collection period as a last attempt to obtain critical survey information. Examples of each of these items can be provided upon request.

As has been done in the past, DEA provides a token of appreciation to all responding agencies regardless of whether they complete the survey. Historically, the costs have ranged from $8-25 each and have included the Drug Identification Bible, Barry Levine’s Principles of Toxicology book, or a customized NFLIS calendar that provides key dates for upcoming events of interest to the forensic community.

4. Testing of Procedures

For the monthly National Sample collection, it is important to note that the NFLIS team accepts data in varying formats to facilitate participating and reporting across laboratories, and to that end, continues to implement a still-growing set of data reporting and integration standards that facilitate laboratory/MEC participation through minimal effort using automation where possible but enabling manual processes when needed. RTI accepts the data format most convenient for the laboratory/MEC to ensure that the burden is minimal to zero, which in turn results in greater coverage and a more robust surveillance system for DEA. NFLIS data management staff continually collaborate with participating drug and MEC laboratories to help develop the most appropriate reporting solutions. Where needed, NFLIS staff will provide technical support to develop and implement a reporting solution. Where these laboratories either have or are in the process of implementing a commercial laboratory information management systems (LIMS), NFLIS team members will collaborate with the laboratories and the LIMS providers to ensure that new or existing NFLIS reporting solutions are integrated into the laboratories’ LIMS solutions and will facilitate the appropriate testing when needed in collaboration with the laboratories, their IT departments, and the software companies. Before any laboratory moves to a reporting status, test files submitted to the NFLIS team are thoroughly evaluated by the NFLIS team in conjunction with laboratory contacts to ensure the appropriate data are being collected, all questions are resolved to ensure high quality data, the reporting solution is of minimal to zero burden moving forward, and thus, the reporting entity’s NFLIS participation and reporting are uninterrupted.

Before administering any NFLIS survey, the previous survey instrument will be thoroughly reviewed by DEA and RTI to ensure that the measures are still of interest and needed to support DEA’s internal mission critical operations and the NFLIS program as a whole. DEA’s contractor, RTI International (RTI), works closely with the DEA to make any necessary changes. In past years, RTI has generally either consulted with 4-5 experts from the field to get inputs on survey measures, as well as conducted a pilot test of the NFLIS-Drug instrument to test the survey items for clarity and test the response options for appropriateness. The pilot tests have generally consisted of three or four experts reviewing the survey, filling out the survey, and providing feedback as part of the pilot test. The pilot testing has historically provided insight into whether respondents provided expected answers, informed our phrasing and response options, and provided a general estimate of the burden. Following the inputs from field experts and the pilot testing, the survey instruments will be revised.

Across all three surveys, a multimode survey approach—including mailing, email, web, and telephone response options—will be used to provide respondents with the ability to complete the survey in the way that they are most comfortable. For the web survey option, RTI thoroughly tests the web-based survey administration system through systematic user testing, including testing skip patterns, attempting to “break” the instrument, and back-end data checks on entered responses.

5. Contacts for Statistical Aspects and Data Collection

RTI Project Staff

Hope Smiley-McDonald, Ph.D.

RTI International

NFLIS Project Director

Jeri Ropero Miller, Ph.D.

RTI International

NFLIS Associate Project Director

BeLinda Weimer, MA

RTI International

NFLIS Data Collection Task Leader

David Heller, BS

RTI International

NFLIS Analysis Task Leader

Paul Mosquin, Ph.D.

RTI International

NFLIS Sampling Task Leader

Contact Number: (202) 307-7183