SUPPORTING STATEMENT: PART A
February 9, 2022
Drug Overdose Surveillance and Epidemiology (DOSE)
OMB #0920-1268
Point of Contact:
Alana Vivolo-Kantor
Contact Information:
Centers for Disease Control and Prevention
National Center for Injury Prevention and Control
4770 Buford Highway NE MS F-62
Atlanta, GA 30341-3724
phone: (770) 488-1244
email: [email protected]
CONTENTS
Section Page
Summary table 3
Justification 4
A.1. Circumstances Making the Collection of Information Necessary 4
A.2. Purpose and Use of Information Collection 10
A.3. Use of Improved Information Technology and Burden Reduction 13
A.4. Efforts to Identify Duplication and Use of Similar Information 15
A.5. Impact on Small Businesses or Other Small Entities 19
A.6. Consequences of Collecting the Information Less Frequently 19
A.7. Special Circumstances Relating to the Guidelines of
5 CFR 1320.5(d)2 20
A.8. Comments in Response to the Federal Register Notice and
Efforts to Consult Outside the Agency 20
A.9. Explanation of Any Payment or Gift to Respondents 21
A.10. Protection of the Privacy and Confidentiality of Information
Provided by Respondents 21
A.11. Institutional Review Board (IRB) and Justification for Sensitive
Questions 22
A.12. Estimates of Annualized Burden Hours and Costs 22
A.13. Estimates of Other Total Annual Cost Burden to Respondents
or Record Keepers 25
A.14. Annualized Cost to the Government 25
A.15. Explanation for Program Changes or Adjustments 27
A.16. Plans for Tabulation and Publication and Project Time Schedule 27
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate 29
A.18. Exceptions to Certification for Paperwork Reduction Act
Submissions 29
Attachments
A1 Authorizing Legislation: Public Health Service Act
A2 Authorizing Legislation: Substance Use-Disorder Prevention that Promotes Opioid Recovery and Treatment for Patients and Communities Act
B Published 60-Day Federal Register Notice
B1 Public comment and response
C Research Determination
D Rapid ED overdose data form
D-2 Crosswalk of Changes
E ED discharge overdose data form
F Privacy Act Applicability
SUMMARY TABLE
Goal of the study – This serves as a revision request for the currently approved ICR. Revisions are requested to revise the number of eligible states, change the data collection template, and revise burden.
Intended use of the resulting data - Improve local, state, and regional situational awareness of drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdose trends and response to acute local and multi-state drug outbreaks.
Methods to be used to collect – The project will leverage ED syndromic data and hospital discharge data on ED visits already routinely collected by state health departments and the District of Columbia health department. No new data will be systematically collected from EDs, and health departments will be reimbursed by CDC for the burden related to sharing ED data with CDC. Fifty-one funded health departments (50 state health departments and the District of Columbia) will rapidly share existing ED data with CDC on a monthly basis using the Rapid ED overdose data form and standard CDC case definitions. Data may come from different local ED data systems, but is expected to cover at least 75% of ED visits in the jurisdiction (e.g., state). Specifically, some health departments conduct rapid overdose surveillance using local ED syndromic systems or hospital discharge data, while others conduct surveillance using CDC’s National Syndromic Surveillance Program (NSSP), which receives near real-time ED data from health departments or their partners on approximately 71% of ED visits in the United States. Regardless of the data source, CDC will require all participating health departments to provide counts of ED visits involving suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepines, and other emerging drug overdoses by county, age group, sex, race/ethnicity, and time (i.e., month and year) in a standardized manner using the Rapid ED overdose data form, which is an Excel data template. This form also collects data quality indicators such as percent of ED visits missing data on key variables (i.e., metadata). In order to assess and improve rapid ED data sharing, all 51 participating health departments will also be asked to share counts of ED visits involving suspected drug, opioid, heroin and stimulant overdoses by county, age group, sex, race/ethnicity, and time (i.e., month and year) from more finalized hospital discharge files, the current surveillance standard. The data will be shared with CDC on a quarterly or yearly basis using a standardized Excel data form, the ED discharge overdose data form, and standard CDC case definitions.
The subpopulation to be studied – Individuals who visit an ED to receive treatment for a drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdose.
How data will be analyzed - Descriptive analyses such as frequencies and changes in the rate of ED visits involving drug, opioid, heroin and stimulant overdoses by region, state, and local jurisdiction. Longitudinal statistical analyses such as Joinpoint regression will be used to describe trends. Also, monthly, quarterly, and yearly changes in key indicators will be monitored to identify outbreaks. Finally, drug overdose counts from rapid ED surveillance and hospital discharge files will be compared to inform improvements.
JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
This is a revision request for the currently approved Drug Overdose Surveillance and Epidemiology (DOSE) - OMB# 0920-1268, expiration date 08/31/2022. CDC is requesting OMB approval for an additional 3 years to continue data collection efforts. The DOSE system originally received approval to collect data from 50 states, the District of Columbia and Puerto Rico, and currently captures data from 47 states and the District of Columbia. We seek to expand to all 50 states and the District of Columbia in 2023. Puerto Rico is being removed from DOSE eligibility and will not be sharing data. DOSE captures and rapidly shares existing ED data on counts of ED visits involving suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses using two standard data forms (i.e., the Rapid ED overdose data form and the ED discharge overdose data form) and standard CDC case definitions. Revisions noted below are for three purposes: (1) to revise the number of eligible recipients from current and future data collection efforts; (2) to change the data collection template to include race/ethnicity and the new drug types; and (3), and revise burden estimates. A future revision or change request will provide final state enrollment numbers in 2023 following the release of a new funding announcement for DOSE (pending award Fall 2023).
Several circumstances make DOSE necessary. First, there have been rapid increases in opioid overdose deaths since 20131 and numerous severe fentanyl and fentanyl analog outbreaks occurring across the United States2,3,4,5,6. In addition, the declaration of the opioid overdose epidemic as a national public health emergency on October 26, 20177 highlighted the urgent need to rapidly establish and enhance timely surveillance of drug overdoses. DOSE provides data critical to inform timely local, state, and regional response, especially to acute and/or widespread multi-state outbreaks.
Second, CDC was appropriated funding in FY 2019 to expand to all 50 states and the District of Columbia drug overdose ED surveillance funded through the CDC Enhanced State Opioid Overdose Surveillance Funding Opportunity (ESOOS; CDC-RFA-CE16-1608).8 Initiated in 12 states in September 2016, ESOOS funded 32 state health departments and the District of Columbia to rapidly share ED data collected by their agency with CDC on the number of suspected drug, opioid, and heroin overdoses. Participating health departments used their own state-based case definitions of ED visits involving suspected drug, opioid, and heroin overdoses and select how they are going to share data with CDC (i.e., are not required to fill out a standardized template). This expansion that occured in FY 2019 was a component of CDC’s Overdose Data to Action Notice of Funding Opportunity (OD2A, CDC-RFA-CE19-1904). DOSE replaced and enhanced ESOOS ED data sharing. OD2A funding expires in September 2023; however, we are requesting three additional years for DOSE data collection in anticipation of continued DOSE activities past the current OD2A funding cycle.
Over the past several years of implementing DOSE, we have learned key lessons to inform the enhanced data sharing that we hope to continue to implement:
Having standard, national case definitions for drug overdose is extremely important for dissemination efforts. With all states using the same definition, we feel more confident that states are more comparable across time.
Using a form [the “Rapid ED overdose data form” (Att. D)”] has improved CDC data processing time. In addition, CDC has developed SAS programs and R code for states to use to analyze the data and output results into the form, which has reduced state data processing time.
Creating and testing new case definitions is paramount to better understanding this evolving epidemic. States are now required under DOSE to share data on suspected all drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses. This responds to sharp increases in deaths involving (for example) stimulants as well as increased co-use and mixing of opioids with stimulants and benzodiazepines over the past few years. 9,10,11 Though this is a revision to the previously approved collection form (Att. D and Att. E), OMB approval was granted in our original request to create and collect data on new drug overdose indicators.
Some states are unable to share syndromic surveillance data, thus allowing flexibilities for sharing of hospital discharage data using the form [the “ED discharge overdose data form” (Att. E)] has been beneficial. ED syndromic systems are designed to collect rapid preliminary data on changes in illness and injuries such as drug overdose. These systems, however, often do not provide an accurate estimate of the full burden of illnesses and injuries because they are based on preliminary data. Responding to this limitation, DOSE will continue to compare counts and rates of drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses from rapid ED data sharing with the same counts calculated using more finalized hospital discharge data files (the current public surveillance standard used to assess the burden of drug overdoses treated in EDs).
States have been working towards improving the data quality of new fields such as those that capture patient race and ethnicity. CDC and states have worked together to assess some of these fields and as completeness and accuracy improves, states are now able to share data on patient race and ethnicity. Thus, CDC would like to now require the sharing of patient race and ethnicity. This is a revision to the previously approved collection form (Att. D and Att. E) and will require OMB approval.
DOSE background
In 2020, a total of 91,799 drug overdose deaths occurred, corresponding to an age-adjusted rate of 28.3 per 100,000 population and a 31% increase from the 2019 rate (21.6).12 From 2013 to 2019, the synthetic opioid-involved death rate increased 1,040%, from 1.0 to 11.4 per 100,000 age-adjusted (3,105 to 36,359). The psychostimulant-involved death rate increased 317%, from 1.2 (3,627) in 2013 to 5.0 (16,167) in 2019.13 Non-fatal overdoses are on the rise as well; ED data from DOSE indicates increases from 2018 to present.14 In response to the growing severity of the opioid overdose epidemic, the US government declared the opioid overdose epidemic a public health emergency on October 26, 2017.15 The opioid overdose epidemic is one of the U.S. Department of Health and Human Services (HHS) top priorities. In 2021, HHS expanded their Overdose Prevention Strategy to focus on four strategic priorities: primary prevention, harm reduction, evidence-based treatment, and recovery support.16
DOSE is a critical element of HHS’s first goal under primary prevention to support research and surveillance to collect timelier and more specific data through accelerating the speed at which CDC’s reports drug overdose data.17 DOSE data collection integrates, expands, and enhances previous data sharing efforts with public health departments initiated under ESOOS. The goal of DOSE is to conduct surveillance of approximately 75% of all ED visits for drug overdoses through the end of the Overdose Data to Action (OD2A) cooperative agreement in 2023. In 2019, OD2A provided funding for 66 jurisdictions; 47 states and the District of Columbia share data with DOSE. Though we had hoped to capture data from all 50 states and the District of Columbia, only 47 states and the District of Columbia applied for this funding announcement. We describe our progress to date in more detail in Section 2.
DOSE is made possible because the vast majority of the participating health departments are already rapidly collecting extensive data on ED visits in their jurisdiction and using these data for the identification of public health concerns including flu, heat-related illness, and hurricane-related health issues. Prior to the implementation of ESOOS, most states, however, were not routinely using these data to identify ED visits related to suspected drug overdoses, nor were they analyzing this data in a timely manner, or sharing these data with CDC. DOSE ensures participating jurisdictions use their data to track suspected overdoses by providing participating jurisdictions standardized definitions of ED visits involving drug, opioid, heroin and stimulant overdoses. This facilitates rapid identification and tracking of ED data on drug overdose.
Also, no single ED surveillance system has national coverage, but almost all of participating health departments use one of three systems - the NSSP BioSense System, local ED syndromic surveillance, or hospital discharge files. DOSE integrates data across these three types of ED surveillance to quickly build a national surveillance system while leveraging existing ED data collection efforts. DOSE can use data across the three types of ED surveillance systems because the key data requirement is the ability to detect change over time (e.g., data consistently collected within the jurisdiction overtime) and not comparability across participating health departments (e.g., same data collection methods deployed across state health departments overtime).
A brief background of the three ED data systems integrated into DOSE is provided below:
CDC National ED Syndromic Surveillance: The Division of Health Informatics and Surveillance (DHIS) in the Center for Surveillance, Epidemiology, and Laboratory Services (CSELS) in CDC operates the National Syndromic Surveillance Program (NSSP) BioSense Platform (OMB #0920-0824) through which state and local health departments share preliminary data such as the chief complaint of the patient seeking care at the ED (e.g., “heroin overdose”) and/or diagnosis codes, primarily International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM)18 diagnosis codes, assigned the ED visit on approximately 71% of ED facilities in the US. Chief complaint data are often received by the NSSP BioSense Platform within 48 hours of date of the ED visit and updated to include ICD-10-CM diagnosis codes within a few weeks, if available. CDC is prioritizing the sharing of data through the NSSP BioSense platform due to the speed at which it currently collects ED data, its high rate of ED participation (>6,000 EDs participating), and its ability to leverage existing CDC efforts (See Efforts to Identify Duplication and Use of Similar Information section).
State or Territorial ED Syndromic Surveillance: Participating health departments may operate their own local ED syndromic system that is not associated with NSSP BioSense. These local ED syndromic systems often collect data very similar to NSSP BioSense such as patient chief complaint and ICD-10-CM diagnostic codes.
Hospital ED Discharge Data: Some health departments have no or limited syndromic surveillance of ED visits. These health departments, however, may be able to leverage hospital discharge data on ED visits that is routinely collected by most states. Hospital discharge data are collected for billing purposes, uses standardized ICD-10-CM coding, and most states use Uniform Billing Version 04 (UB-04) administrative claims data to collect ICD-10-CM diagnosis and procedure codes. Although these data have a two year time lag nationally, CDC has found that some states have access to preliminary hospital discharge data that meet the requirements of DOSE.
Key advantages to DOSE compared to initiating a new ED data collection are:
DOSE can be rapidly implemented and scaled to all 50 states and the District of Columbia with minimum burden on state health departments because it relies on sharing and improving ED data already being collected by state and local health departments.
DOSE ensures local health departments ED syndromic efforts are integrated into national surveillance instead of duplicated.
DOSE leverages instead of duplicating existing CDC work through CDC NSSP and ESOOS to rapidly share state and local health departments ED data with CDC.
DOSE ensures that local health departments are involved in the collection, ownership and use of the ED data collected. This is critical because state and local health departments are primarily responsible for responding to local drug overdose outbreaks and changes in the opioid overdose epidemic, have extensive local knowledge of their local ED data that fosters identification of data quality problems including identifying false positives, and are critical partners in developing tools to monitor illnesses and injury.19, 20
This program is authorized under section 301 (a) [42 U.S.C. 241(a)] of the Public Health Service Act and section 391 (a) [42 U.S.C. 280(b)] of the Public Service Health Act (Att. A1). Also, Subtitle Q in the Substance Use-Disorder Prevention that Promotes Opioid Recovery and Treatment for Patients and Communities Act (SUPPORT Act) specifically grants authority to CDC for overdose data and collection activities including, “Improving the timeliness of reporting data to the public, including data on fatal and nonfatal overdoses of controlled substances,” “Enhancing the comprehensiveness of controlled substance overdose data by collecting information on such overdoses from appropriate sources such as…emergency departments,” and “Working to enable and encourage the access, exchange, and use of information regarding controlled substance overdoses among data sources and entities” (Att. A2).
2. Purpose and Use of Information Collection
All data sharing between CDC and health departments in DOSE is driven by two standardized data forms, the Rapid ED overdose data form and the ED discharge overdose data form, and CDC cases definitions of drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses. The justification of the two forms and the key variables they collects is described in detail below.
Form 1: Rapid ED overdose data forms (Att D)
Health departments able to share syndromic surveillance data with CDC will be required to complete the Rapid ED overdose data form (Att. D) on a monthly basis using data from existing local ED data collection efforts, described above.
Frequency that this data form is reported to CDC. The goal of the program is to have health departments submit monthly reports to CDC in order to detect and respond to drug overdose outbreaks or shifts in trends in a timely manner.
Key variables shared with CDC. Key variables why they are collected is described in Table below.
Variable |
Justification for collecting |
Count of ED visits suspected to involve drug overdoses |
|
Count of ED visits suspected to involve opioid overdoses |
|
Count of ED visits suspected to involve heroin |
|
Count of ED visits suspected to involve stimulants |
|
Count of ED visits suspected to involve other new and emerging drugs |
|
Sex, age group, race/ethnicity, and county level data by drug overdose indicators |
Aggregating data on drug, opioid, heroin, and stimulant overdose by sex, age group, race/ethnicity, and county is critical to assist CDC as well as state and local health departments target interventions on demographic groups and geographic areas impacted by drug overdoses outbreaks or large changes. |
Percent of ED visits with chief complaint text and diagnosis codes |
ED visits involving drug overdose are primarily identified by analyzing patient’s chief complaint and diagnosis codes fields, primarily ICD-10-CM diagnosis codes. Thus, important data quality indicators are the percent of ED visits with chief complaint data and the percent of ED visits with diagnosis codes. |
Median word length of the chief complaint |
The median word length of the chief complaint is tracked because the ability to identify suspected drug overdoses is impacted by the length and quality of text data entered into the chief complaint text field. Based on previous experience working with health departments, chief complaints with fewer words are less likely to contain information on the a) the type of drug involved (e.g., opioids) and b) overdose symptoms (e.g., trouble breathing). |
Mean and maximum number of diagnosis codes |
The mean and maximum of diagnosis codes, primarily ICD-10-CM diagnosis codes, collected by jurisdiction s varies (e.g., one jurisdiction may allow hospitals to enter 10 codes while another allows 16 codes). Since CDC drug overdose case definitions search all diagnosis codes, tracking the number of submitted diagnosis codes is important because they may result in slight differences between jurisdiction ability to identify suspected drug overdose cases (e.g., jurisdictions collected more ICD-10-CM codes might be slightly more likely to identify an ED visit as involving a drug overdose). |
Metadata on local surveillance systems |
Local ED data systems may experience major changes that impact data quality (e.g., ED data sharing delayed due to the implementation of a new EHR system). In order to effectively identify and address these types of changes, DOSE will ask all participating health departments to report major changes in ED participation or data quality each month. |
Use of the data form by CDC. CDC will use this data form to detect outbreaks and rapid changes in drug, opioid, heroin, and stimulant outbreaks to inform response.
Form 2: ED discharge overdose data form (Att E)
Health departments able to share discharge data with CDC will be required to complete the ED discharge overdose data form (Att. E) on a quarterly or yearly basis. The ED discharge overdose data form is similar to the Rapid ED overdose data form and thus the description in this section focuses on its unique characteristics.
Frequency that this data form is reported to CDC. The goal of the program is to have health departments submit quarterly reports to CDC because this will allow a timely analysis of the burden of drug overdoses in the jurisdiction as well as timely analysis of the quality and accuracy of the rapid ED data collection in the jurisdiction. Through previous CDC collaborative work with state health departments, CDC found that a substantial percent of health departments are not involved in the collection of their local hospital discharge data and thus receive the data with large time lags of a year or more. Consequently, about half or 23 participating health departments are expected to only submit hospital discharge data yearly.
Key variables shared with CDC. The variables included in the ED discharge overdose data form are almost identical to the variables included in the Rapid ED overdose data form, discussed above. The few differences are:
Not systematically tracking major changes in hospital participation. Because the hospital discharge data collection is well established in many states and reported quarterly or yearly to CDC instead of monthly, major reporting disruptions are not expected to occur frequently. Thus, metadata about major changes in the hospital discharge data is not collected to reduce burden on participating health departments. The number of submitting hospitals, however, is still collected each quarter in order to detect possible changes requiring follow-up from CDC staff.
Distinguishing ED visits involving unintentional and undetermined-intent drug overdoses from ED visits involving drug overdoses related to intentional self-harm. The ICD-10-CM diagnosis coding system tracks the medically documented intent of the person treated for a drug overdose in the ED (e.g., overdosed as part of a suicide attempt or accidentally overdosed while taking a drug for the euphoric feeling it produces). Since interventions to reduce intentional overdoses related to self-harm may differ from those related to substance abuse, the participating health departments will be asked to aggregate their data by intent (i.e., unintentional versus self-harm).
CDC use of the data form. This form will be used in two ways:
Hospital discharge data is the current standard for tracking drug overdose burden associated with ED visits and hospitalizations. Consequently, comparing trends in ED visits involving suspected drug, opioid, heroin, and stimulant overdoses calculated using hospital discharge data with trends calculated using rapid preliminary ED data will help identify strengths and weaknesses in the rapid preliminary ED data sharing effort. This in turn will inform system improvements. Even if the hospital discharge data has a time lag of a year or more instead of quarterly, these comparisons will still be useful in accessing and improving the general quality of the rapid preliminary ED data reported to CDC.
Current delays in reporting ED data and hospitalization data on drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses from hospital discharge files is approximately two years. This inhibits response and assessment of the overall burden of drug overdose. The current data sharing effort is a now a four-year pilot, with this extension request, to access whether the analysis and dissemination of ED and hospitalization burden data on drug overdoses can be accelerated through targeted data sharing.
To date, the DOSE system has been extremely successful at meetings its stated objectives. Currently, DOSE operates in the 47 states and the District of Columbia currently funded by OD2A (three states did not request CDC funding in the current cycle but may for the next funding cycle in 2023). Of these 48 health departments, 43 share syndromic data with CDC monthly and 26 share at least quarterly discharge data. A total of 33 health departments provide CDC access to their syndromic surveillance data from emergency departments in CDC’s NSSP system. Please see Figure 1 below for a graphic depiction of current state data sharing. Figure 2 depicts the requested state data sharing for DOSE going forward, which includes 50 states and the District of Columbia with 45 health departments sharing syndromic (35 sharing in NSSP) and 28 sharing discharge. Access to this timely data has allowed us to improve situational awareness of federal, state, and local health departments of emerging drug overdose outbreaks and the progression of the opioid overdose epidemic. Health departments have used this data to populate state data dashboards and develop alerts for local communities. In addition, health departments have used this data in concert with public safety partners to gain a better overall picture of outbreaks in their communities.
Figure 1: Current state data sharing for DOSE
Figure 2: Requested and expected state data sharing for DOSE in 2023
Data from the DOSE system has also raised public awareness of the progression of the opioid overdose epidemic and other emerging drug overdose threats in their area. As recently as February 2021, we have updated two areas of CDC’s website using data from DOSE (https://www.cdc.gov/drugoverdose/data/nonfatal/drugs-overall.html; https://www.cdc.gov/drugoverdose/data/nonfatal/states/covid-19.html). Notabily, following the onset of the COVID-19 pandemic, state and local health department reliance on timely syndromic surveillance data to detect trends in all types of injuries and diseases has increased.33 To provide the public information on drug overdoses treated in emergency departments during the onset of the pandemic, CDC quickly developed a new website showcasing the DOSE data (https://www.cdc.gov/drugoverdose/data/nonfatal/states/covid-19.html). This has allowed states to see the changes in drug overdoses from 2019 through September 2021. In addition, CDC staff have used DOSE data to report on drug overdoses involving more than one substance (https://www.cdc.gov/mmwr/volumes/69/wr/mm6934a1.htm) and overdoses involving benzodiazepines (https://www.cdc.gov/mmwr/volumes/70/wr/mm7034a2.htm).
DOSE data has also been used in in 10 peer-reviewed and MMWR publications (see below a list) and was featured in a special supplement in Public Health Reports (https://journals.sagepub.com/toc/phr/136/1_suppl).
Pickens, C. M., Scholl, L., Liu, S., Smith, H., & Snodgrass, S. (2021). Development and Validation of a Syndrome Definition for Suspected Nonfatal Unintentional/Undetermined Intent Stimulant-Involved Overdoses. Public Health Reports, 00333549211054489.
Liu S, O’Donnell J, Gladden RM, McGlone L, Chowdhury F. Trends in Nonfatal and Fatal Overdoses Involving Benzodiazepines — 38 States and the District of Columbia, 2019–2020. MMWR Morb Mortal Wkly Rep 2021;70:1136–1141. DOI: http://dx.doi.org/10.15585/mmwr.mm7034a2
Vivolo-Kantor, A. M., Smith IV, H., & Scholl, L. (2021). Differences and similarities between emergency department syndromic surveillance and hospital discharge data for nonfatal drug overdose. Annals of Epidemiology. doi: 10.1016/j.annepidem.2021.05.008
Vivolo-Kantor, A., Pasalic, E., Liu, S., Martinez, P. D., & Gladden, R. M. (2021). Defining indicators for drug overdose emergency department visits and hospitalisations in ICD-10-CM coded discharge data. Injury prevention, 27(Suppl 1), i56-i61. doi: 10.1136/injuryprev-2019-043521
Scholl, L., Liu, S., Vivolo-Kantor, A., Board, A., Stein, Z., Roehler, D. R., ... & Smith, H. (2021). Development and Validation of a Syndrome Definition to Identify Suspected Nonfatal Heroin-Involved Overdoses Treated in Emergency Departments. Journal of Public Health Management and Practice, 27(4), 369-378. doi: 10.1097/PHH.0000000000001271
Liu, S., & Vivolo-Kantor, A. (2020). A latent class analysis of drug and substance use patterns among patients treated in emergency departments for suspected drug overdose. Addictive behaviors, 101, 106142. doi: 10.1016/j.addbeh.2019.106142
Coffey, W., Hunter, A., Mobley, E., & Vivolo‐Kantor, A. (2020). Rural‐Urban Trends in Opioid Overdose Discharges in Missouri Emergency Departments, 2012‐2016. The Journal of Rural Health, 36(2), 177-186. doi: 10.1111/jrh.12368
Liu, S., Scholl, L., Hoots, B., & Seth, P. (2020). Nonfatal Drug and Polydrug Overdoses Treated in Emergency Departments—29 States, 2018–2019. Morbidity and Mortality Weekly Report, 69(34), 1149. doi: http://dx.doi.org/10.15585/mmwr.mm6934a1
Vivolo-Kantor, A. M., Hoots, B., David, F., & Gladden, R. M. (2019). Suspected heroin overdoses in US emergency departments, 2017–2018. American journal of public health, 109(7), 1022-1024. https://ajph.aphapublications.org/doi/10.2105/AJPH.2019.305053
Vivolo-Kantor, A. M., Seth, P., Gladden, R.M., Mattson, C., Baldwin, G.T., Kite-Powell, A., & Coletta, M. (2018). Vital Signs: Trends in Emergency Department Visits for Suspected Opioid Overdoses—United States, July 2016–September 2017. MMWR. Morbidity and Mortality Weekly Report, 67, 279-285. doi: http://dx.doi.org/10.15585/mmwr.mm6709e1
In addition to reporting out on drug overdoses, CDC DOSE staff have provided extensive technical assistance to health departments participating in DOSE. Our goal was to build a system that captured at least 75% of all non-federal ED visits. At present (January 2022), the DOSE system captures, on average, 85% of ED visits in 47 states and the District of Columbia. We will continue to build state and local health department capacity to both increase and maintain high facility coverage to better track drug overdose trends and outbreaks in their community.
3. Use of Improved Information Technology and Burden Reduction
DOSE is leveraging improved information technology to reduce burden on participating health departments in the following ways:
Data from over 6,000 hospitals and around 71% of ED visits in the US is currently shared with CDC through the NSSP BioSense platform. Just two years ago NSSP included a little over 4,000 hospitals with around 60% coverage. DOSE efforts have significantly helped health departments onboard facilities to increase coverage. Since 2016, the primary computer program used to process and analyze this data is the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE). DOSE expects that 35 health departments (33 state health departments and District of Columbia) will provide CDC access to their ED data within the NSSP BioSense platform using ESSENCE. At present, as of January 2022, 33 health departments (32 state health departments and the District of Columbia) provide CDC access to their data in NSSP and will continue to do so. On at least a monthly basis, CDC DOSE staff will use its access to complete the Rapid ED overdose data form. Participating health departments will only be asked to verify the accuracy of the CDC generated reports and support CDC complete the Rapid ED overdose data form metadata. CDC production of the reports greatly reduces burden on participating health departments. Two other advantages of sharing case-level ED data through this platform are:
NSSP BioSense is constantly improving their data sharing platform and analysis tools such as ESSENCE. This data collection effort will continue to leverage these improvements as they are implemented by CDC.
CDC DOSE staff are closely collaborating with the CDC NSSP BioSense team to better automate analysis of suspected drug, opioid, heroin and stimulant overdoses. This includes creating tools to help CDC as well as state and local health departments identify and respond to data quality issues. Additionally, local and state health departments will be able to apply the standardized DOSE definitions of suspected drug, opioid, heroin and stimulant overdoses to identify trends in their local ED data in near-real time.
For the health departments able to share syndromic data (N=43), 10 are not expected to share case-level ED data with CDC through NSSP BioSense. To assist these states:
DOSE has developed SAS, R, and ESSENCE programming code (Note: some health departments use a local version of ESSENCE that is not part of CDC NSSP) that automatically identifies ED visits involving drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses and formats the data in a manner consistent with the Rapid ED overdose data form. This will substantially reduce the burden on participating health departments of completing the Rapid ED overdose data form.
DOSE will require that the health departments share ED data using the Rapid ED overdose data form and standard CDC case definitions. The Rapid ED overdose data form is an Excel template, which reduces burden on participating health departments in the following ways:
Enables health departments to create computer programs and standard operating procedures for sharing the data with CDC using CDC’s standard format. State health departments have requested CDC design and adhere to standard data sharing protocols.
Excel is widely used and most likely will not require staff from participating health departments to acquire new training,
Many statistical programs can export data into Excel,
Manual data entry is user friendly, and
CDC staff can build multiple automated data quality checks into the Excel data sheet that capture data errors early and prevent the need and burden on participating health departments of submitting revised reports to CDC.
All funded state health departments will be asked to share hospital discharge data with CDC quarterly (N=28 expected) or yearly (N=23 expected) using the ED discharge overdose data form and standard CDC case definitions. Similar to the Rapid ED overdose data form, the ED discharge overdose data form is an Excel template, which reduces burden on participating health departments in multiple ways discussed above.
Participating health departments will share the Rapid ED overdose data form and ED discharge overdose data form with CDC using a National Center for Injury Prevention and Control (NCIPC) interface hosted on the CDC Secure Access Management Service (SAMS) Partner’s Portal, referred to as the NCIPC Partner’s Portal. Two advantages of the NCIPC Partner’s Portal are:
The NCIPC Partner’s Portal will improve data quality and reduce burden on participating health departments by automatically identifying data submission errors by participating health departments. Real-time identification of data submission errors enables rapid fixes and reduces the chance participating health departments will need to make multiple data submissions to CDC.
The NCIPC Partner’s Portal is a website designed to provide centralized access to external users (e.g., state and local health departments) to data and computer applications operated by CDC.34 The NCIPC Partner’s Portal leverages the CDC SAMS Partner’s Portal because CDC SAMS is an established secure method for sharing data that is widely used by state and local health departments. Thus, the time required to gain access to and use of the portal will be minimal.
4. Efforts to Identify Duplication and Use of Similar Information
DOSE maximizes the use of federal government data by leveraging ED data already collected by the NSSP BioSense platform (mentioned previously on A1. Circumstances Making the Collection of Information Necessary) and communicating on an ongoing basis with other federal collections of ED data. CDC’s Division of Overdose Prevention (DOP) is engaged in an intensive collaboration with the Division of Health Informatics (DHIS), the CDC division that operates NSSP BioSense. The collaboration enhances rapid surveillance of suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses by leveraging the approximately 71% of ED facilities in the US that are shared by local health departments or their partners with CDC through the NSSP BioSense Platform.
CDC prefers that participating health departments share ED data with CDC through the NSSP BioSense platform. The NSSP BioSense platform is the preferred mechanism because it minimizes burden on participating health departments by allowing CDC to complete the Rapid ED overdose data form while maximizing data quality by enabling collaborative analyses between CDC and health departments. In CDC’s Overdose Data to Action Notice of Funding Opportunity (OD2A, CDC-RFA-CE19-1904), DOP strongly encouraged all eligible health departments to share ED data with CDC through NSSP BioSense and funded those who agree at higher levels than health departments sharing ED data from other sources (e.g., local ED syndromic systems or hospital discharge).
DOP took a number of actions to identify and contact other federal programs collecting ED data to ensure coordination and avoid duplication. Through previous conversations with OMB, NCHS, SAMHSA, and CDC’s Opioid Response Coordinating Unit (ORCU), five federal government data systems in addition to NSSP BioSense were identified as potentially overlapping with the current data collection. Below, a brief description of each data system is provided as well as why DOSE is not duplicative with the data collection.
ED data system |
Description |
Time lag |
Purpose of ED system |
Additional Value of DOSE |
Recent contact |
Health Care Utilization Project (HCUP) Nationwide Emergency Department Sample (NEDS)35 administered by the Agency of Healthcare Research and Quality (AHRQ)
OMB #: 0935-0206
|
37 states contribute data to NEDS. In 2016, the database contains a sample of around 33 million ED visits that can be used to make national estimates of ED visits involving specific illnesses and injury. Key data include ICD-10-CM diagnosis and procedure codes as well as medical charge and patient demographics. |
~ 2 years |
NEDS data are used to estimate the national burden of ED visits related to drug overdoses.
|
|
|
Health Care Utilization Project (HCUP) State Emergency Department Databases (SEDD)37 administered by the Agency of Healthcare Research and Quality (AHRQ)
OMB #: 0935-0206 |
SEDD includes all ED visits that did not result in a hospitalization from about 37 states. Data and access conditions vary across state. As of May 1, 2019, 22 states provided access to 2016 data and 11 states provided access to 2017 data. |
~ 2-3 years |
SEDD data are used to estimate the burden of ED visits related to drug overdoses by state. Only a subset of states provide public access to their data. |
|
|
The Drug Abuse Warning Network (DAWN) operated by SAMHSA39
OMB # 0930-0078 |
In fall 2018, SAMHSA restarted the DAWN surveillance system after a 7 year period of inactivity. Trained abstractors employed by the DAWN data contractor will mine existing hospital electronic health record (EHR) systems. DAWN began data abstraction in mid-2019 with a 25 hospitals, and grew to a total of 49 hospitals at the end of 2020. The new DAWN will be an early warning system that detects increases in drug-related ED visits, or outbreaks, and characterizes the outbreak using unique information such as specific drug or drug brand, manually abstracted from direct chart review. DAWN will also detect the emergence of new psychoactive substances and identify all ED visits that are related to drugs such as adverse events or injuries caused by drug use as well as drug overdoses. |
Reporting frequency is still be determined, but likely will be monthly to quarterly. |
The new DAWN is bringing in hospitals in three categories: 1) hospitals purposively selected from areas with high drug overdose burden; 2) hospitals sampled from suburban and rural areas with high levels of fatal drug overdoses; and 3) hospitals sampled from areas stratified by region, fatal drug overdose rates and counts. This design provides a framework suitable for sentinel surveillance and for national estimation, should the system expand. |
|
|
National Electronic Injury Surveillance System -- Cooperative Adverse Drug Event Surveillance (NEISS-CADES) project conducted by CDC in collaboration with FDA and Consumer Product Safety Commission.40
OMB #: 3041-0029
|
NEISS-CADES collects nationally representative data from a sample of under 100 hospitals and uses data abstractors to collect in-depth information on ED visits on adverse events related to prescription drug use including overdose. In 2017, NEISS-CADES began collecting ED visits involving prescription opioid overdoses as well as ED visits involving adverse events. NEISS-CADES has abstractors manually review targeted ED visits |
~1 to 2 years |
NEISS-CADES data are used to estimate the national burden of ED visits related to drug overdoses and adverse events. The system cannot make regional or state estimates and does not collect information on opioid overdoses related to illicit opioids such as heroin or illicitly-made fentanyl or allow for state estimates. |
|
|
SAMHSA Emergency Department Surveillance System (SEDSS) leverages data from the National Center for Health Statistics (NCHS) National Hospital Care Survey (NHCS). 41
OMB #: 0920-0212
|
SEDDS draws EDs from the NHCS nationally representative sample of ~500 acute care hospitals. Of this sample, around 200 EDs participate in SEDDS. While not nationally representative, work continues to become nationally representative. SEDDS automatically receives data from hospitals’ electronic health records (EHR) and currently uses diagnostic and procedural codes from the UB-04 administrative claims data to identify and describe drug-related ED visits.42 Approximately 1,000 acute care hospitals outside the NHCS sample have expressed interest in the program, but are not sending data at this time. |
The current time lag in reporting is 3 years, with the most recent data from 2015. The electronic infrastructure of the system, however, lends itself to rapid data sharing over time.
|
SEDSS collects data electronically and in the future may be able to produce national estimates. Also, SEDSS collects identifiable patient information that can be used to link SEDSS to other datasets (e.g., the National Death Index) or track the same patient overtime. |
|
|
As DOSE is implemented, DOP will continue to communicate with other federal ED data collections to avoid duplication and identify opportunities for collaboration. Possible opportunities for collaboration include:
Comparisons of DOSE findings with SEDSS, HCUP or DAWN findings in similar geographic areas or hospitals could help inform revisions and improvements in DOSE’s syndromic definitions of ED visits involving drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses.
If a DAWN hospital reports an outbreak or is located in an area identified by DOSE as experiencing a drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug outbreaks, DAWN data could provide critical in-depth information on specific drugs involved and clinical symptoms of a drug overdose to inform the response. This in-depth data is a unique strength of the DAWN system.
5. Impact on Small Businesses or Other Small Entities
This study does not impact small businesses or other small entities. It impacts state health departments and the District of Columbia whose ED records will be shared with CDC.
6. Consequences of Collecting the Information Less Frequently
If DOSE collects data less frequently, the following adverse consequences will occur:
Federal and state governmental situational awareness of emerging drug overdose outbreaks and the progression of the opioid overdose epidemic, currently a national public health emergency, will be substantially slowed. This will erode the ability of federal and state health departments to rapidly respond to drug overdose outbreaks. Rapid situational awareness is especially critical now as overdoses related to fentanyl and heroin have sharply increased and been accompanied by multiple local and state reports of severe and often widespread opioid overdose outbreaks since 2013. 44,45,46,47,48 Also, the nature and complexity of drug overdoses continues to rapidly evolve with distribution of counterfeit prescription pills laced with fentanyl,49 cocaine products laced with fentanyl,50 increasing overdoses involving synthetic cannabinoids51 and large increases in overdose deaths involving methamphetamines and cocaine.52 Without monthly national data sharing between participating health departments and CDC through DOSE, intervention efforts will continue to fall far behind changes in the drug market and usage patterns driving drug overdoses.
Public situational awareness of emerging drug overdose outbreaks and the progression of the opioid overdose epidemic will be substantially slowed. This may slow intervention efforts by non-governmental organizations and citizens. Currently, limited timely local and state data are available on nonfatal and fatal drug overdoses. The National Center of Health Statistics publishes preliminary drug overdose death data from death certificates with a 7-month delay.53 These data, however, are only available at the state level, reports a 12-month rolling average which will be slow to detect change, and does not provide any information by demographic groups. National and state hospital discharge data on drug overdose ED visits is available from the Health Care Utilization Project with a two to three year delay and not available for all states.54
Local health department surveillance and response to suspected, drug, opioid, heroin and stimulant overdoses would be diminished. First, there would be a longer time lag in local health departments learning about large multi-state outbreaks that threaten to affect their jurisdiction. Second, DOSE reporting is accompanied by data quality efforts. Reducing the frequency of these data quality efforts would likely lead to less timely and effective identification of data quality problems that could diminish the ability of a local health department to accurately detect overdose outbreaks.
7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
This data collection will require monthly reporting of aggregate ED data on suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses using the Rapid ED overdose data form. This is more rapid than quarterly data sharing recommended by OMB. Monthly sharing of ED data is critical to fulfill the mission of DOSE, which is timely response to drug overdose outbreaks and emerging changes in drug overdoses. Data collected less frequently will slow response and consequently may increase harm caused by drug overdose outbreaks.
Rapid situational awareness is especially critical now as overdoses related to fentanyl and heroin have sharply increased and been accompanied by multiple local and state reports of severe and often widespread opioid overdose outbreaks since 2013.55,56,57,58,59 Also, the nature and complexity of the drug overdoses continues to rapidly evolve with distribution of counterfeit prescription pills laced with fentanyl,60 cocaine products laced with fentanyl,61 increasing overdoses involving synthetic cannabinoids62 and large increases in overdose deaths involving methamphetamines and cocaine.63 Without monthly national data sharing between participating health departments and CDC through DOSE, intervention efforts will continue to fall far behind changes in the drug market driving drug overdoses.
DOSE works to mitigate the burden of monthly reporting on participating health departments by:
Providing funding to participating health departments to offset burden related to completing required data sharing forms, the Rapid ED overdose data form and ED discharge overdose data form, on a monthly basis and quarterly basis, respectively,
Providing substantial technical assistance to participating health departments in completing reports. This includes:
CDC staff completing the Rapid ED overdose data form each month for the estimated 35 health departments sharing case-level ED with CDC through NSSP BioSense.
For participating health departments not sharing through NSSP BioSense, CDC is developing SAS, R and ESSENCE programming code (Note: some health departments use a local version of ESSENCE that is not part of CDC NSSP) that will allow the participating health department to identify suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses in their ED data and aggregate the data in a format consistent with the Rapid ED overdose data form. This will substantially reduce the burden of completing the form.
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside
the Agency
Federal Register Notice
A 60-day Federal Register Notice was published in the Federal Register on April 26, 2021, vol. 86, No. 78, pp. 22052-3 (Att. B). For this notice CDC received and responded to four comments. All comments supported the collection, some proposed to include a mental health component to the collection and others proposed ways to mitigate unreported cases in rural areas. This proposed collection is one aspect of a larger investment made by HHS and CDC called Overdose Data to Action (OD2A). OD2A focuses on understanding and tracking the complex and changing nature of the drug overdose epidemic and highlights the need for seamless integration of data into prevention strategies, therefore no changes were made to the data collection instruments based on these comments (Att. B1).
Efforts to Consult Outside the Agency
DOP, NCIPC is currently receiving feedback from state public health departments and the District of Columbia on improving rapid ED surveillance of suspected drug overdoses who are participating in DOSE. Secondly, intensive consultation with Ohio,64 Massachusetts, and Rhode Island65 during three Epi-Aid responses to drug overdoses outbreaks coupled with technical assistance to other states responding to increases in drug overdoses highlight key opportunities and challenges of using ED data for rapid surveillance of overdoses.
DOP, NCIPC also consulted with NCHS, CDC; SAMHSA; NCEZID, CDC; and ORCU, CDC to learn from and avoid duplication with other federal government efforts to collect data on ED visits involving drugs (mentioned previously on A4.Efforts to Identify Duplication and Use of Similar Information)
9. Explanation of Any Payment or Gift to Respondents
No incentives, payments or gifts will be provided to survey participants
10. Protection of the Privacy and Confidentiality of Information Provided by Respondents
The CDC Office of the Chief Information Officer has determined that the Privacy Act does not apply to this information collection request. (Att. F).
Four main strategies will be implemented to maintain the confidentiality of the data.
State health departments and the District of Columbia (the respondents), will only share with CDC aggregate data collected on two standardized forms: the Rapid ED overdose data form and the ED discharge overdose data form. Although CDC will have access to case-level ED data through NSSP BioSense for 35 health departments in order to automate completion of the Rapid ED overdose data form, CDC will only report on aggregate ED data entered into the Rapid ED overdose data form. Case-level ED data will only be used by CDC to assist public health departments complete the Rapid ED overdose data form on a monthly basis and assist health departments and CDC improve data quality and overdose identification.
Participating health departments will submit the Rapid ED overdose data form and the ED discharge overdose data form to CDC using the NCIPC Partner’s Portal hosted on the CDC Secure Access Management Service (SAMS) site. The CDC SAMS Partner’s Portal is a web site designed to provide secure centralized access to external users such as public health departments to data and computer applications operated by CDC. It can also be used to securely exchange data between CDC and participating health departments.66
Only selected staff working in the DOSE program will have access to aggregate data entered into the Rapid ED overdose data form and the ED discharge overdose data form by participating health departments. Also, Excel files as well as analytical statistical files will be stored and managed on secure CDC servers.
DOSE will follow NCHS guidelines on suppression of small sample sizes in data tabulations (e.g., not report any information that involves between 1 and 9 people) to prevent the inadvertent identification of an individual through the combination of various demographic characteristics.
11. Institutional Review Board (IRB) and Justification for Sensitive Questions
The CDC National Center for Injury Prevention and Control’s OMB and human subject’s liaison has determined that the activity is not research and IRB approval is not needed. This data collection is a surveillance effort and human subjects will not be involved (Att. C).
12. A. Estimates of Annualized Burden Hours and Costs
This data collection includes two data forms:
Rapid ED overdose data form (Att D) supports rapid monthly ED surveillance of suspected drug, opioid, heroin and stimulant overdoses to detect outbreaks and provide situational awareness (Att. D). The Rapid ED overdose data form asks jurisdictions to use existing local ED data to calculate the total number of suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdose that occurred each month by county and by age, sex, and race/ethnicity. Also, jurisdictions will be asked to provide metadata including coverage of the local ED surveillance system (i.e., percentage of all ED visits captured by the jurisdiction’s ED surveillance) and recent major changes in the local ED data collection efforts (e.g., large number of hospitals begin or terminate participation). The burden of completing the Rapid ED overdose data form will vary across two groups: 1) health departments sharing case-level ED data with CDC through the NSSP BioSense Platform (OMB #0920-0824) and thus completing the Rapid ED overdose data form with substantial CDC assistance and 2) health departments not sharing data through NSSP BioSense Platform and completing the Rapid ED overdose data form with minimal CDC assistance. The burden for these two groups is described below.
Health departments sharing data with CDC through NSSP BioSense: CDC expects at least 35 of the funded health departments to provide CDC access to case-level ED data through the NSSP BioSense Platform (OMB #0920-0824). Using the case-level data, CDC will complete the vast majority of the Rapid ED overdose data form including calculating the total number of suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses that occurred each month by county and by age, sex, and race/ethnicity. CDC, however, will consult with the jurisdiction when completing metadata and this will result in a small burden. Based on CDC collaborative work with health departments using the NSSP BioSense Platform, the burden will be .5 hours per monthly report per jurisdiction. Thus, the annual burden per health department will be 6 hours (.5 hours x 12 months) or a total of 210 burden hours across the 35 responding health departments.
Health departments sharing local syndromic ED data or hospital discharge data: CDC expects about 10 of the funded health departments (i.e., the health departments not providing CDC NSSP BioSense access) to complete the Rapid ED overdose data form using local ED syndromic or hospital discharge ED data. Based on optional CDC collaborative work with health departments on completing similar forms and the fact the health department will be completing the form instead of CDC, the burden will be 3 hours per report by a health department. Because CDC has created SAS programs that automatically identifies ED visits involving all of these drug categories we do not anticipate any burden increase when adding new drug overdose categories. The form will be completed monthly for an annual burden per health department of 36 hours (3 hours x 12 months) or a total of 360 annual burden hours across all 10 responding health departments.
ED discharge overdose data form (Att E) supports quarterly or yearly hospital discharge surveillance of ED visits involving suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses to evaluate rapid ED surveillance and assess drug overdose burden (Att. E). The preliminary ED data captured by the Rapid ED overdose data form needs to be compared and validated against hospital discharge data that uses ICD-10-CM codes, currently the standard for tracking ED visits. All participating health departments will be asked to complete the ED discharge overdose data form using ED discharge data already routinely collected in their jurisdiction. Specifically, participating health departments will calculate and report the total number of suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses by county and by age, sex, and race/ethnicity and provide metadata such as coverage of the ED hospital discharge surveillance system (i.e., percentage of all ED visits captured by the jurisdiction’s ED surveillance). Because CDC has created SAS programs that automatically identifies ED visits involving all of these drug categories we do not anticipate any burden increase when adding new drug overdose categories. Due to variations in the collection of local ED hospital discharge data across jurisdictions, CDC expects up to 28 health departments to submit data at least quarterly and at least 23 health departments to submit data yearly. The burden associated with each of these groups is described below.
CDC estimates that up to 28 jurisdictions will submit data to CDC quarterly. Based on optional experiences of jurisdictions completing similar templates, the ED discharge overdose data form will take a jurisdiction 3 hours to complete and will be completed quarterly (i.e., every 3 months), for an annual burden per jurisdiction of 12 hours (3 hours x 4 submissions every year) or a total of 336 annual burden hours across 28 jurisdictions.
The remaining 23 jurisdictions are expected to only have the capacity to share hospital discharge data yearly. Based on optional experiences of health departments completing similar templates, the ED discharge overdose data form will take a jurisdiction 3 hours to complete and will be completed yearly, for an annual burden per jurisdiction of 3 hours (3 hours x 1 submissions every year) or a total of 69 annual burden hours across 23 jurisdictions.
This breakdown is also visually depicted below in Figure 2.
Figure 2: Requested and expected state data sharing for DOSE in 2023
Table 1. Estimates of annualized respondent burden hours
Type of respondent |
Form name |
No. of respondents |
Total no. of responses per respondent |
Average burden per response (hours) |
Total annual burden (hours) |
Participating health departments sharing aggregate data from local syndromic or hospital discharge file |
Rapid ED overdose data form (Att. D) |
10 |
12 |
3 |
360 |
Participating health departments sharing case-level ED data with CDC through the NSSP BioSense (OMB #0920-0824)* |
Rapid ED overdose data form (Att. D) |
35 |
12 |
30/60 |
210 |
Participating health department sharing finalized hospital discharge data on a quarterly basis |
ED discharge overdose data form (Att. E) |
28 |
4 |
3 |
336 |
Participating health department sharing finalized hospital discharge data on a yearly basis |
ED discharge overdose data form (Att. E) |
23 |
1 |
3 |
69 |
Total |
|
|
|
|
975 |
* The reporting burden for jurisdictions sharing case-level ED data with CDC is substantially lower because CDC completes most of the form for the jurisdiction and only needs to consult with the jurisdiction on completing the metadata.
Estimates of annualized respondent burden costs:
Because staff retrieving and sharing specified data with CDC will vary substantially across organizations, the mean hourly wage of federal, state, and local government employees ($29.87) as estimated by the Bureau of Labor Statistics (https://www.bls.gov/oes/current/999001.htm#00-0000 , accessed on 18Jan2022) was used to estimate burden costs.
Table 2. Estimates of annualized respondent burden costs
Type of respondent |
No. of respondents |
No. of responses per respondent |
Total burden (hours) |
Hourly wage rate |
Total respondent cost |
Participating health departments sharing aggregate data from local syndromic or hospital discharge file |
10 |
12 |
360 |
$29.87 |
$10,753 |
Participating health departments sharing case-level ED data with CDC through the NSSP BioSense (OMB #0920-0824)* |
35 |
12 |
210 |
$29.87 |
$6,273 |
Participating health department sharing finalized hospital discharge data on a quarterly basis |
28 |
4 |
336 |
$29.87 |
$10,036 |
Participating health department sharing finalized hospital discharge data on a yearly basis |
23 |
1 |
69 |
$29.87 |
$2,061 |
Total |
|
|
|
|
$29,123 |
* The reporting burden for jurisdictions sharing case-level ED data with CDC is substantially lower because CDC completes most of the form for the jurisdiction and only needs to consult with the jurisdiction on completing the metadata.
13. Estimates of Other Total Annual Cost Burden to Respondents or Record
Keepers
Respondents will incur no capital or maintenance costs.
14. Annualized Cost to the Government
These costs fall into several categories, listed below:
Table 3. Cost to government - Contractor
LABOR |
COST |
Contract to fund two data managers (50%) |
$87,500 |
Other Direct Costs |
|
Subcontractors |
$0 |
Travel and subsistence |
$0 |
Total Estimated Contract Costs |
$87,500 |
Government costs
Table 4. Cost to government - Government
Personnel |
Tasks |
Avg. cost/yr |
Senior scientist (75%) |
Program oversight and strategic direction |
$ 92,039 |
6 Epidemiologists (75%) |
|
$468,000 |
14 Public health advisors (5%) |
Programmatic, budgetary, administrative management and oversight of DOSE as part of OD2A NOFO |
$61,250 |
2 Data managers (100%) |
|
$240,000 |
Indirect costs for staff (25%) |
|
$861,289 + $215,322 |
Sub-total |
|
$1,076,611 |
Contract Costs |
|
$87,500 |
Total |
|
$1,164,111 |
Total annual contractual and government staff costs are approximately $1.17 million. This is a four year project.
15. Explanation for Program Changes or Adjustments
This serves as a revision request for the currently approved package (Drug Overdose Surveillance and Epidemiology (DOSE) – OMB# 0920-1268, expiration date 08/31/2022). Funding for DOSE has been extended by one year (new end date: September 2023) and we are requesting an additional three years of data collection. Revisions are requested to revise the number of eligible states, change the data collection template (see Attachment D-2), and revise burden. No additional burden has been added; however, based on current data sharing from states we have decreased our burden estimate to 975 from 1272 hours (i.e., we estimated 19 states would be sharing data outside NSSP when in fact, only 10 states are sharing data outside NSSP).
16. Plans for Tabulation and Publication and Project Time Schedule
Monthly, quarterly, and yearly trends in ED visits involving suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses at the state level will be reported publicly on an ongoing basis by CDC. Additional analyses examining data by age group, sex, race/ethnicity and county will also be conducted as well as comparison of ED trends with other data sets such as drug overdose mortality. These additional analyses will be released in CDC publications such as MMWR or in other peer-reviewed publications as well as available every six months on the CDC nonfatal drug overdose website (https://www.cdc.gov/drugoverdose/nonfatal/index.html). A project time schedule is presented below.
Table 5. Time Schedule
Task |
Time Period |
Ongoing processing of monthly reports of ED visits involving suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses submitted by public health departments to CDC (i.e., Rapid ED overdose data form) |
|
Receive on a monthly basis Rapid ED overdose data form (i.e., Data form 1) from the jurisdiction |
1 – 2 month delay from when the overdose ED visit occurred (e.g., overdoses occurring in January 2020 will be reported to CDC in March 2020). |
Final analysis files validated within 1 month of receipt of data from jurisdiction. Preliminary data is shared with participating health department and CDC/HHS leadership |
2 – 3 month delay from when the overdose ED visit occurred (e.g., analytic file for overdoses occurring in January 2020 will be completed by CDC by the end of April 2020 or earlier) |
At least twice a year, quarterly or monthly changes in ED visits involving suspected drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses will be posted on the web for public access* |
6 – 12 months (e.g., quarterly changes in drug and opioid overdoses occurring from January – March, 2020 to April – June, 2020 will be reported to the web in December, 2020). |
Publish surveillance reports and epidemiologic analyses of DOSE data to support public health prevention efforts |
|
Analyze trends in ED visits to identify important patterns to inform public health action and improve syndromic ED definitions of drug, opioid, heroin, fentanyl, all stimulant, cocaine, methamphetamine, benzodiazepine, and other emerging drug overdoses |
At least three articles per year will be published, starting 1 year after the DOSE system begins operating. |
Conduct analyses to support improved data collection and analysis |
|
Conduct ongoing comparisons of ED data collected on the Rapid ED overdose data form with ED data collected on the ED discharge overdose data form to inform improvements to both data collections |
These analyses will start 6 months after the DOSE system begins operating and sufficient data is available to compare across data sources. |
*Data are posted here: https://www.cdc.gov/drugoverdose/data/nonfatal.html .
Initial publications focused on:
Identifying patterns of polysubstance use among individuals treated in the ED for drug overdoses and how this patterns varies across demographic groups in order to better target interventions.
Determining the extent to which geographically concentrated opioids outbreaks versus gradual changes in opioid overdose contribute to large increases in ED visits involving suspected opioid overdoses. This can inform public health interventions and help identify and respond to emerging public health outbreaks.
Comparing state and country trends in ED visits involving suspected drug, opioid, heroin, and stimulant overdoses with trends observed in drug, opioid, heroin and stimulant overdose deaths. This will help validate and improve the current data collection as well as provide insight into whether efforts to decrease fatal opioid overdoses by enhancing response are working.
Comparing the data sources collected – syndromic and discharge – to better understand strengths and weaknesses of the two different, yet connected data sources.
Future publications will focus on:
Exploration of trends in other drug types such as fentanyl, benzodiazepine, cocaine, and methamphetamine (separate from all stimulants)
Assessment of relationships between health inequities and social determinents of health and suspected drug overdose.
17. Reason(s) Display of OMB Expiration Date Is Inappropriate
There are no standard paper data collection forms to be used in this data collection. Instead, the participating health departments share the requested ED data with CDC using two Excel files, the Rapid ED overdose data form and the ED discharge overdose data form. The OMB number will be displayed on the Rapid ED overdose data form and the ED discharge overdose data form distributed to state and local health departments.
18. Exceptions to Certification for Paperwork Reduction Act Submissions
This collection of information involves no exception to the Certification for Paperwork Reduction Act Submissions.
1 Hedegaard H, Miniño AM, Warner M. Drug overdose deaths in the United States, 1999–2017. NCHS Data Brief, no 329. Hyattsville, MD: National Center for Health Statistics. 2018.
2 Gladden, R.M., Martinez, P., and Seth, P. (2016). “Fentanyl Law Enforcement Submissions and Increases in Synthetic Opioid–Involved Overdose Deaths — 27 States, 2013–2014”. Morbidity and Mortality Weekly Report 65(33):837–843. DOI: http://dx.doi.org/10.15585/mmwr.mm6533a2.
3 CDC. Rising numbers of deaths involving fentanyl and fentanyl analogs deaths, including carfentanil, and increased usage and mixing with non-opioids. HAN no. 413. Atlanta, GA: US Department of Health and Human Services, CDC; 2018. https://emergency.cdc.gov/han/han00413.asp.
4 Somerville, N.J., et al. (2017). “Characteristics of Fentanyl Overdose — Massachusetts, 2014–2016.” Morbidity and Mortality Weekly Report 66(14):382–386. DOI: http://dx.doi.org/10.15585/mmwr.mm6614a2.
5 O’Donnell, J, et al. (2018). “Notes from the Field: Overdose Deaths with Carfentanil and Other Fentanyl Analogs Detected — 10 States, July 2016–June 2017”. Morbidity and Mortality Weekly Report 67(27):767–768. DOI: http://dx.doi.org/10.15585/mmwr.mm6727a4.
6 Spencer, MR. et al. Drug overdose deaths involving fentanyl, 2011–2016. National Vital Statistics Reports. 68(3). Hyattsville, MD: National Center for Health Statistics. 2019.
7 Additional information on President Trump’s Initiative to Stop Opioid Abuse, https://www.whitehouse.gov/opioids/.
8 Additional information on the Enhanced State Opioid Overdose Surveillance (ESOOS) program, https://www.cdc.gov/drugoverdose/foa/state-opioid-mm.html.
9 Seth, P., et al. (2018). “Overdose Deaths Involving Opioids, Cocaine, and Psychostimulants — United States, 2015–2016”. Morbidity and Mortality Weekly Report 67(12):349–358.
10 CDC. Rising numbers of deaths involving fentanyl and fentanyl analogs deaths, including carfentanil, and increased usage and mixing with non-opioids. HAN no. 413. Atlanta, GA: US Department of Health and Human Services, CDC; 2018. https://emergency.cdc.gov/han/han00413.asp.
12 Hedegaard H, Miniño AM, Spencer MR, Warner M. Drug overdose deaths in the United States, 1999–2020. NCHS Data Brief, no 428. Hyattsville, MD: US Department of Health and Human Services, CDC, National Center for Health Statistics; 2021. https://www.cdc.gov/nchs/products/databriefs/db428.htm
13 Additional information on the opioid overdose epidemic, https://www.cdc.gov/drugoverdose/epidemic/index.html. Mattson CL, Tanz LJ, Quinn K, Kariisa M, Patel P, Davis NL. Trends and Geographic Patterns in Drug and Synthetic Opioid Overdose Deaths — United States, 2013–2019. MMWR Morb Mortal Wkly Rep 2021;70:202–207. DOI: http://dx.doi.org/10.15585/mmwr.mm7006a4 .
14 https://www.cdc.gov/drugoverdose/nonfatal/index.html
15 Additional information on President Trump’s Initiative to Stop Opioid Abuse, https://www.whitehouse.gov/opioids/.
18 Additional information on the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) coding system, https://www.cdc.gov/nchs/icd/icd10cm.htm.
19 Gould, D. W., Walker, D., and Yoon, P. W. (2017). The evolution of BioSense: lessons learned and future directions. Public Health Reports, 132(1_suppl), 7S-11S.
20 US Government Accountability Office (2015). Information technology: federal agencies face challenges in implementing initiatives to improve public health infrastructure. http://www.gao.gov/new.items/d05308.pdf.
21 O’Donnell JK, et al. (2017). “Deaths Involving Fentanyl, Fentanyl Analogs, and U-47700 — 10 States, July–December 2016”. Morbidity and Mortality Weekly Report 66(43):1197–1202. DOI: http://dx.doi.org/10.15585/mmwr.mm6643e1.
22 O’Donnell, J, et al. (2018). “Notes from the Field: Overdose Deaths with Carfentanil and Other Fentanyl Analogs Detected — 10 States, July 2016–June 2017”. Morbidity and Mortality Weekly Report 67(27):767–768. DOI: http://dx.doi.org/10.15585/mmwr.mm6727a4.
23 Law, R, et al. (2015). “Notes from field: Increase in reported adverse health effects related to synthetic cannabinoid use – United States, January – May 2015”. Morbidity and Mortality Weekly Report 64(22):618–619. DOI: https://www.cdc.gov/mmwr/pdf/wk/mm6422.pdf.
24 Spies, E., et al. (2016). Undetermined risk factors for fentanyl-related overdose deaths – Ohio, 2015 (EpiAid 2016-003) trip report. https://odh.ohio.gov/wps/portal/gov/odh/know-our-programs/violence-injury-prevention-program/media/CDC_EpiAid_Ohio. Accessed April 26, 2019.
25 Vivolo-Kantor, AM, et al. (2018). “Vital Signs: Trends in Emergency Department Visits for Suspected Opioid Overdoses — United States, July 2016–September 2017.” Morbidity and Mortality Weekly Report 67:279–285. DOI: http://dx.doi.org/10.15585/mmwr.mm6709e1External.
26 CDC. Trends in emergency department visits for suspected opioid overdose, Q2 2017 to Q2 2018: CDS’s Enhanced State Opioid Overdose Surveillance Program, data current as of October 15, 2018. https://www.cdc.gov/drugoverdose/data/nonfatal.html. Accessed April 26, 2019.
27 CDC. Influx of Fentanyl-laced Counterfeit Pills and Toxic Fentanyl-related Compounds Further Increases Risk of Fentanyl-related Overdose and Fatalities. HAN no. 395. Atlanta, GA: US Department of Health and Human Services, CDC; 2016. https://emergency.cdc.gov/han/han00395.asp.
28 Somerville, N.J., et al. (2017). “Characteristics of Fentanyl Overdose — Massachusetts, 2014–2016.” Morbidity and Mortality Weekly Report 66(14):382–386. DOI: http://dx.doi.org/10.15585/mmwr.mm6614a2.
29 Hedegaard H, Miniño AM, Warner M. Drug overdose deaths in the United States, 1999–2017. NCHS Data Brief, no 329. Hyattsville, MD: National Center for Health Statistics. 2018.
30 CDC. Increases in Fentanyl Drug Confiscations and Fentanyl-related Overdose Fatalities. HAN no. 384. Atlanta, GA: US Department of Health and Human Services, CDC; 2015. https://emergency.cdc.gov/han/han00384.asp.
31 CDC. Rising numbers of deaths involving fentanyl and fentanyl analogs deaths, including carfentanil, and increased usage and mixing with non-opioids. HAN no. 413. Atlanta, GA: US Department of Health and Human Services, CDC; 2018. https://emergency.cdc.gov/han/han00413.asp.
32 Seth, P., et al. (2018). “Overdose Deaths Involving Opioids, Cocaine, and Psychostimulants — United States, 2015–2016”. Morbidity and Mortality Weekly Report 67(12):349–358. DOI: http://dx.doi.org/10.15585/mmwr.mm6712a1External.
33 https://jamanetwork.com/journals/jamapsychiatry/article-abstract/2775991
34 Additional information on SAMS, https://auth.cdc.gov/sams/SAMSUserGuide.pdf.
35 Additional information on NEDS, https://www.hcup-us.ahrq.gov/nedsoverview.jsp.
36 Additional information on CDC data hub, https://www.cdc.gov/ophss/csels/dhis/documents/dhis-data-hub-508.pdf.
37 Additional information on SEDD, https://www.hcup-us.ahrq.gov/seddoverview.jsp.
38 Additional information on CDC data hub, https://www.cdc.gov/ophss/csels/dhis/documents/dhis-data-hub-508.pdf.
39Additional information on DAWN, https://www.samhsa.gov/data/data-we-collect/dawn-drug-abuse-warning-network.
40 Geller, A.I., et al. (2019). “U.S. emergency department visits resulting from nonmedical use of pharmaceuticals, 2016.” American Journal of Preventive Medicine. 56(5): 639-647. DOI: https://doi.org/10.1016/j.amepre.2018.12.009.
41 Brown, A.M., et al. (2018). Identification of substance-involved emergency department visits using data from
the National Hospital Care Survey. National Health Statistics Reports. 114. Hyattsville, MD: National Center for Health Statistics.
42 Ibid
43 Gould, D. W., Walker, D., and Yoon, P. W. (2017). The evolution of BioSense: lessons learned and future directions. Public Health Reports, 132(1_suppl), 7S-11S.
44 Gladden, R.M., Martinez, P., and Seth, P. (2016). “Fentanyl Law Enforcement Submissions and Increases in Synthetic Opioid–Involved Overdose Deaths — 27 States, 2013–2014”. Morbidity and Mortality Weekly Report 65(33):837–843. DOI: http://dx.doi.org/10.15585/mmwr.mm6533a2.
45 CDC. Rising numbers of deaths involving fentanyl and fentanyl analogs deaths, including carfentanil, and increased usage and mixing with non-opioids. HAN no. 413. Atlanta, GA: US Department of Health and Human Services, CDC; 2018. https://emergency.cdc.gov/han/han00413.asp.
46 Somerville, N.J., et al. (2017). “Characteristics of Fentanyl Overdose — Massachusetts, 2014–2016.” Morbidity and Mortality Weekly Report 66(14):382–386. DOI: http://dx.doi.org/10.15585/mmwr.mm6614a2.
47 O’Donnell, J, et al. (2018). “Notes from the Field: Overdose Deaths with Carfentanil and Other Fentanyl Analogs Detected — 10 States, July 2016–June 2017”. Morbidity and Mortality Weekly Report 67(27):767–768. DOI: http://dx.doi.org/10.15585/mmwr.mm6727a4.
48 Spencer, MR, et al. Drug overdose deaths involving fentanyl, 2011–2016. National Vital Statistics Reports. 68(3). Hyattsville, MD: National Center for Health Statistics. 2019.
49 DEA intelligence brief. (2016). Counterfeit Prescription Pills Containing Fentanyls: A Global Threat. DEA strategic intelligence section. DEA-DCT-DIB-021-16.
50 CDC. Rising numbers of deaths involving fentanyl and fentanyl analogs deaths, including carfentanil, and increased usage and mixing with non-opioids. HAN no. 413. Atlanta, GA: US Department of Health and Human Services, CDC; 2018. https://emergency.cdc.gov/han/han00413.asp.
51 Law, R, et al. (2015). “Notes from field: Increase in reported adverse health effects related to synthetic cannabinoid use – United States, January – May 2015”. Morbidity and Mortality Weekly Report 64(22):618–619. DOI: https://www.cdc.gov/mmwr/pdf/wk/mm6422.pdf.
52 Seth, P., et al. (2018). “Overdose Deaths Involving Opioids, Cocaine, and Psychostimulants — United States, 2015–2016”. Morbidity and Mortality Weekly Report 67(12):349–358. DOI: http://dx.doi.org/10.15585/mmwr.mm6712a1External.
53 Ahmad FB, Rossen LM, Spencer MR, Warner M, Sutton P. Provisional drug overdose death counts. National Center for Health Statistics. 2019. Retrieved April 29, 2019 from https://www.cdc.gov/nchs/nvss/vsrr/drug-overdose-data.htm.
54 Additional information on current availability of HCUP data by state, https://www.hcup-us.ahrq.gov/db/availability_public.jsp.
55 Gladden, R.M., Martinez, P., and Seth, P. (2016). “Fentanyl Law Enforcement Submissions and Increases in Synthetic Opioid–Involved Overdose Deaths — 27 States, 2013–2014”. Morbidity and Mortality Weekly Report 65(33):837–843. DOI: http://dx.doi.org/10.15585/mmwr.mm6533a2.
56 CDC. Rising numbers of deaths involving fentanyl and fentanyl analogs deaths, including carfentanil, and increased usage and mixing with non-opioids. HAN no. 413. Atlanta, GA: US Department of Health and Human Services, CDC; 2018. https://emergency.cdc.gov/han/han00413.asp.
57 Somerville, N.J., et al. (2017). “Characteristics of Fentanyl Overdose — Massachusetts, 2014–2016.” Morbidity and Mortality Weekly Report 66(14):382–386. DOI: http://dx.doi.org/10.15585/mmwr.mm6614a2.
58 O’Donnell, J, et al. (2018). “Notes from the Field: Overdose Deaths with Carfentanil and Other Fentanyl Analogs Detected — 10 States, July 2016–June 2017”. Morbidity and Mortality Weekly Report 67(27):767–768. DOI: http://dx.doi.org/10.15585/mmwr.mm6727a4.
59 Spencer, MR, et al. Drug overdose deaths involving fentanyl, 2011–2016. National Vital Statistics Reports. 68(3). Hyattsville, MD: National Center for Health Statistics. 2019.
60 DEA intelligence brief. (2016). Counterfeit Prescription Pills Containing Fentanyls: A Global Threat. DEA strategic intelligence section. DEA-DCT-DIB-021-16.
61 CDC. Rising numbers of deaths involving fentanyl and fentanyl analogs deaths, including carfentanil, and increased usage and mixing with non-opioids. HAN no. 413. Atlanta, GA: US Department of Health and Human Services, CDC; 2018. https://emergency.cdc.gov/han/han00413.asp.
62 Law, R, et al. (2015). “Notes from field: Increase in reported adverse health effects related to synthetic cannabinoid use – United States, January – May 2015”. Morbidity and Mortality Weekly Report 64(22):618–619. DOI: https://www.cdc.gov/mmwr/pdf/wk/mm6422.pdf.
63 Seth, P., et al. (2018). “Overdose Deaths Involving Opioids, Cocaine, and Psychostimulants — United States, 2015–2016”. Morbidity and Mortality Weekly Report 67(12):349–358. DOI: http://dx.doi.org/10.15585/mmwr.mm6712a1External.
64 Spies, E., et al. (2016). Undetermined risk factors for fentanyl-related overdose deaths – Ohio, 2015 (EpiAid 2016-003) trip report. https://odh.ohio.gov/wps/portal/gov/odh/know-our-programs/violence-injury-prevention-program/media/CDC_EpiAid_Ohio. Accessed April 26, 2019.
65 Mercado-Crespo, M. C., et al. (2014). "Notes from the Field: Increase in fentanyl-related overdose deaths - Rhode Island, November 2013 - March 2014." Morbidity and Mortality Weekly Report 63(24): 531.
66 Additional information on SAMS, https://auth.cdc.gov/sams/SAMSUserGuide.pdf.
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
File Title | Title: Test of Four Intimate Partner Violence (IPV) Measurement Scales |
Author | CDC User |
File Modified | 0000-00-00 |
File Created | 2022-02-23 |