Ssb_mat_1218

Broadly, the respondent universe includes individuals in the United States who receive some form of opioid use disorder (OUD) treatment. The sample from this broad universe has been operationalized by selecting 14 metropolitan statistical areas (MSAs) and recruiting four to five OUD treatment facilities per area. Study recruitment has ended, and the study has enrolled a total of 1,975 participants across 4 treatment groups. All participants have been screened for inclusion and exclusion criteria, signed an approved consent form, and completed a Baseline Patient Questionnaire. To achieve the desired statistical power, the study sought to enroll a target of 594 participants in each of the 4 treatment groups, for a total of 2,376 participants, however the target was not achieved in two of the 4 groups (naltrexone and counseling without medication). Therefore, comparisons will primarily be made between the two groups that exceeded the target (buprenorphine and methadone) (discussed in section A.15 of the SSA). This will be explained in more detail in section B.4. Study area is defined as the combination of metropolitan statistical area plus, in some instances, surrounding cities or corresponding larger cities within the same state.

B.1.1 Site Selection

Within each area, the study team began by building a list of treatment facilities that includes information on sampling criteria and contact information for recruitment. No single list of U.S. health facilities or practices that offer OUD treatment specifies the type of OUD treatments offered and the patient load. Therefore, the sampling frame for treatment facilities was built by triangulating a variety of sources (e.g., National Survey of Substance Abuse Treatment Services, Substance Abuse and Mental Health Services Administration, Drug Enforcement Agency list of buprenorphine providers, National Provider Index) to identify eligible facilities in the 14 study areas and facility contact information (e.g., address, phone number, clinic director or proprietor, treatments offered, patient load). The study team aimed to create a list of up to 20 OUD treatment facilities (sites) per study area from which five to six OUD treatment facilities were selected. The following criteria are being used to determine site inclusion:

Sufficient patient flow, about 10 new patients per month, to reach the approximate enrollment target of 66 participants per treatment facility.
Site diversity related to:
1. Organizational structure (e.g., clinic, group practice)
2. Management structure
3. Facility operations and locations
4. Numbers and types of providers
5. Patient characteristics such as age, gender, race or ethnicity, residence (urban or rural), and OUD treatment characteristics.

B.1.2 Patient Selection within Site

Once a treatment facility agreed to participate, potential participants who initiating a new treatment episode for OUD were identified. Prospective participants were eligible if they were 18 years of age or older and initiating one of four primary treatments for OUD: methadone maintenance treatment (MMT), buprenorphine (BUP), naltrexone (NTX), or counseling treatment without medication (COUN).^¹ Client referral information are maintained within a secure server environment for retrieval by approved RTI field staff (see Sections A.10 and B.2 for further information). Exhibit 1 summarizes the selected study areas and number of sites and patients participating in this study.

Exhibit 1. Sampling Stages for MAT Study

Sample Unit Description
14 geographic areas
Alabama (Birmingham)	Massachusetts (Boston)
Arizona (Phoenix)	North Carolina (Raleigh, Durham)
Colorado (Denver)	New York (New York City)
California (San Francisco, Los Angeles)	Ohio (Cincinnati)
Illinois (Chicago)	Texas (Dallas)
Washington DC Metro Area	Utah (Salt Lake City)
Kentucky/West Virginia (Huntington, Ashland)	Washington (Seattle)
62 treatment sites Four to five sites per study area on average
1,975 completed Patient Baseline Questionnaires Average of 31 participants per site within each study area Average of 138 participants with each study area

MAT Study = Medication-Assisted Treatment Study; OUD = opioid use disorder.

The study team received 3,536 patient referrals of which 1,975 were confirmed to be eligible and agreed to participate. The 3-month and 6-month Study Check-Ins were completed by 1,414 patients and 1,341 patients respectively, or 71.6% and 67.9% of the 1,975 Baseline Patient Questionnaire respondents. Finally, expected participation rates for the baseline and 12- and 18-month follow-up Patient Questionnaires are shown in Exhibit 2.

Of the 4 OUD treatment groups, two of the groups (NTX and COUN) failed to reach the target of 594 patients needed to ensure adequate statistical power. Preliminary data collected in the focus groups indicates a number of reasons for this. Clinicians and patients in our focus groups reported that they viewed counseling without MAT as a suboptimal course of treatment, explaining our observation of most opting for treatment that includes MAT. Naltrexone is more commonly used in inpatient settings and within the criminal justice healthcare system and is less commonly chosen for outpatient settings such as MAT Study sites. Because this study design relied on non-random assignment to each treatment group, we were unable to overcome these recruiting challenges.

The data collection approach outlined in Section B.3 will support the expected participation rates shown in Exhibit 2. Participation rates instead of response rates are used in acknowledgment of the nonprobability sample design and of standard definitions recommended by The American Association for Public Opinion Research (AAPOR, 2016; Baker et al., 2013). These rates are based on past studies such as Hser et al. (2015), and Desmond et al. (1995). For example, Hser et al (2015) conducted a long-term follow-up among patients initially randomized to receive buprenorphine or methadone. They found 89.4% of their targeted sample over an average 4.5-year follow-up period. In a case study of best practices for recruitment, Desmond et al studied illicit opioid users starting in MMT. Overall, 12-month follow-up was obtained for 98% of the sample. Broken down, 100% of those in treatment were followed up and 96% of those discharged. Of those discharged, 80% were followed up within 2 months of the anniversary window.

Exhibit 2. Patient Sampling for the Medication-Assisted Treatment Study

Patient Contact and Questionnaire	Number of Completed Interviews	Participation Rates
Patient Contact and Questionnaire	Number of Completed Interviews	Referrals (%)¹	Baseline (%)²
Patient referrals	3,536	100.0
Screened patients	2,338	66.1
Study eligible	2,307	65.2
Baseline	1,975	55.9	100.0
3-month check-in	1,414	40.0	71.6
6-month check-in	1,341	37.9	67.9
18-month follow-up	324³	9.2	16.4

MAT Study = Medication-Assisted Treatment Study.

²¹ Participation rates based on the number of patient referrals.

² Participation rates based on the number of baseline interviews. Note that participants are retained in the study even if they are unable to complete an interview between baseline and the 18-month follow-up.

³18-month follow-up data collection is ongoing. Based on prior experience plus findings and protocols discussed in for example, Novak et al. (2015), Novak et al. (2009), and Desmond et al. (1995), we anticipate completing approximately 674 18-month interviews by 2/28/2021 and 987 18-month interviews by 5/31/2021. The rates are not assumed to vary significantly across treatment regimen.

B.2 Procedures for the Collection of Information

Information is collected and stored on the Management System and is accessible to staff as needed. Identifying information is collected and stored separately from all other data collected, and a random ID are used to identify patient records. The data collection effort uses a secure data collection web system, Voxco, hosted on Enhanced Security Network (ESN), which meets federal requirements for high-security risk data. Identifying information will only be hosted on the ESN. Non-identifying information are collected through a separate web-based tool, also developed using Voxco, and a random patient ID is used to identify these records.

B.3 Methods to Maximize Participation Rates

B.3.1 Participation Rates

Various procedures are used to help maximize participation rates across the study; the procedures will help ensure a participation rate of 50.0% for the 18-month follow-up questionnaire. Additionally, the analysis techniques used will incorporate methods to account for nonresponse; analysis approaches are discussed in B.3.3. For patients, FIs will

build and maintain a relationship through calls, emails, and in person visits;
observe all other needed measures to protect patients’ rights as research subjects, their privacy, and the confidentiality of their data;
respond appropriately to patient distress;
follow up with patients should they not respond to email requests to complete the 18-month follow-up Patient Questionnaires; and
be available to patients via phone should patients have any questions.

If FIs lose contact with a patient, the FI will seek to locate the patient and reestablish contact. Methods used to locate subjects include reaching out to alternative contacts (e.g., parent, friend) obtained during enrollment; and checking local data sources (e.g., newspapers, court dockets). The patient will also be referred to a Tracing Unit, which conducts online searches (e.g., U.S. Postal Service change-of-address records, Department of Motor Vehicles records, credit reports). Tracking and tracing efforts have succeeded in retaining up to 10% of subjects lost to follow-up in other similar studies.

B.3.2 Nonparticipation

The project team has been able to successfully recruit and retain sites and patients, however due to nonparticipation it has been necessary to make adjustments to the timeline, inclusion criteria, statistical power calculations, and follow-up schedule. Two non-substantial change requests were submitted and approved to increase the number of MSAs included in the study from 11 to 14. The additional MSAs directly resulted in the addition of 13 new treatment sites and the recruitment of 667 additional patients. More information on these change requests can be found in section A.15 of the SSA. As a result of these adjustments, we recruited 1,210 patients into the MMT treatment group and 632 patients into the BUP treatment group, both of which are above the 594 target that we identified as sufficient for a 10% detectable difference with at least 90% power. This target relies on retaining no less than 50% of our patients through the 18-month follow-up period, which is currently being exceeded (61.6% of patients enrolled before January 2019 have completed their 18-Month Patient Questionnaire). However, as discussed above in section B.1.2, two of the treatment groups (NTX and COUN) failed to reach the recruitment target of 594 patients. We consider this a valuable finding, as it provides insight into the choices being made by patients and clinicians when selecting between treatment options for opioid use disorder. Each treatment option offers unique benefits, risks, and other factors, and we believe the findings of this study will provide insight into the reasons for these disparities in recruitment. For study participants, nonresponse will first be limited by the various patient retention strategies discussed in Section B.3.1. Negative effects linked to nonresponse will be examined through information collected in the Patient Screen former; if found, such characteristics will be included in the analytic models to mitigate any such biasing factors. In addition to unit nonresponse, imputation is another tool examined to address item nonresponse. The RTI team will incorporate models for non-random missingness into quantitative analyses to assess sensitivity to different assumptions about missing data. Non-ignorable missing at random methods generally fall under two categories: selection models (e.g., Heckman, 1979) and pattern mixture models (Hedeker & Gibbons, 1997), and both will be incorporated as appropriate during data analysis.

B.3.3 Analysis

Descriptive Statistics

Our analyses will begin by examining descriptive statistics for key variables. Specifically, descriptive statistics will include frequencies (e.g., sample sizes), measures of dispersion (e.g., standard deviation) and central tendency (e.g., mean, proportion), and an assessment of the variation within and among the treatment facilities (e.g., intraclass correlations). Key variables will include patients’ social, economic, clinical, and demographic characteristics and treatment facility’s operational structure, service focus, and approach to care.

Multilevel Analysis

We propose to estimate a variety of advanced statistical models for the population at large and for relevant subpopulations that consider key health, quality of life, and socioeconomic outcomes and that control for a rich set of patient- and treatment facility-level covariates. This study will be conducted in real-world settings and will rely on longitudinal primary data sources. Therefore, our approach can result in a more comprehensive, generalizable, and representative analysis and can therefore be more relevant to the health care decisions of policymakers, treatment facilities, and patients. The observational nature of our research design and the multilevel structure of our longitudinal data sources also presents challenges that will require statistical tools to ensure appropriate estimation and inference.

The primary quantitative analysis methods will focus on multilevel latent growth models, which will incorporate the longitudinal data collected by the Patient Check-In and Patient Questionnaires. This approach will allow for repeated patient measures to be linked from baseline, 3, 6, 12 months, and 18- months in a model that also accounts for hierarchal data structure (e.g., patient, treatment, site, study area). This longitudinal data will be used in the model by adding a regression parameter for each patient that captures the relationship between the risk factor (e.g., depression) and the outcome for that patient.

Our aim is understanding which type of patients do better in which type of treatment, and which treatment program characteristics matter for patient success. To that end, we will estimate regression models that account for the different treatment types (MMT and BUP). The dependent variables of interest will include relevant outcomes such as opioid misuse and abuse, overdose, and mortality, among others. The independent variables will include relevant patient and program characteristics that could matter for patient success.

Several quantitative methods will be implemented to help address the challenges raised by conducting an observational study, including propensity-score matching, and advanced mediation analysis. Propensity scoring approaches may be used to estimate the probability of being captured in our study and to align our sample with the population seeking OUD treatment using known characteristics. These scores can then be used to weight outcome analyses to produce estimates with the intended goal of improving their statistical properties (e.g., lower bias), given the study design (Harder, Stuart, & Anthony, 2010; McCaffrey, Ridgeway, & Morral, 2004; Rosenbaum & Rubin, 1983; Shadish, 2010). Where appropriate, we will compare the study estimates with external data sources to assess the effectiveness of this procedure.

Advanced mediation analysis will support the assessment of indirect linkages between treatment and the outcome variables. OUD outcomes will be modeled as growth processes, with random intercepts and slopes over time for both the mediator and outcomes under the longitudinal mediation framework of Cheong, MacKinnon, and Khoo (2003).

B.4 Justification for Target Sample Sizes

The goal of the study is to conduct an epidemiologic, mixed-methods evaluation of MAT in real-world outpatient settings, incorporating characteristics for the area and site along with patient-level covariates. The study is not designed to conduct site-level comparisons nor comparisons across areas.

A conservative approach was initially used for the power calculation to determine the number of patients needed to remain in the study at the end of the 18-month observation period. However, later in the study, the power calculations were revised to reflect the addition of more sites and to take a less conservative approach, in response to lower than anticipated patient recruitment levels (Att 9). These changes to the power calculation were based on review of recent literature, and included the following changes: (1) Decrease of the client-level intracluster correlation coefficient (ICC) from 0.40 to 0.10; (2) Added a model-based component to the descriptive statistics approach used in the initial power calculation; and (3) Decrease of 0.10 in the covariance between to opioid abstinence rates to account for sites that have multiple treatment options available. The study was established to detect a 10-percentage point difference in two-sided statistical tests to evaluate pairwise comparisons across the treatments; the tests were set at 0.05 significance and 90-percent power. The sensitivity of site-level intracluster correlations (ICC) was evaluated; ICC=0.01 was consistent with prior studies of a similar nature (see discussion in Section B.1.2). Though the treatments will not be directly compared to identify the “best” regimen, the sample size targets will ensure efficient estimates for each regimen. After data collection is complete, we will examine patient characteristics and attempt to determine if the assumptions included in our power calculation assumptions were justified.

We will examine descriptive statistics by subgroups such as gender, age, race/ethnicity, urban/rural, and insurance coverage; those subgroups with at least 300 participants are projected to have sufficient size to produce reliable estimates. Estimates with a percent coefficient of variation (CV, calculated as the square root of the estimated variance / estimated statistic) exceeding 30 percent will be labeled as unstable and interpreted with caution. In keeping with CDC guidelines, estimates with a CV at or above 50 percent are unreliable and will be suppressed from publication.

B.5 Test of Procedures or Methods to Be Undertaken

Data collection tools used in this study have been adapted from previously administered tools and instruments available from the literature (see Section A.2).

B.6 Individuals Consulted on Statistical Aspects and/or Analyzing Data

As noted in Section A.8, CDC has consulted extensively with a federal expert panel who will continue to provide expert advice throughout the course of the evaluation. In addition, the contractor team is composed of several experts who will be directly involved in the data collection and statistical analysis. In addition, contracting in-house experts will be consulted throughout the program on various statistical aspects of the design, methodological issues, economic analysis, database management, and data analysis. Exhibit 3 provides details of these team members and advisors.

Exhibit 3. Data Collection and Analysis Team Members and Advisors

MAT Staff
Laura Dunlap, PhD Project Director Behavioral Economics RTI International Research Triangle Park, NC Phone: 919-541-7310	Jill Dever, PhD Director, Social Statistics Program RTI International Washington, DC Phone: 202-974-7846
Mark Edlund, MD, PhD Associate Project Director Epidemiologist RTI International Research Triangle Park, NC Phone: 919-597-5132	Hannah Knudsen, PhD Associate Professor University of Kentucky Lexington, KY Phone: 859-323-3947
CDC Project Leadership
Bradley Biggers, MPH Technical Monitor/Behavioral Scientist Division of Overdose Prevention Centers for Disease Control and Prevention Atlanta, GA Phone: 770-488-0562	Terry Davis, EdD Contracting Officer’s Representative II Division of Overdose Prevention Centers for Disease Control and Prevention Atlanta, GA Phone: 770-488-3940

References

The American Association for Public Opinion Research (2016). Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 9th edition. AAPOR. http://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf, accessed 02 February 2018.

Baker, R., Brick, J. M., Bates, N. A., Battaglia, M., Couper, M. P., Dever, J. A., et al. (2013). Report of the AAPOR task force on non-probability sampling (June 2013). Available at http://www.aapor.org/AM/Template.cfm?Section=Reports1&Template=/CM/ContentDisplay.cfm&ContentID=6055.

Brewer, K.R. (1999). Design-based or Prediction-based Inference? Stratified Random vs Stratified Balanced Sampling. International Statistical Review, 67(1): 35-47.

Cheong, J., MacKinnon, D. P., & Khoo, S. T. (2003). Investigation of mediational processes using parallel process latent growth curve modeling. Structural Equation Modeling, 10(2), 238-262.

Desmond, D.P., Maddux, J.F., Johnson, T.H., and Confer, B.A. (1995). Obtaining Follow-Up Interviews for Treatment Evaluation. Journal of Substance Abuse Treatment, 12(2): 95-102.Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological methods, 15(3): 234.

Heckman, J.J. 1979. Sample Selection Bias as a Specification Error. Econometrica 47(1): 153-162.

Hedeker, D., and R.D. Gibbons. 1997. “Application of Random-effects Pattern-mixture Models for Missing Data in Longitudinal Studies.” Psychological Methods 2(1): 64.

McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological methods, 9(4): 403.

Miles, M. B., Huberman, A. M. & Saldana, J. (2003). Qualitative data analysis. Sage: Los Angeles, CA.

Novak, S. P., Herman-Stahl, M., Flannery, B., & Zimmerman, M. (2009). Physical pain, common psychiatric and substance use disorders, and the non-medical use of prescription analgesics in the United States. Drug and alcohol dependence, 100(1): 63-70.

Novak, S. P., Wenger, L., Lorvick, J., & Kral, A. (2015). The misuse, abuse and diversion of opioid replacement therapies among street abusers. Drug & Alcohol Dependence, 146, e54.

Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 41-55.

Shadish, W. R., & Steiner, P. M. (2010). A primer on propensity score analysis. Newborn and Infant Nursing Reviews, 10(1): 19-26.

1 Initiating OUD treatment, for the purposes of this study, will be defined as starting an OUD treatment not received by the patient in the prior three months.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Author
File Modified	0000-00-00
File Created	2021-01-28