SSB_MAT_Clean_2.08.18

Broadly, the respondent universe includes individuals in the United States who receive some form of opioid use disorder (OUD) treatment. The sample from this broad universe has been operationalized by selecting 11 areas and. recruiting five to six OUD treatment facilities per area. The study targets enrolling a sufficient number of participants to yield 3,560 baseline Patient Questionnaires. Study area is defined as the combination of metropolitan statistical area plus, in some instances, surrounding cities or corresponding larger cities within the same state.

B.1.1 Site Selection

Within each are, the study team began by building a list of treatment facilities that includes information on sampling criteria and contact information for recruitment. No single list of U.S. health facilities or practices that offer OUD treatment specifies the type of OUD treatments offered and the patient load. Therefore, the sampling frame for treatment facilities was built by triangulating a variety of sources (e.g., National Survey of Substance Abuse Treatment Services, Substance Abuse and Mental Health Services Administration, Drug Enforcement Agency list of buprenorphine providers, National Provider Index) to identify eligible facilities in the 11 study areas and facility contact information (e.g., address, phone number, clinic director or proprietor, treatments offered, patient load). The study team aimed to create a list of up to 20 OUD treatment facilities (sites) per study area from which five to six OUD treatment facilities will be selected. The following criteria are being used to determine site inclusion:

Sufficient patient flow, about 10 new patients per month, to reach the approximate enrollment target of 66 participants per treatment facility.
Site diversity related to:
1. Organizational structure (e.g., clinic, group practice)
2. Management structure
3. Facility operations and locations
4. Numbers and types of providers
5. Patient characteristics such as age, gender, race or ethnicity, residence (urban or rural), and OUD treatment characteristics.

B.1.2 Patient Selection within Site

Once a treatment facility has agreed to participate, RTI International (RTI) will train and support facility staff to identify eligible, potential participants who are initiating a new treatment episode for OUD and to collect contact information. Prospective participants will be eligible if they are 18 to 64 years of age and initiating one of four primary treatments for OUD: methadone maintenance treatment (MMT), buprenorphine (BUP), naltrexone (NTX), or counseling treatment without medication (COUN).^¹ Client referral information will be maintained within a secure server environment for retrieval by approved RTI field staff (see Sections A.10 and B.2 for further information). Exhibit 1 summarizes the selected study areas and number of sites and patients expected to participate.

Exhibit 1. Sampling Stages for MAT Study

Sample Unit Description
11 geographic areas
Alabama (Birmingham)	Kentucky/West Virginia (Huntington, Ashland)
Arizona (Phoenix)	New York (New York City)
California (San Francisco, Los Angeles)	Ohio (Cleveland, Cincinnati)
Florida (Miami)	Oklahoma (Oklahoma City)
Illinois (Chicago)	Washington (Seattle)
Washington DC Metro Area
60 treatment sites^* Five to six sites per study area
3,560 completed Patient Baseline Questionnaires Average of 60 participants per site within each study area Average of 324 participants with each study area

MAT Study = Medication-Assisted Treatment Study; OUD = opioid use disorder.

^*Few, if any, primary care providers in the sampling frame will meet the volume recruitment of up to 10 new OUD patients per month. We plan to lower the volume requirement, if necessary, and make every effort to recruit and enroll several (up to four) primary care sites in the sample of participating sites. These sites will be in addition to the 60 sites called for in our original sampling plan.

Patients who consent to have their contact information released to RTI, referred to as referrals in the subsequent discussions, will be contacted by a trained RTI field interviewer (FI). The FI administers the Patient Screener Form (discussed in Section A.2) to verify study eligibility and to collect demographic information. Not only will this information be used to assess bias in enrollment by comparing participants characteristics with those who decline to participate, but the data will be used for classification into quota cells. Initial sample quotas, set within each OUD treatment (MMT, BUP, NTX, and COUN), reflect marginal distributions on the admissions file from the Treatment Episode Data Set (TEDS-A) such as those shown in Exhibit 2, in addition to the management of enrollment burden at each site. Quotas will be periodically reviewed for changes required based on distributions exhibited in and across the study sites and based on preliminary analysis of the Baseline Interview responses. Once quotas are met for the characteristics of a particular patient type, they are no longer eligible for the study. The participating staff members at the site will be informed of these characteristics.

Exhibit 2. U.S. Opioid-related substance abuse treatment admissions in 2014: TEDS-A

Characteristic	Distribution (%)	Characteristic	Distribution (%)
Age		Race
18-24	13.3	White	71.1
25-34	37.7	Other	29.0
35-44	20.8	Ethnicity
45-54	17.8	Non-Hispanic	82.4
55+	10.4	Hispanic	17.6
Gender		Opioid Type
Male	59.5	Rx Opioids	33.0
Female	40.5	Heroin	81.3
Region
Northeast	57.5
Midwest	2.9
South	8.4
West	31.2

TEDS-A = admissions file from the Treatment Episode Data Set, 2014.

The use of sample quotas has several advantages for The MAT Study. For example, the take-all approach will enable efficient recruiting of study participants within the compressed enrollment period. Second, sample quotas can be easily understood and implemented by FIs in the field, unlike advanced sampling methods that require interaction with project statisticians. Third, for satisfied quotas, site staff are told to stop referring patients with certain traits, thus reducing burden. Finally, sample quotas are set in line with the MAT analytic models in an effort to increase the statistical power of the analyses (see, e.g., Brewer, 1999).

A major assumption behind the effective use of quota sampling is that there is no systematic bias in how patients are enrolled. Study training will emphasize the study eligibility criteria and the need for such level of unbiasedness. Additionally, the relatively short MAT enrollment period in our opinion precludes the concern for a seasonality effect that would require a consistent sampling rate across the time period.

The study team anticipates receiving approximately 5,500 patient referrals of which 3,560 are confirmed to be eligible and agree to participate. The Study Check-In is expected to be completed by at least 80% of the 3,560 Baseline Patient Questionnaire respondents. Finally, expected participation rates for the baseline and 12- and 24-month follow-up Patient Questionnaires are shown in Exhibit 3.

The data collection approach outlined in Section B.3 will support the expected participation rates shown in Exhibit 3. Participation rates instead of response rates are used in acknowledgment of the nonprobability sample design and of standard definitions recommended by The American Association for Public Opinion Research (AAPOR, 2016; Baker et al., 2013). These rates are based on past studies such as Hser et al. (2015), and Desmond et al. (1995). For example, Hser et al (2015) conducted a long-term follow-up among patients initially randomized to receive buprenorphine or methadone. They found 89.4% of their targeted sample over an average 4.5-year follow-up period. In a case study of best practices for recruitment, Desmond et al studied illicit opioid users starting in MMT. Overall, 12-month follow-up was obtained for 98% of the sample. Broken down, 100% of those in treatment were followed up and 96% of those discharged. Of those discharged, 80% were followed up within 2 months of the anniversary window.

Exhibit 3. Patient Sampling for MAT Study

Patient Contact and Questionnaire	Target Number of Interviews	Target Participation Rates¹
Patient Contact and Questionnaire	Target Number of Interviews	Referrals (%)²	Baseline (%)³
Patient referrals	5,567	100.0
Screened patients	4,732	85.0
Study eligible	4,684	99.0
Consent	4,450	95.0
Baseline	3,560	80.0	100.0
Check-In	2,848	56.0	80.0
12-month follow-up	2,791	59.0	78.4
24-month follow-up	2,233	47.7	62.7

MAT Study = Medication-Assisted Treatment Study.

¹ Participation rates are average estimates obtained from prior experience, plus findings and protocols discussed in, for example, Novak et al. (2015), Novak et al. (2009), and Desmond et al. (1995). The rates are not assumed to vary significantly across treatment regimen.

² Estimated participation rates based on the number of patient referrals.

³ Estimated participation rates based on the number of baseline interviews. Note that participants are retained in the study even if they are unable to complete an interview between baseline and the 24-month follow-up.

B.2 Procedures for the Collection of Information

Data collection from providers and patients will be centrally coordinated and supported by the Centers for Disease Control and Prevention’s (CDC’s) contractor, RTI. In each study area, RTI will hire experienced, local FIs to contact patients and collect data.

RTI will create a Case Management System to support all data collection activities and will be housed on secure RTI servers. Information stored on the system will be accessible to RTI FIs and other RTI staff as needed. Identifying information will be collected and stored separately from all other data collected, and a random ID will be used to identify patient and site records. The data collection effort will use a secure data collection web system, Voxco, hosted on RTI’s Enhanced Security Network (ESN), which meets federal requirements for high-security risk data. Identifying information will only be hosted on the ESN. Non-identifying information will be collected through a separate web-based tool, also developed using Voxco, and a random patient ID will be used to identify these records. The secure data collection methods are summarized in Exhibit 4.

Exhibit 4. Respondent and Data Collection Methods

Tool	Respondent	Method
Patient Screener Form	RTI FI ask prospective patients screening questions and complete the form, typically via telephone.	Secure web-based Voxco tool.
Patient Check-In	Patient self-report with RTI FI support if needed.	Secure web-based Voxco tool.
Patient Questionnaire: Baseline and 12-Month and 24-Month Follow-Ups	Patient self-report with RTI FI support if needed. For consenting patients, Social Security Numbers (SSNs) are separately collected in conjunction with the Patient Questionnaire Baseline.	Secure web-based Voxco tool. For SSNs, a separate secure web-based Voxco tool with storage in RTI’s ESN.
Patient Focus Group Protocol	Patients recruited by RTI FI and treatment facility staff.	Secure tele-conference.
Provider Focus Group Protocol	Treatment facility staff recommended by the facility’s key contact for the MAT study.	Secure tele-conference.

B.3 Methods to Maximize Participation Rates

B.3.1 Participation Rates

Various procedures will be used to help maximize participation rates across the study; RTI FIs will be central in implementing these well-tested procedures. The procedures will help ensure the following interview-specific participation rates among participants who complete the Baseline interview: (1) 80.0% for the Patient Check-In, (2) 78.4% for the 12-month follow-up, and (3) 62.7% for the 24-month follow-up. Additionally, the analysis techniques used will incorporate methods to account for nonresponse; analysis approaches are discussed in B.3.3. For patients, FIs will

engage patient interest in MAT study;
build a relationship through calls, emails, and in person visits;
be responsible for ensuring informed consent of participants;
observe all other needed measures to protect patients’ rights as research subjects, their privacy, and the confidentiality of their data;
assist patients to complete the Baseline Patient Questionnaire (in person) and provide incentives;
respond appropriately to patient distress;
follow up with patients should they not respond to email requests to complete the Check-In and follow-up Patient Questionnaires; and
be available to patients via phone should they have any questions.

If FIs lose contact with a patient, the FI will seek to relocate the patient and reestablish contact. Methods used to relocate subjects include reaching out to alternative contacts (e.g., parent, friend) obtained during enrollment; and checking local data sources (e.g., newspapers, court dockets). The patient will also be referred to RTI’s Tracing Unit, which conducts online searches (e.g., U.S. Postal Service change-of-address records, Department of Motor Vehicles records, credit reports). Tracking and tracing efforts have succeeded in retaining up to 10% of subjects lost to follow-up in other similar studies.

B.3.2 Nonparticipation

At the treatment facility level, nonresponse may occur if a facility does not agree to participate. Here, the sampling team will review our list of facilities for another facility to invite that is similar in regard to type of treatment offered and volume of patient flow. Our initial and replacement facility selection will keep an eye toward achieving the primary sampling objectives of approximately 66 patients completing a Baseline interview from each treatment facility in each study area (assuming 60 study sites), and 890 patients from each treatment modality (i.e., MMT, BUP, NTX, COUN) across all the study areas (3,560 total patients across all 4 modalities and all 11 study areas). Secondary sampling objectives include reaching our targeted distribution of subjects by characteristics such as age, gender, race or ethnicity, and OUD characteristics implemented via sample quotas (see Section B.1.2).

For study participants, nonresponse will first be limited by the various patient recruitment and retention strategies discussed in Section B.3.1. Negative effects linked to nonresponse will be examined through information collected in the Patient Screen former; if found, such characteristics will be included in the analytic models to mitigate any such biasing factors. In addition to unit nonresponse, imputation is another tool examined to address item nonresponse. The RTI team will plan to incorporate models for non-random missingness into quantitative analyses to assess sensitivity to different assumptions about missing data. Non-ignorable missing at random methods generally fall under two categories: selection models (e.g., Heckman, 1979) and pattern mixture models (Hedeker & Gibbons, 1997), and both will be incorporated as appropriate during data analysis.

B.3.3 Analysis

Descriptive Statistics

Our analyses will begin by examining descriptive statistics for key variables. Specifically, descriptive statistics will include frequencies (e.g., sample sizes), measures of dispersion (e.g., standard deviation) and central tendency (e.g., mean, proportion), and an assessment of the variation within and among the treatment facilities (e.g., intraclass correlations). Key variables will include patients’ social, economic, clinical, and demographic characteristics and treatment facility’s operational structure, service focus, and approach to care.

Multilevel Analysis

We propose to estimate a variety of advanced statistical models for the population at large and for relevant subpopulations that consider key health, quality of life, and socioeconomic outcomes and that control for a rich set of patient- and treatment facility-level covariates. This study will be conducted in real-world settings and will rely on longitudinal primary data sources. Therefore, our approach can result in a more comprehensive, generalizable, and representative analysis and can therefore be more relevant to the health care decisions of policymakers, treatment facilities, and patients. The observational nature of our research design and the multilevel structure of our longitudinal data sources also presents challenges that will require statistical tools to ensure appropriate estimation and inference.

The primary quantitative analysis methods will focus on multilevel latent growth models, which will incorporate the longitudinal data collected by the Patient Check-In and Patient Questionnaires. This approach will allow for repeated patient measures to be linked from baseline, 3, 6, 12 months, and 24 months in a model that also accounts for hierarchal data structure (e.g., patient, treatment, site, study area). This longitudinal data will be used in the model by adding a regression parameter for each patient that captures the relationship between the risk factor (e.g., depression) and the outcome for that patient.

Our aim is understanding which type of patients do better in which type of treatment, and which treatment program characteristics matter for patient success. To that end, we will estimate separate multivariate regression models for patients in each treatment type (MMT, BUP, NTX, and COUN). The dependent variables of interest will include relevant outcomes such as opioid misuse and abuse, overdose, and mortality, among others. The independent variables will include relevant patient and program characteristics that could matter for patient success.

Several quantitative methods will be implemented to help address the challenges raised by conducting an observational study, including propensity-score matching, and advanced mediation analysis. Propensity scoring approaches may be used to estimate the probability of being captured in our study and to align our sample with the population seeking OUD treatment using known characteristics. These scores can then be used to weight outcome analyses to produce estimates with the intended goal of improving their statistical properties (e.g., lower bias), given the study design (Harder, Stuart, & Anthony, 2010; McCaffrey, Ridgeway, & Morral, 2004; Rosenbaum & Rubin, 1983; Shadish, 2010). Where appropriate, we will compare the study estimates with external data sources to assess the effectiveness of this procedure.

Advanced mediation analysis will support the assessment of indirect linkages between treatment and the outcome variables. OUD outcomes will be modeled as growth processes, with random intercepts and slopes over time for both the mediator and outcomes under the longitudinal mediation framework of Cheong, MacKinnon, and Khoo (2003).

Qualitative Analysis

Upon completion of each Focus Group, transcripts will be extracted by RTI qualitative analysts and compared with the recordings for verification. Members of the team will upload the data into NVivo, a qualitative analysis software program. The study’s qualitative analysis approach will be based on coding or categorizing data using carefully defined and tested codes. Each focus group transcript will be independently double coded with an established list of codes and then checked for reliability between analysts. During the initial stages of coding, codes will be tested and refined in process expected to follow the Miles and Huberman (1994) interactive model of qualitative data review, in which we simultaneously collect, display, and reduce data; draw conclusions; and verify our assertions. The final version of the qualitative data will be a condensed set of themes, one for the treatment facility staff and another for patients, which will highlight the factors that influence OUD treatment outcomes.

B.4 Justification for Target Sample Sizes

The goal of the study is to conduct an epidemiologic, mixed-methods evaluation of MAT in real-world outpatient settings, incorporating characteristics for the area and site along with patient-level covariates. The study is not designed to conduct site-level comparisons nor comparisons across areas.

A conservative approach was used for the power calculation to determine the number of patients needed to remain in the study at the end of the 2-year observation period, i.e., 2,233 shown in Exhibit 3. Inputs to the power calculation for the four MAT treatments included a 50-percent abstinence rate and descriptive instead of model-based statistics; without the benefits of covariates to lower the model errors, our approach resulted slightly larger variance than expected. The study was established to detect a 10-percentage point difference in two-sided statistical tests to evaluate pairwise comparisons across the treatments; the tests were set at 0.05 significance and 90-percent power. The sensitivity of site-level intracluster correlations (ICC) was evaluated; ICC=0.01 was consistent with prior studies of a similar nature (see discussion in Section B.1.2). Though the treatments will not be directly compared to identify the “best” regimen, the sample size targets will ensure efficient estimates for each regimen.

We will examine descriptive statistics by subgroups such as gender, age, race/ethnicity, urban/rural, and insurance coverage; those subgroups with at least 450 participants are projected to have sufficient size to produce reliable estimates. Estimates with a percent coefficient of variation (CV, calculated as the square root of the estimated variance / estimated statistic) exceeding 30 percent will be labeled as unstable and interpreted with caution. In keeping with CDC guidelines, estimates with a CV at or above 50 percent are unreliable and will be suppressed from publication.

B.5 Test of Procedures or Methods to Be Undertaken

Data collection tools used in this study have been adapted from previously administered tools and instruments available from the literature (see Section A.2). Similarly, the Focus Group guides for both patients and staff have been developed from questionnaires used in similar studies and have been adapted to fit the specific needs of this study.

B.6 Individuals Consulted on Statistical Aspects and/or Analyzing Data

As noted in Section A.8, CDC has consulted extensively with a federal expert panel who will continue to provide expert advice throughout the course of the evaluation. In addition, the contractor team is composed of several experts who will be directly involved in the data collection and statistical analysis. In addition, contracting in-house experts will be consulted throughout the program on various statistical aspects of the design, methodological issues, economic analysis, database management, and data analysis. Exhibit 5 provides details of these team members and advisors.

Exhibit 5. Data Collection and Analysis Team Members and Advisors

MAT Staff
Laura Dunlap, PhD Project Director Behavioral Economics RTI International Research Triangle Park, NC Phone: 919-541-7310	Victoria Albright, MA Senior Survey Researcher and Analyst RTI International Research Triangle Park, NC Phone: 919-541-7129
Mark Edlund, MD, PhD Associate Project Director Epidemiologist RTI International Research Triangle Park, NC Phone: 919-597-5132	Hannah Knudsen, PhD Associate Professor University of Kentucky Lexington, KY Phone: 859-323-3947
CDC Project Leadership
Marci Hertz, MSEd Technical Contracting Officer’s Representative/ Lead Health Scientist Division of Analysis, Research, and Practice Integration Centers for Disease Control and Prevention Atlanta, GA Phone: 770-488-2547	Donovan Newton, MPA Contracting Officer’s Representative Division of Analysis, Research, and Practice Integration Centers for Disease Control and Prevention Atlanta, GA Phone: 770-488-3987

References

The American Association for Public Opinion Research (2016). Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 9th edition. AAPOR. http://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf, accessed 02 February 2018.

Baker, R., Brick, J. M., Bates, N. A., Battaglia, M., Couper, M. P., Dever, J. A., et al. (2013). Report of the AAPOR task force on non-probability sampling (June 2013). Available at http://www.aapor.org/AM/Template.cfm?Section=Reports1&Template=/CM/ContentDisplay.cfm&ContentID=6055.

Brewer, K.R. (1999). Design-based or Prediction-based Inference? Stratified Random vs Stratified Balanced Sampling. International Statistical Review, 67(1): 35-47.

Cheong, J., MacKinnon, D. P., & Khoo, S. T. (2003). Investigation of mediational processes using parallel process latent growth curve modeling. Structural Equation Modeling, 10(2), 238-262.

Desmond, D.P., Maddux, J.F., Johnson, T.H., and Confer, B.A. (1995). Obtaining Follow-Up Interviews for Treatment Evaluation. Journal of Substance Abuse Treatment, 12(2): 95-102.Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological methods, 15(3): 234.

Heckman, J.J. 1979. Sample Selection Bias as a Specification Error. Econometrica 47(1): 153-162.

Hedeker, D., and R.D. Gibbons. 1997. “Application of Random-effects Pattern-mixture Models for Missing Data in Longitudinal Studies.” Psychological Methods 2(1): 64.

McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological methods, 9(4): 403.

Miles, M. B., Huberman, A. M. & Saldana, J. (2003). Qualitative data analysis. Sage: Los Angeles, CA.

Novak, S. P., Herman-Stahl, M., Flannery, B., & Zimmerman, M. (2009). Physical pain, common psychiatric and substance use disorders, and the non-medical use of prescription analgesics in the United States. Drug and alcohol dependence, 100(1): 63-70.

Novak, S. P., Wenger, L., Lorvick, J., & Kral, A. (2015). The misuse, abuse and diversion of opioid replacement therapies among street abusers. Drug & Alcohol Dependence, 146, e54.

Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 41-55.

Shadish, W. R., & Steiner, P. M. (2010). A primer on propensity score analysis. Newborn and Infant Nursing Reviews, 10(1): 19-26.

1 Initiating OUD treatment, for the purposes of this study, will be defined as starting an OUD treatment not received by the patient in the prior three months.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Author
File Modified	0000-00-00
File Created	2021-01-21