SUPPORTING STATEMENT: PART B
OMB #
Comprehensive Evaluation of the Implementation and Uptake of the CDC Clinical Practice Guideline for Prescribing Opioids for Pain
Date: November 21, 2024
Point of Contact:
Kristine M. Schmit, MD, MPH
Contact Information:
Centers for Disease Control and Prevention
National Center for Injury Prevention and Control
Division of Overdose Prevention
B. INFORMATION COLLECTION PROCEDURES 1
B1. Respondent Universe and Sampling Methods 1
B2. Procedures for the Collection of Information 8
B3. Methods to Maximize Response Rates and Address Nonresponse 9
B4. Test of Procedures or Methods to be Undertaken 10
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and Analyzing Data 10
This evaluation of the 2022 CDC Clinical Practice Guideline for Prescribing Opioids for Pain (2022 CDC Clinical Practice Guideline) will include systematic collection and analysis of a range of primary and secondary data sources. To answer the research questions, we will employ qualitative synthesis and analytic approaches, quantitative analyses, and various mixed-methods approaches.
The research questions for the project will be answered through collecting primary data and analyzing secondary medical and prescription claims data.
Primary data collection efforts include a web-based survey conducted among a national sample of clinicians, virtual interviews with clinicians, virtual interviews with dentists,1 virtual interviews with leaders from professional organizations, payers, medical boards, and health systems, and virtual focus groups with patients and caregivers.
For secondary data analyses, we will use commercial claims data to measure changes in opioid and MOUD prescribing practices, variously defined, from before and after the release of the 2022 CDC Clinical Practice Guideline. We will obtain claims data for a cohort of patients, which will include records of all medical services, including dates, diagnoses, procedures, National Drug Code (NDC) of prescribed medications, prescribing clinician’s National Provider Identifier (NPI), and patient’s enrollment and demographic information.
Clinician Survey
We will conduct a web-based survey of approximately 600 outpatient clinicians who are able to prescribe opioids to their patients with acute, subacute and chronic pain. Survey data will be collected from a respondent universe composed of 3,000 total possible recipients of an invitation to participate, with a target response of up to 600 clinicians (assuming 20% response rate of 3,000 and supported by power analysis described in detail later in this section). Inclusion criteria are that the clinicians: 1) practice in an ambulatory, outpatient, and/or emergency department at least once a week; 2) primarily treat adults; 3) treat patients with acute, subacute, and chronic pain other than pain management related to sickle cell disease, cancer-related pain treatment, palliative care, and/or end of life care; 4) practice in the following care areas: family medicine, internal medicine, emergency medicine, surgery, occupational medicine, physical medicine and rehabilitation, neurology, obstetrics and gynecology. Participants who primarily treat patients within the Veterans Health Administration (VA) will be excluded because the VA has its own pain management guidelines.2 Dentists will also be excluded from the clinician survey due to the fact that dentists are included in a separate database than the database from which the clinician sample will be drawn. The survey design will be cross-sectional, conducted at a single point in time, and web based. The survey is expected to take 10 minutes to complete.
Interviews with Clinicians
We will conduct virtual interviews with 30 outpatient clinicians who completed the clinician survey and agreed to participate in a follow-up interview. Interview participants who agree to participate will be selected based on their demographics and responses to the survey to maximize different experiences in the interview respondents. The interviews with clinicians will be cross-sectional, conducted at a single point in time.
Interviews with Dentists
We will conduct five virtual interviews with dentists to understand their experience managing acute, subacute, and chronic pain. Potential interview participants will be recruited through referrals from the American Dental Association. The interviews with dentists will be cross-sectional, conducted at a single point in time.
Interviews with Leaders from Health Systems
We will conduct 10 virtual interviews with leaders from health systems to understand their experience implementing the 2022 CDC Clinical Practice Guideline. We will identify and recruit participants through CDC and the project team’s networks as well as through Internet searches. Potential respondents can include: chief medical officer (CMO), VP of Quality and Safety, and VP for Primary Care, or similar roles. We will conduct interviews with up to 10 individuals at two time periods, approximately two years apart.
Interview with Payers
We will conduct 10 virtual interviews with individuals that represent public and private payers (i.e., Medicaid, Medicare, private health plans providing employer, marketplace, managed care or Medicare Advantage plans) to understand their experience implementing the 2022 CDC Clinical Practice Guideline. We will identify and recruit participants through CDC and the project team’s networks as well as through Internet searches. We will conduct interviews with up to 10 individuals at two time periods, approximately two years apart.
Interviews with Leaders from Professional Associations
We will conduct 10 virtual interviews with leaders from professional associations, such as the American Association of Family Practitioners, National Association of Community Health Centers, and Society for General Internal Medicine, to understand their experience implementing the 2022 CDC Clinical Practice Guideline. We will identify and recruit participants through CDC and the project team’s networks as well as through Internet searches. We will conduct interviews with up to 10 individuals at two time periods, approximately two years apart.
Interviews with Leaders from Medical Boards
We will conduct 10 virtual interviews with leaders from state medical boards to understand their experience implementing the 2022 CDC Clinical Practice Guideline. We will identify and recruit participants through CDC and the project team’s networks as well as through Internet searches. We will conduct interviews with up to 10 individuals at two time periods, approximately two years apart.
Focus Groups with Patients
Focus groups with patients provide an in-depth understanding of experiences with pain management in their real-world contexts. Examining the experiences of caregivers can provide a deeper understanding of real-world behavior within a specific healthcare context to elucidate perceptions of whether and/or how changes occurred in overall treatment and/or pain management, including opioid prescribing. We will conduct three virtual focus groups with patients at three time points, each containing 15 participants
Focus Groups with Caregivers
Focus groups with caregivers provide an in-depth understanding of experiences with pain management in their real-world contexts. Examining the experiences of caregivers can provide a deeper understanding of real-world behavior within a specific healthcare context to elucidate perceptions of whether and/or how changes occurred in overall treatment and/or pain management, including opioid prescribing. We will conduct two virtual focus groups of caregivers at three time points, each containing 15
Exhibit 1. Estimated Duration of Primary Data Collection Activities
Type of Respondent |
Data Collection Activity Name |
Mode |
Average Duration of Activity (in minutes) |
Clinicians |
Clinician Survey |
Web-based |
10 |
Clinicians |
Clinician Interview |
Virtual |
60 |
Dentist |
Dentist Interview |
Virtual |
60 |
Health System Leaders |
Health System Leaders Interview |
Virtual |
60 |
Payer |
Payer Interview |
Virtual |
60 |
Professional Association Leaders |
Professional Association Leaders Interview |
Virtual |
60 |
Medical Board Leaders |
Medical Board Leaders Interview |
Virtual |
60 |
Patients |
Patient Focus Groups |
Virtual |
60 |
Caregivers |
Caregiver Focus Groups |
Virtual |
60 |
Abstraction of Secondary Data
To complement the primary data collection efforts described above, secondary data from one commercial medical claims vendor will be used to estimate the impact of the 2022 CDC Clinical Practice Guideline on several different outcomes using commercial claims data on patients’ diagnoses, treatment codes, and prescriptions dispensed before and after the release of the 2022 CDC Clinical Practice Guideline.
A more detailed description of the purpose and methods to complete these primary and secondary data collection efforts is presented in the Supporting Statement A (SSA) documentation.
Exhibit 2. Data Sources, Sampling Approach, and Topics
Data Sources |
Approach and Topics |
National Survey of Clinicians |
Sampling: We will use a random sample for the survey comprising 3,000 clinicians. We will use IQVIA’s national database of clinicians. The IQVIA medical professionals database is primarily based on commercial claims data. Therefore, the sample may not incorporate all potential groups of interest from the full target population of clinicians who treat patients with pain (e.g., clinicians from the Indian Health Service may be missing from the sample). As such, we will work carefully with CDC to determine potential strategies for assessing and managing potential bias in the resulting sample. IQVIA’s dataset is continuously expanding, and includes a diverse set of clinicians with respect to their occupation, geographic region, and practice type.
We will request a sample from IQVIA that includes clinicians (including physicians, nurse practitioners, and physician assistants) that practice in the following care areas: family medicine, internal medicine, emergency medicine, surgery, occupational medicine, physical medicine and rehabilitation, neurology, obstetrics and gynecology.
Topics:
Analysis: Quality checks will be performed on the data as responses are collected to address any issues that arise. An analyst will clean the dataset and prepare it for analysis once the survey closes. We will analyze the survey data to describe clinicians’ awareness, use, and facilitators or barriers to implementation of the 2022 CDC Clinical Practice Guideline. We will report descriptive statistics, including the mean, median, and standard deviation of continuous variables and frequencies for categorical variables, in tables and visualized in figures and graphics. We will qualitatively analyze open-ended responses, using content analysis to identify common themes. In addition to overarching results, we will provide tables of stratified results by geographic region, clinician type, and practice type. We will work closely with CDC to identify other sub-analyses and stratification approaches we should incorporate into a revised analysis plan so that results are timely and meaningful for the primary goals of the survey. Findings from this survey will inform CDC regarding prescriber knowledge and behavior related to the 2022 CDC Clinical Practice Guideline for clinicians who treat patients with pain. Sample size and power estimation: For sample size estimation, we relied on Cochran’s sample size calculation with corrections for finite populations. Assuming an assessed survey impact of the 2022 CDC Clinical Practice Guideline of 50-60% (the proportion expected to answer that they are very or moderately familiar with the practice guideline), between 365 and 1,066 surveys would be necessary to achieve a representative sample of the target population, with a confidence level of 95% and a precision level of 5% - 3% regarding responses to that question across all providers. A sample size of 600 is achievable given 3,000 contacts and a response rate of 20%. A sample size of 600 will likely provide sufficient power (80% or greater) to test at least one of the following hypotheses:
H01:
There is no difference in primary
care clinicians (i.e., NPs, PAs, MDs in family medicine or
internal medicine) as
compared with specialty
clinicians, including emergency medicine familiarity
with specific recommendations in the 2022 CDC Clinical Practice
Guideline as measured by survey question 9. BLS data suggests that primary care providers (Family medicine physicians, internal medicine physicians, nurse practitioners, and physician’s assistants) make up approximately 80% of the target population. A sample size of 600 would contain approximately 20% specialty care physicians (emergency medicine, internal medicine, neurologists, obstetricians and gynecologists, and surgeons). Assuming 480 primary care providers, and 120 specialty care physicians in the sample, and assuming equal variance with a standard deviation of 1, an average difference of 0.3 on question 9’s scale is expected to yield a Type II error rate under 20% for a two-sample T-test.
H02: There is no difference in clinicians with less than 10 years practicing medicine as compared with clinicians with 10 years or greater practicing medicine familiarity with specific recommendations in the 2022 CDC Clinical Practice Guideline as measured by survey question 9.
There is no data regarding target sample proportions for years of experience available. Assuming relatively equal proportions of these groups, equal variance with standard deviation of 1, and a difference of up to 0.3 on question 9’s scale, the type II error rate is expected to be under 10% for a two-sample T-test for a total sample of 600. The following hypothesis may benefit from oversampling for groups of interest and may have lower power, but could also be tested with an overall sample size of 600: H03: There is no difference in physicians (MD and DO) as compared with nurse practitioners (NP) and physicians assistants (PA) in terms of familiarity with specific recommendations in the 2022 CDC Clinical Practice Guideline as measured by survey question 9.
BLS data suggest that physicians (MD and DO) make up approximately 59% of the target population. NP’s make up 27% and PA’s make up approximately 15%. A sample of 600 is likely to yield representative samples of both groups (physicians and NPs combined with PAs). Assuming a total sample size of 600, containing approximately 59% physicians and 41% NPS/PAs, equal variance with a standard deviation of 1, the type 2 error rate to detect differences of up to 0.3 on the scale of question 9 is expected to be under 10%.
Generalizability: Characteristics of respondents and non-respondents will be compared, and, if necessary, non-response weights will be applied to adjust for differences by region and clinician type. If needed, this weighting will support generalization of results to the target population of clinicians.
|
Interviews with Clinicians |
Sampling: Respondents to the clinician survey (drawn from IQVIA clinician sampling frame) who have agreed to be contacted for a follow-up interview, with target of 30 interviews conducted.
Topics:
Analysis: All qualitative data will be entered into NVivo 12 to allow for standardized coding by topic and theme. Thematic analysis will be conducted through dual coding (i.e., two independent coders) of interview notes using thematic codebook and NVivo 12 software, to identify common themes and examine divergence and convergence of themes across interviewee types and other characteristics. |
Interviews with Dentists |
Sampling: We will work with the American Dental Association to identify dentists to participate in the interview.
Topics:
Analysis: All qualitative data will be entered into NVivo 12 to allow for standardized coding by topic and theme. Dual coding (i.e., two independent coders) of interview notes using thematic codebook and NVivo 12 software, to identify common themes and examine divergence and convergence of themes across interviewee types and other characteristics. |
Interviews with Leaders from Health Systems |
Sampling: We will use a convenience sample of individuals identified through contacts at the CDC, from previous work, and through internet searches.
Topics:
Analysis: All qualitative data will be entered into NVivo 12 to allow for standardized coding by topic and theme. Dual coding (i.e., two independent coders) of interview notes using thematic codebook and NVivo 12 software, to identify common themes and examine divergence and convergence of themes across interviewee types and other characteristics. |
Interviews with Payers |
Sampling: We will use a convenience sample of individuals identified through contacts at the CDC, from previous work, and through internet searches.
Topics:
Analysis: All qualitative data will be entered into NVivo 12 to allow for standardized coding by topic and theme. Dual coding (i.e., two independent coders) of interview notes using thematic codebook and NVivo 12 software, to identify common themes and examine divergence and convergence of themes across interviewee types and other characteristics. |
Interviews with Leaders from Professional Associations |
Sampling: We will use a convenience sample of individuals identified through contacts at the CDC, from previous work, and through internet searches.
Topics:
Analysis: All qualitative data will be entered into NVivo 12 to allow for standardized coding by topic and theme. Dual coding (i.e., two independent coders) of interview notes using thematic codebook and NVivo 12 software, to identify common themes and examine divergence and convergence of themes across interviewee types and other characteristics. |
Interviews with Medical Board Leaders |
Sampling: We will use a convenience sample of individuals identified through contacts at the CDC, from previous work, and through internet searches.
Topics:
Analysis: All qualitative data will be entered into NVivo 12 to allow for standardized coding by topic and theme. Dual coding (i.e., two independent coders) of interview notes using thematic codebook and NVivo 12 software, to identify common themes and examine divergence and convergence of themes across interviewee types and other characteristics. |
Focus groups with Patients |
Sampling: We will partner with patient advocacy organizations to identify patient from their member lists.
Topics:
Analysis: All qualitative data will be entered into NVivo 12 to allow for standardized coding by topic and theme. Dual coding (i.e., two independent coders) of interview notes using thematic codebook and NVivo 12 software, to identify common themes and examine divergence and convergence of themes across interviewee types and other characteristics. |
Focus groups with Caregivers |
Sampling: We will partner with patient advocacy organizations to identify caregivers from their member lists.
Topics:
Analysis: All qualitative data will be entered into NVivo 12 to allow for standardized coding by topic and theme. Dual coding (i.e., two independent coders) of interview notes using thematic codebook and NVivo 12 software, to identify common themes and examine divergence and convergence of themes across interviewee types and other characteristics. |
Secondary data populated from a commercial claims database (Merative MarketScan) |
Sampling: Random selection of all patient medical claims from 2020 to 2023 for a total of 5 million patients. All medical claim records, prescription records, and treatment procedures for these 5 million patients with 3-year capture period will be included in sample.
Topics:
Analysis: All the outcomes listed in this section will be measured quantitatively and will be statistically analyzed by computed rates per 100,000 residents using Bureau of Census population estimates. We will develop descriptive tables reporting frequencies by state and quarter. Following the approach used by Bohnert, Guy, and Losby in their analysis of the 2016 CDC Guideline impacts,3 we will conduct interrupted time series analyses with segmented regressions using monthly repeated measures to estimate changes in each of the outcomes, if any, following release of the 2022 CDC Clinical Practice Guideline. As the data permits, we may also explore impacts by prescriber type/specialty, diagnosis categories, patient demographics (e.g., age and gender), pain conditions, and/or indications for opioid use. Health equity analysis related to pain management and care will be cross-sectional, using ordinary least squares regression to estimate the extent to which attributes of the resident population and the supply of health services are correlated with the various measured outcomes. |
Outpatient Clinician Survey
As described above and in greater detail in the SSA documentation, we will conduct a survey of clinicians. Inclusion criteria are that the clinicians: 1) practice in an ambulatory, outpatient, and/or emergency department at least once a week; 2) primarily treat adults; 3) treat patients with acute, subacute, and/or chronic pain other than pain management related to sickle cell disease, cancer-related pain treatment, palliative care, and/or end of life care; 4) practice in the following care areas: family medicine, internal medicine, emergency medicine, surgery, occupational medicine, physical medicine and rehabilitation, neurology, obstetrics and gynecology.
To identify a sample, we will use IQVIA’s national database of clinicians which has a diverse set of clinicians with respect to their occupation, geographic region, and practice type. Clinicians identified through the IQVIA national database will be recruited via email by project team staff. Survey data from clinicians will be collected electronically.
Interviews with Clinicians, Dentists, and Leaders from Health Systems, Payers, Professional Associations, and Medical Boards
We will identify and conduct interviews with clinicians and dentists, as well as leaders from health systems, payers, professional associations, and medical boards to focus on the awareness and implementation of the 2022 CDC Clinical Practice Guideline, unintended consequences of the 2022 CDC Clinical Practice Guideline, and other considerations. Interviews will be conducted by teleconference and are expected to take 60 minutes. All interviews will be conducted using Microsoft Teams. All interviews will be recorded with respondents’ permission and transcribed. If permission to record the interview is not given, we will rely on the notes taken by a notetaker.
Patient Focus Group Interviews
We will conduct focus groups with patients to provide an in-depth understanding of a single or small number of cases set in their real-world contexts. Examining the experiences of patients can provide a deeper understanding of real-world behavior within a specific healthcare context to elucidate perceptions of whether and/or how changes occurred in overall treatment and/or pain management, including opioid prescribing. Working with patient and caregiver advocacy groups, we will identify potential focus group participants.
Caregiver Focus Group Interviews
We will conduct focus groups with caregivers to provide an in-depth understanding of a single or small number of cases set in their real-world contexts. Examining the experiences of caregivers can provide a deeper understanding of real-world behavior within a specific healthcare context to elucidate perceptions of whether and/or how changes occurred in overall treatment and/or pain management, including opioid prescribing.
Because response rates in surveys of clinicians have declined in recent years,4 we will use several approaches to improve the survey response rate. Given Abt Global’s recent experience with web surveys for clinicians and using e-mail addresses obtained through IQVIA’s national database of providers,5 we believe a 20% response rate is achievable.
Plan to Address Nonresponse
To the extent possible, if we have enough information on the respondents within the system who were asked to complete the survey, we will provide a description of sampling strategy and nonresponse patterns, and implications for survey report (and apply weights if we have sufficient detail in order to reduce potential bias). The criterion used for weighting will be a logistic regression model on nonresponse (y/n) with region and clinician type as predictors. If the logit model finds these indicators significant, the contractor will perform an adjustment. Should significant survey effects exist, the base will be adjusted to realign the sample and reduce bias in data analysis. All weighting work will be performed using STATA software.
Strategies to increase clinician participation and minimize nonresponse in surveys include:
Brevity of survey questionnaire (25 or fewer items), expected to take 10 minutes
Incentive for participation
Survey addressing topics that are relevant to clinicians
Follow-up email reminders from project staff
Strategies to increase participation and minimize nonresponse in clinician interviews include:
Invitation to participate made to clinicians already responding to survey invitation
Interview questions and topics that are relevant to their practice, based on the clinical and opioid/MOUD-related experience of the Abt and CDC team.
Incentive for participation
Strategies to increase participation and minimize nonresponse in interviews with dentists, health system leaders, payers, professional association leaders, and medical board leaders include:
Interview questions and topics that are relevant to their work
Incentive for participation
Strategies to increase patient/caregiver focus group participation and minimize nonresponse include:
Engagement of national advocacy groups supporting patients experiencing chronic pain and/or long-term opioid therapy
Engagement of national stakeholders directly involved with advocacy for patients with chronic pain
Incentive for participation
The data collection instruments were designed collaboratively by CDC staff and selected contractors to refine wording, increase efficiency, and verify burden estimates. Contractors consulted included researchers, statisticians, clinicians, and pharmacists with expertise in evaluation research and survey methodology, as well as opioid prescribing. Finally, primary data collection materials were also reviewed and are still under review by the Abt Global’s Institutional Review Board (IRB) for appropriateness of content, burden, clarity, respondents, and data availability.
The estimate for burden hours presented is based on the sum of the total time burden of the outpatient clinician survey, interviews with clinicians, interviews with dentists, interviews with leaders from professional organizations, payers, medical boards, health systems, and focus groups with patients and caregivers. The Clinician Survey is expected to take 10 minutes to complete based on project staff testing and experience with similar data collection efforts. All interviews and focus groups are expected to take 60 minutes to complete based on experience with similar data collection efforts. The burden estimates for these primary data collection efforts are based on Abt Global’s experience with similar data collection efforts in the past.
Abt Global is the subcontractor who will develop data collection tools and perform all analysis on behalf of CDC. The professionals from Abt Global have over 50 years of experience providing high quality, timely, and cost-effective data collection for federal agencies. Abt Global employs many statisticians, health economists, and experienced research methodologists. Kimberly Groover, PhD, from Abt Global, reviewed the proposed statistical analyses. Dr. Groover has designed several rigorous, practice-based research studies for CDC and other federal agencies. She is available should any questions regarding the statistical analyses for this project arise. The key project contact at Abt Global is Ellen Childs.
Contact information for individuals consulted on statistical aspects of the design:
Ellen Childs, PhD, Senior Manager – Research and Evaluation
Abt Global
Role: Project Director
Kimberly Groover, PhD, Associate
Abt Global
Role: Aim 2: Impact Evaluation Task Lead
Sarah Shoemaker-Hunt, PhD, PharmD; Science and Research Director
Abt Global
Role: Senior Technical Advisor
Claire Lay, PhD; Data Analytics Senior Manager
Abt Global
Role: Clinician Survey Technical Advisor
1 Dentists are included as a separate group from clinicians because the research design includes a separate sampling approach and interview guide specific to dentists.
2 US Department of Veterans Affairs. 2022. “Use of Opioids in the Management of Chronic Pain.” Use of Opioids in the Management of Chronic Pain (2022) - VA/DoD Clinical Practice Guidelines
3 Bohnert AS, Guy Jr GP, Losby JL. Opioid prescribing in the United States before and after the Centers for Disease Control and Prevention's 2016 opioid guideline. Annals of internal medicine. 2018;169(6):367-375.
4 McLeod CC, Klabunde CN, Willis GB, Stark D. Health care provider surveys in the United States, 2000–2010: a review. Evaluation & the health professions. 2013 Mar; 36(1):106-26.
5 AHRQ Project “Identifying and Testing Strategies for Management of Opioid Use and Misuse in Older Adults in Primary Care Practices” OMB # it is 0935-0258
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
Author | Leigh Mathias |
File Modified | 0000-00-00 |
File Created | 2025-01-07 |