Evaluation of the CMS Quality
Improvement Organizations:
Medication Safety and
Adverse Drug Event Prevention (CMS-10675)
The purpose of this Information Collection Request (ICR) is to collect data to inform the program evaluation of the Centers for Medicare & Medicaid Services (CMS) Quality Improvement Organizations (QIO) current contract known as the 11th Scope of Work (SOW).1 The current ICR focuses on evaluating the national implementation of the quality improvement activities of the Quality Innovation Network Quality Improvement Organizations (QIN-QIOs) and is part of a larger evaluation of the overall national impact of the QIO program.2 This ICR aims to assess the QIN-QIO Task which focuses on Medication Safety and Adverse Drug Event (ADE) Prevention. For this evaluation, we are using a mixed methods design to compare quality improvement activities of pharmacists and healthcare providers (physicians, physician assistants, and nurse practitioners) participating in the QIN-QIO program (participating group) with those not participating in the QIN-QIO program (non-participating group).
As mandated by Sections 1152-1154 of the Social Security Act, CMS directs the QIO program, which is one of the largest federal programs dedicated to improving health quality for Medicare beneficiaries. QIOs are groups of health quality experts, clinicians, and consumers who work to assist Medicare providers with quality improvement throughout the spectrum of care and to review quality concerns for the protection of beneficiaries and the Medicare Trust Fund.3 This program is a key component of the U.S. Department of Health and Human Services’ (HHS) National Quality Strategy4 and the CMS Quality Strategy.5 The work is aligned with the current HHS and CMS administration priorities to empower patients and doctors to make decisions about their health care; usher in a new era of state flexibility and local leadership; support innovative approaches to improve quality, accessibility, and affordability; and improve the CMS customer experience. In the current SOW, 14 QIN-QIOs coordinate the work in 53 U.S. states and territories.
CMS evaluates the quality and effectiveness of the QIO program as authorized in Part B of Title XI of the Social Security Act.6 CMS created the Independent Evaluation Center (IEC) to provide CMS and its stakeholders with an independent and objective program evaluation of the 11th SOW.
ADEs are defined as “injury resulting from medical intervention related to a drug,”7 and cause the majority of preventable deaths in hospitals.8 ADEs escalate healthcare costs and utilization, increasing admission and readmission rates, emergency department (ED) visits, and physician visits.9 ADEs are particularly problematic for older adults who have multiple chronic conditions and interact with many care settings. In particular, opioid misuse and overdose is a significant cause of ADEs and was declared a public health emergency by the White House in 2017. In 2016, over 14 million Medicare Part D beneficiaries received opioid prescriptions,10 and many of these beneficiaries received extreme amounts of the drugs.11 The Medicare population has one of the highest and fastest-growing rates of diagnosed opioid use disorder.12
As part of the HHS Opioid Initiative launched in March 2015, CMS developed a multipronged approach to address misuse and promote programs that support treatment and recovery support services for clinicians, beneficiaries, and families. CMS also worked with HHS and other health agencies to develop a National Action Plan for Adverse Drug Prevention (2014).13 In addition to opioids, the Action Plan focused on ADEs caused by other high-risk medication (HRM) groups: anticoagulants and diabetic medications. Given the burden of ADEs caused by these three classes of drugs, focusing prevention efforts in these areas could have a significant impact on reducing harm and improving population health among Medicare beneficiaries.
The QIO program provides technical assistance to reduce ADEs in beneficiaries resulting from polypharmacy, specifically those who use three or more medications including a prescription in a HRM) drug groups. In the 11th SOW, specific interventions include training providers through Learning Action Networks; developing collaborations among local providers across care settings; providing materials and information resources; and helping providers collect data to monitor prescribing practices.
To evaluate the effectiveness of this program, we will use a mixed method evaluation combining secondary data analysis of Medicare claims with a community provider survey. We plan to conduct an online survey of 1,200 community-based pharmacists and providers (physicians, nurse practitioners, and physician assistants). These participants were selected based on the volume and settings of professionals recruited by QIOs to support this QIN-QIO Task C.3.6 Medication Safety and Adverse Drug Event (ADE) Prevention. The survey will include professionals participating in the QIN-QIO program (participating group) and a similar sample that are not participating in the QIN-QIO program (non-participating group). Contact information for the participating group will be collected from QIO staff. The non-participating group of providers will be contacted through the Medscape Market Research Panel, an online panel of physicians and other healthcare professionals in the U.S. States where QIN-QIOs do not work with group practice physicians related to ADEs will be excluded from the survey.
Our proposed survey assesses the extent to which best practices have been used (as summarized by the Joint Commission on Accreditation of Healthcare Organizations patient safety event taxonomy model, the basis for the National Action Plan Section 3: Prevention Approaches), the level of engagement with the QIO, and other influences that can help explain progress towards the goals of the QIN-QIO SOW. The questions used for these constructs related to program and non-program influences have been adopted from previously used and/or validated instruments, including the IEC Nursing Home Survey that was approved by OMB (OMB control number 0938-1330, Expiration date 6/30/20).
The survey will also support estimates of the perceived attribution of the QIN-QIO program impact in improving ADE prevention from the perspective of healthcare providers. The perceived influence on quality improvement efforts will be quantified and, along with econometric modeling methods, will be used to assess program impact. Estimating attribution is a contract requirement for the IEC and helps provide evidence of impact of the QIN-QIO program. Since current analytical methods do not adequately address the overlap of quality improvement initiatives targeting medication safety and ADE prevention, the IEC developed an innovative approach, combining survey input with modeling, to estimate the relative importance of the QIN-QIO program. The concept is supported at the highest level of administration for Quality Improvement at CMS and has been presented at national conferences14,15 and to CMS/CCSQ leadership.16 The survey data is an essential component of this analytic method.
An overview of our proposed survey, including survey topics, participants, and frequency and duration of data collection is shown in Table 1. The survey instrument is included in Appendix A. The information collected through the survey will complement the existing data by helping identify factors associated with ADE outcomes of interest from existing data sets such as Medicare claims. For example, claims data can provide information on whether the number of prescriptions for opioids has decreased, but not what has helped to facilitate the decrease. Please see Attachment 1 that provides a crosswalk of how the existing and new data sources will address the evaluation questions.
Table 1: Overview of IEC Community-Provider Survey
Survey Mode |
Survey Topics |
Participants |
Frequency |
Online survey community- based pharmacists and healthcare providers |
|
|
Once (May-June 2019)
|
The QIO program is mandated by Sections 1152-1154 of Part B of Title XI of the Social Security Act, as amended by the Peer Review Improvement Act of 1982 and by the Trade Adjustment Assistance reauthorization bill (Pub. L. 112-40) signed by the President in October 2011. This law includes language authorizing evaluation of the QIO program:
§ Social Security Section. 1153. [42 U.S.C. 1320c–2] c(2): “the Secretary shall have the right to evaluate the quality and effectiveness of the organization in carrying out the functions specified in the contract.”17
Our proposed data collection is necessary for CMS to evaluate the QIO program and provide reports on the performance of QIOs. Sections 1152-1154 of Part B of Title XI of the Social Security Act requires CMS to “regularly furnish each quality improvement organization with a contract under this section with a report that documents the performance of the organization in relation to the performance of other such organizations.”
As required by this law, CMS published the general criteria and standards used for evaluating the efficient and effective performance of contract obligations for the program and provided the opportunity for public comment in the following Federal Register notice:
Medicare Program; Evaluation Criteria and Standards for Quality Improvement Networks Quality Improvement Program Contracts [Base and Task Order(s)]: 60-day Notice published on August 11, 2014 in Federal Register Volume 79, Number 154, pg. 46830-46835 (CMS–3300–NC); Final Notice published on December 30, 2014 in Federal Register Volume 79, Number 249, pg. 78440-78442 CMS–3300–FN).
These evaluation criteria are required for contract monitoring rather than evaluation of overall impact of the program. We will use the data required for contract monitoring along with the proposed data in outcome and impact analyses. Attachment 1 provides a crosswalk between the evaluation data needs, existing data sources, and the proposed data to be collected under this information collection request to show how the data will be integrated efficiently to assess the impact of the program.
The purpose of this data collection is to inform the program evaluation of the QIO program’s 11th SOW as required in Sections 1152-1154 of the Social Security Act. The current data collection will focus on the impact of the QIO program on medication safety and reducing ADEs through collaborations with community-based healthcare providers. The findings will also be used to inform CMS annual reports to Congress, reports and briefings to OMB, and other stakeholder groups. The results from this data collection may be published in annual program reports and peer-reviewed journal publications.
CMS can use these findings to improve the delivery of the QIN-QIO program. For example, if we identify covariates, such as engagement with QIO, that are influential in predicting improved outcomes (e.g., reduced admissions), CMS can focus on ways to increase engagement with healthcare providers or strategically recruit facilities that are most likely to benefit from the program. We will use this information to identify QIO program elements that help healthcare providers comply with the National Action Plan for Adverse Drug Prevention Strategies. CMS can also use this information to modify future Scopes of Work for continued improvement of the QIO Program. For example, a strategic goal for the 12th SOW is to empower patient and providers to make informed decisions.18 If involving beneficiaries and families is found to be correlated with effectiveness or engagement with QIOs, CMS can use this finding to support adding this aim to other programs addressing substance abuse.
Since medication safety and ADE reduction will continue to be a focus in the 12th SOW,18 the survey will provide a baseline for activities, resources used, and perceived effectiveness of the QIO program. Finally, CMS may use findings from the survey in Reports to Congress or responses to investigations from the General Accounting Office to address acceptance and use of the QIO program by community-based providers who are responsible for prescribing and filling the majority of prescriptions for high-risk medications to the Medicare population.
We will conduct an online survey to effectively balance the need for program information with the costs of data collection and potential burden on program staff and stakeholders. The online survey will be brief to minimize burden on participants and to minimize break-offs. We conducted pre-tests to improve clarity and understandability of the survey questions, to reduce participant burden, and to enhance survey administration. Tested recruitment and data collection procedures will be used to maximize completion and to achieve the desired response rate.
This data collection request does not require a signature from participants. Consent will be obtained implicitly when they choose to continue the survey after reading the OMB statement.
This information collection does not duplicate any other effort, and the information cannot be obtained from any other source. Administrative data are regularly collected from QIN-QIOs, but not from community-based healthcare providers related to these quality improvement efforts. To prevent duplication of data collection, the IEC developed a Baseline Current Environment Report and Data Inventory, which outlined the current CMS and secondary datasets and sources that can be used to inform the evaluation, including Medicare claims data. Additional data collection is only proposed when data necessary to inform the QIO evaluation questions are not available in current data sets, program reports, or other sources. The IEC is collaborating with the QIN National Coordinating Center (NCC), the Home Health Quality Initiative, and other quality improvement programs to share information and data to avoid any duplication of data collections from QIO providers and beneficiaries.
The existing datasets and newly collected data will be integrated to address the evaluation questions and meet the goals of the data collection as previously described in the Background Section. The IEC has conducted preliminary analysis of outcomes associated with ADEs using claims data (see Attachment 2). These findings showed significant reductions in hospital readmission rates for Medicare beneficiaries at risk for ADEs served by providers who participate in the QIN-QIO program, compared to at-risk beneficiaries whose providers are not involved with the program. However, no differences in rates were observed for readmissions related to specific high-risk medications (anticoagulants, opioids or diabetes agents). This survey will help identify factors which may be influencing the outcomes and could be used to adjust the analysis.
The survey instrument, as seen in Appendix A, has similar questions to the survey conducted to support the evaluation of the CMS QIO program “Reducing Healthcare Acquired Conditions in Nursing Homes”, which is under OMB control number 0938-1330. The similarity of questions is due to similar objectives and guiding evaluation questions common to both evaluations, while tailored to the specific objectives of this data collection. Also, responses to the first fielding of the survey under OMB 0938-1330 (from September to November 2017) validated the majority of questions, while providing guidance for minor revisions of other items.
As seen in Appendix A, the content of this survey is tailored to the activities and resources relevant to the settings and health professionals participating in the QIO program to increase medication safety and reduce adverse drug events—pharmacists working in retail pharmacies, and providers working in either independent or group practices. Another difference from the previous survey is the mode of administration—the current data collection request is for an online survey, whereas the survey under OMB 0938-1330 is by telephone—and this change is also reflected in the wording.
Survey participants may be employed by small or large businesses based on the definition of the Small Business Paperwork Relief Task Force as having 500 or fewer employees or $6M or less in receipts.19 For example, approximately 58% of physicians work in practices owned by 10 or fewer physicians,20 and more than 22,000 community pharmacies are considered small businesses.21
To reduce the impact on these small businesses and entities, data collection will be streamlined, focused, and limited to only the collection of data required to answer the evaluation questions. Surveys will require 20 minutes or less to complete for respondents participating in the QIN-QIO program, and 10 minutes or less to complete for respondents not participating in the QIN-QIO program, for an average of 15 minutes or less. Pre-notification emails or letters will be sent out to respondents prior to data collection to inform them about the purpose of the data collection, expected time required, and to provide other elements of informed consent (Appendix B).
If these information collection activities are not conducted, CMS will not be able to fulfill the mandates of the Social Security Act Title XI Section 1153 to evaluate the QIO program and provide a report on performance to contracted QIOs. This survey will be administered only once to provide CMS and QIOs feedback to improve their services and programs in future efforts.
There are no special circumstances relating to the Guidelines of 5 CFR 1320.5.
The 60-day Federal Register notice was published on 07/20/2018 (83 FR 34593). The 30-day Federal Register notice published on November 29, 2018 (83 FR 61383). In addition to the required public notices, we pre-tested instruments among 8 participants to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format, and on the data elements to be recorded, disclosed, or reported. We have made updates to the survey instrument based on the survey pretest (see Appendix A for the revised survey instrument, Appendix A1 for screen shots with an example, and Attachment 3 for a Crosswalk of Survey Instrument Changes).
We received two public comments about the ICR from the general public. One comment expressed concerns over access to medication for people with chronic pain conditions, and we referred them to the HHS Pain Management Best Practices Inter-Agency Task Force. The second comment provided background on and requested endorsement of new bill to increase access to providers for opioid treatment. Since these comments were not directly related to the data collection process under way, they did not influence the design (see Attachment 4 for Response to Public Comments).
We also received additional comments from one professional society and one QIN-QIO and discussed questions about the survey with QIN-QIOs during a regularly scheduled “Think Tank” meeting for the ADE Prevention task. The major questions and concerns from QIOs related to (1) sampling and (2) methods for discerning attribution toward quality improvement efforts from QIN-QIO resources, which are summarized below. We have made changes to the sample and survey instrument based on these comments.
Comments on Sampling (S1 and S2 on Appendix A Survey Instrument)
One comment mentioned was that the sample was not representative of all settings and professional types that have been recruited across QIN-QIOs, such as “home health, social work, care transitions nurses, EMS, and non-medical home care, to name a few”. It is not feasible to develop a comparison group across each setting and professional type recruited for the program, and that our choices of community-based pharmacies and physician practices were based on assessing the volume of recruiting efforts from administrative records. However, in response to this comment, we expanded the sample to include nurse practitioners and physician assistants working in physician practices based on the input. We also consulted with the QIN-QIOs about obtaining their help with contact lists of community-based pharmacies and practices that participate in the QIO program and received these lists along with information to help participants identify QIO resources.
A second comment was about including nursing homes in the sample; although these facilities were the settings for a large volume of technical assistance (TA) provided for increasing medication safety and ADE prevention, QIN-QIOs also recruited nursing homes for a different task. Commenters speculated that survey participants may confuse the TA provided for these tasks. They also noted “Nursing homes are over-burdened with surveys, QAPI, composite measures, and National Healthcare Safety Network (NHSN) reporting, not-withstanding the enormous burden of staff turnover.” We agree with these comments and for these reasons have removed nursing homes from the sample.
Comments on Attribution Methods (Q6 & Q7)
Commenters noted that it may be difficult for participants to identify TA provided by the QIN-QIOs because the program frequently collaborates with other CMS programs as well as state and local initiatives. For this reason, we reached out to the QIOs to obtain the names by which they would be recognized by participants (e.g., local QIO name, campaign name, etc.). We also asked for the names of the partnering organizations, which can be used to adjust analysis if participants identify QIO activities but not the name of the QIO.
Another comment was about the attribution measure requiring participants to enter percentages representing how helpful they found each resource they use to prevent ADEs. Commenters believed this process would be too confusing and suggested using 5-point scales for each resource instead. We had used both percentages and scales for a similar question in the survey of nursing home administrators (OMB control number 0938-1330) and found that participants could manage both methods of assigning attribution and that responses were highly correlated. However, in pretests of the current survey instrument we observed that the percentage method was slightly more challenging than in the nursing home survey, partly because pharmacists and physicians tend to use more sources of information than did nursing home administrators. For these reasons, we changed the responses to this question to a nominal scale.
Comments on Other Content
Other helpful comments that led to changes on the instrument were to add a definition of ADEs in the introduction and to include other quality improvement activities that are best practices (Question 5). These changes were tested in the pretests and added.
Although the burden of response is low, we will provide an honorarium to each panel member to increase response rates. Providing small incentives has been shown effective in increasing response rates, especially for online surveys.22 Honoraria values will be based on industry standard practices for different professions using the Medscape online panel. Using Medscape’s established rates, physicians who respond will receive $35, and pharmacists, physician assistants, and nurse practitioners who respond will receive $30. These incentives, along with evidence-based strategies for successful recruiting (e.g., using relationships among QIN-QIO program staff, establishing CMS as the sponsor of the study to improve provider confidence in the work), will be used to encourage participation.
To protect the privacy of participant data, each survey respondent will be de-identified and given a unique identification (ID) number. This ID number will be the only information that is recorded on survey instruments, and the survey instruments will be stored separately from other data collected within this project. National Provider Identification numbers will be stored separately from data files and will only be accessed by authorized team members for further analysis (e.g., using survey results in further analysis of beneficiary outcomes).
No one outside the evaluation team will have access to the individual responses, nor will anyone outside the team be able to identify any individual participant with their responses. Reports on data collected will be presented in aggregate form only. At the end of the project, the Primary Investigator will arrange for the proper storage and destruction of all data in compliance with all relevant government regulations and policies.
The survey does not include any sensitive questions related to private matters.
The estimated annual burden (number of burden hours per year) and costs for the data collection are outlined in Table 2 below.
Table 2: Estimated Annual Burden Hours and Cost
Survey with QIN-QIO Participating Respondents
Data Collection Activity and Type of Participant |
Estimated Number of
Participants |
Number of Responses per
Participant |
Hours per Response |
Estimated Annual Burden
Hours |
Hourly Wage Rate23 |
Estimated Total Annual
Participant Cost |
Pharmacists in a community setting |
300 |
1 |
0.33 |
99 |
$117.04 |
$11,587 |
Physicians in a primary care practice |
201 |
1 |
0.33 |
72.60 |
$190.74 |
$12,652 |
Physician Assistants |
42 |
1 |
0.33 |
11.55 |
$100.74 |
$1,396 |
Nurse Practitioners |
57 |
1 |
0.33 |
14.85 |
$103.36 |
$1,944 |
Survey with Non-Participating Respondents24
Data Collection Activity and Type of Participant |
Estimated Number of
Participants |
Number of Responses per
Participant |
Hours per Response |
Estimated Annual Burden
Hours |
Hourly Wage
Rate23 |
Estimated Total Annual
Participant Cost |
Pharmacists in a community setting |
300 |
1 |
0.17 |
51 |
$117.04 |
$5,969 |
Physicians in a primary care practice |
201 |
1 |
0.17 |
34 |
$190.74 |
$6,518 |
Physician Assistants |
42 |
1 |
0.17 |
5.95 |
$100.74 |
$719 |
Nurse Practitioners |
57 |
1 |
0.17 |
7.65 |
$103.36 |
$1,002 |
Total for all respondents |
1,200 |
-- |
-- |
300 |
-- |
$41,787 |
The estimated number of survey respondents reflects the planned sample of 1,200 healthcare providers (600 in the participating group and 600 in the non-participating group). The burden hour estimates for the survey are based on pre-tests of the length of time each type of respondent is likely to need to complete the survey screener and questions. The survey is expected to take approximately 10 minutes with community-based providers that are not participating in the QIN-QIO program and approximately 20 minutes to complete if the provider is participating in the QIO program, with an overall average of 15 minutes. The number of questions asked of a respondent will vary depending on factors like the number of quality improvement objectives their organization is addressing and whether their organization is interacting with a QIN-QIO.
The estimated annual hour and cost burden is based on the 2017 hourly wage rate of the categories of participants for these data collections (Table 2). The total cost is calculated by multiplying the number of responses by the average time per response by the adjusted hourly wage. The costs are then summed to derive the total cost for all participants.
There are no capital costs.
The cost of this information collection effort to the Federal government consists of the costs for government (CMS) activity and CMS’s contractor activity (Table 3). The costs to CMS involve labor costs for overseeing the contractor’s work and reviewing and providing guidance on data collection instruments, OMB clearance package, and other materials. The costs to CMS’s contractors are the costs to carry out the data collection and analysis, develop written reports, and present the findings to CMS and other stakeholders. These costs include labor hours for survey development and testing; sample recruitment, screening, and scheduling; survey administration and management; data cleaning and analysis; and developing reports. Operational expenses include overhead, survey scripting, data processing, and coding. Survey costs increase over time due to the escalation in rates and increased level of effort related to analysis and reporting.
For purposes of OMB review and approval, we have annualized the number. The estimated annual cost to the Federal government over a standard 3-year OMB approval period will be $90,285.
Table 3: Cost to the Federal Government
Activity |
Total |
Government
Activity |
$20,000 |
Contractor
Activity |
$160,570 |
Total |
$180,570 |
Annual Cost
|
$90,285 |
This is a new information collection.
Data collection will take place between May 2019 and June 2019. Analysis of survey findings will take place from June 2019 through August 2019. Our plans and timeline for reports and publications are outlined in Table 4. Reports include program management reports that provide ongoing performance data that can guide CMS’ program decisions regarding continuation or modification of contract recruitment and performance targets, measurement strategies, and recommended evidence-based interventions. In addition, we will develop documents and reports suitable for presentation to various audiences, national stakeholders, and policymakers, including presentation at professional meetings, and publications in peer-reviewed journals.
Table 4: Deliverable Schedule for Data Collection and Reporting Activities
Deliverables |
Timeline |
Data Collection and Analysis Report |
5/2019 – 8/2019 |
Presentation(s) |
9/2019 |
Publications (e.g., brief reports, analytic memos, and peer‑reviewed manuscripts) |
9/2019 – 3/2020 |
Final Data Collection and Analysis Reports |
8/2019, 8/2020 |
Statement B provides an overview of our statistical techniques used to analyze survey data.
The expiration date and will be displayed on collection instruments.
There are no exceptions to the certification statement.
1 CMS. (2018). Current Work: QIO Program 11th SOW (2014-2019). Retrieved February 2, 2018 from https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityImprovementOrgs/Current.html
2 CMS previously submitted ICR 10622 and was approved to evaluate another QIN-QIO task focusing on reducing Healthcare Acquired Conditions in Nursing Homes.
3 CMS. (2017). Quality Improvement Organizations. Retrieved February 2, 2018 from https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityImprovementOrgs/index.html?redirect=/qualityimprovementorgs/
4 Department of Health and Human Services (DHHS). (2015). 2015 Report to Congress: National Quality Strategy for Quality Improvement in Health Care. Retrieved February 2, 2018 from https://www.ahrq.gov/workingforquality/reports/2015-annual-report.html
5 CMS. (2015). CMS Quality Strategy 2016. Retrieved February 2, 2018 from https://www.cms.gov/medicare/quality-initiatives-patient-assessment-instruments/qualityinitiativesgeninfo/downloads/cms-quality-strategy.pdf
6 Social Security Administration. Contracts with Quality Improvement Organizations. Retrieved February 2, 2018 from https://www.ssa.gov/OP_Home/ssact/title11/1153.htm
7 IOM. A Blueprint for Transforming Prevention, Care, Education and Research. Institute of Medicine, 2011.
8 Levinson DR. Adverse events in hospitals: national incidence among Medicare beneficiaries. U.S. Department of Health and Human Services, Office of Inspector General, 2010 Contract No.: OEI-06-09-00090.
9 Beijer HJ, de Blaey CJ. Hospitalisations caused by adverse drug reactions (ADR): a meta-analysis of observational studies. Pharm World Sci. 2002;24(2):46-54. PubMed PMID: 12061133.
10 United States Government Accountability Office. (2017, October). Prescription Opioids: Medicare needs to expand oversight efforts to reduce the risk of harm. Retrieved from https://www.gao.gov/assets/690/687628.pdf
11 U.S. Department of Health and Human Services – Office of Inspector General (OIG) (2017, July). HHS OIG Data Brief: Opioids in Medicare Part D – Concerns about extreme use and questionable prescribing. Retrieved from https://oig.hhs.gov/oei/reports/oei-02-17-00250.pdf
12 Lembke, A. and Chen, J. (2016). Use of opioid agonist therapy for Medicare patients in 2013. JAMA Psychiatry. 73(9), 990- 992. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5526587/
13 National Action Plan for Adverse Drug Event Prevention. Washington D.C.: U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion, 2014.
14 Yu P, Sonnenfeld N. Innovative Evaluation Methods. Presented at CMS Quality Conference, February 2018 in Baltimore, MD.
15 Tregear S, Sonnenfeld N. Market Share approach for assessing program attribution. Presented at American Evaluation Association Conference, October 2016 in Atlanta, GA.
16 Yu, P. Briefing to CMS Chief Medical Officer for Quality Improvement on July 25, 2016 in Baltimore, MD.
17 Social Security Administration. Contracts with Quality Improvement Organizations. Retrieved February 8, 2018 from https://www.ssa.gov/OP_Home/ssact/title11/1153.htm
18 Remarks by Dennis Wagner at the 2018 CMS Quality Conference special session: Aligning Future Quality Improvement Work with Administrative Priorities.
19 Final Report of the Small Business Paperwork Relief Task Force. (2003). Retrieved February 6, 2018 from https://www.sba.gov/sites/default/files/Final%20Task%20Force%20Report_June%202003.pdf
20 American Medical Association. (2017). AMA 2016 Physician Practice Benchmark Survey. Accessed February 12, 2018 from https://www.ama-assn.org/sites/default/files/media-browser/public/health-policy/PRP-2016-physician-benchmark-survey.pdf.
21 Letter to Congressional leadership from National Community Pharmacists Association to address tax fairness for small business owners. Accessed February 18, 2018 from http://www.ncpanet.org/newsroom/details/2017/11/14/ncpa-supports-expanded-tax-relief-to-small-businesses
22 Goritz AS (2006). Incentives in web studies: methodological issues and a review. International Journal of Internet Science. 1:58-70.
23 Based May 2017 National Industry-Specific Occupational Employment and Wage Estimates, Bureau of Labor Statistics (https://www.bls.gov/oes/current/oes_nat.htm#29-0000). To account for benefits and overhead, 100% x the hourly wage was added to the hourly wage to provide the adjusted hourly wage for each group. Pharmacists (29-1051) on average earned $58.52 in 2017 (adjusted hourly wage: $117.04). Physicians-General Internists (29-1063) on average earned $95.37 (adjusted hourly wage: $190.74). Physician Assistants (29-1071) on average earned $50.37 in 2017 (adjusted hourly wage: $100.74). Nurse practitioners (29-1171) on average earned $51.68 in 2017 (adjusted hourly wage: $103.36).
24 The survey is estimated to take approximately 10 minutes to complete if the facility is not participating in the QIN-QIO program. The survey is estimated to take approximately 20 minutes to complete if the facility is participating in the QIO program, because the survey includes additional questions related to interactions with the QIO.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement – Part A Evaluation of the CMS Quality Improvement Organizations: Medication Safety and Adverse Drug Event |
| Subject | Medication Safety and Adverse Drug Event Prevention (CMS-10675) |
| Author | Booz Allen Hamilton |
| File Modified | 0000-00-00 |
| File Created | 2021-01-20 |