CAHPS for MIPS SS Part B - Passback_CAHPS 4-5-17 clean

Supporting Statement – Part B

Consumer Assessment of Healthcare Providers and Systems (CAHPS) Survey

for the Merit-Based Incentive Payment System (MIPS)

CMS-10450, OCN 0938-1222

Collections of Information Employing Statistical Methods

Introduction

The Centers for Medicare & Medicaid Services (CMS) requests a revision to a previously approved survey under the Paperwork Reduction Act of 1995 to implement the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Survey for the Merit-based Incentive Payment System (MIPS). Specifically, CMS will use the CAHPS for MIPS survey to collect data on fee-for-service Medicare beneficiaries’ experiences of care with providers participating in the MIPS for use in quality reporting, the Physician Compare website, and annual statistical reports describing MIPS data for all MIPS eligible clinicians.

Under MIPS, groups of two or more eligible clinicians may choose to use the CAHPS for MIPS survey to fulfill part of the requirements for data submission for the quality performance category. Groups that elect to include the CAHPS for MIPS survey as a quality measure must use a CMS-approved survey vendor to submit CAHPS data but other quality data may be reported by any single one of the other available submission mechanisms for the quality performance category. For the 12-month performance period, a group who wishes to voluntarily elect to participate in the CAHPS for MIPS survey measures must use a survey vendor that is approved by CMS for a particular performance period to collect and transmit survey measures data to CMS.

Under MIPS, groups may opt to, but are not required to, submit CAHPS for MIPS data as one of their quality measures. In contrast, the PQRS required groups of over 100 professionals to submit CAHPS for PQRS data, and allowed groups of 25-99 professionals to opt to submit CAHPS data.

The CAHPS for MIPS survey counts for one measure towards the MIPS quality performance category. Groups that elect the CMS-approved vendor reporting mechanism must select an additional group data submission mechanism in order to meet the data submission criteria for the MIPS quality performance category. CAHPS for MIPS can affect a group’s total score through both the quality and improvement activities performance categories. CAHPS for MIPS counts as one of the six required measures for the quality performance category. In instances where an outcome measure is not available, CAHPS for MIPS will be considered a patient experience measure and a high priority measure. If a group or clinician reported on an outcome measure or another high priority measure, reporting on a patient experience measure such as CAHPS for MIPS would result in two bonus points that are counted towards the score in the quality performance category. In addition, the use of a patient experience measure (including CAHPS for MIPS) would be counted as a high weighted activity under the improvement activities performance category.

This request for a revision to a previously approved package takes the OMB control number 0938-1222. The currently approved CAHPS for Physician Quality package expires on 02/28/2017.

1. Describe (including a numerical estimate) the potential respondent universe and any sam­pling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Because historical participation rates for quality data submission under PQRS have never reached 100 percent, we anticipate that MIPS will not achieve full participation. Under the 2014 PQRS, 834,358 of approximately 1.3 million eligible professionals (EPs) (including those who belonged to group practices that participated under the group practice reporting option (GPRO), EPs within an accountable care organization (ACO) participating under the Shared Savings Program or Pioneer ACO Model, and EPs participating through the comprehensive primary care (CPC) initiative. The 2014 PQRS participation rate was 62.8 percent, quadruple the 15 percent participation rate in the program’s first year (2007).¹

Given that in the first MIPS performance period, the CAHPS for MIPS survey instrument will have the same questions as the CAHPS for PQRS survey, we assume that the group practices that previously contracted with a CMS-approved survey vendor to collect and report CAHPS data under PQRS data will continue to do so under MIPS. We assume that some practices will voluntarily submit quality performance category data, including CAHPS for MIPS data, because MACRA permits any EP under PQRS who is not a MIPS eligible clinician the option to volunteer to report on applicable measures and activities under MIPS. ² Voluntary reporters will be scored and receive performance feedback under MIPS, but will not be subject to payment adjustments.

Because the MIPS program has not yet been implemented, we estimate the potential universe of practices containing MIPS eligible clinicians required to report and the potential universe of practices that can voluntarily report using 2014 data from the PQRS, VM Program and other CMS data. The potential universe of MIPS eligible clinicians, within these group practices, subject to reporting requirements includes between approximately 592,119 and 642,119 (among the 1,060,901-1,110,901 clinicians in MIPS eligible specialties). The potential universe of clinicians whose group practice may voluntarily report use of a CAHPS survey includes approximately 668,090 clinicians.³ Clinicians may voluntarily submit MIPS quality data to get feedback on their performance.

2. Describe the procedures for the collection of information including:

- Statistical methodology for stratification and sample selection,

- Estimation procedure,

- Degree of accuracy needed for the purpose described in the justification,

- Unusual problems requiring specialized sampling procedures, and

- Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Groups that elect to use the CAHPS for MIPS survey as a MIPS quality measure must use a CMS-approved survey vendor to administer and report the CAHPS for MIPS survey. The CMS-approved survey vendor is required to meet the data submission requirements on the sample of the Medicare Part B patients. Based on reporting year 2015 data in which 461 group practices elected to participate in the CAHPS for PQRS survey, we anticipate that 461 groups will contract with CMS-approved survey vendors to collect CAHPS for MIPS data for the MIPS transition year. Groups that elect to participate in the CAHPS for MIPS survey must submit additional quality measures via another submission mechanism (EHR, QCDR, registry, or CMS Web Interface) to ensure that the group meets the requirement for the minimum number of measures. Groups that do not elect to participate in the CAHPS for MIPS survey are required to submit the minimum number of required measures through a single submission mechanism (EHR, QCDR, registry, or CMS Web Interface).

TABLE 1: Summary of Proposed Quality Data Submission Criteria for CAHPS for MIPS Survey

Performance Period	Measure Type	Submission Criteria, including Sampling	Data Completeness
Jan 1 – Dec 31	Groups of 2 or more eligible clinicians	CMS-approved survey vendor would have to be paired with another reporting mechanism to ensure the minimum number of measures are reported. CAHPS for MIPS survey would fulfill the requirement for high priority measure (if no outcome measure is available) towards the MIPS quality data submission criteria. CAHPS for MIPS survey will only count for one measure.	Sampling requirements for their Medicare Part B patients

For the CAHPS for MIPS survey, CMS plans to use the same sampling methodology as the CAHPS for PQRS survey. ⁴ CMS will identify beneficiaries eligible for the survey from the pool of beneficiaries assigned to the group. CMS assigns original Medicare beneficiaries to a practice based on the plurality of the primary care claims during the first three quarters of the performance period. CMS will then randomly select samples from those assigned beneficiaries to create the sample for the CAHPS for MIPS survey. The sample will be limited to beneficiaries aged 18 or older and who are not known to be institutionalized or deceased. The sample is drawn at the level of the group, not at the individual provider level. The survey names a specific MIPS eligible clinician who delivered primary care to the beneficiary over multiple visits in the performance period, to help orient the beneficiary to the care he or she received.

In keeping with guidance for CAHPS for PQRS, the number of patients sampled for the 2017 MIPS survey varies based on the size of the group:

• For large groups of 100 or more MIPS eligible clinicians:

o    CMS will draw a sample of 860 beneficiaries

o    If the group has fewer than 860 beneficiaries, but more than 415 beneficiaries, all eligible beneficiaries will be surveyed

o    If the group has fewer than 416 beneficiaries, the survey cannot be conducted

• For groups with 25 to 99 MIPS eligible clinicians:

o    CMS will draw a sample of 860 beneficiaries

o    If the group has fewer than 860 beneficiaries, but more than 254 beneficiaries, all eligible beneficiaries will be surveyed

o    If the group has fewer than 255 beneficiaries, the survey cannot be conducted

• For groups with 2 to 24 MIPS eligible clinicians:

o    CMS will draw from a sample of 860 beneficiaries

o    If the group has fewer than 860 beneficiaries, but more than 124 beneficiaries, all eligible beneficiaries will be surveyed

o    If the group has fewer than 125 beneficiaries, the survey cannot be conducted

      

The sample sizes recommended above are based on analysis of 2012 CAHPS for ACOs survey data.  Specifically, we set a target number of completed questionnaires for each group practice to obtain a desired level of interunit reliability (IUR) for most survey measures. The IUR is defined as 1-V/(V+t2), where V is the variance of the estimate for a specific unit and t2 is the between-unit variance of population means. For CAHPS for ACOs and CAHPS for PQRS, IUR=0.75 is regarded as adequate reliability for public reporting; IUR between 0.60 and 0.75 and in the lowest 12% of reliability for ACOs or practices is considered low, while IURs below 0.60 are deemed very low.  Measure scores with lower than adequate IURs can still provide practices with useful information about patient experience and potential areas for improvement.  The target sample for group practices (regardless of their number of eligible clinicians) is 860, as a sample of this size is anticipated to produce measure scores meeting the adequate reliability threshold for most measures; a minimum sample size threshold is set for each practice size category to ensure that practices do not pursue the survey if they have so few beneficiaries that most measures would be expected to have very low reliability.  These recommendations reflect a conservative approach that suggests sampling the same sample size for medium and small groups as is recommended for large groups when it is feasible, but lowers the minimum sample size threshold.  During 2017, CMS will evaluate sample size recommendations for practices participating in the CAHPS for MIPS survey.

        The historical response rate for the CAHPS for PQRS survey ranged from 48 percent in 2013 to 37 percent in 2015.  Factors that contributed to the lower response rate include survey administration by a single vendor vs. multiple vendors, and time of year of survey administration.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, OMB guidance requires that a non-response bias assessment be conducted to determining if the results are generalized to the universe studied.

The CAHPS for MIPS survey is collected via a mixed-mode data collection protocol that uses a pre-notification letter alerting sample members that a survey will be mailed to them shortly, a first mailing of the full questionnaire booklet, followed by a second mailing to those who do not respond to the earlier mailing of the questionnaire. For those who also do not respond to the second mailing of the questionnaire, CAHPS employs a telephone follow-up through which it offers sample members the opportunity to complete the survey by phone. The mailing materials to all sample members also include a toll-free telephone number that allows recipients to call in to ask questions about the survey. CMS-approved survey vendors would be supplied with mail and telephone versions of the survey in electronic form, and text for beneficiary pre-notification and cover letters. Further, the CAHPS for MIPS survey can be administered in English, Spanish, Cantonese, Mandarin, Korean, Russian and/or Vietnamese. The survey vendors will be required to administer the survey according to established protocols to ensure valid and reliable results. Survey vendors will be required to use appropriate quality control, encryption, security and backup procedures to maintain survey response data. The data would then be securely sent back to CMS for scoring and/or validation in accordance with applicable law. To ensure that a survey vendor possesses the ability to transmit survey measures data for a particular performance period, we propose to require survey vendors to undergo this approval process for each year in which the survey vendor seeks to transmit survey measures data to us.

With regard to assessment of potential non-response bias, we will compare the characteristics of survey respondents with the characteristics of the sample frame using the standardized mean difference, which compares the mean of a beneficiary characteristic among respondents to the mean among the sample frame. A standardized mean difference of greater than 0.2 indicates that the respondents differ from the sample frame and there is potential for non-response bias. We will also fit a logistic regression model predicting beneficiary-level response using fixed effects for beneficiary characteristics. Any characteristics that are strongly associated with response are potential drivers of nonresponse bias. Differential non-response across practices is directly addressed using case-mix adjustment, ensuring that the comparison of practices is valid in the presence of nonresponse bias. The case-mix model includes various demographic and health questions that are used to adjust the SSM scores at the practice level. Using case-mix adjustments allows for fair comparisons across practices and across reporting years.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separate­ly or in combination with the main collection of information.

No tests of procedures or methods will be undertaken as part of this data collection. AHRQ released updated version of the core Clinician & Group CAHPS Survey (CG-CAHPS) from version 2.0, on which the CAHPS for MIPS survey is based) to version 3.0. Version 3.0 CG-CAHPS reflects wording improvements (e.g., use of “contact” instead of “phone” to reflect all the ways patients communicate with clinicians) and a shorter survey. CMS will review the results of a test of CG-CAHPS 3.0 language and other survey refinements (e.g., reduction in items, changes in skip instructions) conducted for the Shared Savings Program CAHPS for ACOs survey to determine if the CAHPS for MIPS survey should adopt 3.0 CG-CAHPS revisions or other tested refinements.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The CAHPS for MIPS survey consists of the core Clinician & Group CAHPS Survey (CG-CAHPS), version 2.0, which was developed by the Agency for Healthcare Quality Research (AHRQ) and additional supplemental items covering domains of patient experience specific to the information needs of CMS and MIPS.

For the transition year of MIPS, the CAHPS for MIPS survey will be the same as the current CAHPS for PQRS survey with additional questions that meet CMS program needs. The survey, sampling approach, and data collection procedures for the CAHPS for PQRS survey were designed by the RAND Corporation.

1 The 2014 PQRS data are from the most recent PQRS Experience Report, available at: https://www.cms.gov/medicare/quality-initiatives-patient-assessment-instruments/pqrs/analysisandpayment.html

2 Section 1848(q)(1)(C) of the Act defines a MIPS eligible clinician for payment years 1 and 2 as a physician, physician’s assistant, nurse practitioner, or clinical nurse anesthetist, or a group that includes such clinicians. Specialties not listed as eligible in the Act for payment period 1 or 2 include: Audiologists, Certified Nurse Midwives, Clinical Psychologists/Counselors, Clinical Social Workers, Physical/Occupational Therapists, and Registered Dieticians/Nutritionists.

3 The 668,090 clinicians permitted to voluntarily submit data includes 199,308 ineligible clinician types, 85,268 newly enrolled Medicare clinicians, and 383,514 low-volume clinicians. See Table 57 in the Regulatory Impact Analysis of the final rule with comment period for additional details on the estimated counts of clinicians excluded from or ineligible for MIPS.

4 http://www.pqrscahps.org/en/faqs/#Sampling

Page 3 of 3