CAHPS for MIPS Supporting Statement Part B clean 7.12.2017

Supporting Statement – Part B

Consumer Assessment of Healthcare Providers and Systems (CAHPS) Survey

for the Merit-Based Incentive Payment System (MIPS)

CMS-10450, OMB Control Number 0938-1222

Collections of Information Employing Statistical Methods

Introduction

The Centers for Medicare & Medicaid Services (CMS) requests a three-year clearance from the Office of Management and Budget (OMB) under the Paperwork Reduction Act (PRA) of 1995 to implement the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey for the Merit-based Incentive Payment System (MIPS). CMS is submitting the shortened CAHPS for MIPS survey (version 2.0) to OMB for approval under the PRA as a revision of the previously approved CAHPS for MIPS package (0938-1222). Specifically, CMS requests a revision to the previously approved CAHPS for MIPS survey (version 1.0) used in Quality Payment Program (QPP) transition year to collect data on fee-for-service Medicare beneficiaries’ experiences of care with eligible clinicians participating in MIPS. The survey information is used for quality reporting, the Physician Compare website, and annual statistical reports describing MIPS data for all MIPS eligible clinicians.

CMS will be requesting approval for two additional PRA packages associated with the CY 2018 Quality Payment Program proposed rule. The collection of information associated with the CY 2018 Quality Payment Program proposed rule (other than virtual group election and CAHPS-related data collection) will be submitted as a revision of the currently approved MIPS PRA package (0938-1314). CMS is proposing to allow the formation of virtual groups that can elect to submit via any quality submission mechanism available to groups, including CMS-approved survey vendors administering to CAHPS for MIPS survey. The collection of information associated with virtual group election will be submitted under a new OMB control number.

1. Describe (including a numerical estimate) the potential respondent universe and any sam­pling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Because historical participation rates for quality data submission under Physician Quality Reporting System (PQRS) have never reached 100 percent, we anticipate that MIPS will not achieve full participation.

Based on 2015 data from the PQRS, 2017 preliminary MIPS eligibility determination data, and 2017 preliminary Qualifying APM Participant (QP) determination data, we estimate that there will be 554,000 MIPS eligible clinicians in the 2018 MIPS performance period. Further, we estimate that at least 92 percent of MIPS eligible clinicians will submit quality performance category data, including those participating as individual clinicians, or as part of a MIPS APM entity, group, or virtual group. Groups and virtual groups can elect to contract with a CMS-approved survey vendor to collect and submit CAHPS for MIPS survey version 2.0 as one of their quality performance category measures.

Because data from the CAHPS for MIPS survey are not yet available, we use CAHPS for PQRS data to inform our estimate about the number of groups and virtual groups that will contract with a CMS-approved survey vendor to collect and report CAHPS for MIPS data. Because selecting the CAHPS for MIPS survey as a quality measure is optional, the use of a CMS-approved survey vendor is optional under MIPS. We anticipate that the number of groups and virtual groups electing to contract with a CMS-approved survey vendor may be somewhat lower than the number of groups contracting with a CMS-approved survey vendor under the PQRS. Under the PQRS, groups of 100 or more eligible professionals were required to contract with a CMS-approved survey vendor to collect the CAHPS for PQRS survey. Because participation in the CAHPS for MIPS survey is not required and MIPS eligible clinicians can voluntarily elect to participate as one quality measure, we assume that the number of groups and virtual groups selecting to use the CAHPS for MIPS survey version 2.0 will be equivalent to the second highest participation rate for the CAHPS for PQRS survey. This occurred in Reporting Year (RY) 2015 (regarding the 2014 PQRS performance year) when 461 groups used the survey. To date, the most popular year of the CAHPS for PQRS survey was RY 2016 (regarding the 2015 PQRS performance year), when 514 groups contracted with a CMS-approved survey vendor.¹

2. Describe the procedures for the collection of information including:

- Statistical methodology for stratification and sample selection,

- Estimation procedure,

- Degree of accuracy needed for the purpose described in the justification,

- Unusual problems requiring specialized sampling procedures, and

- Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Groups and virtual groups submitting quality measures data using a CMS-approved survey vendor to report the CAHPS for MIPS survey would need to meet the data submission requirements on the sample of the Medicare Part B fee-for-service (FFS) beneficiaries. Based on 2015 data in which 461 group practices administered the CAHPS for PQRS survey, we anticipate that 461 groups and virtual groups will contract with CMS-approved survey vendors to collect the CAHPS for MIPS survey version 2.0 data. Groups and virtual groups that elect to participate in the CAHPS for MIPS survey must submit using a CMS-approved survey vendor and must still also submit their other quality measures via another submission mechanism (electronic health record (EHR), qualified clinical data registry (QCDR), qualified registry, or CMS Web Interface) to ensure that the group meets the requirement for the minimum number of measures. Groups and virtual groups that do not elect to participate in the CAHPS for MIPS survey version 2.0 may just choose to submit the minimum number of required measures through one or more other submission mechanisms (EHR, QCDR, qualified registry, or CMS Web Interface).²

TABLE 1: Summary of Proposed Quality Data Submission Criteria for the CAHPS for MIPS Survey Year 2

Performance Period	Measure Type	Submission Criteria, including Sampling	Data Completeness
Jan 1 – Dec 31, 2018	Groups or Virtual Groups of 2 or more eligible clinicians	CMS-approved survey vendor would have to be paired with another data submission mechanism to ensure the minimum number of measures are reported. The CAHPS for MIPS survey would fulfill the requirement for a high priority measure (if no outcome measure is available) towards the MIPS quality performance category data submission criteria. The CAHPS for MIPS survey will only count for one quality measure.	Sampling requirements for their Medicare Part B beneficiaries

For the CAHPS for MIPS survey version 2.0, CMS will use the same sampling methodology as the CAHPS for MIPS survey 1.0. For groups and virtual groups that elect to contract with a CMS-approved survey vendor, CMS will identify beneficiaries eligible for the survey from the pool of beneficiaries assigned to the group or virtual group. CMS assigns original Medicare beneficiaries to a practice based on the plurality of the primary care claims during the first two quarters of the performance period. CMS will then randomly select samples from those assigned beneficiaries to create the sample for the CAHPS for MIPS survey. The sample will be limited to beneficiaries aged 18 or older and who are not known to be institutionalized or deceased. The sample is drawn at the level of the group, not at the individual provider level. To complete the survey, the survey form specifies the MIPS eligible clinician who delivered primary care to the beneficiary over multiple visits in the performance period, to help orient the beneficiary to the care he or she received.

CMS will oversample high utilizers of care, defined as beneficiaries who accounted for the highest 10 percent of total primary care charges within each practice. High utilizers represented 25 percent of the survey sample. For practices where the top 10 percent of users comprised fewer than 215 beneficiaries, we sampled all users in the top 10 percent. The reasons for oversampling are two-fold. First, oversampling increased the likelihood that the survey items that measure less-common experiences received adequate numbers of responses to enable analysis. Second, because one of the goals of the MIPS program is to incentivize high quality and efficient service delivery, it is particularly useful to capture the patient experiences of those with high levels of health care utilization.

The number of beneficiaries sampled may vary based on the size of the group or virtual group.

For large groups of 100 or more MIPS eligible clinicians :

• CMS will draw a sample of 860 beneficiaries

• If the group has fewer than 860 beneficiaries, but more than 415 beneficiaries, all eligible beneficiaries will be surveyed in Performance Year (PY) 2017

• If the group has fewer than 416 beneficiaries, the survey cannot be conducted

For groups with 25 to 99 MIPS eligible clinicians:

• CMS will draw a sample of 860 beneficiaries

• If the group has fewer than 860 beneficiaries, but more than 254 beneficiaries, all eligible beneficiaries will be surveyed in PY 2017

• If the group has fewer than 255 beneficiaries, the survey cannot be conducted

For groups with 2 to 24 MIPS eligible clinicians:

• CMS will draw from a sample of 860 beneficiaries

• If the group has fewer than 860 beneficiaries, but more than 124 beneficiaries, all eligible beneficiaries will be surveyed in PY 2017

• If the group has fewer than 125 beneficiaries, the survey cannot be conducted

The sample sizes recommended above are based on analysis of 2012 CAHPS for Accountable Care Organizations (ACOs) survey data. Specifically, we set a target number of completed questionnaires for each group practice to obtain a desired level of interunit reliability (IUR) for most survey measures. The IUR is defined as 1-V/(V+t2), where V is the variance of the estimate for a specific unit and t2 is the between-unit variance of population means. For CAHPS for ACOs and CAHPS for PQRS, IUR=0.75 is regarded as adequate reliability for public reporting; IUR between 0.60 and 0.75 and in the lowest 12% of reliability for ACOs or practices is considered low, while IURs below 0.60 are deemed very low. Measure scores with lower than adequate IURs can still provide practices with useful information about patient experience and potential areas for improvement. The target sample for group practices (regardless of their number of eligible clinicians) is 860, as a sample of this size is anticipated to produce measure scores meeting the adequate reliability threshold for most measures; a minimum sample size threshold is set for each practice size category to ensure that practices do not pursue the survey if they have so few beneficiaries that most measures would be expected to have very low reliability. These recommendations reflect a conservative approach that suggests sampling the same sample size for medium and small groups as is recommended for large groups when it is feasible, but lowers the minimum sample size threshold.

The historical response rate for beneficiaries invited to participate in the CAHPS for PQRS survey has ranged from 47 percent in RY 2013 to 37 percent in RY 2016. Factors that contribute to the lower response rate include a switch from survey administration by a single vendor to using multiple vendors, and time of year of survey administration.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, OMB guidance requires that a non-response bias assessment be conducted to determining if the results are generalized to the universe studied.

The CAHPS for MIPS survey version 2.0 will be collected via a mixed-mode data collection protocol that uses a pre-notification letter alerting sample members that a survey will be mailed to them shortly, a first mailing of the full questionnaire booklet, followed by a second mailing to those who do not respond to the earlier mailing of the questionnaire. For those who also do not respond to the second mailing of the questionnaire, CMS-approved survey vendors employ a telephone follow-up through which it offers sample members the opportunity to complete the survey by phone. The mailing materials to all sample members also include a toll-free telephone number that allows recipients to call in to ask questions about the survey. CMS-approved survey vendors are supplied with mail and telephone versions of the survey in electronic form, and text for beneficiary pre-notification and cover letters. Further, CAHPS for MIPS surveys can be administered in English, Spanish, Cantonese, Mandarin, Korean, Russian and/or Vietnamese. Across reporting years 2013-2016, CAHPS for PQRS has achieved a 42 percent response rate on average, slightly higher than some other CAHPS surveys of Medicare beneficiaries.

CMS-approved survey vendors will continue to be required to administer the survey according to established protocols to ensure valid and reliable results. Survey vendors will be required to use appropriate quality control, encryption, security and backup procedures to maintain survey response data. The data would then be securely sent back to CMS for scoring and validation in accordance with applicable law. To ensure that a survey vendor possesses the ability to transmit survey measures data for a particular performance period, we have proposed to require survey vendors to undergo this approval process for each year in which the survey vendor seeks to transmit survey measures data to us. The approval process includes submitting an application, meeting minimum business requirements, participation in training(s), passing post-training evaluation(s), submitting a Quality Assurance Plan, and following the schedule and procedures for survey administration. Additional details about the vendor approval process can be found at https://www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS/mips.html.

With regard to assessment of potential non-response bias, we will compare the characteristics of survey respondents with the characteristics of the sample frame using the standardized mean difference, which compares the mean of a beneficiary characteristic among respondents to the mean among the sample frame. A standardized mean difference of greater than 0.2 indicates that the respondents differ from the sample frame and there is potential for non-response bias. We will also fit a logistic regression model predicting beneficiary-level response using fixed effects for beneficiary characteristics. Any characteristics that are strongly associated with response are potential drivers of nonresponse bias. Differential non-response across practices is directly addressed using case-mix adjustment, recalculated every year ensuring that the comparison of practices is valid in the presence of potential nonresponse bias for that year. The case-mix model includes various demographic and health questions that are used to adjust the summary survey measure (SSM) scores at the practice level. Using case-mix adjustments allows for fair comparisons across practices and across reporting years.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separate­ly or in combination with the main collection of information.

The proposed revisions to the CAHPS for MIPS survey version 2.0 reflect two sets of tests. Several of the changes reflect testing done by AHRQ to the core Clinician & Group CAHPS Survey (CG-CAHPS) from version 2.0 to version 3.0. As a result of the testing, AHRQ eliminated 11 items and 2 SSMs from the survey, changed the wording of other items, and added an SSM while maintaining the reliability and validity of the measurement properties of the survey. Version 3.0 CG-CAHPS reflects wording improvements (e.g., use of “contact” instead of “phone” to reflect all the ways beneficiaries communicate with providers) and a shorter survey.

The second set of testing was a pilot test done under the auspices of the CAHPS for ACOs pilot survey, which was identical to the proposed CAHPS for MIPS survey version 2.0. The CAHPS for ACOs pilot field testing was conducted from November 2016 through February 2017 with a specific goal to determine whether a shorter survey affects SSM scores, response rates, and reliability. The pilot study participation included 18 ACOs served by seven vendors. The vendor and ACO participants were selected to represent ACOs with high and low CAHPS scores in 2015, ACOs with high and low response rates in 2015, and vendors with many and few ACO clients. Vendors followed standard CAHPS for ACOs data collection protocols and specifications to administer the ACO Pilot Survey.

Currently, the ACO pilot survey data are being analyzed with results expected by August 2017. This will include scoring of the revised SSMs, and comparisons with the RY 2016 CAHPS for ACOs (ACO-9 version) regarding response rates, scores, reliability, and other patterns. CMS also plans to determine if an adjustment is required to trend prior data to the new, shortened survey to inform calculation of benchmarks. CMS will consider the findings of the CAHPS for ACO survey pilot and the public comments we receive on the proposed CAHPS for MIPS survey version 2.0 and present findings in the final rule published in November 2017.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The CAHPS for MIPS survey version 2.0 consists of the core Clinician & Group CAHPS Survey (CG-CAHPS), version 3.0, which was developed by the Agency for Healthcare Quality Research (AHRQ) and additional supplemental items covering the information needs of CMS and MIPS.

The survey administration, sampling approach, and data collection procedures were designed by the RAND Corporation.

1 http://www.pqrscahps.org/

2 In the CY 2017 Quality Payment Program final rule (81 FR 77091), we finalized that MIPS eligible clinicians and groups submitting on behalf of MIPS eligible clinicians could submit information via one submission mechanism or, for groups that elect to include the CAHPS for MIPS survey as a quality measure, one submission mechanism and a CMS-approved survey vendor. However, in the CY 2018 Quality Payment Program proposed rule, we propose to allow MIPS eligible clinicians, groups, and virtual groups submitting on behalf of MIPS eligible clinicians to submit quality data via one or more submission mechanisms (other than a CMS-approved survey vendor).

Page 3 of 3