CMS-10450 Supporting Statement B (12-22-20)

Supporting Statement – Part B

Consumer Assessment of Healthcare Providers and Systems (CAHPS) Survey

for the Merit-Based Incentive Payment System (MIPS)

CMS-10450, OMB Control Number 0938-1222

Collections of Information Employing Statistical Methods

Introduction

CMS is submitting updates to one ICR associated with the CAHPS for MIPS survey to OMB for approval under the PRA as a revision of the previously approved CAHPS for MIPS package (0938-1222; CMS-10450). The CAHPS for MIPS survey is used in the Quality Payment Program (QPP) to collect data on fee-for-service Medicare beneficiaries’ experiences of care with eligible clinicians participating in MIPS and is designed to gather only the necessary data that CMS needs for assessing physician quality performance, and related public reporting on physician performance, and should complement other data collection efforts. The CAHPS for MIPS survey consists of the core Agency for Healthcare Research and Quality (AHRQ) CAHPS Clinician & Group Survey, version 3.0, plus additional survey questions to meet CMS’s information and program needs. The survey information is used for quality reporting, the Physician Compare website, and annual statistical experience reports describing MIPS data for all MIPS eligible clinicians.

1 Describe (including a numerical estimate) the potential respondent universe and any sam­pling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Because historical participation rates for quality data submission under Physician Quality Reporting System (PQRS) have never reached 100 percent, we anticipate that MIPS will not achieve full participation.

Based on participation data from the 2020 MIPS performance period eligibility data, we assume that 890,742 clinicians who participated in MIPS and who are not QPs in Advanced APMs in the 2020 MIPS performance period will continue to submit quality data in the 2021 MIPS performance period. We assume that 100 percent of APM Entities in MIPS APMs will submit quality data to CMS as required under their models. Groups and virtual groups can elect to contract with a CMS-approved survey vendor to collect and submit the CAHPS for MIPS survey as one of their quality performance category measures and also as a high-weighted activity under the improvement activities performance category. Virtual groups are subject to the same requirements as groups, therefore we will only refer to groups as an inclusive term for both unless otherwise noted.

For the 2021 MIPS performance period, we assume that 282 groups will register to conduct the CAHPS for MIPS survey based on the number of groups which elected to register during the CY 2020 registration period; this is the same number of groups currently approved by OMB under the aforementioned control number (82 FR 53917). With regard to beneficiary respondents, we assume that 117 groups will elect to report on the CAHPS for MIPS survey, which is equal to the number of groups that have registered and have a sufficient beneficiary sample size to conduct the CAHPS for MIPS survey in the 2020 MIPS performance period; a decrease of 26 from the 143 groups currently approved by OMB. Based on the number of complete and partially complete surveys for groups participating in CAHPS for MIPS survey administration for the 2019 MIPS performance period, we assume that an average of 256 beneficiaries will respond per group for the 2021 MIPS performance period. Therefore, the CAHPS for MIPS survey will be administered to approximately 29,952 beneficiaries per year (117 groups x an average of 256 beneficiaries per group responding). This is a decrease of 9,087 from our currently approved 39,039 beneficiary estimate.

2 Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Groups submitting quality measures data using a CMS-approved survey vendor to report the CAHPS for MIPS survey would need to meet the data submission requirements on the sample of the Medicare Part B fee-for-service (FFS) beneficiaries. Based on the number of groups electing to register during the 2020 registration period that were found to have a sufficient beneficiary sample size, we anticipate that 117 groups will contract with CMS-approved survey vendors to collect CAHPS for MIPS survey data. Groups that elect to participate in the CAHPS for MIPS survey must submit using a CMS-approved survey vendor and must still also submit their other quality measures to ensure that the group meets the requirement for the minimum number of measures. Groups that do not elect to participate in the CAHPS for MIPS survey may just choose to submit the minimum number of required measures through one or more other submission types.¹

TABLE 1: Summary of Quality Data Submission Criteria Finalized for the CAHPS for MIPS Survey Year 5

Performance Period	Measure Type	Submission Criteria, including Sampling	Data Completeness
Jan 1 – Dec 31, 2021	Groups	CMS-approved survey vendor would have to be paired with another collection type to ensure the minimum number of measures are reported. The CAHPS for MIPS survey would fulfill the requirement for a high priority measure (if no outcome measure is available) towards the MIPS quality performance category data submission criteria. The CAHPS for MIPS survey will only count for one quality measure. The CAHPS for MIPS survey will count as a high-weighted activity for the improvement activities performance category.	Sampling requirements for the group’s Medicare Part B patients

For groups that elect to contract with a CMS-approved survey vendor, CMS will identify beneficiaries eligible for the survey from the pool of Medicare fee-for-service (FFS) beneficiaries assigned to the group.² CMS uses retrospective beneficiary assignment determined at the end of the registration period to identify beneficiaries eligible to receive the CAHPS for MIPS survey. If a beneficiary receives at least one primary care service by a primary care clinician who is part of the group or virtual group, the beneficiary is eligible to be assigned to the group or virtual group based on a two-step process³:

• The first step assigns a beneficiary to the group or virtual group if the beneficiary receives the plurality of his or her primary care services from primary care clinicians who are part of the group or the virtual group. Primary care clinicians are defined as those with one of seven specialty designations: internal medicine, general practice, family practice, geriatric medicine, nurse practitioner, clinical nurse specialist, and physician assistant.

• The second step only considers beneficiaries who have not had any primary care service furnished by a primary care clinician, including primary care clinicians external to the group or the virtual group. Under this second step, we assign a beneficiary to the group or the virtual group if the beneficiary receives the plurality of his or her primary care services from clinicians who are not primary care clinicians within the group or virtual group.

A plurality means a greater proportion of primary care services was provided from clinicians who are part of the group or the virtual group than any other entity, measured in terms of allowed charges. A plurality may be less than the majority of services.

CMS assigns Medicare FFS beneficiaries to a group and then randomly samples from those assigned beneficiaries to create the sample for the CAHPS for MIPS survey. The sample will be limited to beneficiaries age 18 or older, who are not known to be institutionalized or deceased, and who had at least two visits for care to the group. The sample is drawn at the group level, not at the individual clinician level. The survey names a specific clinician, who delivered primary care to the beneficiary over multiple visits in the performance year to help orient the beneficiary to the care he or she received. The named provider can be a physician, specialist, nurse practitioner, physician assistant, or clinical nurse specialist.

CMS will oversample high utilizers of care, defined as beneficiaries who accounted for the highest 10 percent of total primary care charges within each practice. High utilizers represent 25 percent of the survey sample. For practices where the top 10 percent of users comprise fewer than 215 beneficiaries, we will sample all users in the top 10 percent. The reasons for oversampling are two-fold. First, oversampling increases the likelihood that the survey items that measure less-common experiences receive adequate numbers of responses to enable analysis. Second, because one of the goals of the MIPS program is to incentivize high quality and efficient service delivery, it is particularly useful to capture the patient experiences of those with high levels of health care utilization.

The number of beneficiaries sampled may vary based on the size of the group.

For large groups of 100 or more MIPS eligible clinicians:

• CMS will draw a sample of 860 beneficiaries

• If the group has fewer than 860 beneficiaries, but more than 415 beneficiaries, all eligible beneficiaries will be surveyed in Performance Year (PY) 2021

• If the group has fewer than 416 beneficiaries, the survey cannot be conducted

For groups with 25 to 99 MIPS eligible clinicians:

• CMS will draw a sample of 860 beneficiaries

• If the group has fewer than 860 beneficiaries, but more than 254 beneficiaries, all eligible beneficiaries will be surveyed in PY 2021

• If the group has fewer than 255 beneficiaries, the survey cannot be conducted

For groups with 2 to 24 MIPS eligible clinicians:

• CMS will draw from a sample of 860 beneficiaries

• If the group has fewer than 860 beneficiaries, but more than 124 beneficiaries, all eligible beneficiaries will be surveyed in PY 2021

• If the group has fewer than 125 beneficiaries, the survey cannot be conducted

The sample sizes recommended above are based on analysis of 2012 CAHPS for Accountable Care Organizations (ACOs) survey data. Specifically, we set a target number of completed questionnaires for each group practice to obtain a desired level of interunit reliability (IUR) for most survey measures. The IUR is defined as 1-V/(V+t2), where V is the variance of the estimate for a specific unit and t2 is the between-unit variance of population means. For CAHPS for ACOs and CAHPS for PQRS, IUR=0.75 is regarded as adequate reliability for public reporting; IUR between 0.60 and 0.75 and in the lowest 12% of reliability for ACOs or practices is considered low, while IURs below 0.60 are deemed very low. Measure scores with lower than adequate IURs can still provide practices with useful information about patient experience and potential areas for improvement. The target sample for group practices (regardless of their number of eligible clinicians) is 860, as a sample of this size is anticipated to produce measure scores meeting the adequate reliability threshold for most measures; a minimum sample size threshold is set for each practice size category to ensure that practices do not pursue the survey if they have so few beneficiaries that most measures would be expected to have very low reliability. These recommendations reflect a conservative approach that suggests sampling the same sample size for medium and small groups as is recommended for large groups when it is feasible, but lowers the minimum sample size threshold.

The historical response rate for beneficiaries invited to participate in the CAHPS for MIPS survey has ranged from 34.7 percent in RY 2017 to 32.7 percent in RY 2019. Factors that contribute to the lower response rate over time include use of multiple vendors, and time of year of survey administration.

3 Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, OMB guidance requires that a non-response bias assessment be conducted to determining if the results are generalized to the universe studied.

The CAHPS for MIPS survey will be collected via a mixed-mode data collection protocol that uses a pre-notification letter alerting sample members that a survey will be mailed to them shortly, a first mailing of the full questionnaire booklet, followed by a second mailing to those who do not respond to the earlier mailing of the questionnaire. For those who also do not respond to the second mailing of the questionnaire, CMS-approved survey vendors employ a telephone follow-up through which it offers sample members the opportunity to complete the survey by phone. The mailing materials to all sample members also include a toll-free telephone number that allows recipients to call in to ask questions about the survey. CMS-approved survey vendors are supplied with mail and telephone versions of the survey in electronic form, and text for beneficiary pre-notification and cover letters. Further, CAHPS for MIPS surveys can be administered in English, Spanish, Cantonese, Mandarin, Korean, Russian and/or Vietnamese. Across reporting years 2017-2019, CAHPS for MIPS has achieved a 34.2 percent response rate on average, similar to other CAHPS surveys of Medicare beneficiaries.

CMS-approved survey vendors will continue to be required to administer the survey according to established protocols to ensure valid and reliable results. Survey vendors will be required to use appropriate quality control, encryption, security and backup procedures to maintain survey response data. The data would then be securely sent back to CMS for scoring and validation in accordance with applicable law. To ensure that a survey vendor possesses the ability to transmit survey measures data for a particular performance period, we have proposed to require survey vendors to undergo this approval process for each year in which the survey vendor seeks to transmit survey measures data to us. The approval process includes submitting an application, meeting minimum business requirements, participation in training(s), passing post-training evaluation(s), submitting a Quality Assurance Plan, and following the schedule and procedures for survey administration. Additional details about the vendor approval process can be found at https://www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS/mips.html.

With regard to assessment of potential non-response bias, we will compare the characteristics of survey respondents with the characteristics of the sample frame using the standardized mean difference, which compares the mean of a beneficiary characteristic among respondents to the mean among the sample frame. A standardized mean difference of greater than 0.2 indicates that the respondents differ from the sample frame and there is potential for non-response bias. We will also fit a logistic regression model predicting beneficiary-level response using fixed effects for beneficiary characteristics. Any characteristics that are strongly associated with response are potential drivers of nonresponse bias. Differential non-response across practices is directly addressed using case-mix adjustment, recalculated every year ensuring that the comparison of practices is valid in the presence of potential nonresponse bias for that year. The case-mix model includes various demographic and health questions that are used to adjust the summary survey measure (SSM) scores at the practice level. Using case-mix adjustments allows for fair comparisons across practices and across reporting years.

4 Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separate­ly or in combination with the main collection of information.

We do not anticipate any testing to occur under this PRA. We are proposing one additional survey item to the survey instrument, which has been cognitively tested.

5 Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The CAHPS for MIPS survey consists of the core Clinician & Group CAHPS Survey (CG-CAHPS), version 3.0, which was developed by the Agency for Healthcare Quality Research (AHRQ) and additional supplemental items covering the information needs of CMS and MIPS.

The survey administration, sampling approach, and data collection procedures were designed by the RAND Corporation.

1 In the CY 2017 Quality Payment Program final rule (81 FR 77091), we finalized that MIPS eligible clinicians and groups submitting on behalf of MIPS eligible clinicians could submit information via one submission mechanism or, for groups that elect to include the CAHPS for MIPS survey as a quality measure, one submission mechanism and a CMS-approved survey vendor. However, in the CY 2018 Quality Payment Program final rule, we establish a policy that allows MIPS eligible clinicians and groups submitting on behalf of MIPS eligible clinicians to submit quality data via one or more submission mechanisms (other than a CMS-approved survey vendor) beginning in the 2019 MIPS performance period.

2 https://qpp-cm-prod-content.s3.amazonaws.com/uploads/925/2020%20CAHPS%20for%20MIPS%20Overview%20Fact%20Sheet.pdf

3 https://qpp-cm-prod-content.s3.amazonaws.com/uploads/1234/2020%20MIPS%20Assignment%20for%20CMS%20Web%20Interface%20and%20CAHPS%20for%20MIPS%20Survey.pdf

Page 3 of 3