CMS-10488-Supporting Statement QHP Survey Final x

The Affordable Care Act (ACA) authorized the creation of Health Insurance Exchanges (Exchanges) (also known as Marketplaces) to help individuals and small employers shop for, select, and enroll in high quality, affordable private health plans beginning October 2013.¹ Section 1311(c)(4) of the ACA requires the Department of Health & Human Services (HHS) to develop an enrollee satisfaction survey system that assesses consumer experience with qualified health plans (QHPs) offered through an Exchange. It also requires public display of enrollee satisfaction information by the Exchange to allow individuals to easily compare enrollee satisfaction levels between comparable plans. In 2014, HHS established the Qualified Health Plan Enrollee Experience Survey (QHP Enrollee Survey). The main purpose of the QHP Enrollee Survey is to assess enrollee experience with their QHP around areas such as access to care, access to information, care coordination, cultural competence, doctor communication and plan administration. Under OMB Control Number 0938-1221, a psychometric test and beta test were performed in 2014 and 2015, respectively. The QHP Enrollee Survey was conducted nationwide in 2016 and 2017. This request is to continue nationwide collection and administration of the survey in 2018 – 2020.

The Centers for Medicare & Medicaid Services (CMS) initially intended to display the results of the 2016 data collection as part of the Quality Rating System (QRS) star ratings for the 2017 Open Enrollment Period. However, CMS decided to conduct an additional year of focused consumer testing of the display of QRS star ratings to maximize the clarity and consistency of the information provided to the public and to assess how the QHP quality rating information is displayed on HealthCare.gov.

In April 2016, CMS announced this change in the public reporting timeline of quality ratings information by the Federally-facilitated Exchange (FFE), including FFEs where the state performs plan management functions and State-based Exchanges on the Federal Platform (SBE-FPs), and announced that a limited pilot would be conducted during the 2017 open enrollment period. During the pilot, CMS displayed the QRS star ratings in select states with consumers using HealthCare.gov during the 2017 open enrollment period. In September 2016, CMS finalized selection of the following states to display star ratings on healthcare.gov: Virginia and Wisconsin. CMS selected these states because they have ample participation of QHP issuers on their respective Exchanges and relative variation in QRS star ratings based on 2016 QRS results. FFEs not in the pilot did not display star ratings during the 2017 open enrollment period. CMS anticipates displaying the 2017 QRS global rating and three summary indicator ratings on the HealthCare.gov website for eligible QHPs operating in the pilot States. State-based Exchanges (SBEs) whose consumers do not use HealthCare.gov could choose to display QHP quality information during the open enrollment period for the 2017 plan year or follow the revised timeframe.

CMS goals for the QRS pilot include: obtaining further details about consumer access and the use of QHP quality rating information during an actual open enrollment period, to inform the display of QRS star ratings; and informing the development of comprehensive technical assistance and education related to the QRS for assisters, navigators, agents, brokers and consumer groups prior to QRS public reporting. Information collection associated with QRS display consumer testing was approved under OMB Control Number 0938-1247. CMS is assessing information obtained regarding the display of QRS ratings during the pilot and will consider the timeframes for public reporting of quality rating information.

CMS renewed OMB approval for the information collection related to the QHP Enrollee Experience Survey in 2018-2020. These activities are necessary to ensure that CMS fulfills legislative mandates established by section 1311(c)(4) of the Affordable Care Act to develop an “enrollee satisfaction survey system” and provide such information on Exchange websites.

CMS is requesting approval for a non-substantive change request to update the information collection burden associated with the 2020 QHP Enrollee Survey. The non-substantive change does not introduce new policy or any fundamental program changes. This non-substantive change reduces burden for this year (2020) and is being submitted for approval by OMB as a result of the recent CMS decision to suspend data collection for the 2020 QHP Enrollee Survey for Plan Year 2021².

QHP Enrollee Survey

The goals of the QHP Enrollee Survey are to:

Provide comparable and useful information to consumers about the quality of health care services and enrollee experience with QHPs offered through the Exchanges,
Facilitate oversight of QHP issuer compliance with quality reporting standards set forth in the Patient Protection and Affordable Care Act and implementing regulations, and
Provide actionable information that QHP issuers can use to improve quality and performance.

Based on the requirements for the QHP Enrollee Survey, CMS developed a survey to capture information about enrollees’ experience with health plans offered via the Exchange. Since the Marketplace population was new, and continuing to evolve, there was no existing survey that could adequately measure the experiences of this population. In depth, formative research was conducted, including a comprehensive review of the literature and related surveys; consumer focus groups; stakeholder discussions; and input from a technical expert panel (TEP). Research indicated use of the CAHPS^® Health Plan 5.0 Adult Medicaid Survey as the QHP Enrollee Survey core, with supplemental items drawn from the CAHPS® Health Plan 5.0 Adult Supplemental Item Set, the CAHPS Health Plan 4.0 Supplemental Item Set, and the CAHPS® 5.0 HEDIS Survey to provide the full set of information needed to evaluate QHP performance.

Due to the nature and needs of this new population, additional items were developed specifically for the QHP Enrollee Survey to fill in the topics not covered by existing CAHPS® items. Inclusion of these additional questions was a result of a psychometric test in 2014, a beta test in 2015, and additional testing during the 2016 administration. Thus, nine questions that were removed for the beta Test were returned to the question set when results of psychometric testing determined they were vital for understanding enrollee experiences with their QHP. To comply with the requirements of Section 4302 Data Collection Standards of the ACA, CMS will be including six disability status items. The disability status items were tested in the 2014 psychometric test and have been extensively tested by the US Census Bureau for administration in the American Community Survey.

The unique nature of the QHP enrollee population and its application to the QRS required a customized survey instrument. While the purpose of the QHP Enrollee Survey is not to make comparisons between Exchange plans and commercial plans, the detailed question set allows insurers to make item-level comparisons across product lines. Changes to the QHP Enrollee Survey must be weighed against any loss in utility to look at trends in the insurance Marketplace. CMS will continue to refine the QHP Enrollee Survey, including potential reduction in the number of questions, based upon TEP feedback, other stakeholder feedback (i.e., through the Call Letter Process), and continued analysis (e.g., psychometric testing of measures, response rates, etc.)

The questionnaire submitted for clearance is available in English, Spanish, and Traditional Chinese for use in a mixed-mode methodology that includes mail, telephone, and Internet survey modes.

The QHP Enrollee Survey will be conducted by HHS-approved survey vendors who meet minimum business requirements. A similar system is currently used for other CMS surveys, including Medicare CAHPS, Hospital CAHPS (HCAHPS), Home Health CAHPS (HHCAHPS), the CAHPS Survey for Accountable Care Organizations, and the Health Outcomes Survey (HOS). Under this model, all QHPs that are required to conduct the QHP Enrollee Survey must contract with an HHS-approved survey vendor to collect the data and submit it to CMS on the issuer’s behalf (45 CFR § 156.1125(a)). CMS is responsible for approving and training vendors, providing technical assistance to vendors, overseeing vendors to ensure that they are following the data collection protocols, collecting and analyzing the data from vendors, and producing reports that QHP issuers can use for quality improvement. The survey vendor program was tested in the 2015 beta test and refined for the 2016 survey administration.

Reducing Respondent Burden

CMS is proposing to remove eight questions from the QHP Enrollee Survey. Based on proposed question reduction, the 2018 QHP Enrollee Survey includes 82 questions, this is inclusive of the 29 questions required for the QRS. The proposed question set includes seven demographic questions, six required disability status questions, and skip-logic that could potentially reduce fifteen or more questions based upon individual respondent characteristics. Question sets on other CAHPS surveys often examine a narrower topic, the Hospital CAHPS or Nursing Home CAHPS for example. As such, they have a smaller set of questions. CMS will continue to review the utility of the QHP Enrollee Survey, looking for opportunities to reduce respondent burden while maintaining a robust set of measures.

The rationale for changes to the 2018 QHP Enrollee Survey is discussed below.

Removing Access to After Hours Care Questions

7. In the last 6 months, did you need to visit a doctor’s office or clinic after regular office hours?

8. In the last 6 months, how often were you able to get care you needed from a doctor’s office or clinic after regular office hours?

Due to low screen-in rates among respondents (less than nine percent of respondents) and poor inter-unit reliability, CMS decided to remove the item about after-hours care from the QRS Access to Care measure beginning in 2018. Before finalizing this proposal, CMS sought public comment. The comments that CMS received agreed with this proposed change.

Given that this question (#7 above) will no longer be utilized for the QRS and both (Questions #7 and #8 above) have a low screen-in rate of less than nine percent, CMS proposes removing these survey questions from the 2018 QHP Enrollee Survey.

Removing Question about Recommending Health Plan to Friends and Family

53. Using any number from 0 to 10, where 0 is not at all likely and 10 is extremely likely, how likely is it that you would recommend this health plan to a friend or family member?

Based on stakeholder feedback, CMS added a question about the respondents’ likelihood of recommending their current health plan to friends or family (i.e., “recommend question”) to the 2016 QHP Enrollee Survey. This question was included because of public comment, however analysis of the 2016 QHP Enrollee Survey demonstrated that this item was very highly correlated (0.96) with the enrollee rating of their health plan item. In addition, this question is not utilized by the QRS. Given this correlation, CMS has removed this question (#53 above) to reduce respondent burden.

Removing Survey Questions for Aspirin Use and Discussion Measure

Do you take aspirin daily or every other day?
Do you have a health problem or take medication that makes taking aspirin unsafe for you?
Has a doctor or health provider ever discussed with you the risks and benefits of aspirin to prevent heart attack or stroke?
Are you aware that you have any of the following conditions? Mark one or more.
1. High cholesterol
2. High blood pressure
3. Parent or sibling with heart attack before the age of 60
Has a doctor ever told you that you have any of the following conditions? Mark one or more.
1. A heart attack
2. Angina or coronary heart disease
3. A stroke
4. Any kind of diabetes or high blood sugar

Due to revised recommendations by the U.S. Preventive Services Task Force (USPSTF), CMS retired the Aspirin Use and Discussion measure for the QRS measure set beginning in 2017. Accordingly, CMS is proposing to remove the survey questions associated with this measure (#65 through #69 above) in the 2018 QHP Enrollee Survey.

B. Justification

1. Need and Legal Basis

Section 1311(c)(3) of the Patient Protection and Affordable Care Act³ directs the Secretary of HHS to develop a quality rating for each QHP offered through an Exchange, based on quality and price. Section 1311(c)(4) of the Patient Protection and Affordable Care Act directs the Secretary to establish an enrollee satisfaction survey that will assess enrollee satisfaction with each QHP offered through the Exchanges with more than 500 enrollees in the prior year.

Based on this authority, CMS issued a regulation in May 2014 to establish standards and requirements related to QHP issuer data collection and public reporting of quality rating information in every Exchange.⁴ As a condition of certification and participation in the Exchanges, CMS requires that QHP issuers submit QRS clinical measure data and QHP Enrollee Survey response data for their respective QHPs offered through an Exchange in accordance with CMS guidelines.⁵ Exchanges are also required to display QHP quality rating information on their respective websites.⁶

2. Information Users

Beginning with the 2017 national implementation of the QHP Enrollee Survey, the survey results were publicly reported (as part of the Quality Rating System) on select Exchange websites to aid consumers in choosing a QHP as part of a limited display consumer pilot test of the QRS. Beginning with the 2018 individual market open enrollment period and similar to last year, CMS anticipates displaying the 2017 QRS global rating and three summary indicator ratings on the HealthCare.gov website for eligible QHPs operating in the pilot States of Virginia and Wisconsin. The QHP Enrollee Survey data are provided to QHP issuers to improve their performance and better tailor their efforts to improve the experiences of the QHP enrollee population. Additionally, the data is used by HHS, State-based Exchanges, and state insurance commissioners to aid in effective regulatory and oversight efforts.

3. Use of Information Technology

The current data collection protocol for the QHP Enrollee Survey includes the use of an online survey as well as the use of Computer Assisted Telephone Interviewing (CATI). Beginning with the 2016 QHP Enrollee Survey, the survey vendors have the option of offering the web survey in Spanish (it is required in English). CMS will continue to evaluate methods to increase the use of online surveys.

Survey vendors are required to submit the final data files to CMS for analysis and scoring through a secure portal on the QHP Enrollee Survey website. This process ensures that the data files meet established specifications. Additionally, after analysis the survey data is submitted into the Exchange Quality Module (MQM) within CMS’ Health Insurance Oversight System (HIOS) for calculation of star ratings produced by the QRS for public reporting.

4. Duplication of Efforts

There is no duplication of efforts. The QHP Enrollee Survey is the only survey being conducted by HHS to measure patient experiences with QHPs offered through the Exchanges.

5. Small Businesses

The survey population for the QHP Enrollee Survey includes individuals who enrolled in QHPs through an individual Exchange, a Small Business Health Options Program (SHOP) Exchange, or directly with the issuer for plans offered through the Exchange. The sample frame is developed by issuers, few if any of whom are small businesses. CMS expects that this will not have an impact on small businesses. Some survey vendors who will apply to field the QHP Enrollee Survey will be small businesses, but conducting CAHPS® surveys is their business and the decision to apply for approval as a vendor for the QHP Enrollee Survey is voluntary. Furthermore, the survey vendor application process imposes a minimal burden on any applicant, including small businesses. Thus, there is no reason to expect that the survey will burden small businesses; it offers them a business opportunity if they choose to apply for participation.

6. Less Frequent Collection

Annual data collection of the QHP Enrollee Survey is needed to meet the objectives of providing feedback to Exchanges, issuers, and regulators for quality improvement; providing information for consumers’ choice; and to track performance.

7. Special Circumstances

There are no special circumstances associated with this data collection.

8a. Federal Register

This is a renewal of the information collection approved under OMB Control Number 0938-1221. As required by 5 CFR 1320.8(d), CMS has solicited comments on these revisions to the QHP Enrollee Survey, through a Federal Register Notice. There were 24 comments submitted; however, the comments were not applicable since they did not relate to the QHP Enrollee Survey and its associated information collection.

On July 20, 2017, a 30-day comment period was announced in the Federal Register (81 FR 45167). One comment was received on August, 24, 2017 from America’s Health Insurance Plans (AHIP). AHIP believes the QHP Enrollee Survey prevents health plans from making direct comparisons between Exchange and other commercial lines of business and that utility would be improved by eliminating all non-CAHPS questions and replacing the current survey with the CAHPS Health Plan survey.

In response, CMS is continually looking for ways to improve the QHP Enrollee Survey and reduce burden by removing questions, when appropriate. CMS has addressed AHIPs concern regarding plan comparison in the Background section above. Additionally, CMS has clarified the new and unique nature of the QHP Enrollee population in the Background section above, how changes to the survey could make it difficult to perform trend analyses across the Marketplace and measure quality improvements of reporting units over time. CMS has elected to remove eight questions from the 2018 QHP Enrollee Survey, as described above, and will continue to engage stakeholders and perform analysis to refine the survey instrument for future fielding. At this time, CMS does not recommend making further reductions to the 2018 QHP Enrollee Survey.

8b. Outside Consultation

CMS is working with a variety of outside organizations and individuals to aid in the development and implementation of the QHP Enrollee Survey. Chief among these organizations is the American Institutes for Research (AIR), the National Committee for Quality Assurance (NCQA), and Booz Allen Hamilton. In addition, a Technical Expert Panel composed of consumer advocates, health plan representatives, Exchange administrators, survey design experts, state regulators, and providers provides ongoing feedback on technical issues. The panel meets approximately three times a year to provide guidance.

9. Payments/Gifts to Respondents

No payments or gifts will be made to any respondents.

10. Confidentiality

Individual survey respondents will be told the purposes for which the information is collected and that, in accordance with section 934(c) of the Public Health Service Act, 42 USC 299c-3(c), any identifiable information about them will not be used or disclosed for any purpose beyond conducting the survey. The confidentiality of individual’s replies is further assured under 5 U.S.C.552 (Freedom of Information Act), 5 U.S.C.552a (Privacy Act of 1974), and OMB Circular No. A-130. SORN: Health Insurance Exchange Program - 78 FR 8538 Publication Date: 02/06/2013.

11. Sensitive Questions

There are no sensitive questions associated with this information collection.

12. Burden Estimates (Hours & Wages)

Estimated burden hours for the QHP Enrollee Survey in 2018 are presented in Exhibit A1 and are based on the following assumptions and definitions.

Units. The reporting unit has been defined at the level of product type (i.e., Exclusive Provider Organization [EPO], Health Maintenance Organization [HMO], Preferred Provider Organization [PPO], Point of Service [POS]) offered by a QHP issuer through the Exchange in a particular state. For example, XYZ issuer’s HMOs offered through the Exchange in Florida would be considered a single reporting unit. Depending on the way a QHP issuer packages its plan offerings, the reporting unit might include a single QHP or many QHPs spanning all categories of coverage (i.e., bronze, silver, gold, platinum, catastrophic). QHP issuers will create a sample frame for each product type they offer through the Exchange within a particular state, or reporting unit.

For the 2018 survey, CMS estimates that no more than 300 reporting units will be required to administer the QHP Enrollee Survey. This is lower than previous estimates based on two factors: (1) the actual numbers of reporting units that were required to administer the 2017 survey was 262 and (2) several QHP issuers have left the Health Insurance Exchange.

Respondents per unit. Based on the results of the 2014 Psychometric Test, 2015 Beta Test, and 2016 implementation, CMS plans to collect 300 responses per reporting unit. As this survey program continues, CMS will explore whether the number of responses can be reduced.

Total respondents. The total number of respondents equals the product of the completed surveys per reporting unit and the current estimate of the number of QHP reporting units.

Hours per response. Based on testing of the QHP Enrollee Survey, and after implementing corrections to the 2019 questionnaire, it takes 12.5 minutes to complete.

Survey vendors. Survey vendors who want to participate in collecting QHP Enrollee Survey data must complete a Survey Vendor Participation Form. CMS anticipates that approximately 15 survey vendors will apply to field the QHP Enrollee Survey annually. The Survey Vendor Participation Form is designed to be completed in 90 minutes.

Exhibit A1. Estimated Burden Hours for 2018-2020 Implementation of QHP Enrollee Survey

Source	Num. of Reporting Units	Completes per Reporting Unit	Total Sample¹	Burden Hours	Total burden hours
2020 Survey Respondents	300	255	76,500	0.208	15,912
2020 Survey Vendors	15	1	15	1.5	22.5
2018 TOTAL	300		90,015		22,523
2019 TOTAL	300		90,015	0.208	18,772.5
2020 TOTAL	300		76,515		15,934.5
3-year TOTAL	900				57,230

¹ Total Sample = Num. of Reporting Units x Completes per Reporting Unit

In 2018, the total annual burden hours for the 2018 QHP Enrollee Survey are estimated to be 22,523 hours. Because only minimal adjustments to the questionnaire are expected for 2019 we estimate a burden of 18,772.5 hours for 2019. Based on the April announcement of the suspension of 2020 data collection for the survey that would normally have been submitted in May 2020 for PY 2021, CMS estimates an annual burden hour reduction from an estimated 18,772.5 hours to 15,934.5 hours for the 2020 QHP Enrollee Survey which accounts for 85% of the original estimated 90,000 responses since the telephone phase of survey administration was not completed.⁷ We estimate a total burden of 57,230 hours over three years.

The Bureau of Labor Statistics reported the average hourly wage for civilian workers in the United States was $26.00 as of January 2017. To estimate the burden costs for survey vendors, CMS used the average hourly wage for employees in the business and professional services sector which was $31.24 as of January 2017. See exhibit A2 for estimated burden costs which reflects the reductions based on the April 2020 announcement of the discontinuation of survey data collection for this year.

Exhibit A2. QHP Enrollee Survey Estimated Burden Costs

Source	Number of Respondents	Total Burden Hours	Average Hourly Wage Rate	Total Cost Burden
2020 Survey Respondents	76,500	15,912	$26.00	$413,712.00 .
2020 Survey Vendors	15	22.5	$31.24	$702.90
2018 TOTAL	90,015	22,523		$585,702.90
2019 TOTAL	90,015	18,772.5		$488,202.90
2020 TOTAL	76,515	15,934.5		$414,414.90
3-Year TOTAL	256,530	57,230		$1,488,320.7

13. Capital Costs

There are no direct capital costs to respondents other than their time to participate in the survey.

14. Cost to Federal Government

The only cost to the Government of these data collections that would not otherwise have been incurred is the cost of the Booz Allen Hamilton contract. The portion of this contract attributable to the QHP Enrollee Survey is approximately $1.7 million for the 2018 national implementation. We expect the 2018 and 2019 implementation to also be approximately $1.7 million as well for a 3-year total of $5.1 million. This cost includes soliciting and approving survey vendors, developing quality assurance guidelines and technical specifications for survey vendors, providing technical assistance and training to survey vendors, conducting oversight of approved survey vendors, providing technical assistance to QHP issuers, scoring and analyzing the survey data, and development of final reports for QHP issuers.

15. Changes to Burden

Based on the April announcement of the suspension of 2020 data collection for the survey that would normally have been submitted in May 2020 for PY 2021, CMS estimates an annual burden hour reduction from an estimated 18,772.5 hours to 15,934.5 hours (a total decrease of 2,838). This decrease applies to the 2020 QHP Enrollee Survey which accounts for 85% of the original estimated 90,000 responses since the telephone phase of survey administration was not completed.

In addition, due to an internal oversight, we erroneously calculated the currently approved burden hours estimate. Therefore, CMS is adjusting the existing burden hour estimate from 22,523 to 60,068 hours. However, due to the 2020 suspension of data collection related to the COVID-19 pandemic, the final burden hour total is 57,230 hours.

16. Publication/Tabulation Dates

Publication of the QHP Enrollee Survey results will occur in the fall of 2018, following the data collection. Reporting of the survey results will include distribution of survey reports for each sampling unit to QHP issuers, summary reports to Exchanges, and the Office of Personnel Management (OPM). CMS also anticipates displaying the 2017 QRS global rating and three summary indicator ratings on the HealthCare.gov website for eligible QHPs operating in the pilot States. CMS also publishes updates about the survey through its Exchange Quality Initiatives webpage.⁸

17. Expiration Date

The expiration date will be displayed.

18. Certification Statement

There are no exceptions to the certification statement identified in Item 19, "Certification for Paperwork Reduction Act Submissions," of OMB Form 83-I.

1 Health Insurance Exchange^SM and Exchange^SM are service marks of the U.S. Department of Health & Human Services.

2 COVID-19 Marketplace Quality Initiatives memo available at https://www.cms.gov/files/document/covid-qrs-and-marketplace-quality-initiatives-memo-final.pdf

3 The Patient Protection and Affordable Care Act (Pub. L. 111–148) as amended by the Health Care and Education Reconciliation Act of 2010 (Pub. L. 111–152) (collectively referred to as the Patient Protection and Affordable Care Act).

4 Patient Protection and Affordable Care Act; Exchange and Insurance Market Standards for 2015 and Beyond, Final Rule, 79 Fed. Reg. 30240 at 30352 (May 27, 2014), 45 C.F.R. §§ 156.1120 and 156.1125.

5 45 C.F.R. §§§ 156.200(b)(5),(h); 156.1120; and 156.1125.

6 45 C.F.R. §§ 155.1400 and 155.1405.

7 We recommend 85% of the original annual estimates because in 2019, 20% of completed responses came from the telephone survey and 60% of reporting units had some telephone fielding prior to April 18^th.

8 https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityInitiativesGenInfo/Health-Insurance-Exchange-Quality-Initiatives.html

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2021-01-14

CMS-10488-Supporting Statement QHP Survey Final x