Health Insurance Exchange Consumer Experience Surveys: Qualified Health Plan Enrollee Experience Survey
Supporting
Statement—Part A
Supporting Statement for Information
Collection the Enrollee Satisfaction Survey and Exchange Survey Data
Collection
Centers for Medicare & Medicaid Services
TABLE OF CONTENTS
A. Background 2
B. Justification 5
3. Use of Information Technology 5
9. Payments/Gifts to Respondents 6
12. Burden Estimates (Hours & Wages) 7
14. Cost to Federal Government 8
16. Publication/Tabulation Dates 9
18. Certification Statement 10
The Affordable Care Act (ACA) authorized the creation of Health Insurance Exchanges (Exchanges) (also known as Marketplaces) to help individuals and small employers shop for, select, and enroll in high quality, affordable private health plans beginning October 2013.1 Section 1311(c)(4) of the ACA requires the Department of Health & Human Services (HHS) to develop an enrollee satisfaction survey system that assesses consumer experience with qualified health plans (QHPs) offered through an Exchange. It also requires public display of enrollee satisfaction information by the Exchange to allow individuals to easily compare enrollee satisfaction levels between comparable plans. In 2014, HHS established the Qualified Health Plan Enrollee Experience Survey (QHP Enrollee Survey). The main purpose of the QHP Enrollee Survey is to assess enrollee experience with their QHP around areas such as access to care, access to information, care coordination, cultural competence, doctor communication and plan administration. Under OMB Control Number 0938-1221, a psychometric test and beta test were performed in 2014 and 2015, respectively. The QHP Enrollee Survey was conducted nationwide in 2016 and 2017. This request is to continue nationwide collection and administration of the survey in 2018 – 2020.
The Centers for Medicare & Medicaid Services (CMS) initially intended to display the results of the 2016 data collection as part of the Quality Rating System (QRS) star ratings for the 2017 Open Enrollment Period. However, CMS decided to conduct an additional year of focused consumer testing of the display of QRS star ratings to maximize the clarity and consistency of the information provided to the public and to assess how the QHP quality rating information is displayed on HealthCare.gov.
In April 2016, CMS announced this change in the public reporting timeline of quality ratings information by the Federally-facilitated Exchange (FFE), including FFEs where the state performs plan management functions and State-based Exchanges on the Federal Platform (SBE-FPs), and announced that a limited pilot would be conducted during the 2017 open enrollment period. During the pilot, CMS displayed the QRS star ratings in select states with consumers using HealthCare.gov during the 2017 open enrollment period. In September 2016, CMS finalized selection of the following states to display star ratings on healthcare.gov: Virginia and Wisconsin. CMS selected these states because they have ample participation of QHP issuers on their respective Exchanges and relative variation in QRS star ratings based on 2016 QRS results. FFEs not in the pilot did not display star ratings during the 2017 open enrollment period. CMS anticipates displaying the 2017 QRS global rating and three summary indicator ratings on the HealthCare.gov website for eligible QHPs operating in the pilot States. State-based Exchanges (SBEs) whose consumers do not use HealthCare.gov were able to choose to display QHP quality information during the open enrollment period for the 2017 plan year or follow the revised timeframe.
CMS goals for the QRS pilot include: obtaining further details about consumer access and the use of QHP quality rating information during an actual open enrollment period, so as to inform the display of QRS star ratings; and informing the development of comprehensive technical assistance and education related to the QRS for assisters, navigators, agents, brokers and consumer groups prior to QRS public reporting. Information collection associated with QRS display consumer testing was approved under OMB Control Number 0938-1247. CMS is assessing information obtained regarding the display of QRS ratings during the pilot and will consider the timeframes for public reporting of quality rating information.
At this time, CMS is seeking to renew approval for the information collection related to the QHP Enrollee Experience Survey in 2018-2020. These activities are necessary to ensure that CMS fulfills legislative mandates established by section 1311(c)(4) of the Affordable Care Act to develop an “enrollee satisfaction survey system” and provide such information on Exchange websites. CMS is proposing to remove several questions from the QHP Enrollee Survey. The rationale for these changes is discussed below.
7. In the last 6 months, did you need to visit a doctor’s office or clinic after regular office hours?
8. In the last 6 months, how often were you able to get care you needed from a doctor’s office or clinic after regular office hours?
Due to low screen-in rates among respondents (less than nine percent of respondents) and poor inter-unit reliability, CMS decided to remove the item about after-hours care from the QRS Access to Care measure beginning in 2018. Before finalizing this proposal, CMS sought public comment. The comments that CMS received agreed with this proposal.
Given that this question (#7 above) will no longer be utilized for the QRS and has a low screen-in rate of less than nine percent, CMS proposes dropping this survey question from the 2018 QHP Enrollee Survey. The accompanying question (#8 above) used to measure access to after regular office hours would also be removed from the 2018 QHP Enrollee Survey.
53. Using any number from 0 to 10, where 0 is not at all likely and 10 is extremely likely, how likely is it that you would recommend this health plan to a friend or family member?
Based on stakeholder feedback, CMS added a question about the respondents’ likelihood of recommending their current health plan to friends or family (i.e., “recommend question”) to the 2016 QHP Enrollee Survey. The recommend question had previously been included in the questionnaire for the 2014 psychometric test, but CMS dropped it from the questionnaire used for the 2015 Beta Test to reduce respondent burden.
In analyses of the 2016 QHP Enrollee Survey, CMS found a strong, positive correlation (0.96) between the recommend question and the Consumer Assessment of Healthcare Providers & Systems (CAHPS®) Health Plan Survey global rating question at the reporting unit level. Given that the recommend question is not used by the QRS, whereas the CAHPS® Health Plan Survey global rating question is used by the QRS, CMS proposes eliminating this question (#53 above) from the 2018 QHP Enrollee Survey.
Do you take aspirin daily or every other day?
Do you have a health problem or take medication that makes taking aspirin unsafe for you?
Has a doctor or health provider ever discussed with you the risks and benefits of aspirin to prevent heart attack or stroke?
Are you aware that you have any of the following conditions? Mark one or more.
High cholesterol
High blood pressure
Parent or sibling with heart attack before the age of 60
Has a doctor ever told you that you have any of the following conditions? Mark one or more.
A heart attack
Angina or coronary heart disease
A stroke
Any kind of diabetes or high blood sugar
In light of revised recommendations by the U.S. Preventive Services Task Force (USPSTF), CMS retired the Aspirin Use and Discussion measure for the QRS measure set beginning in 2017. Accordingly, CMS is proposing to remove the survey questions associated with this measure (#65 through #69 above) in the 2018 QHP Enrollee Survey.
QHP Enrollee Survey
As required by section 1311(c)(4) of the Affordable Care Act, CMS established the QHP Enrollee Survey with the goals of (1) informing consumer decision making in choosing a QHP, (2) providing actionable information that the QHP issuers can use to improve performance, and (3) providing information that state and federal regulators and Exchanges need for effective oversight.
The development of the QHP Enrollee Survey included a comprehensive review of the literature and related surveys, consumer focus groups, stakeholder discussions, and input from a technical expert panel (TEP). As a result of this formative research, CMS decided that the QHP Enrollee Survey would use the CAHPS® Health Plan 5.0 Adult Medicaid Survey as its core, with supplemental items drawn from the CAHPS® Health Plan 5.0 Adult Supplemental Item Set, the CAHPS Health Plan 4.0 Supplemental Item Set, and the CAHPS® 5.0 HEDIS Survey to provide the full set of information needed to evaluate QHP performance. Also, a few additional items were developed specifically for the QHP Enrollee Survey to fill in the topics not covered by existing CAHPS® items. All selected items underwent two rounds of cognitive testing and revisions before being further evaluated with a psychometric test in 2014, a beta test in 2015, and additional testing during the 2016 survey administration.
The questionnaire submitted for clearance is available in English, Spanish, and Traditional Chinese for use in a mixed-mode methodology that includes mail, telephone, and web survey modes.
The QHP Enrollee Survey will be conducted by HHS-approved survey vendors who meet minimum business requirements. A similar system is currently used for other CMS surveys, including Medicare CAHPS, Hospital CAHPS (HCAHPS), Home Health CAHPS (HHCAHPS), the CAHPS Survey for Accountable Care Organizations, and the Health Outcomes Survey (HOS). Under this model, all QHPs that are required to conduct the QHP Enrollee Survey must contract with an HHS-approved survey vendor to collect the data and submit it to CMS on the issuer’s behalf (45 CFR § 156.1125(a) ). CMS is responsible for approving and training vendors, providing technical assistance to vendors, overseeing vendors to ensure that they are following the data collection protocols, collecting and analyzing the data from vendors, and producing reports that QHP issuers can use for quality improvement. The survey vendor program was tested in the 2015 beta test and refined for the 2016 survey administration.
Section 1311(c)(4) of the Affordable Care Act (ACA) requires HHS to establish an enrollee satisfaction survey to be administered to members of each QHP offered through an Exchange. The QHP Enrollee Survey meets the goal of measuring enrollee satisfaction with their health plan. Additionally, in accordance with section 1311(c)(4), the results of this survey will be available on each State Exchange’s web portal, as well as the Federally-facilitated Exchange’s web portal, in a manner that allows applicants for coverage to compare plans.
Beginning with the 2017 national implementation of the QHP Enrollee Survey, the survey results were publicly reported (as part of the Quality Rating System) on select Exchange websites to aid consumers in choosing a QHP as part of a limited display consumer pilot test of the QRS. Beginning with the 2018 individual market open enrollment period and similar to last year, CMS anticipates displaying the 2017 QRS global rating and three summary indicator ratings on the HealthCare.gov website for eligible QHPs operating in the pilot States of Virginia and Wisconsin. The QHP Enrollee Survey data are provided to QHP issuers to improve their performance and better tailor their efforts to improve the experiences of the QHP enrollee population. Additionally, the data is used by HHS, State-based Exchanges, and state insurance commissioners to aid in effective regulatory and oversight efforts.
The current data collection protocol for the QHP Enrollee Survey includes the use of an online survey as well as the use of Computer Assisted Telephone Interviewing (CATI). Beginning with the 2016 QHP Enrollee Survey, the survey vendors have the option of offering the web survey in Spanish (it is required in English). CMS will continue to evaluate methods to increase the use of online surveys.
Survey vendors are required to submit the final data files to CMS for analysis and scoring through a secure portal on the QHP Enrollee Survey website. This process ensures that the data files meet established specifications. Additionally, after analysis the survey data is submitted into the Exchange Quality Module (MQM) within CMS’ Health Insurance Oversight System (HIOS) for calculation of star ratings produced by the QRS for public reporting.
There is no duplication of efforts. The QHP Enrollee Survey is the only survey being conducted by HHS to measure patient experiences with QHPs offered through the Exchanges.
The survey population for the QHP Enrollee Survey includes individuals who enrolled in QHPs through an individual Exchange, a Small Business Health Options Program (SHOP) Exchange, or directly with the issuer for plans offered through the Exchange. The sample frame is developed by issuers, few if any of whom are small businesses. CMS expects that this will not have an impact on small businesses. Some survey vendors who will apply to field the QHP Enrollee Survey will be small businesses, but conducting CAHPS® surveys is their business and the decision to apply for approval as a vendor for the QHP Enrollee Survey is voluntary. Furthermore, the survey vendor application process imposes a minimal burden on any applicant, including small businesses. Thus, there is no reason to expect that the survey will burden small businesses; it offers them a business opportunity if they choose to apply for participation.
Annual data collection of the QHP Enrollee Survey is needed to meet the objectives of providing feedback to Exchanges, issuers, and regulators for quality improvement; providing information for consumers’ choice; and to track performance.
There are no special circumstances associated with this data collection.
As required by 5 CFR 1320.8(d), CMS has solicited comments on these revisions to the QHP Enrollee Survey, through a Federal Register Notice (April 14, 2017 – 82 FR 17997). There were 24 comments submitted; however, the comments were not applicable since they did not relate to the QHP Enrollee Survey and its associated information collection.
CMS is working with a variety of outside organizations and individuals to aid in the development and implementation of the QHP Enrollee Survey. Chief among these organizations is the American Institutes for Research (AIR), the National Committee for Quality Assurance (NCQA), and Booz Allen Hamilton. In addition, a Technical Expert Panel composed of consumer advocates, health plan representatives, Exchange administrators, survey design experts, state regulators, and providers provides ongoing feedback on technical issues. The panel meets approximately three times a year to provide guidance.
No payments or gifts will be made to any respondents.
Individual survey respondents will be told the purposes for which the information is collected and that, in accordance with section 934(c) of the Public Health Service Act, 42 USC 299c-3(c), any identifiable information about them will not be used or disclosed for any purpose beyond conducting the survey. The confidentiality of individual’s replies is further assured under 5 U.S.C.552 (Freedom of Information Act), 5 U.S.C.552a (Privacy Act of 1974), and OMB Circular No.A-130. SORN: Health Insurance Exchange Program - 78 FR 8538 Publication Date: 02/06/2013.
There are no sensitive questions associated with this information collection.
Estimated burden hours for the QHP Enrollee Survey in 2018 are presented in Exhibit A1 and are based on the following assumptions and definitions.
Units. The reporting unit has been defined at the level of product type (i.e., Exclusive Provider Organization [EPO], Health Maintenance Organization [HMO], Preferred Provider Organization [PPO], Point of Service [POS]) offered by a QHP issuer through the Exchange in a particular state. For example, XYZ issuer’s HMOs offered through the Exchange in Florida would be considered a single reporting unit. Depending on the way a QHP issuer packages its plan offerings, the reporting unit might include a single QHP or many QHPs spanning all categories of coverage (i.e., bronze, silver, gold, platinum, catastrophic). QHP issuers will create a sample frame for each product type they offer through the Exchange within a particular state, or reporting unit.
For the 2018 survey, CMS estimates that no more than 300 reporting units will be required to administer the QHP Enrollee Survey. This is lower than previous estimates based on two factors: (1) the actual numbers of reporting units that were required to administer the 2017 survey was 262 and (2) a number of QHP issuers have left the Health Insurance Exchange.
Respondents per unit. Based on the results of the 2014 Psychometric Test, 2015 Beta Test, and 2016 implementation, CMS plans to collect 300 responses per reporting unit. As this survey program continues, CMS will explore whether the number of responses can be reduced.
Total respondents. The total number of respondents equals the product of the completed surveys per reporting unit and the current estimate of the number of QHP reporting units.
Hours per response. Based on testing of the QHP Enrollee Survey it takes 15 minutes to complete.
Survey vendors. Survey vendors who want to participate in collecting QHP Enrollee Survey data must complete a Survey Vendor Participation Form. CMS anticipates that approximately 15 survey vendors will apply to field the QHP Enrollee Survey annually. The Survey Vendor Participation Form is designed to be completed in 90 minutes.
Exhibit A1. Estimated Burden Hours for 2018-2020 Implementation of QHP Enrollee Survey
Source |
Num. of Reporting Units |
Completes per Reporting Unit |
Total Sample1 |
Burden Hours |
Total burden hours |
2018 Survey Respondents |
300 |
300 |
90,000 |
0.25 |
22,500 |
2018 Survey Vendors |
15 |
1 |
15 |
1.5 |
22.5 |
2018 TOTAL |
300 |
|
90,015 |
|
22,523 |
2019 TOTAL |
300 |
|
90,015 |
|
22,523 |
2020 TOTAL |
300 |
|
90,015 |
|
22,523 |
3-year TOTAL |
900 |
|
|
|
67,569 |
1 Total Sample = Num. of Reporting Units x Completes per Reporting Unit
In 2018, the total annual burden hours for the 2018 QHP Enrollee Survey are estimated to be 22,523 hours. Because only minimal adjustments to the questionnaire are expected for 2019 and 2020, we estimate an annual burden of 22,523 hours for 2019 and 2020. We estimate a total burden of 67,569 hours over three years.
The Bureau of Labor Statistics reported the average hourly wage for civilian workers in the United States was $26.00 as of January 2017. To estimate the burden costs for survey vendors, CMS used the average hourly wage for employees in the business and professional services sector which was $31.24 as of January 2017. See exhibit A2 for estimated burden costs.
Exhibit A2. Estimated Burden Costs
Source |
Number of Respondents |
Total Burden Hours |
Average Hourly Wage Rate |
Total Cost Burden |
2017 Survey Respondents |
90,000 |
22,500 |
$26.00 |
$585,000.00 |
2017 Survey Vendors |
15 |
22.5 |
$31.24 |
$702.90 |
2017 TOTAL |
90,015 |
22,523 |
|
$585,702.90 |
2018 TOTAL |
90,015 |
22,523 |
|
$585,702.90 |
2019 TOTAL |
90,015 |
22,523 |
|
$585,702.90 |
3-Year TOTAL |
270,045 |
67,569 |
|
$1,757,108.70 |
There are no direct capital costs to respondents other than their time to participate in the survey.
The only cost to the Government of these data collections that would not otherwise have been incurred is the cost of the American Institutes for Research (AIR) contract. The portion of this contract attributable to the QHP Enrollee Survey is approximately $1.7 million for the 2018 national implementation. We expect the 2018 and 2019 implementation to also be approximately $1.7 million as well for a 3-year total of $5.1 million. This cost includes soliciting and approving survey vendors, developing quality assurance guidelines and technical specifications for survey vendors, providing technical assistance and training to survey vendors, conducting oversight of approved survey vendors, providing technical assistance to QHP issuers, scoring and analyzing the survey data, and development of final reports for QHP issuers.
The forecasted burden for implementing the 2018 - 2020 QHP Enrollee Survey has been reduced for two reasons.
First, CMS has reduced its estimate for the number of reporting units that will be required to administer the survey. In previous Information Collection Reviews, CMS had been estimating that 350 reporting units would administer, while CMS is now estimating that 300 reporting units will be required to administer the QHP Enrollee Survey. This change reflects the reduced number of QHP issuers operating in the Exchange and the reduction of different product types offered by each issuer. There are currently 262 reporting units administering the 2017 QHP Enrollee Survey and CMS anticipates a similar number of reporting units in future administrations
Secondly, CMS has revised its estimate for the amount of time needed to complete the QHP Enrollee Survey to 15 minutes. Following the 2016 survey administration, HHS-approved survey vendors reported that on average telephone interviews took 16 minutes and respondents who completed the survey online completed the survey in approximately 13 minutes. Given that the 2018 questionnaire will be shortened by eight questions, CMS anticipates that respondents will be able to complete the QHP Enrollee Survey in 15 minutes or less.
As a result of these changes CMS has reduced the 3-year burden estimates by 45,900 burden hours.
Publication of the QHP Enrollee Survey results will occur in the fall of 2018, following the data collection. Reporting of the survey results will include distribution of survey reports for each sampling unit to QHP issuers, summary reports to Exchanges, and the Office of Personnel Management (OPM). CMS also anticipates displaying the 2017 QRS global rating and three summary indicator ratings on the HealthCare.gov website for eligible QHPs operating in the pilot States. CMS also publishes updates about the survey through its Exchange Quality Initiatives webpage2 and through the QHP Enrollee Survey project webpage.3
The expiration date will be displayed on each instrument (top right-hand corner).
1 Health Insurance ExchangeSM and ExchangeSM are service marks of the U.S. Department of Health & Human Services.
2 https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityInitiativesGenInfo/Health-Insurance-Exchange-Quality-Initiatives.html
3 https://qhpcahps.cms.gov/
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |