Health Insurance Marketplace Consumer Experience Surveys: Qualified Health Plan Enrollee Experience Survey
Supporting
Statement—Part A
Supporting Statement for Information
Collection the Enrollee Satisfaction Survey and Marketplace Survey
Data Collection
October 3, 2016
Centers for Medicare & Medicaid Services
TABLE OF CONTENTS
A. Background 2
B. Justification 4
3. Use of Information Technology 4
9. Payments/Gifts to Respondents 6
12. Burden Estimates (Hours & Wages) 7
14. Cost to Federal Government 8
16. Publication/Tabulation Dates 9
The Affordable Care Act (ACA) authorized the creation of Health Insurance Marketplaces (Marketplaces) to help individuals and small employers shop for, select, and enroll in high quality, affordable private health plans beginning October 2013. By 2020, more than 27 million individuals and employees of small firms are expected to obtain their health insurance through Marketplaces. Section 1311(c)(4) of the ACA requires the Department of Health and Human Services (HHS) to develop an enrollee satisfaction survey system that assesses consumer experience with qualified health plans (QHPs) offered through a Marketplace. It also requires public display of enrollee satisfaction information by the Marketplace to allow individuals to easily compare enrollee satisfaction levels between comparable plans. In 2014, HHS established the Qualified Health Plan (QHP) Enrollee Experience Survey. The main purpose of the QHP Enrollee Survey is to assess enrollee experience with their QHP around areas such as access to care, access to information, care coordination, cultural competence, doctor communication and plan administration. Under OMB Control Number 0938-1221, a psychometric test and beta test were performed in 2014 and 2015 respectively. The QHP Enrollee Survey was conducted nationwide in 2016. This request is to continue nationwide collection in 2017.
The Centers for Medicare & Medicaid Services (CMS) intended to display the results of the 2016 data collection as part of the Quality Rating System (QRS) star ratings for the upcoming (2017) open season. However, CMS has decided to conduct an additional year of focused consumer testing of the display of QRS star ratings to maximize the clarity and consistency of the information provided to the public and to assess how the QHP quality rating information is displayed on HealthCare.gov. Although the public reporting of QRS results will be delayed (anticipated to begin during the 2018 open enrollment period), issuers are expected to continue collection of the QRS data during 2017 using the established QRS methodology, which is designed to encourage the delivery of high quality health care services and improve health outcomes of QHP enrollees over time.
In April 2016, CMS announced this change of the public reporting timeline of quality ratings information by the Federally-facilitated Marketplaces (FFMs), including FFMs where the State performs plan management functions and State-based Marketplaces on the Federal Platform (SBM-FPs), and announced that a limited pilot would be conducted during the 2017 open enrollment period. During the pilot, CMS will display the QRS star ratings in select States whose consumers use HealthCare.gov during the 2017 open enrollment period. In September 2016, CMS finalized selection of the following States to display star ratings on healthcare.gov: Virginia and Wisconsin. CMS selected these states because they have ample participation of QHP issuers on their respective Marketplaces and relative variation in QRS star ratings based on 2016 QRS results. FFMs not in the pilot will not display star ratings during the 2017 open enrollment period; it is anticipated that these states will display star ratings during the national implementation beginning during the 2018 open enrollment period. State-based Marketplaces (SBMs) whose consumers do not use HealthCare.gov may display QHP quality information during the open enrollment period for the 2017 plan year or follow the revised timeframe.
CMS goals for the QRS pilot include: obtaining further details about consumer access and the use of QHP quality rating information during an actual open enrollment period, so as to inform the display of QRS star ratings; and informing the development of comprehensive technical assistance and education related to the QRS for assisters, navigators, agents, brokers and consumer groups prior to QRS public reporting. CMS intends to interview consumers in pilot States to evaluate if they used quality rating information while shopping for QHPs and if they understood and interpreted the star ratings appropriately. If consumers did not understand the quality rating information displayed, CMS would like to assess if and how consumers sought assistance to obtain quality rating educational materials. Information collection associated with QRS display consumer testing was approved under OMB Control Number 0938-1247. Revisions to currently approved information collections would be submitted to OMB for approval.
At this time, CMS is seeking approval for the revisions to the information collection related to conducting the QHP Enrollee Experience Survey in 2017-2019. These revisions include deleting an item assessing whether the consumer was previously enrolled in insurance and adding six disability status items required by the ACA section 4302 data collection standards. These disability status items were tested in the 2014 Field Test.
QHP Enrollee Survey
As required by section 1311(c) (4) of the Affordable Care Act, CMS established the QHP Enrollee Survey with the goals of (1) informing consumer decision making in choosing a QHP, (2) providing actionable information that the QHP issuers can use to improve performance, and (3) providing information that state and federal regulators and Marketplaces need for effective oversight.
The development of the QHP Enrollee Survey included a comprehensive review of the literature and related surveys, consumer focus groups, stakeholder discussions, and input from a technical expert panel (TEP). As a result of this formative research, CMS decided that the QHP Enrollee Survey would use the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Health Plan 5.0 Adult Medicaid Survey as its core, with supplemental items drawn from the CAHPS Health Plan 5.0 Adult Supplemental Item Set, the CAHPS Health Plan 4.0 Supplemental Item Set, and the CAHPS 5.0 HEDIS Survey to provide the full set of information needed to evaluate QHP performance. Also, a few additional items were developed specifically for the QHP Enrollee Survey to fill in the topics not covered by existing CAHPS items. All selected items underwent two rounds of cognitive testing and revisions before being further evaluated with a Psychometric Test and a Beta Test.
The questionnaire submitted for clearance is available in English, Spanish, and Traditional Chinese for use in a mixed-mode methodology that includes mail, telephone, and web survey modes.
The QHP Enrollee Survey will be conducted by HHS-approved survey vendors who meet minimum business requirements. A similar system is currently used for other CMS surveys, including Medicare CAHPS, Hospital CAHPS (HCAHPS), Home Health CAHPS (HHCAHPS), the CAHPS Survey for Accountable Care Organizations, and the Health Outcomes Survey. Under this model, all QHPs that are required to conduct the QHP Enrollee Survey must contract with a HHS-approved survey vendor to collect the data and submit it to CMS on the issuer’s behalf (45 CFR § 156.1125(a) ). CMS is responsible for approving and training vendors, providing technical assistance to vendors, overseeing vendors to ensure that they are following the data collection protocols, collecting and analyzing the data from vendors, and producing reports that QHP issuers can use for quality improvement. The Survey Vendor process was tested in the 2015 Beta Test.
Section 1311(c)(4) of the Affordable Care Act (ACA) requires HHS to establish an enrollee satisfaction survey to be administered to members of each QHP offered through a Marketplace. The QHP Enrollee Survey meets the goal of measuring enrollee satisfaction with their health plan. Additionally, in accordance with section 1311(c)(4), the results of this survey will be available on each State Marketplace’s web portal, as well as the Federally-facilitated Marketplace’s web portal, in a manner that allows applicants for coverage to compare plans.
Beginning with the 2016 national implementation of the QHP Enrollee Survey, the data collected from the survey will be publicly reported on the Marketplace websites to aid consumers in choosing a QHP. The QHP Enrollee Survey data will also be used by QHP issuers to improve their performance and better tailor efforts to the QHP enrollee population. Additionally, the data will be used by HHS, State Based Marketplaces, and state insurance commissioners to aid in effective regulatory and oversight efforts. Finally, a de-identified, public-use dataset will be made available for use by health services researchers.
The current data collection protocol for the QHP Enrollee Survey includes the use of an online survey as well as the use of Computer Assisted Telephone Interviewing (CATI). Beginning with the 2016 QHP Enrollee Survey, the survey vendors have the option of offering the web survey in Spanish. CMS will continue to evaluate methods to increase the use of online surveys.
In addition to the actual data collection methodologies used by survey vendors, survey vendors are required to submit the final data files to CMS for analysis and scoring through a secure portal on the QHP Enrollee Survey website. This process ensures that the data files meet established specifications. Additionally, after analysis the survey data will be submitted into the Marketplace Quality Module (MQM) within CMS’ Health Insurance Oversight System (HIOS) for use in the Quality Rating System (QRS) and for public reporting.
There is no duplication of efforts. The QHP Enrollee Survey is the only survey being conducted by HHS to measure patient experiences with QHPs offered through the Marketplaces.
The survey population for the QHP Enrollee Survey includes individuals who enrolled in QHPs through an individual Marketplace, a Small Business Health Options Program (SHOP) Marketplace, or directly with the issuer. The sample frame is developed by issuers, few if any of whom are small businesses. CMS expects that this will not have an impact on small businesses. Some survey vendors who will apply to field the QHP Enrollee Survey will be small businesses, but conducting CAHPS surveys is their business and the decision to apply for approval as a vendor for the QHP Enrollee Survey is voluntary. Furthermore, the survey vendor application process imposes a minimal burden on any applicant, including small businesses. Thus, there is no reason to expect that the survey will burden small businesses; it offers them a business opportunity if they choose to apply for participation.
Annual data collection of the QHP Enrollee Survey is needed to meet the objectives of providing feedback to Marketplaces, issuers, and regulators for quality improvement; providing information for consumers’ choice; and to track performance.
There are no special circumstances associated with this data collection.
This is a revised collection to 0938-1221. As required by 5 CFR 1320.8(d), CMS solicited comments on these revisions to the QHP Enrollee Survey, through a Federal Register Notice which was posted on April 29, 2016.
There was one comment from the Disability Rights Education & Defense Fund (DREDF) who expressed support for the addition of the six disability items. DREDF expressed that the addition of these items will allow for better understanding of the experience of enrollees with disabilities in the Marketplaces.
CMS solicited further comments through a second Federal Register Notice which was posted on July 12, 2016. The comment period was open for 30 days.
There was one comment received from America’s Health Insurance Plans (AHIP) who expressed concern that the survey is too long. They believe the length will negatively impact the response rate and recommend removing two areas: the “About You” items that have a low response rates (e.g. the disability items), and the items that they feel do not relate to health plan performance (e.g. care coordination, doctor communication, affordability items).
CMS recognizes that this survey is long, but there are many factors that affect response rates. Previous research has shown that the effect of additional questions up to even 95 questions had minimal effect on response rates and other CAHPS® surveys are of similar length. For example, the Accountable Care Organization CAHPS® questionnaire is currently 80 questions and the Medicare Advantage Prescription Drug Plan CAHPS® survey contains 95 questions
Many of the items included in the “About You” section are required for the aspirin use, tobacco cessation, and flu shot measures that are National Committee for Quality Assurance (NCQA) Healthcare Effectiveness Data and Information Set (HEDIS) items and are also included in the Quality Rating System (QRS) measures and must be included in this implementation. CMS has added the disability status items that are required by section 4302 of the Affordable Care Act. We have dropped the item about previous health insurance status.
The doctor communication and care coordination items are core CAHPS questions that have been previously validated in cognitive testing and field testing. These are included to allow CMS, policy makers, and health plans to make direct comparisons between QHP and Medicaid populations.
CMS has added questions to measure out-of-pocket costs that were included in the 2014 Psychometric Test questionnaire, because they provide critical information about enrollee experiences requested by consumer advocate groups and the project’s Technical Expert Panel. Furthermore, analysis from the 2014 Psychometric Test found meaningful differences between and across reporting units, which suggests that these survey results could be used by QHP issuers to better understand plan ratings and improve communication with enrollees about cost-sharing aspects of QHPs. Given CMS’ belief that the results from these questions will be used for evaluation and quality improvement efforts, they have not been included in the QRS and will not be publicly reported.
While CMS appreciates the comments, there were no revisions to the survey instrument based on the comments received.
CMS is working with a variety of outside organizations and individuals to aid in the development and implementation of the QHP Enrollee Survey. Chief among these organizations is the American Institutes of Research (AIR) and the National Committee for Quality Assurance (NCQA). In addition, a Technical Expert Panel composed of consumer advocates, health plan representatives, Marketplace administrators, survey design experts, state regulators, and providers provides ongoing feedback on technical issues. The panel meets approximately three times a year to provide guidance.
No payments or gifts will be made to any respondents.
Individual survey respondents will be told the purposes for which the information is collected and that, in accordance with section 934(c) of the Public Health Service Act, 42 USC 299c-3(c), any identifiable information about them will not be used or disclosed for any purpose beyond conducting the survey. The confidentiality of individual’s replies is further assured under 5 U.S.C.552 (Freedom of Information Act), 5 U.S.C.552a (Privacy Act of 1974), and OMB Circular No.A-130. SORN: Health Insurance Exchange Program - 78 FR 8538 Publication Date: 02/06/2013.
There are no sensitive questions associated with this information collection.
Estimated burden hours for the QHP Enrollee Survey in 2016-2018 are presented in Exhibit A1 and are based on the following assumptions and definitions.
Units. The sampling/reporting unit has been defined at the level of product type (i.e., Exclusive Provider Organization [EPO], Health Maintenance Organization [HMO], Preferred Provider Organization [PPO], Point of Service [POS]) offered by a QHP issuer through the Marketplace in a particular state. For example, XYZ issuer’s HMOs offered through the Marketplace in Florida would be considered a single sampling unit. Depending on the way a QHP issuer packages its plan offerings, the sampling unit might include anywhere from a single QHP to many QHPs spanning all categories of coverage (i.e., bronze, silver, gold, platinum, catastrophic). QHP issuers will create a sample frame for each product type they offer through the Marketplace within a particular state.
For the 2017 survey, CMS estimates that no more than 350 reporting units will be required to field the QHP Enrollee Survey. This is lower than previous estimates based on two factors: (1) the actual numbers of reporting units that were required to administer the 2015 and 2016 QHP Enrollee Surveys were lower (298 reporting units in 2015 and 311 reporting units in 2016) and (2)a number of QHP issuers have announced their intention to reduce the number of health plans that they offer through the Health Insurance Marketplace in 2017.
CMS is exploring collecting data at a more granular level of QHP issuer coverage (e.g., HMO bronze level) in the future, keeping in mind the need to balance the value of this information for consumers with enrollment volume and QHP issuer data collection, validation, and reporting burden. CMS will revise this Information Collection Request if changes are implemented.
Respondents per unit. Based on the results of the 2014 Psychometric Test and 2015 Beta Test, CMS plans to collect 300 responses per reporting unit. As this survey program continues, CMS will explore whether the number of responses can be reduced.
Total respondents. The total number of respondents equals the product of the completed surveys per sampling unit and the current estimate of the number of QHP sampling units.
Hours per response. Based on testing of the QHP Enrollee Survey it takes 22 minutes to complete.
Survey vendors. Survey vendors who want to participate in collecting QHP Enrollee Survey data must complete a Survey Vendor Participation Form. CMS anticipates that approximately 15 survey vendors will apply to field the QHP Enrollee Survey annually. The Survey Vendor Participation Form is designed to be completed in 90 minutes.
Exhibit A1. Estimated Burden Hours for 2017-2019 National Implementation of QHP Enrollee Survey
Source |
Num. of Reporting Units |
Completes per Reporting Unit |
Total Sample1 |
Burden Hours |
Total burden hours |
2017 Survey Respondents |
350 |
300 |
105,000 |
0.36 |
37,800 |
2017 Survey Vendors |
15 |
1 |
15 |
1.5 |
22.5 |
2017 TOTAL |
350 |
|
|
|
37,823 |
2018 TOTAL |
365 |
|
105,015 |
|
37,823 |
2019 TOTAL |
365 |
|
105,015 |
|
37,823 |
3-year TOTAL |
1,095 |
|
|
|
113,469 |
1 Total Sample = Num. of Reporting Units x Completes per Reporting Unit
In 2017, the total annual burden hours for the 2017 QHP Enrollee Survey are estimated to be 37,823 hours. Because only minimal adjustments to the questionnaire are expected for 2018 and 2019, we estimate an annual burden of 37,823 hours for 2017 and 2018. We estimate a total burden of 113,469 hours over three years.
The Bureau of Labor Statistics reported the average hourly wage for civilian workers in the United States was $25.35 as of February 2016. To estimate the burden costs for survey vendors, CMS used the average hourly wage for employees in the business and professional services sector as of February 2016. See exhibit A2 for estimated burden costs.
Exhibit A2. Estimated Burden Costs
Source |
Number of Respondents |
Total Burden Hours |
Average Hourly Wage Rate |
Total Cost Burden |
2017 Survey Respondents |
105,000 |
37,800 |
$25.35 |
$958,230.00 |
2017 Survey Vendors |
15 |
22.5 |
$30.36 |
$683.10 |
2017 TOTAL |
105,015 |
37,823 |
|
$958,913.10 |
2018 TOTAL |
105,015 |
37,823 |
|
$958,913.10 |
2019 TOTAL |
105,015 |
37,823 |
|
$958,913.10 |
3-Year TOTAL |
315,045 |
113,469 |
|
$2,876,739.30 |
There are no direct capital costs to respondents other than their time to participate in the survey.
The only cost to the Government of these data collections that would not otherwise have been incurred is the cost of the American Institutes for Research (AIR) contract. The portion of the AIR contract attributable to the QHP Enrollee Survey is approximately $1.7 million for the 2017 national implementation. We expect the 2018 and 2019 implementation to also be approximately $1.7 million as well for a 3-year total of $5.1 million. This cost includes soliciting and approving survey vendors, developing quality assurance guidelines and technical specifications for survey vendors, providing technical assistance and training to survey vendors, conducting oversight of approved survey vendors, providing technical assistance to QHP issuers, scoring and analyzing the survey data, and development of final reports for QHP issuers.
While CMS is proposing to add six new questions to the survey, at the same time there has also been a reduction in one item associated with assessing whether the consumer was previously enrolled in insurance. The proposed six new questions are being added to comply with section 4302 of the Affordable Care Act and ensure that appropriate data collection standards are used in the QHP Enrollee Survey.1 The estimated number of QHP issuer reporting units that will need to field the survey has decreased from 400 to 350. As a result, there is a net reduction of 5,400 burden hours over three years.
Publication of the QHP Enrollee Survey results will occur in the fall of 2017, following the data collection. Reporting of the survey results will include distribution of survey reports for each sampling unit to QHP issuers, summary reports to Marketplaces, and the Office of Personnel Management (OPM), as well as public reporting of survey results through Marketplace websites. CMS also publishes updates about the survey through its Marketplace Quality Initiatives webpage2 and through the QHP Enrollee Survey project webpage.3
All reporting websites under CMS’ control will provide Marketplace consumers with the overall response rate and the minimum and maximum response rates obtained by reporting units nationwide. This information will also include a statement of findings from the nonresponse bias analysis and CMS’ assessment of the potential implications of those findings for use of the response rates by consumers in choosing a QHP. CMS will report back to OMB before posting results publicly regarding how it intends to communicate these concepts to consumers within the context of the Quality Rating System (QRS).
The expiration date will be displayed.
There are no exceptions to the certification statement identified in Item 19, "Certification for Paperwork Reduction Act Submissions," of OMB Form 83-I.
1 Section 4302 requires the Secretary of Department of Health and Human Services to establish data collection standards for race, ethnicity, sex, primary language, and disability status. The law requires that, once established, these data collection standards be used, to the extent practicable, in all national population health surveys
2 https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityInitiativesGenInfo/Health-Insurance-Marketplace-Quality-Initiatives.html
3 https://qhpcahps.cms.gov/
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |