Health Insurance Exchange Consumer Experience Surveys: Qualified Health Plan Enrollee Experience Survey
Supporting
Statement—Part A
Supporting Statement for the Enrollee
Satisfaction Survey and Exchange Survey Data Collection
OMB Control Number: 0938-1221
Centers for Medicare & Medicaid Services
TABLE OF CONTENTS
A. Background 1
B. Justification 3
3. Use of Improved Information Technology and Burden Reduction 3
4. Efforts to Identify Duplication and Use of Similar Information 3
5. Impact on Small Businesses or Other Small Entities 4
6. Consequences of Collecting the Information Less Frequently 4
9. Payments/Gifts to Respondents 5
12. Burden Estimates (Hours & Wages) 5
14. Cost to Federal Government 7
Section 1311(c)(4) of the Patient Protection and Affordable Care Act (PPACA) directs the Secretary of the Department of Health & Human Services (HHS) to establish an enrollee satisfaction survey to assess enrollee satisfaction with each Qualified Health Plan (QHP) offered through the Health Insurance Exchanges (Exchanges) (also known to consumers as Health Insurance Marketplaces)1 and Small Business Health Options Program (SHOP) with more than 500 enrollees in the prior year. Additionally, Section 1311(c)(3) of the PPACA directs the Secretary to develop a quality rating for each QHP offered through an Exchange.
Based on this authority, CMS issued a regulation in May 2014 to establish standards and requirements related to QHP issuer data collection and public reporting of quality rating information in every Exchange.2 As a condition of certification and participation in the Exchanges, CMS requires that QHP issuers submit QHP Enrollee Experience Survey (QHP Enrollee Survey or survey) response data and Quality Rating System (QRS) clinical data for their respective QHPs offered through an Exchange in accordance with CMS guidelines.3 Exchanges are also required to display QHP quality rating information on their respective websites.4 The 2020 QRS and QHP Enrollee Survey Technical Guidance includes all relevant statutory and regulatory citations for the QRS and the QHP Enrollee Survey.
The QHP Enrollee Survey assesses enrollees’ experience with their QHPs around such areas as access to care, access to information, care coordination, cultural competence, doctor communication, and plan administration. The goals of the QHP Enrollee Survey are to:
Provide comparable and useful information to consumers about the quality of health care services and enrollee experience with QHPs offered through the Exchanges,
Facilitate oversight of QHP issuer compliance with quality reporting standards set forth in the PPACA and implementing regulations, and
Provide actionable information that QHP issuers can use to improve quality and performance.
Based on the requirements for the QHP Enrollee Survey, CMS developed a survey to capture information about enrollees’ experience with QHPs offered through an Exchange. CMS conducted in-depth formative research including: a comprehensive literature review, review of existing CMS survey instruments, consumer focus groups, stakeholder discussions, and input from a Technical Expert Panel (TEP). Under Office of Management and Budget (OMB) Control Number 0938-1221, CMS performed a psychometric test and beta test in 2014 and 2015, respectively. CMS began fielding the QHP Enrollee Survey nationwide in 2016 and this request is to continue nationwide collection and administration of the statutorily-required survey in 2021 through 2023.
Due to the unique nature of the QHP enrollee population and its application to the QRS, CMS determined that the QHP Enrollee Survey required a customized survey instrument. The QHP Enrollee Survey includes questions from the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Health Plan 5.0 Adult Medicaid Survey, the CAHPS 5.0 Healthcare Effectiveness Data and Information Set (HEDIS®) Survey, the CAHPS Health Plan 4.0 and, and the CAHPS 5.0 Adult Supplemental Item Sets. The survey also includes additional items developed specifically for the QHP Enrollee Survey. These additional items include questions around enrollee experience with costs and customer service to capture topics not covered by existing CAHPS items and disability status items to comply with the requirements of Section 4302, Data Collection Standards, of the PPACA.
Currently, CMS proposes adding two questions and additional instructions to existing questions and section introductions included in the QHP Enrollee Survey for the 20212023 administrations within this request (see the Crosswalk of Changes to the QHP Enrollee Survey). These updates were informed by stakeholder input CMS received through the TEP, issuer and consumer focus groups5, and discussions with survey vendors. CMS continually seeks stakeholder input and conducts analyses of results to inform future refinements to the survey (e.g., changes that may improve survey usability for issuers in supporting quality improvement efforts, changes to reduce burden to issuers by removing questions with high levels of missingness). CMS conducted focus groups and cognitive testing to support identification of future refinements for the QHP Enrollee Survey. If any refinements are proposed that may impact the 2022 or 2023 administrations of the survey, CMS will submit a revision of this information collection request.
The QHP Enrollee Survey is conducted by HHS-approved survey vendors that meet minimum business requirements. A similar system is currently used for other CMS surveys, including Medicare CAHPS, Hospital CAHPS (HCAHPS), Home Health CAHPS (HHCAHPS), the CAHPS Survey for Accountable Care Organizations, and the Health Outcomes Survey (HOS).
Under this model, all QHP issuers that are required to conduct the QHP Enrollee Survey must contract with an HHS-approved survey vendor to collect the data and submit it to CMS on the issuer’s behalf (45 CFR § 156.1125(a)). CMS is responsible for approving and training survey vendors, providing technical assistance to vendors, overseeing vendors to ensure that they are following the data collection protocols, collecting and analyzing the data from vendors, and producing reports that QHP issuers can use for quality improvement.
At this time, CMS is seeking a routine, three-year renewal of the approval for the information collection related to the QHP Enrollee Experience Survey. These activities are necessary to ensure that CMS fulfills legislative mandates established by Section 1311(c)(4) of the Affordable Care Act to develop an “enrollee satisfaction survey system” and provide such information on Exchange websites. At this time, CMS is proposing the addition of two questions and additional instructions to existing questions and section introductions included in the QHP Enrollee Survey. The questionnaire submitted for clearance is available in English, Spanish, and Chinese for use in a mixed-mode methodology that includes mail, telephone, and Internet survey modes.
Section 1311(c)(4) of the Affordable Care Act requires HHS to establish an enrollee satisfaction survey to be administered to members of each QHP offered through an Exchange. The QHP Enrollee Survey meets the goal of measuring enrollees’ satisfaction with their health plan. Additionally, in accordance with Section 1311(c)(4) and outlined in 45 CFR 155.205(b)(1)(iv) and 45 CFR 155.1405, the results of this survey will be available by displaying the QRS information (which incorporates member experience data from the QHP Enrollee Survey), on each state Exchange’s web portal, as well as on the Federally-facilitated Exchange (FFE) web portal (HealthCare.gov), in a manner that allows applicants for coverage to compare plans.
After each QHP Enrollee Survey administration year, CMS produces Quality Improvement (QI) Reports summarizing the item-level results for each reporting unit and state participating in the QHP Enrollee Survey. These reports also include comparative benchmark data so that QHP issuers can see their results relative to the national level results.
A subset of survey questions is included in the QRS measure set and accompanying QHP quality rating information for public display. Beginning with the 2020 open enrollment period, CMS displayed the QHP quality rating information for all Exchanges that use the HealthCare.gov platform, including the FFEs, inclusive of FFE states where the state performs plan management functions and State-based Exchanges on the Federal Platform (SBE-FPs). SBEs were required to display QHP quality ratings for the 2020 open enrollment period, but had some flexibility to customize the display of the QHP quality rating information.
The current data collection protocol for the QHP Enrollee Survey includes the use of an online survey, as well as the use of computer-assisted telephone interviewing (CATI). Beginning with the 2019 QHP Enrollee Survey, survey vendors were required to field the web survey in Spanish (it was previously only required in English) and optimize the web survey for mobile devices.
Also beginning with the 2019 QHP Enrollee Survey, CMS implemented an email protocol. Survey vendors were required to send a notification email and two reminder emails during the fielding period. CMS will continue to evaluate methods to increase the utility of email outreach and online surveys.
Survey vendors are required to submit the final data files to CMS for analysis and scoring through a secure file transfer protocol (SFTP) or a similar secure protocol approved by CMS. This process ensures the data files meet established specifications. Additionally, after analysis, the survey data is submitted into the SAS Viya analytic environment which connects to CMS’ Health Insurance Oversight System (HIOS) for calculation of quality ratings based on QRS methodology for public reporting.
There is no duplication of efforts. The QHP Enrollee Survey is the only survey being conducted by HHS to measure enrollee experiences with QHPs offered through the Exchanges.
CMS does not anticipate the QHP Enrollee Survey will have an impact on small businesses. The sample frame is developed by issuers, few, if any, of which are small businesses. Some survey vendors that will apply to field the QHP Enrollee Survey may be small businesses, but conducting CMS surveys, such as the CAHPS surveys, is part of these vendors’ business model and the decision to apply for approval as a vendor for the QHP Enrollee Survey is voluntary. Furthermore, the survey vendor application process imposes a minimal burden on any applicant, including small businesses.
Annual data collection of the QHP Enrollee Survey is required to meet the objectives of providing feedback to Exchanges, issuers, and regulators for quality improvement; providing information for consumers’ choice; tracking QHP performance; and complying with applicable legislation.
There are no special circumstances associated with this data collection.
This is a revised information collection approved under OMB Control Number 0938-1221. As required by 5 CFR 1320.8(d), CMS provided the opportunity for the public to comment for 60 days through publication of a Federal Register Notice and received no comments.
CMS further solicited comments through a second Federal Register Notice which was posted on August 10, 2020. The comment period was open for 30 days. CMS received three public comments from the Center for the Study of Services (CSS), Anthem, and Kaiser Permanente.
Commenters were supportive of the addition of two questions, as well as instructions related to the pandemic and the increase use of telehealth but did recommend refinements to the new questions and instructions. CMS agreed with the recommendations to revise the new questions to more effectively collect data on access to care and telehealth appointments during the public health emergency. CMS appreciates commenters concerns about including instructions to consider telephone and video appointments for specific survey questions but will not remove these instructions to ensure enrollees consider these appointment types when answering survey questions. Please review the Responses to Public Comments to the Qualified Health Plan Enrollee Survey for a comprehensive summary of the public comments received and CMS’ responses.
CMS is working with a variety of outside organizations and individuals to aid in the development and implementation of the QHP Enrollee Survey. Chief among these organizations are Booz Allen Hamilton (Booz Allen) and the National Committee for Quality Assurance (NCQA). Booz Allen and NCQA are the contractor and subcontractor, respectively, and develop, implement and oversee the administration of the QHP Enrollee Survey.
In addition, a TEP composed of consumer advocates, health plan representatives, Exchange administrators, survey design experts, and state regulators provides ongoing feedback to CMS’ contractor (Booz Allen Hamilton) to inform future refinements to the QHP Enrollee Survey. The panel meets approximately twice a year to provide input on topics such as survey development and refinement; technical and methodological issues related to development, testing and fielding of the survey instrument; and survey findings.
No payments or gifts will be made to any survey respondents.
Individual survey respondents will be told the purposes for which the information is collected and that, in accordance with Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c), any identifiable information about them will not be used or disclosed for any purpose beyond conducting the survey. The confidentiality of individuals’ replies is further assured under 5 U.S.C.552 (Freedom of Information Act), 5 U.S.C.552a (Privacy Act of 1974), and OMB Circular No. A-130. System of Records Notice (SORN): Health Insurance Exchange Program - 78 FR 8538 Publication Date: 02/06/2013.
The QHP Enrollee Survey collects race and ethnicity data to identify health disparities. Per OMB standards, race and ethnicity information is collected separately and meet the minimum set of response categories.
Estimated burden hours for the implementation and administration of the QHP Enrollee Survey in 2021 are presented in Exhibit A1. CMS has reduced the estimated total burden hours from 57,230 to 49,550.10 and the estimated total cost burden from $1,659,608.70 to $1,403,554.41since the previous information collection request submitted in April 2018. See Section 15 Changes to Burden for a detailed description of these changes. The estimated burden hours and cost burden are based on the following assumptions and definitions:
Units. CMS has established the reporting unit as the product type (i.e., Exclusive Provider Organization [EPO], Health Maintenance Organization [HMO], Preferred Provider Organization [PPO], Point of Service [POS]) offered by a QHP issuer through an Exchange in a particular state. For example, XYZ issuer’s HMOs offered through the Exchange in Florida would be considered a single reporting unit.
Depending on the way a QHP issuer packages its plan offerings, the reporting unit might include anywhere from a single QHP to many QHPs spanning all categories of coverage (i.e., bronze, silver, gold, platinum, catastrophic). QHP issuers will create a sample frame for each product type they offer through the Exchange within a particular state or reporting unit. Child-only QHPs and standalone dental plans (SADPs) are excluded from the QHP Enrollee Survey requirements.
For the 2021 survey, CMS estimates that no more than 275 reporting units will be required to administer the QHP Enrollee Survey. This number is lower than previous estimates for this data collection and is based on actual numbers of reporting units that were required to administer the survey in previous years. In 2019, there were 214 reporting units fielding the survey and in 2020 there were 238 reporting units fielding the survey. In April 2020 CMS suspended data collection for the 2020 QHP Enrollee Survey due to the COVID-19 pandemic6.
Respondents per unit. CMS expects to collect 300 responses per reporting unit from a minimum sample size of 1,300 survey respondents. The average number of completed surveys per reporting unit in 2019 was 241.
Total respondents. CMS calculated the total number of respondents by multiplying the planned number of completed surveys (300) for each reporting unit by the planned number of completed reporting units.
Hours per response. Based on testing of the QHP Enrollee Survey, the survey takes on average 12 minutes for respondents to complete.
Survey vendors. Survey vendors that want to participate in collecting QHP Enrollee Survey data must complete a Survey Vendor Participation Form. CMS anticipates that approximately 10 survey vendors will apply to field the QHP Enrollee Survey annually and that it takes 100 minutes to complete the Survey Vendor Participation Form. CMS reduced the estimated number of survey vendors from 15 to 10 based on the consolidation of several vendors in this market area and the actual number of survey vendors that applied for HHS-approval. In 2020, there were nine survey vendor applicants.
Exhibit A1. Estimated Burden Hours for 2021-2023 Implementation of QHP Enrollee Survey
Source |
Number of Reporting Units/ Survey Vendors |
Completes per Reporting Unit |
Total Sample1 |
Burden Hours |
Total Burden Hours |
2021 Survey Respondents |
275 |
300 |
82,500 |
0.2 |
16,500.00 |
2021 Survey Vendors |
10 |
1 |
10 |
1.670 |
16.70 |
2021 TOTAL |
285 |
|
82,510 |
|
16,516.70 |
2022 Survey Respondents |
275 |
300 |
82,500 |
0.2 |
16,500.00 |
2022 Survey Vendors |
10 |
1 |
10 |
1.670 |
16.70 |
2022 TOTAL |
285 |
|
82,510 |
|
16,516.70 |
2023 Survey Respondents |
275 |
300 |
82,500 |
0.2 |
16,500.00 |
2023 Survey Vendors |
10 |
1 |
10 |
1.670 |
16.70 |
2023 TOTAL |
285 |
|
82,510 |
|
16,516.70 |
3-year TOTAL |
855 |
|
247,530 |
|
49,550.10 |
1 Total Sample = Number of Reporting Units x Completes per Reporting Unit
In 2021, the total annual burden hours for the 2021 QHP Enrollee Survey are estimated to be 16,516.70 hours. CMS estimates a total burden of 49,550.10 hours over three years.
The Bureau of Labor Statistics reported the average hourly wage for civilian workers in the United States was $28.32 as of December 2019. To estimate the burden costs for survey vendors, CMS used the average hourly wage for employees in the business and professional services sector which was $34.22 as of December 2019.7 See Exhibit A2 for estimated burden costs.
Exhibit A2. Estimated Burden Costs
Source |
Number of Respondents |
Total Burden Hours |
Average Hourly Wage Rate |
Total Cost Burden |
2021 Survey Respondents |
82,500 |
16,500.00 |
$28.32 |
$467,280.00 |
2021 Survey Vendors |
10 |
16.7 |
$34.22 |
$571.47 |
2021 TOTAL |
82,510 |
16,516.70 |
|
$467,851.47 |
2022 Survey Respondents |
82,500 |
16,500.00 |
$28.32 |
$467,280.00 |
2022 Survey Vendors |
10 |
16.7 |
$34.22 |
$571.47 |
2022 TOTAL |
82,510 |
16,516.70 |
|
$467,851.47 |
2023 Survey Respondents |
82,500 |
16,500.00 |
$28.32 |
$467,280.00 |
2023 Survey Vendors |
10 |
16.7 |
$34.22 |
$571.47 |
2023 TOTAL |
82,510 |
16,516.70 |
|
$467,851.47 |
3-Year TOTAL |
247,530 |
49,550.10 |
|
$ 1,403,554.41 |
There are no direct capital costs to respondents other than their time to participate in the survey.
The only cost to the Government of these data collections that would not otherwise have been incurred is the cost of the Booz Allen Hamilton contract, which is approximately $2.4 million for the 2021 national implementation. CMS estimates a three-year total of $7.2 million. This cost includes soliciting and approving survey vendors, developing quality assurance guidelines and technical specifications for survey vendors, providing technical assistance and training to survey vendors, conducting oversight of approved survey vendors, providing technical assistance to QHP issuers, scoring and analyzing the survey data, and development of final reports for QHP issuers.
The forecasted burden for implementing the 2021-2023 QHP Enrollee Survey has been reduced for four reasons.
First, CMS has reduced its estimate for the number of reporting units that will be required to administer the survey. In previous information collection requests, CMS estimated that 300 reporting units would be required to contract with a survey vendor to field the QHP Enrollee Survey. CMS is now estimating that 275 reporting units will be required to administer the QHP Enrollee Survey. This change reflects the reduced number of QHP issuers operating in the Exchange and the reduction of different product types offered by each issuer. There were 238 reporting units administering the 2020 QHP Enrollee Survey and CMS anticipates a similar number of reporting units in future administrations (2021 through 2023).
Second, CMS has reduced the estimate burden hours to complete the survey from 12.5 to 12 minutes. CMS collected data on the time respondents spent completing the 2019 QHP Enrollee Survey and found that it took an average of 11 minutes to complete the survey. However, with the emergence of the COVID-19 pandemic, CMS updated the survey to include two new questions and instructions related to the pandemic and the increase use of telehealth (see the Crosswalk of Changes to the QHP Enrollee Survey). Consequently, CMS updated the estimate burden hours to 12 minutes.
Third, CMS has reduced the number of survey vendors applying to become HHS-approved QHP Enrollee Survey vendors. In previous Information Collection Reviews, CMS estimated that 15 survey vendors would apply. CMS now estimates that 10 vendors will apply due to the consolidation of several survey vendors in recent years. Nine survey vendors applied to become vendors for the 2020 QHP Enrollee Survey.
Fourth, CMS has minimally increased the estimated burden hours to complete the Survey Vendor Participation Form. This increase is offset by the reductions in the estimated number of required reporting units, survey vendors, and survey burden hours mentioned above.
CMS added additional minimum business requirements that survey vendors must confirm they meet in order to receive HHS-approval. CMS added minimum business requirements for QHP Enrollee Survey vendors to align with other CMS surveys’ minimum business requirements for vendors to maintain similar standards for survey vendors across CMS. In addition, CMS added minimum business requirements based on the refinements to the email and internet survey protocols.
As a result of these changes CMS has reduced the three-year burden from 57,230 to 49,550 hours, a reduction of 7,680 burden hours.
Reporting of the QHP Enrollee Survey results will occur in the fall of 2021, following the data collection period. Reporting of the survey results will include distribution of QI Reports for each reporting unit to QHP issuers and summary reports to Exchanges. CMS also anticipates displaying the 2021 QRS global rating and three summary indicator ratings on the HealthCare.gov website for eligible QHPs and a subset of QHP Enrollee Survey questions are included in the QRS measure set. CMS also publishes updates about the survey through its Marketplace Quality Initiatives webpage.8
The expiration date and OMB control number will be displayed on the first page of the survey instrument.
There are no exceptions to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I.
1 Unless the context indicates otherwise, the term “Exchanges” refers to the Federally-facilitated Exchanges (FFEs) (inclusive of FFEs where the state performs plan management functions [FFE-SPM]) and the State-based Exchanges (SBEs) (inclusive of State-based Exchanges on the Federal Platform [SBE-FPs]).
2 Patient Protection and Affordable Care Act; Exchange and Insurance Market Standards for 2015 and Beyond, Final Rule, 79 Fed. Reg. 30240 at 30352 (May 27, 2014), 45 C.F.R. §§ 156.1120 and 156.1125.
3 45 C.F.R. §§ 156.200(b)(5),(h); 156.1120; and 156.1125.
4 45 C.F.R. §§ 155.1400 and 155.1405
5 OMB No. 0938-1185 expiration 11/30/2022
6 COVID-19 Marketplace Quality Initiatives memo available at https://www.cms.gov/files/document/covid-qrs-and-marketplace-quality-initiatives-memo-final.pdf
8 https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityInitiativesGenInfo/Health-Insurance-Exchange-Quality-Initiatives.html
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Tuchman, Hallie [USA] |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |