Supporting
Statement—Part A
Supporting Statement for the Enrollee
Satisfaction Survey and Exchange Survey Data Collection
OMB Control Number: 0938-1221
June 14, 2023
Centers for Medicare & Medicaid Services
A. Background 1
B. Justification 3
3. Use of Improved Information Technology and Burden Reduction 3
4. Efforts to Identify Duplication and Use of Similar Information 4
5. Impact on Small Businesses or Other Small Entities 4
6. Consequences of Collecting the Information Less Frequently 4
9. Payments/Gifts to Respondents 5
12. Burden Estimates (Hours & Wages) 6
14. Cost to Federal Government 8
Section 1311(c)(4) of the Patient Protection and Affordable Care Act (PPACA) directs the Secretary of the Department of Health & Human Services (HHS) to establish an enrollee satisfaction survey to be administered to consumers of Qualified Health Plans (QHPs) through the Health Insurance Exchanges (Exchanges) (also known to consumers as Health Insurance Marketplaces)1. The survey must assess enrollee satisfaction with each QHP offered through the Exchanges as well as any Small Business Health Options Program (SHOP) with more than 500 enrollees in the prior year. Additionally, Section 1311(c)(3) of the PPACA directs the Secretary to develop a quality rating for each QHP offered through an Exchange.
Based on this authority, CMS issued a regulation in May 2014 to establish standards and requirements related to QHP issuer data collection and public reporting of quality rating information in every Exchange.2 As a condition of certification and participation in the Exchanges, CMS requires that QHP issuers submit QHP Enrollee Experience Survey (QHP Enrollee Survey or survey) response data and Quality Rating System (QRS) clinical data for their respective QHPs offered through an Exchange in accordance with CMS guidelines.3 Exchanges are also required to display QHP quality rating information on their respective websites.4 The QRS and QHP Enrollee Survey Technical Guidance for 2023 includes all relevant statutory and regulatory citations for the QRS and the QHP Enrollee Survey.
The QHP Enrollee Survey assesses enrollees’ experience with their QHPs around such areas as access to care, access to information, care coordination, cultural competence, doctor communication, enrollee experience with cost, prevention and plan administration. The goals of the QHP Enrollee Survey are to:
Provide comparable and useful information to consumers about the quality of health care services and enrollee experience with QHPs offered through the Exchanges,
Facilitate oversight of QHP issuer compliance with quality reporting standards set forth in the PPACA and implementing regulations, and
Provide actionable information that QHP issuers can use to improve quality and performance.
Based on the requirements for the QHP Enrollee Survey, CMS developed a survey to capture information about enrollees’ experience with QHPs offered through an Exchange. CMS conducted in-depth formative research including: a comprehensive literature review, review of existing CMS survey instruments, consumer focus groups, stakeholder discussions, and input from a Technical Expert Panel (TEP). Under Office of Management and Budget (OMB) Control Number 0938-1221, CMS performed a psychometric test and beta test in 2014 and 2015, respectively. As a result of the psychometric and beta tests, CMS identified changes in the questionnaire items, data collection procedures, and sampling specifications that obtained approval for the 2016 implementation of the survey. CMS began fielding the QHP Enrollee Survey nationwide in 2016, adding six disability status items in 2017 and removing several questions with low screen-in rates or that had been removed from the QRS in 2017 and 2018. This request is to continue nationwide collection and administration of the statutorily required survey in 2024 through 2026.
Due to the unique nature of the QHP enrollee population and its application to the QRS, CMS determined that the QHP Enrollee Survey required a customized survey instrument. The QHP Enrollee Survey includes questions from the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Health Plan 5.0 Adult Medicaid Survey, the CAHPS 5.0 Healthcare Effectiveness Data and Information Set (HEDIS®) Survey, the CAHPS Health Plan 4.0 and, and the CAHPS 5.0 Adult Supplemental Item Sets. The survey also includes additional items developed specifically for the QHP Enrollee Survey. These additional items include questions around enrollee experience with costs and customer service to capture topics not covered by existing CAHPS items and disability status items to comply with the requirements of Section 4302, Data Collection Standards, of the PPACA.
Currently, CMS proposes removing the public health emergency question, removing the flu vaccine question and revising the race and ethnicity questions to align with the 2011 HHS Data Collection Standard for the QHP Enrollee Survey 2024 administration (see the Crosswalk of Changes to the QHP Enrollee Survey). These updates were informed by stakeholder input CMS received through the TEP, public comment via the 2022 and 2023 Call Letter for the Quality Rating System (QRS) and Qualified Health Plan (QHP) Enrollee Experience Survey (2023 Call Letter), and discussions with survey vendors. CMS continually seeks stakeholder input and conducts analyses of results to inform future refinements to the survey (e.g., changes that may improve survey usability for issuers in supporting quality improvement efforts, changes to reduce burden to issuers by removing questions with high levels of missingness). CMS will continue to conduct focus groups and cognitive testing to support identification of future refinements for the QHP Enrollee Survey. If any refinements are proposed that may impact future administrations of the survey, CMS will submit a revision of this information collection request.
The QHP Enrollee Survey is conducted by HHS-approved survey vendors that meet minimum business requirements. A similar system is currently used for other CMS surveys, including Medicare CAHPS, Hospital CAHPS (HCAHPS), Home Health CAHPS (HHCAHPS), the CAHPS Survey for Accountable Care Organizations, and the Health Outcomes Survey (HOS).
Under this model, all QHP issuers that are required to conduct the QHP Enrollee Survey must contract with an HHS-approved survey vendor to collect the data and submit it to CMS on the issuer’s behalf (45 CFR § 156.1125(a)). CMS is responsible for approving and training survey vendors, providing technical assistance to vendors, overseeing vendors to ensure that they are following the data collection protocols, collecting and analyzing the data from vendors, and producing reports that QHP issuers can use for quality improvement.
At this time, CMS is seeking a routine, three-year renewal of the approval for the information collection related to the QHP Enrollee Experience Survey. These activities are necessary to ensure that CMS fulfills legislative mandates established by Section 1311(c)(4) of the Affordable Care Act to develop an “enrollee satisfaction survey system” and provide such information on Exchange websites. At this time, CMS is proposing the removal of one question about delays in care due to the public health emergency, removal of one question about the flu vaccine and the revision of the race and ethnicity questions to align with the 2011 HHS Data Collection Standard for the QHP Enrollee Survey 2024 administration. The questionnaire submitted for clearance is available in English, Spanish, and Chinese for use in a mixed-mode methodology that includes mail, telephone, and Internet survey modes. The English and Spanish questionnaire is available in all three modes of completion, while the Chinese questionnaire is available to be completed via mail and telephone.
Section 1311(c)(4) of the Affordable Care Act requires HHS to establish an enrollee satisfaction survey to be administered to members of each QHP offered through an Exchange. The QHP Enrollee Survey meets the goal of measuring enrollees’ satisfaction with their health plan. Additionally, in accordance with Section 1311(c)(4) and outlined in 45 CFR 155.205(b)(1)(iv) and 45 CFR 155.1405, the results of this survey will be available by display of the QRS information (which incorporates member experience data from the QHP Enrollee Survey) on each state Exchange’s web portal, as well as on the Federally-facilitated Exchange (FFE) web portal (HealthCare.gov), in a manner that allows applicants for coverage to compare plans.
After each QHP Enrollee Survey administration year, CMS produces Quality Improvement (QI) Reports summarizing the item-level results for each reporting unit and state participating in the QHP Enrollee Survey. These reports also include comparative benchmark data so that QHP issuers can see their results relative to the national level results.
A subset of survey questions is included in the QRS measure set and accompanying QHP quality rating information for public display. Beginning with the 2020 open enrollment period, CMS displayed the QHP quality rating information for all Exchanges that use the HealthCare.gov platform, including the FFEs (inclusive of FFE states where the state performs plan management functions and State-based Exchanges on the Federal Platform (SBE-FPs)). SBEs were required to display QHP quality ratings for the 2020 open enrollment period but had some flexibility to customize the display of the QHP quality rating information.
The data collection protocol for the QHP Enrollee Survey employs a mixed-mode methodology that combines internet, mail, and telephone (which utilizes computer-assisted telephone interviewing (CATI) technology) surveys. All sampled enrollees receive a pre-notification letter that informs them that they have been sampled for the survey and provides them with information about the survey and how the data collected will be used. The pre-notification letter contains information on completing the survey online, including the website URL and the sample member’s login credentials (which are unique to each sample member). Sampled enrollees may receive up to two reminder letters, or six telephone attempts, depending on the mode of completion. Vendors have the option of using a quick-response (QR) code to link enrollees to the online survey.
Beginning with the 2019 QHP Enrollee Survey, survey vendors were required to field the web survey in Spanish (it was previously only required in English) and optimize the web survey for mobile devices. Also beginning with the 2019 QHP Enrollee Survey, CMS implemented an email protocol. Survey vendors were required to send a notification email and two reminder emails during the fielding period. CMS will continue to evaluate methods to increase the utility of email outreach and online surveys.
Beginning in 2023, survey vendors are required to submit the final data files to CMS for analysis and scoring through a secure website which adheres to all CMS IT system requirements and includes automated data validation checks. This process ensures the data files meet established specifications. Additionally, after analysis, the survey data is submitted into the SAS Viya analytic environment which connects to CMS’ Health Insurance Oversight System (HIOS) for calculation of quality ratings based on QRS methodology for public reporting.
There is no duplication of efforts. The QHP Enrollee Survey is the only survey being conducted by HHS to measure enrollee experiences with QHPs offered through the Exchanges.
CMS does not anticipate the QHP Enrollee Survey will have an impact on small businesses. The sample frame is developed by issuers, few, if any, of which are small businesses. Some survey vendors that will apply to field the QHP Enrollee Survey may be small businesses, but conducting CMS surveys, such as the CAHPS surveys, is part of these vendors’ business model and the decision to apply for approval as a vendor for the QHP Enrollee Survey is voluntary. Furthermore, the survey vendor application process imposes a minimal burden on any applicant, including small businesses.
Annual data collection of the QHP Enrollee Survey is required to meet the objectives of providing feedback to Exchanges, issuers, and regulators for quality improvement; providing information for consumers’ choice; tracking QHP performance; and complying with applicable legislation.
There are no special circumstances associated with this data collection.
A 60-day notice published in the Federal Register on March 20, 2023 (88 FR 16634). CMS received four total comments (one comment was duplicative). Comments have been address in the attached response to comment document.
A 30-day notice published in the Federal Register on July 7, 2023 (88 FR 43355).
CMS is working with a variety of outside organizations and individuals to aid in the development and implementation of the QHP Enrollee Survey. Chief among these organizations are American Institutes for Research (AIR) and General Dynamics Information Technology, Inc. (GDIT). AIR is the contractor, and GDIT is the subcontractor; they develop, implement, and oversee the administration of the QHP Enrollee Survey.
In addition, a TEP composed of consumers, consumer advocates, health plan representatives, Exchange administrators, survey design experts, and subject matter experts provide ongoing feedback to CMS’ contractor (AIR) to inform future refinements to the QHP Enrollee Survey. The panel meets approximately twice a year to provide input on topics such as survey development and refinement; technical and methodological issues related to development; testing and fielding of the survey instrument; and survey findings.
No payments or gifts will be made to any survey respondents. Consumer members of the TEP will receive a $75 gift card as a thank you for their participation.
Individual survey respondents will be told the purposes for which the information is collected and that, in accordance with Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c), any identifiable information about them will not be used or disclosed for any purpose beyond conducting the survey. The confidentiality of individuals’ replies is further assured under 5 U.S.C.552 (Freedom of Information Act), 5 U.S.C.552a (Privacy Act of 1974), and OMB Circular No. A-130. System of Records Notice (SORN): Health Insurance Exchange Program - 78 FR 8538 Publication Date: 02/06/2013.
The QHP Enrollee Survey collects race and ethnicity data to identify health disparities. Per OMB standards, race and ethnicity information is collected separately and meets the minimum set of response categories. Beginning with the 2024 survey administration, the race and ethnicity questions align with the 2011 HHS Data Collection Standard. For example, the race question will include more specific subcategories of Asian (Asian Indian, Chinese, Filipino, Japanese, Korean, Vietnamese, Other Asian) and Native Hawaiian or Pacific Islander (Native Hawaiian, Guamanian or Chamorro, Samoan, Other Pacific Islander). Additionally, the previous ethnicity questions were combined into one question which allows the respondent to select more than one answer choice.
Estimated burden hours for the implementation and administration of the QHP Enrollee Survey in 2024 are presented in Exhibit A1. CMS has reduced the estimated total burden hours from 49,550.10 to 48,872.55 and increased the estimated total cost burden from $1,403,554.41 to $1,604,164.43 since the previous information collection request submitted in 2021. See Section 15. Changes to Burden for a detailed description of these changes. The estimated burden hours and cost burden are based on the following assumptions and definitions:
Units. CMS has established the reporting unit as the product type (i.e., Exclusive Provider Organization [EPO], Health Maintenance Organization [HMO], Preferred Provider Organization [PPO], Point of Service [POS]) offered by a QHP issuer through an Exchange in a particular state. For example, XYZ issuer’s HMOs offered through the Exchange in Florida would be considered a single reporting unit.
Depending on the way a QHP issuer packages its plan offerings, the reporting unit might include anywhere from a single QHP to many QHPs spanning all categories of coverage (i.e., bronze, silver, gold, platinum, catastrophic). QHP issuers will create a sample frame for each product type they offer through the Exchange within a particular state or reporting unit. Child-only QHPs and standalone dental plans (SADPs) are excluded from the QHP Enrollee Survey requirements.
For the 2024 survey, CMS estimates that no more than 325 reporting units will be required to administer the QHP Enrollee Survey. This estimate is primarily based on trends in the number of reporting units required to collect and report survey data along with review of several data sources used to update the list of eligible reporting units maintained by CMS. In 2021, 265 reporting units submitted survey data, in 2022 it was 297, and CMS anticipates that approximately 325 reporting units will participate in the 2023 survey data collection. Updates to the reporting unit list are based on a combination of reports provided to CMS by issuers, CMS review of enrollment data maintained by CCIIO, and other sources.
Respondents per unit. For the 2024 administration, CMS continues to propose a sample size of 1,300 enrollees per reporting unit. CMS expects to collect 300 responses per reporting unit, based on the average number of responses received by reporting units that had sufficient enrollment to produce the full sample of 1,300 enrollees in 2022.
Total respondents. CMS calculated the total number of respondents by multiplying the planned number of completed surveys (300) for each reporting unit by the planned number of reporting units.
Hours per response. Based on testing of the QHP Enrollee Survey, the survey takes on average 10 minutes for respondents to complete.
Survey vendors. Survey vendors that want to participate in collecting QHP Enrollee Survey data must complete a Survey Vendor Participation Form. CMS anticipates that approximately 5 survey vendors will apply to field the QHP Enrollee Survey annually and that it will take 100 minutes to complete the Survey Vendor Participation Form. CMS estimates the number of eligible CAHPS survey vendors at 11 based on the consolidation of several vendors in this market area and the actual number of survey vendors that applied for HHS-approval. For the 2023 survey administration, there were three survey vendor applicants.
Exhibit A1. Estimated Burden Hours for 2024-2026 Implementation of QHP Enrollee Survey
Source |
Number of Reporting Units/ Survey Vendors |
Completes per Reporting Unit |
Total Sample1 |
Burden Hours |
Total Burden Hours |
2024 Survey Respondents |
325 |
300 |
97,500 |
0.167 |
16,282.50 |
2024 Survey Vendors |
5 |
1 |
5 |
1.67 |
8.35 |
2024 TOTAL |
330 |
|
97,505 |
|
16,290.85 |
2025 Survey Respondents |
325 |
300 |
97,500 |
0.167 |
16,282.50 |
2025 Survey Vendors |
5 |
1 |
5 |
1.67 |
8.35 |
2025 TOTAL |
330 |
|
97,505 |
|
16,290.85 |
2026 Survey Respondents |
325 |
300 |
97,500 |
0.167 |
16,282.50 |
2026 Survey Vendors |
5 |
1 |
5 |
1.67 |
8.35 |
2026 TOTAL |
330 |
|
97,505 |
|
16,290.85 |
3-year TOTAL |
990 |
|
292,515 |
|
1 Total Sample = Number of Reporting Units x Completes per Reporting Unit
In 2024, the total annual burden hours for the 2024 QHP Enrollee Survey are estimated to be 16,290.85 hours. CMS estimates a total burden of 48,872.55 hours over three years.
The Bureau of Labor Statistics reported the average hourly wage for civilian workers in the United States was $32.82 as of December 2022. To estimate the burden costs for survey vendors, CMS used the average hourly wage for employees in the business and professional services sector which was $39.50 as of November 2022.5 See Exhibit A2 for estimated burden costs.
Exhibit A2. Estimated Burden Costs
Source |
Number of Respondents |
Total Burden Hours |
Average Hourly Wage Rate |
Total Cost Burden |
2024 Survey Respondents |
97,500 |
16,282.50 |
$32.82 |
$534,391.65 |
2024 Survey Vendors |
5 |
8.35 |
$39.50 |
$329.83 |
2024 TOTAL |
97,505 |
16,290.85 |
|
$534,721.48 |
2025 Survey Respondents |
97,500 |
16,282.50 |
$32.82 |
$534,391.65 |
2025 Survey Vendors |
5 |
8.35 |
$39.50 |
$329.83 |
2025 TOTAL |
97,505 |
16,290.85 |
|
$534,721.48 |
2026 Survey Respondents |
97,500 |
16,282.50 |
$32.82 |
$534,391.65 |
2026 Survey Vendors |
5 |
8.35 |
$39.50 |
$329.83 |
2026 TOTAL |
97,505 |
16,290.85 |
|
$534,721.48 |
3-Year TOTAL |
292,515 |
48,872.55 |
|
$1,604,164.43 |
There are no direct capital costs to respondents other than their time to participate in the survey.
The only cost to the Government of these data collections that would not otherwise have been incurred is the cost of the AIR contract, which is approximately $2.3 million for the 2024 national implementation. CMS estimates a three-year total of $6.7 million. This cost includes soliciting and approving survey vendors, developing quality assurance guidelines and technical specifications for survey vendors, providing technical assistance and training to survey vendors, conducting oversight of approved survey vendors, providing technical assistance to QHP issuers, scoring and analyzing the survey data, and development of final reports for QHP issuers.
The forecasted burden hours for implementing the 2024-2026 QHP Enrollee Survey has been slightly reduced, while the forecasted cost burden has been increased, for several reasons.
First, CMS has increased its estimate for the number of reporting units that will be required to administer the survey. In previous Information Collection Reviews, CMS estimated that 275 reporting units would be required to contract with a survey vendor to field the QHP Enrollee Survey. CMS is now estimating that 325 reporting units will be required to administer the QHP Enrollee Survey. This change is based on the trend in the number of reporting units described above in Section A.12. CMS anticipates this trend of increasing number of reporting units required to administer the survey will continue in future administrations (2024 through 2026). However, this increase is offset by other factors listed below, including reduced burden hours to complete the survey and reduced number of survey vendors applying to become HHS-approved QHP Enrollee Survey Vendors.
Second, CMS has reduced the estimated time to complete the survey from 12 to 10 minutes. CMS collected data on the time respondents spent completing the 2022 QHP Enrollee Survey and found that it took an average of 10 minutes to complete the survey.
Third, CMS has reduced the number of survey vendors applying to become HHS-approved QHP Enrollee Survey vendors. In previous Information Collection Reviews, CMS estimated that 10 survey vendors would apply. CMS now estimates that 5 vendors will apply due to the consolidation of several survey vendors in recent years. Three survey vendors applied to become vendors for the 2022 and 2023 QHP Enrollee Survey administration.
Fourth, CMS has not changed the estimated burden hours to complete the Survey Vendor Participation Form.
CMS estimates the same number of completes per reporting unit for the 2024-2026 QHP Enrollee Survey administrator as for the 2021-2023 administration (300).
As a result of these changes CMS has reduced the three-year burden from 54,050 to 48,872 hours, a reduction of 5,178 burden hours. Conversely, CMS has increased the three-year cost burden from $1,403,554.41 to $1,604,164.43, due to the increase in The Bureau of Labor Statistics reported average hourly wages.
.
Reporting of the QHP Enrollee Survey results will occur in the fall of 2024, following the data collection period. Reporting of the survey results will include distribution of QI Reports for each reporting unit to QHP issuers and summary reports to Exchanges. CMS also anticipates displaying the 2024 QRS global rating and three summary indicator ratings on the HealthCare.gov website for eligible QHPs and a subset of QHP Enrollee Survey questions are included in the QRS measure set. CMS also publishes updates about the survey through its Marketplace Quality Initiatives webpage.6
The expiration date and OMB control number will be displayed on the first page of the survey instrument.
There are no exceptions to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I.
1 Unless the context indicates otherwise, the term “Exchanges” refers to the Federally-facilitated Exchanges (FFEs) (inclusive of FFEs where the state performs plan management functions [FFE-SPM]) and the State-based Exchanges (SBEs) (inclusive of State-based Exchanges on the Federal Platform [SBE-FPs]).
2 Patient Protection and Affordable Care Act; Exchange and Insurance Market Standards for 2015 and Beyond, Final Rule, 79 Fed. Reg. 30240 at 30352 (May 27, 2014), 45 C.F.R. §§ 156.1120 and 156.1125.
3 45 C.F.R. §§ 156.200(b)(5),(h); 156.1120; and 156.1125.
4 45 C.F.R. §§ 155.1400 and 155.1405
6 https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityInitiativesGenInfo/ACA-MQI/ACA-MQI-Landing-Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Health Insurance Exchange Consumer Experience Surveys: Qualified Health Plan Enrollee Experience Survey - Supporting Statement P |
| Subject | Supporting Statement—Part A: Supporting Statement for the Enrollee Satisfaction Survey and Exchange Survey Data Collection |
| Author | Centers for Medicare & Medicaid Services |
| File Modified | 0000-00-00 |
| File Created | 2023-08-01 |