SUPPORTING STATEMENT

Status: Original and derived artifacts are available for this document.

Download: docx | pdf | html




SUPPORTING STATEMENT

Part B






Collection of Information for
Agency for Healthcare Research and Quality’s (AHRQ)
Consumer Assessment of Healthcare Providers and Systems (CAHPS) 
Health Plan Survey Database




Version 5-27-2026








Agency of Healthcare Research and Quality (AHRQ)

Table of contents
Contents
B. Collections of Information Employing Statistical Methods	1
1.	Respondent Universe and Sampling Methods	1
2.	Information Collection Procedures	7
3.	Methods to Maximize Response Rates	9
4.	Tests of Procedures	11
5.	Statistical Consultants	11

B. Collections of Information Employing Statistical Methods

1.	Respondent Universe and Sampling Methods

Universe of health plans and representativeness of the data. While there are many survey vendors that collect Consumer Assessment of Healthcare Providers and Systems (CAHPS) Health Plan Survey data and may maintain a database of their own clients’ data, the Agency for Healthcare Research and Quality (AHRQ) is the only entity that serves as a comprehensive repository of CAHPS survey data. However, the CAHPS Health Plan Database is comprised of data that are voluntarily submitted by health plans that have administered the CAHPS Health Plan Survey and is not a statistically selected sample, nor is it a representative sample of all health plans in the U.S. Voluntary participants include public and private employers, State Medicaid agencies, Children's Health Insurance Programs (CHIP), the Centers for Medicare & Medicaid Services (CMS), and individual health plans. AHRQ collaborates with CMS and the National Committee for Quality Assurance (NCQA) to combine their CAHPS Health Plan Survey data with the CAHPS Health Plan Database.
The CAHPS Health Plan Survey Database is a voluntary, comprehensive database of CAHPS Health Plan Survey responses gathered directly from Medicaid plans and CMS. NCQA is the managed care plan accreditor and requires yearly CAHPS results for accreditation. Medicare conducts a yearly survey of all Medicare plans and provides the CAHPS Health Plan Database these results. Many Medicaid program health plans do not seek NCQA accreditation, but submit instead directly to the CAHPS Health Plan Database.

Because the organizations that voluntarily contribute data to the CAHPS Database are not from a statistically representative sample of all U.S. health plans, and a limited number of plans may choose to participate, the submitting organizations are not representative of all U.S. health plans or enrollee populations1. Estimates based on these voluntarily submitted data sets may produce biased estimates of the U.S. health plan and enrollee populations; it is not possible to compute estimates of precision from these data. In addition, the number and mix of sponsors contributing data vary slightly from year to year, and therefore comparisons over time should be made with these limitations and variations in mind. Comparisons of results across populations should also take into account that variations in benefit design and other factors might affect survey responses across populations.

Table B-1 presents the number of Medicaid and CHIP, and Medicare survey respondents and health plan submissions included in the CAHPS Health Plan Database for 2025. The number of health plan submissions is indicated in parentheses.


Table B-1. Number of Survey Respondents and Health Plan Submissions: 2025

Year 
Medicaid
CHIP
(Version)
Adult
Child
Child
2025 (5.1)
82,042 (257)
143,235 (291)
37,035 (96)

The 5.1 CAHPS Health Plan Survey results for the Medicaid sector were obtained from data submitted directly to the CAHPS Health Plan Database by State Medicaid agencies and individual health plans. The 2025 database consists of submissions from 51 states, of which a total of 26 Medicaid State Agencies submitted data. Medicaid results are based on survey data collected from July through June.


Table B-2 presents the number of survey respondents and health plan submissions obtained for the 5.1 results within each major sector by State, including U.S. territories and the District of Columbia.
 
Table B-2. 2025 Survey Respondents and Health Plan Submissions by State (5.1 Results)

State 
Medicaid Adult
Medicaid Child
CHIP
Alabama
340
491
504
Alaska
181
510
-
Arizona
2,454
2,535
1,958
Arkansas
502
731
669
California
11,356
16,135
519
Colorado
1413
1,777
1,036
Connecticut
446
476
437
Delaware
758
1,396
482
District Of Columbia
1,537
1,877
-
Florida
2,815 
7,017
2,662
Georgia
716
3,110
1,023
Hawaii
307
1,831
-
Idaho
228
-
982
Illinois
429
1,160
711
Indiana
2,487
2,540
1,405
Iowa
1,095
2,653
2,155
Kansas
872
1,574
1,618
Kentucky
1,378
2,905
518
Louisiana
1,557
3,847
191
Maine
-
833
-
Maryland
2,744
10,292
-
Massachusetts
2,569
2,297
400
Michigan
3,441
2,950
-
Minnesota
-
-
-
Mississippi
747
1,711
968
Missouri
1,200
3,758
1,387
Montana
171
232
227
Nebraska
709
1,439
1,161
Nevada
998
2,066
1,097
New Hampshire
1,072
1,966
-
New Jersey
1,479
4,646
2,791
New Mexico
683
1,312
-
New York
411
2,188
1,080
North Carolina
4,780
5,298
-
North Dakota
514
693
-
Ohio
4,827
6,974
-
Oklahoma
1,213
4,703
-
Oregon
5,719
4,213
2,692
Pennsylvania
5,031
3,726
3,983
Puerto Rico
1,510
2,303
-
Rhode Island
1,007
922
-
South Carolina
1,006
2,955
-
South Dakota
344
321
393
Tennessee
721
3,126
1,325
Texas
892
3,446
1,202
Utah
246
479
-
Vermont
365
488
-
Virginia
2,247
4,141
241
Washington
1,058
3,921
-
West Virginia
465
1,494
292
Wisconsin
2,476
4,909
926
Wyoming
526
868
-
Total
82,042
143,235
37,035

The CAHPS Health Plan Database currently contains 27 years of data from the CAHPS Health Plan Survey. Table B-3 shows data submissions to the CAHPS Database from 1998 to 2025. The total number of respondents is presented by population sector, with the number of health plan submissions given in parentheses.
Table B-3. Data Submissions to the CAHPS Health Plan Database From 1998-2025
Year
(CAHPS Version)
Commercial
Medicaid
CHIP
Medicare

Adult
Child
Adult
Child
Child
Adult
2025 (5.1)
N/A
N/A
82,042 
(257)
143,235 
(291)
37,035 (96)
282,553 
(513)
2024 (5.1)
N/A
N/A
69,505
(233)
111,833
(234)
20,345
(48)
296,671
(543)
2023 (5.1)
N/A
N/A
65,261
(221)
103,515
(233)
23,008
(69)
266,334
(544)
2022 (5.0/5.1)
N/A
N/A
50,336
(197)
66,182
(166)
18,592
(59)
246,189 
(513)
2021 (5.0/5.1)
N/A
N/A
49,997
(162)
86,597
(175)
17,615
(52)
233,141
(475)
2020 (5.0)
N/A
N/A
51,211
(159)
56,311
(137)
12,448
(42)
N/A
2019 (5.0)
N/A
N/A
43,588
(131)
72,429
(152)
18,090
(48)
180,568
(402)
2018 (5.0)
N/A
N/A
54,362
(146)
79,736
(150)
13,933
(25)
190,838
(388)
2017 (5.0)
N/A
N/A
65,053
(152)
103,283
(169)
15,221
(23)
194,916
(397)
2016 (5.0)
N/A
N/A
73,155
(157)
79,058
(132)
14,999
(21)
147,908
(382)
2015 (5.0)
N/A
N/A
61,369 (133)
91,049 (136)
13,466 
(19)
155,095 (431)
2014 (5.0)
N/A
N/A
68,234 (149)
60,153 (100)
11,762 
(15)
195,748 (443)
2013 (5.0)
N/A
N/A
60,249 
(124)
66,804 
(105)
9,149 
(12)
198,350 (451)
2011 (4.0)
168,341 (376)
900
(1)
73,820 (148)
85,003 (129)
26,232 
(41)
163,182 (445)
2010 (4.0)
139,156 (288)
1,474
(2)
97,626 (132)
88,694 (132)
N/A
221,120 (431)
2009 (4.0)
179,528 (405)
751
(2)
63,391 (126)
68,697 (107)
N/A
206,647 (405)
2008 (4.0)
174,307
(410)
0
 (0)
59,840
(120)
9,755
 (29)
0
 (0)
207,366
 (343)
2008 (3.0)
0
 (0)
1,882
 (4)
0
 (0)
37,347
 (64)
0
 (0)
0
 (0)
2007 (4.0)
106,811
(239)
0
 (0)
45,979
 (109)
4,647 
(16)
0 
(0)
115,910 
(296)
2007 (3.0)
0
(0)
1,659
 (4)
0
 (0)
64,039 
(103)
0 
(0)
0 
(0)
2006 (3.0)
124,585
(271)
2,400
 (7)
43,174
 (119)
50,204 
(95)
9,303 
(30)
97,955 
(273)
2005 (3.0)
123,272
(254)
2,661
(4)
32,115
(76)
40,204
(65)
1,252
(3)
127,930
(276)
2004 (3.0)
111,680
(223)
7,024
(12)
59,515
(149)
86,159
(128)
16,657
(29)
132,420
(288)
2003 (3.0)
114,063
(216)
1,866
(4)
39,275
(112)
31,081
(69)
19,061
(49)
141,421
(295)
2002 (2.0)
94,546
(219)
5,600
(10)
48,109
(136)
60,534
(122)
18,910
(43)
153,172
(321)
2001 (2.0)
165,500
(266)
9,913
(24)
45,127
(142)
36,940
(124)
0
(0)
179,451
(381)
2000 (2.0)
135,479
(270)
2,760
(8)
49,327
(156)
41,400
(140)
0
(0)
166,072
(367)
1999 (2.0)
168,234
(307)
42,879
(149)
28,420
(77)
14,106
(66)
0
(0)
0
(0)
1998 (1.0)
34,965
(54)
0
(0)
23,519
(31)
9,871
(33)
0
(0)
0
(0)
TOTALS
1,840,467
81,769
1,286,791
1,490,283
236,690
3,655,399

Most of the CAHPS Health Plan Survey questions ask respondents to report on their experiences with different aspects of their care. These reporting questions are combined into groups that address the same aspect of care or service to arrive at a broader assessment. The 5.1 version of the CAHPS Adult and Child Health Plan Surveys reporting questions fall into four major “composites” that summarize consumer experiences in the following areas: 1) Getting needed care, 2) Getting care quickly, 3) How well doctors communicate, and 4) Health plan information & customer service.

The CAHPS Health Plan Survey collects four separate global ratings to distinguish between important aspects of care. The four questions ask plan enrollees to rate their experiences in the past 6 months with: 1) their personal doctor, 2) the specialist they saw most often, 3) health care received from all doctors and, 4) their health plan. Ratings are scored on a 0 to 10 scale, where 0 is the “worst possible” and 10 is the “best possible.” The ratings are analyzed and presented in the three-part bar chart display used in the CAHPS Health Plan Survey reports: the percentage of respondents who gave a rating of either 0-6, 7-8, or 9-10. This three-part scale is used because testing by the CAHPS team determined that these cut-points improve the ability to discriminate among plans while simplifying the presentation of results.

Case-Mix Adjustment. Case-mix refers to the respondents’ health status and other socio-demographic characteristics that have been shown to affect enrollee reports and ratings of health plans. Characteristics used to case-mix adjust CAHPS Health Plan scores, where applicable, are respondent age, education, and self-reported physical and mental health status.

Testing for Statistical Differences. The individual participant’s comparative reports test for statistically significant differences between top box  scores for composite measures, items and ratings for individual health plans and the top box scores of all plans’ mean top box scores in the CAHPS Health Plan Database using the t-test. A significance level of 0.05 or less is considered statistically significant. As described in the previous sections, the top box  scores are adjusted for case-mix differences before the statistical tests are applied.  Top box scores are based on the most positive response for the items and composite measures (“Always”) and for the top two responses for the rating items (“9”, “10”). Individual plan results that differ significantly from the overall  are denoted by arrows, either pointing up (significantly higher than the overall mean top box score) or down (significantly lower than the overall mean top box score).

2.	Information Collection Procedures
Information collection for the AHRQ CAHPS Health Plan Survey Database occurs in a regular data collection cycle each year in June/July. The information collection procedure for submitting and processing data for the database is shown in Figure B-1. Each of the steps is described below. Screen shots of each step are provided in Attachment E.
Figure B-1. CAHPS Health Plan Database Data Submission Process






Step 1: Call for Data Submission. State Medicaid agencies and health plans that have administered the CAHPS Health Plan survey are recruited through multiple outlets and asked to submit to the database. The call for data submission is done through various publicity sources such as AHRQ’s electronic newsletters, GovDelivery messages, and communication with prior year’s participants. Organizations that have administered the Health Plan Survey and are interested in participating in the CAHPS Health Plan Database communicate with Westat through a dedicated email address ([email protected]) that routes directly to Westat and a toll-free phone number (1-888-808-7108) to indicate their interest in participation.

Step 2: Complete Registration Form. A database submission extranet has been set up so that interested parties such as, state Medicaid agencies, coalitions, vendors, and health plans register for the data submission process. Information about eligibility requirements, benefits of participation, data use agreement, and data file specifications regarding how to prepare their data file for inclusion in the CAHPS Health Plan Database are posted on the extranet. The data file specifications ensure that data files received from users are standardized and consistent in the way variables are coded and formatted. Potential participants’ online registration information will be reviewed by Westat staff. Upon approval of the registrant, an automated email that contains a username and password to the data submission extranet is sent to the registered user. The automatic email informs registrants of the information needed in the next steps of the data submission process.

Step 3: Complete Health Plan Information Form. This step requires each health plan that administered the CAHPS Health Plan survey, submit the requested characteristics including such details as the name of the plan, product type (e.g., HMO, PPO), the population surveyed (e.g., adult Medicaid, child Medicaid) and plan state.

Step 4: Upload Data Use Agreement. To protect the confidentiality of all respondents and entities that are included in any CAHPS Database, all participating institutions must sign a data use agreement (DUA) that has been reviewed and approved by AHRQ. The data use agreement specifies how the submitted data will be used, provides assurance that the identity of the participating institution will be protected and ensures the confidentiality of the data. Data are not included in the database without this signed data use agreement. Users must upload a copy of the signed agreement. Data collection vendors may not sign and submit the DUA on behalf of an institution (even if they have been given permission by the entity to handle the actual submission of data). Only a duly appointed representative may sign the DUA.

Step 5: Upload Questionnaire. Each health plan must upload a copy of the questionnaire used. The CAHPS Database reviews the questionnaire to ensure that it meets CAHPS Health Plan Survey standards (the survey instrument must include all core questions, not alter the wording of any core questions, and must not omit any of the survey items related to respondent characteristics that are used for case mix adjustment.) Once the questionnaire is reviewed by CAHPS Database staff, an email notification is sent to the registrant within three business days with an approval or rejection. Only health plans that receive questionnaire approval may submit data files.

Step 6: Secure Online Data Submission. To enable participants to transmit their CAHPS survey data to Westat in a secure manner, an online data submission extranet has been developed. The online system will be expanded and adapted to include data submission for all CAHPS surveys. Data are accepted in csv format. Data files must conform to the Data File Layout Specifications provide by the CAHPS Database. Since the unit of analysis is at the health plan level, users must upload one data file per health plan.

Data File Approval. Once a data file is successfully uploaded, a separate load program developed in Visual Basic (VB) reads the submitted files and loads them into the SQL database that stores the data. Upon submission, a data file status report is produced and made available to the participant. This report displays item frequencies and flags out-of-range values. If there are any out of range values or problems with the data file the submitter may review the Data File Status Report for further detail. Participants are expected to fix any errors and resubmit their data file(s) for processing. If the data have been properly received, a CAHPS Database staff member then reviews the report to conduct data quality checks. If any data problems are discovered, users will be notified immediately along with a description of the problem. If there are no problems with the data file the CAHPS Database staff review all aspects of the submission for an account final approval status and an email will be sent to the participant contact via the database submission extranet indicating their data will be included in the CAHPS Health Plan Database.

Step 7: Account Status. Participants have the opportunity to check the status of their account at any time during the submission process. Only accounts that receive the CAHPS Database Final Approval status will be included in the CAHPS Health Plan Database.

3.	Methods to Maximize Response Rates

AHRQ promotes the voluntary participation in the CAHPS Health Plan Survey Database using several methods to different target audiences. We continually conduct general marketing through existing CAHPS channels and targeted outreach to existing and previous health plan participants. The CAHPS Database staff also contact national quality initiatives to promote the Database and have sought data partners that result in the yearly data contributions from NCQA and CMS.


Ongoing general marketing includes:

    • CAHPS Database Web Site: Announcements regarding data submission and reporting timetables;
    • CAHPS Database Annual Report and Related Press Release(s): Announcements for release of Annual Report that includes CAHPS Database contact information for plans and purchasers interested in participating; and
    • AHRQ Web Site: Brief summary of CAHPS Database products and benefits of participation, with link to the CAHPS Database website.
    • GovDelivery Messages: Send GovDelivery messages to participants who signed up for CAHPS News and Events.

In addition to the direct contact of health plans themselves, the CAHPS Database staff contact many national leaders and programs and direct them to the annual chartbook and references to the AHRQ National Healthcare Quality and National Healthcare Disparities Reports. These organizations and programs often cite and use CAHPS Database information. These include:

National Quality Initiatives
National Forum on Health Care Quality Measurement and Reporting (board members)
Quality Interagency Coordinating Committee (e.g., federal agencies such as HHS, Labor, Defense, Veterans Affairs, Federal Trade Commission, etc.)
Institute of Medicine Quality of Health Care in America Project

Federal and State Health Policy Leaders
Appropriate Federal and State Agency Administrators (including public health)
Federal and State Congressional Staffs
National Governors Association
National Conference of State Legislatures
National Association of Health Data Organizations
National Association for State Health Policy
State Medicaid Directors Association

Consumer Advocacy Groups
American Association of Retired Persons
Consumer Coalition for Quality Health Care
Families USA
Family Voices

Business Leaders on Health 
National Business Coalition on Health
Managed Health Care Association
Leapfrog Group
Washington Business Group on Health
Midwest Business Group on Health
National Health Care Purchasers Institute

Health Care Industry Leaders
American Association of Health Plans
Health Insurance Association of America
American Health Quality Association
National Association of Insurance Commissioners

CAHPS Users and Researchers

CAHPS User Network
CAHPS Consortium


Health Policy and Health Services Researchers
Grant Makers in Health
Academy for Health Services Research and Health Policy

4.	Tests of Procedures

The CAHPS Database staff talks with submitters about their experience and use their feedback to improve the collection process.
5.	Statistical Consultants
Joann Sorra, PhD
Westat
1600 Research Blvd.
Rockville, MD 20850
[email protected]

Naomi Yount, PhD
Westat
1600 Research Blvd.
Rockville, MD 20850
[email protected]