SUPPORTING STATEMENT
Part A
Collection of Information for
Agency for Healthcare Research and Quality’s (AHRQ)
Consumer Assessment of Healthcare Providers and Systems (CAHPS) Health Plan Survey Comparative Database
Version 11-21-2013
Agency of Healthcare Research and Quality (AHRQ)
Table of contents
A. Justification 1
1. Circumstances That Make the Collection of Information Necessary 1
2. Purpose and Use of Information 3
3. Use of Improved Information Technology 4
4. Efforts to Identify Duplication 4
5. Involvement of Small Entities 5
6. Consequences if Information Collected Less Frequently 5
7. Special Circumstances 5
8. Federal Register Notice and Outside Consultants 5
9. Payments/Gifts to Respondents 5
10. Assurance of Confidentiality 5
11. Questions of a Sensitive Nature 6
12. Estimates of Annualized Burden Hours and Costs 6
13. Estimates of Annualized Respondent Capital and Maintenance Costs 7
14. Estimates of Annualized Cost to the Government 7
16. Time Schedule, Publication and Analysis Plans 8
17. Exemption for Display of Expiration Date 8
List of Attachments 8
AHRQ’s mission. The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care; and
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
Request for information collection approval. The Agency for Healthcare Research and Quality (AHRQ) requests that the Office of Management and Budget (OMB) reapprove, under the Paperwork Reduction Act of 1995, AHRQ’s collection of information for the AHRQ Consumer Assessment of Healthcare Providers and Systems (CAHPS) Database for Health Plans: OMB Control number 0935-0165, expiration July 31, 2013. The CAHPS Health Plan Database consists of data from the AHRQ CAHPS Health Plan Survey. Health plans in the U.S. are asked to voluntarily submit data from the survey to AHRQ, through its contractor, Westat. The CAHPS Database was developed by AHRQ in 1998 in response to requests from health plans, purchasers, the Centers for Medicare and Medicaid Services (CMS) to provide comparative data to support public reporting of health plan ratings, health plan accreditation and quality improvement.
Background on the CAHPS Health Plan Survey. The CAHPS Health Plan Survey is a tool for collecting standardized information on enrollees’ experiences with health plans and their services. The development of the CAHPS Health Plan Survey began in 1995, when AHRQ awarded the first set of CAHPS grants to Harvard, RTI, and RAND. In 1997 the CAHPS 1.0 survey was released by the CAHPS Consortium. The CAHPS Consortium refers to the research organizations involved in the development, dissemination, and support of CAHPS products. The current Consortium includes AHRQ, the Centers for Medicare & Medicaid Services (CMS), RAND, Yale School of Public Health, and Westat.
Since that time, the Consortium has clarified and updated the survey instrument to reflect field test results; feedback from industry experts; reports from health plan participants, data collection vendors, and other users; and evidence from cognitive testing and focus groups. In November 2006, the CAHPS Consortium released the latest version of the instrument: the CAHPS Health Plan Survey 4.0. The development of this update to the Health Plan Survey has been part of the "Ambulatory CAHPS (A-CAHPS) Initiative," which arose as a result of extensive research conducted with users. AHRQ released the CAHPS Health Plan Survey 4.0, along with guidance on how to customize and administer it. The National Quality Forum endorsed the 4.0 version of the Health Plan Survey in July 2007.
Rationale for the information collection. The CAHPS Health Plan Database uses data from AHRQ’s standardized CAHPS Health plan survey to provide comparative results to health care purchasers, consumers, regulators and policy makers across the country. The Database also provides data for AHRQ’s annual National Healthcare Quality and National Healthcare Disparities Reports. Voluntary participants include public and private employers, State Medicaid agencies, State Children's Health Insurance Programs (SCHIP), the Centers for Medicare & Medicaid Services (CMS), and individual health plans.
Each year State Medicaid agencies, and individual health plans decide whether to participate in the database and prepare their materials and dataset for submission to the CAHPS Health Plan Database. Participating organizations are typically State Medicaid agencies with multiple health plans. However, individual health plans are also encouraged to submit their data to the CAHPS Database. The number of data submissions per registrant varies from participant to participant and year to year because some participants submit data for multiple health plans, while others may only submit survey data for one plan.
Each organization that decides to participate in the database must have their point-of-contact (POC) complete a registration form providing their contact information for access to the on-line data submission system, sign and submit a DUA, and provide health plan characteristics such as health plan name, product type, type of population surveyed, health plan state, and plan name to appear in the reporting of their results.
Each vendor that submits files on behalf of a Medicaid agency or individual health plan must also complete the registration form in order to obtain access to the on-line submission system. The vendor, on behalf of their client, may also complete additional information about survey administration (CAHPS survey version used, mode of survey administration, total enrollment count, description of how the sample was selected), submit a copy of the questionnaire used, and submit one data file per health plan. Commercial health plan data is received directly from NCQA. Medicare health plan data is received from CMS.
The collection of information for the CAHPS Database for Health Plans is being conducted pursuant to AHRQ’s statutory authority to conduct and support research on health care and systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of health care services. See 42 U.S.C. 299a(a)(1).
Survey data from the CAHPS Health Plan Database is used to produce four types of products: 1) an annual chartbook available to the public on the CAHPS Database web site (https://www.cahpsdatabase.ahrq.gov/CAHPSIDB/Public/Chartbook.aspx); 2) individual participant comparative reports that are confidential and customized for each participating organization (e.g., health plan, Medicaid agency) that submits their data; 3) a research database available to researchers wanting to conduct additional analyses; and 4) data tables provided to AHRQ for inclusion in the National Healthcare Quality and National Healthcare Disparities Reports.
Information for the CAHPS Health Plan Database has been collected by AHRQ through its contractor Westat, on an annual basis since 1998. Health plans and State Medicaid agencies are asked to voluntarily submit their data to the comparative database in June/July of each year. To help facilitate State Medicaid CHIP programs meet new requirements of CHIPRA legislation, in the future AHRQ may open a second data submission for just the State Medicaid CHIP plans that were not able to submit data in June/July.
Once data are submitted the data are cleaned with standardized programs, then aggregated and used to produce comparative results for commercial (adult and child), Medicaid (adult and child), and Medicare (adult) populations for the two most recent years. In addition, individual participant reports are produced that display the participating organizations’ own results compared to appropriate comparisons derived from the National, regional and product-type distributions on a password-protected section of the online reporting system.
The CAHPS Health Plan Database receives the data from three sources: 1) commercial health plan data is purchased by the CAHPS Health Plan Database directly from the National Committee for Quality Assurance (NCQA). The data is collected by NCQA from those who participate in their accreditation program, 2) Medicare data is provided by the Centers for Medicare and Medicaid Services (CMS) through an agency Data Use Agreement. The Medicare data is collected by CMS and their contractor from beneficiaries who were enrolled in a managed care health plan and 3) Medicaid data is collected by the CAHPS Health Plan Database. Medicaid agencies and their vendors directly submit their Medicaid health plan survey data to the CAPHS Health Plan Database through an online data submission system. Data submitted by Medicaid plans, are compiled along with the data received from CMS and NCQA to comprise the CAHPS Health Plan Survey comparative database.
AHRQ has produced CAHPS Health Plan Chartbook reports since 2000. The most recent Health Plan results reported in the online reporting system includes Medicaid, commercial and Medicare data from two years, 2010 and 2011. For 2010, 985 health plans are included covering 548,070 respondents and for 2011, 1,140 health plans are included covering 517,478 respondents. The Health Plan comparative results can be viewed in the online reporting system on the CAHPS Database web site at: https://www.cahpsdatabase.ahrq.gov/CAHPSIDB/Public/about.aspx.
Submitted data is also used to give individual participants their own customized feedback report presenting their own plan results compared to the latest comparative data results. Trend data is also available to those plans that provide data across consecutive years.
Chartbook results and individual participant reports are used for a variety of purposes:
Identify areas for quality improvement and contracting goals
Brief senior leadership
Support public reporting
Compare care across payers (commercial, Medicaid, Medicare)
Combine with other quality measures to examine outcomes
The research database has provided health plan data to 55 researchers thus far. Results of this research have been used to examine:
Disparities in CAHPS scores by racial and ethnic characteristics of respondents
Comparison of adult and child CAHPS survey results
Analysis of case-mix factors affecting CAHPS scores
Differences in CAHPS scores at the state level
Comparison of CAHPS scores between civilian and military beneficiaries
All information collection for the CAHPS Health Plan Database is done electronically, except the Data Use Agreement (DUA) that health plans sign in hard copy and fax or mail back. Registration is handled online and data submission information and data upload is handled through a secure web site. Delivery of confidential health plan survey results is also done electronically by having submitters enter a username and password to access their reports from a secure web site.
While there are many survey vendors that collect the CAHPS Health Plan Survey and may maintain databases of their own clients’ data, AHRQ is the only entity that serves as a comprehensive repository of CAHPS Health Plan survey data. AHRQ collaborates with CMS and NCQA to combine their CAHPS Health Plan survey data with the CAHPS Health Plan Database in an effort to reduce the burden of submission on health plans.
The collection of information associated with data submission is very basic and does not unduly burden small business or health plans. The information being requested has been held to the absolute minimum required for the intended uses and is likely already being compiled by survey vendors for their health plan clients.
Regulations issued by CMS and NCQA require health plans to follow a regular annual data collection and reporting cycle. As a result, the CAHPS Health Plan Database follows the same cycle for data submission. Because data are used to monitor patient experience and to evaluate specific quality initiatives, timely data is imperative to monitoring and quality improvement.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d) (2). No special circumstances apply.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on August 14th, 2013 for 60 days, and again on November 18th, 2013 for 30 days (see Attachment A). No substantive comments were received.
The CAHPS Database maintains an Advisory Committee of 16 members that meets regularly to provide guidance on the databases. The current composition of the Advisory Group is included in Attachment B.
No payment or remuneration is provided to health plans for submitting data to the CAHPS Health Plan Database.
Individuals and organizations will be assured of the confidentiality of their replies under Section 944(c) of the Public Health Service Act. 42 U.S.C. 299c-3(c). That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied.
Confidentiality of the Point-of-Contact for a Health Plan. The health plans’ POC, who submits data on behalf of their plan, is asked to provide his/her name, phone number and email address during the data submission process to ensure that the plan’s individual survey feedback report is delivered back to that person for use by the health plan. In addition, the POC’s contact information is important when any clarifications or corrections of the submitted data set are required and follow up is needed. However, the name of the POC and name of the plan is kept confidential and not publicly reported. Only aggregated, de-identified results are displayed in any reports.
Confidentiality of the Survey Data Submitted by a Health Plan. Health plans are assured of the confidentiality of their CAHPS Health Plan survey data through a DUA that they must sign that has been approved by AHRQ (see Attachment C). The DUA states that their data will be handled in a secure manner using necessary administrative, technical and physical safeguards to limit access to it and maintain its confidentiality. In addition, the DUA states the data will be used for the purposes of the database, that only aggregated results will be reported, and that the health plan will not be identified by name.
There are no questions of a sensitive nature.
Exhibit 1 shows the estimated burden hours for the respondent to participate in the database. The burden hours pertain only to the collection of Medicaid data from State Medicaid agencies and individual Medicaid health plans because those are the only entities that submit data through the data submission process (other data are obtained directly from NCQA and CMS as noted earlier in Section 2). The 80 POCs in exhibit 1 are a combination of an estimated 60 State Medicaid agencies and individual health plans, and 20 estimated vendors.
Each State Medicaid agency, health plan or vendor will register online for submission. The online Registration form (see Attachment D) will require about 5 minutes to complete. Each submitter will also complete a Health Plan information form of information about each Health Plan such as the name of the plan, the product type (e.g., HMO, PPO), the population surveyed (e.g., adult Medicaid or child Medicaid), the health plan State, total enrollment at the time the sample frame was generated, mode of survey administration (mail, telephone, IVR) and how the sample was selected. The online Health Plan Information form (see Attachment E) takes on average 30 minutes to complete per health plan with each POC completing the form for 4 plans on average. The data use agreement will be completed by the 60 participating State Medicaid agencies or individual health plans. Vendors do not sign or submit DUAs. The DUA requires about 3 minutes to sign and return by fax or mail. Each submitter will provide a copy of their questionnaire and the survey data file in the required file format. Survey data files must conform to the data file layout specifications provide by the CAHPS Database. Since the unit of analysis is at the health plan level, submitters will upload one data file per health plan. Once a data file is uploaded the file will be automatically checked to ensure it conforms to the specifications and a data file status report will be produced and made available to the submitter. Submitters will review each report and will be expected to fix any errors in their data file and resubmit if necessary. It will take about one hour to submit the data for each plan, and each POC will submit data for 4 plans on average (see Attachments F and G). The total burden is estimated to be 490 hours annually.
Exhibit 1. Estimated Annualized Burden Hours
Form Name |
Number
of respondents/ |
Number of responses per POC |
Hours per response |
Total burden hours |
Registration Form |
80 |
1 |
5/60 |
7 |
Health Plan Information Form |
80 |
4 |
30/60 |
160 |
Data Use Agreement |
60 |
1 |
3/60 |
3 |
Data Files Submission |
80 |
4 |
1 |
320 |
Total |
300 |
NA |
NA |
490 |
Exhibit 2 shows the estimated annualized cost burden based on the respondents’ time to complete one submission process. The cost burden is estimated to be $20,202 annually.
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of respondents |
Total Burden hours |
Average Hourly Wage Rate* |
Total Cost burden |
Registration Form |
80 |
7 |
47.34a |
$331 |
Health Plan Information Form |
80 |
160 |
47.34a |
$7,574 |
Data Use Agreement |
60 |
3 |
85.02b |
$255 |
Data Files Submission |
80 |
320 |
37.63c |
$12,042 |
Total |
300 |
490 |
NA |
$20,202 |
* National Compensation Survey: Occupational wages in the United States May 2012, “U.S. Department of Labor, Bureau of Labor Statistics.”
a) Based on the mean hourly wage for Medical and Health Services Managers (11-9111).
b) Based on the mean hourly wage for Chief Executives (11-1011).
c) Based on the mean hourly wages for Computer Programmer (15-1131).
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
Exhibit 3 shows the estimated annualized cost to the government for developing, maintaining and managing the Health Plan Database and analyzing the data and reporting results. The cost for one annual data submission is estimated to be $310,000. Annualized costs for collecting and processing the CAHPS Health Plan Database are based upon 14 years of historical project costs. Start-up costs were present in the early years of the database only.
Exhibit 3. Estimated Annualized Cost
Cost Component |
Total Cost |
Annualized Cost |
Database Maintenance |
$150,000 |
$50,000 |
Data Submission |
$360,000 |
$120,000 |
Data Analysis and Reporting |
$420,000 |
$140,000 |
Total |
$930,000 |
$310,000 |
Participation is expected to slightly increase during the 3 year period covered by this renewal, however, the total burden will decrease from 722 hours to 490 hours due to a more accurate estimate of the time required to submit data to the database.
Information for the CAHPS Health Plan Database is collected by AHRQ through its contractor, Westat, on an annual basis since 2000 and is planned to continue yearly through at least 2017. Health plans are asked to voluntarily submit their CAHPS Health Plan survey data to the CAHPS Database in June/July each year. The data are then cleaned and aggregated and used to produce annual chartbooks and individual participants comparative reports each Fall. Results for 2010 and 2011 are available on the Online Reporting System at https://cahpsdatabase.ahrq.gov/CAHPSORS.aspx Results for the participating Medicaid agencies and individual health plans are provided in a password protected portion of the web site.
AHRQ does not seek this exemption.
List of Attachments:
Attachment A: Federal Register Notice
Attachment B: Advisory Group List
Attachment C: Data Use Agreement
Attachment D: Registration Form
Attachment E: Health Plan Information Form
Attachment F: Screen Shots of Data Submission
Attachment G: Notification Emails
Attachment H: User Feedback Report
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |