B1. Development of Concepts, Methods, and Design
As part of its Federal Employees Health Benefits (FEHB) Program Plan Performance Assessment (PPA), OPM is requiring all FEHB Carriers to utilize the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) commercial survey using a certified CAHPS vendor (more on certification below). The PPA ties FEHB Carriers profit to performance on key metrics.
The Agency for Healthcare Research and Quality (AHRQ) CAHPS surveys provide a uniform and industry accepted survey vehicle to assess member satisfaction. According to the AHRQ website (https://www.ahrq.gov/cahps/surveys-guidance/hp/index.html):
The CAHPS Health Plan Survey is a tool for collecting standardized information on enrollees' experiences with health plans and their services. It was designed to support consumers in assessing the performance of health plans and choosing the plans that best meet their needs. Health plans can also use the survey results to identify their strengths and weaknesses and target areas for improvement. Survey results can be used to:
Support consumers in assessing the performance of health plans and choosing the plans that best meet their needs.
Identify the strengths and weaknesses of health plans and target areas for improvement.
OPM is using the Adult Commercial Survey. The survey utilizes a sampling methodology as required by the survey licensee, the National Committee for Quality Assurance (NCQA), outlined in extensive detail in the HEDIS Volume 3: Specifications for Survey Measures includes surveys and protocols for the CAHPS 5.0H Survey. Information can also be found on the AHRQ website at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance/hp/fielding-the-survey-hp50-2013.pdf. Recommended data collection modes include mail, email and telephone, and protocols have been developed for each mode.
Two modes of survey administration are allowed during the national implementation to give facilities options in how they would like to administer the survey, based on their goals and resources. These two modes are described below:
Mail-only Mode
Mailing of the questionnaire and cover letter to all sampled members.
Reminder postcard mailing
Second mailing of the questionnaire and cover letter to sampled members who do not respond to the first questionnaire mailing within 5 weeks
Second reminder postcard mailing
Third mailing of the questionnaire and cover letter to sampled members who do not respond to the first or second questionnaire mailing within 3 weeks of the second questionnaire mailing.
Mixed Mode (Mail with Telephone Follow-up)
Mailing of the questionnaire and cover letter to all sampled members.
Reminder postcard mailing
Second mailing of the questionnaire and cover letter to sampled members who do not respond to the first questionnaire mailing within 5 weeks
Second reminder postcard mailing
Telephone follow-up with all sampled patients who do not respond to one of the questionnaire mailings. A maximum of six telephone contact attempts per sampled patient will be made to complete the survey.
AHRQ versions of the survey are administered by the Centers for Medicare and Medicaid Services (CMS) as part of their public reporting and reimbursement programs. The CAHPS surveys utilized by CMS have received PRA approval. CAHPS is also utilized by many private health insurance purchasers and is part of the NCQA accreditation process.
B2. Collection of Data
To collect the data, health plans contract with NCQA-Certified survey vendors to administer the CAHPS survey on their behalf. To become an NCQA-Certified survey vendor, “an organization must demonstrate that it has the capabilities, experience and expert personnel to collect and report survey results accurately. NCQA issues an annual request for proposal (RFP) for new organizations to apply for certification.”
The surveys collected and processed by NCQA-Certified survey vendors, and are submitted through the NCQA Portal, and access is restricted by NCQA. NCQA validates the submission, determines benchmarks for each measure, and provides results to health plans and purchasers. Respondents are informed about the data collection in the instructions for the survey. OPM contracts with a vendor to further analyze and validate the aggregated results, and then produces a dataset for OPM’s use.
FEHB Carriers are required to report their CAHPS results, and many report the results for their entire book of business of which FEHB is a part. Participation in the survey is entirely voluntary. The burden on participants is not considered onerous, as the estimated time to complete the survey is fifteen minutes.
The results are aggregated and used as part of the formula to determination of FEHB Plans profit factor. In addition, aggregated CAHPS survey results are provided on two OPM webpages to help inform selection of FEHB health plans. Specific Carrier scores are reported within a range.
B3. Processing and Editing of Data
Standard data cleaning and analysis procedures are performed by the NCQA Certified Vendor. The vendor is responsible for validating the results, including making sure that the sample protocol is followed, and all other requirements are followed. Only then is the data submitted to NCQA. The OPM vendor does additional validation and analysis under a separate contract with OPM. Currently, the OPM vendor is a NCQA-Certified survey vendor.
B4. Production of Estimates and Projections
The CAHPS sampling protocols are set by NCQA, and applied by the NCQA-Certified vendor for each health plan. This ensures that they are statistically valid, and comparisons of results can be made across all carriers. NCQA- Certified vendors identify any biased results, and those results are excluded from use. For a specific measure identified as biased, the results are not used in the analysis or for the calculation of profit factor. They are also not made available to consumers.
B5. Data Analysis
Basic survey analysis techniques (e.g., comparing to commercial benchmarks produced by NCQA for CAHPS, calculating frequencies, patterns, etc.) will be used to interpret and explain the data provided by respondents. The results will be used in the FEHB Plan Performance Assessment, and ranges for each FEHB Plan posted on the OPM website (https://www.opm.gov/healthcare-insurance/healthcare/plan-information/compare-plans/quality) and included in the results for the FEHB Plan Comparison Tool (https://www.opm.gov/healthcare-insurance/healthcare/plan-information/compare-plans/).
B6. Review and Evaluation Procedures
As noted above, the survey tool was developed by AHRQ and is the industry standard, used for multiple government healthcare programs including Medicare and Medicaid, by NCQA for accreditation, by states for their healthcare programs, and throughout the industry. PRA approval has been granted on numerous occasions for CAHPS in CMS programs.
B7. Data Dissemination
Aggregated reporting on the CAHPS survey results are provided on two OPM webpages to help inform selection of FEHB health plans. Specific carrier scores are reported within a range.
No PII is reported on the CAHPS results. The results are aggregated, and no individual responses are reported.
B8. Contact Person(s)
Michael Kaszynski, Senior Policy Analyst
Healthcare and Insurance
U.S. Office of Personnel Management
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Sobek, Lauren M. |
| File Modified | 0000-00-00 |
| File Created | 2022-06-20 |