Consumer Assessment of Healthcare Providers and Systems Outpatient and Ambulatory Surgery Survey (OAS CAHPS)
Since 1995, the Agency for Healthcare Research and Quality (AHRQ) and its Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Consortium, in conjunction with CMS, have developed standardized CAHPS Surveys and tools for a variety of patient populations, including commercially insured ambulatory patients, patients whose care is covered by Medicare and Medicaid, dialysis patients, home health patients, hospital inpatients, dental patients, and patients who receive behavioral health care and services. The purpose of the CAHPS family of surveys is to collect data about patients’ assessment and rating of the care they receive from their health care provider or health care system.
CMS began looking at Medicare beneficiaries’ experience of care with outpatient surgery via the Patient Satisfaction with Outpatient Survey: A National Survey of Medicare Beneficiaries conducted by the U.S. Office of the Inspector General in 1989. However, until recent years, the focus has been on patients’ experience with other types of health care. In 2006, CMS began implementing the Hospital CAHPS (HCAHPS) Survey, which collects data about hospital inpatients’ rating of and experience with hospital inpatient care. CMS began publicly reporting HCAHPS Survey results on the Hospital Compare link on the Medicare.gov website in 2008. The HCAHPS Survey, however, includes data from samples of patients who receive inpatient hospital care. It does not include patients who received outpatient surgical care from hospital-based outpatient surgical departments (HOPDs), nor does it include patients who receive outpatient surgery from independently owned, freestanding ambulatory surgical centers (ASCs). In 2009, AHRQ reviewed and approved a Surgical Care CAHPS (S-CAHPS) Survey, the development of which was sponsored by the American College of Surgeons and the Surgical Quality Alliance. However, that survey, which is applicable to patients who receive surgery in inpatient and outpatient settings, focuses on care provided by a specific surgeon, not on the care received at the facility level.
In 2012, CMS partnered with a contractor (RTI International) to develop and test a standardized survey to measure the experiences of patients who receive outpatient surgical care from hospital-based outpatient surgical departments and independently owned ambulatory surgery centers. Prior to the pilot testing phase, significant background research was conducted, including the following steps:
Literature Review. An exhaustive literature review was performed, gathering information on outpatient surgery surveys and issues to inform the development of the survey or protocols for implementation. The purpose of the literature review was to obtain information about the dimensions of domains of care that may be of interest to consumers when choosing an outpatient surgery center and issues that would affect the development of the survey. In addition, the literature review was used to identify other issues that would affect the development of the survey, including sampling approaches, data collection, reporting, and quality improvement issues.
TEP Input. RTI convened a meeting with a Technical Expert Panel (TEP) comprising individuals from industry, professional associations affiliated with the outpatient surgery industry, and academia in February 2013. The purpose of the meeting was to discuss the goal of the survey and to understand how to encourage facilities to participate in the field test. The TEP was also given the opportunity to provide feedback on the focus and content of the survey. The team also met and communicated with the CAHPS Consortium multiple times during in the various stages of the survey development process to solicit feedback.
Federal Register Notice. On January 25, 2013, CMS published a Federal Register Notice soliciting the submission of survey domains and topic areas in the public domain measuring outpatient surgery patients’ experience of care. The notice of request for measures closed on March 26, 2013.
Review of Submitted and Existing Instruments. RTI reviewed all of the responses for their relevance for inclusion in this survey. Submitted items were entered into a comprehensive database, allowing comparisons across domains and topic areas. Other existing CAHPS Survey instruments that are publicly available were also reviewed for relevancy. Individual items from both the submitted and other existing CAHPS Surveys were examined by the team’s methodologists for possible inclusion in the OAS CAHPS draft questionnaire.
Public Comment. In early October 2013, another Federal Register notice was published seeking public comment on the draft instrument and protocol. CMS received two comments by the end of the 60-day window (in early December 2013). The 30-day notice was published in late December 2013 and did not produce any additional feedback. The additional feedback received through this process was also used to refine the survey instrument.
Field Test. As part of a previous information collection request in 2014, CMS received OMB approval to conduct a 6-week field test of OAS CAHPS to test the reliability and validity of the survey items and implementation procedures. Survey participants included patients who had a recent outpatient surgery (in May 2014) at one of the participating facilities. Patients who had a recent diagnostic procedure, such as a colonoscopy were also eligible. The survey questionnaire that was tested contained questions about the check-in process, facility environment, patient’s experience communicating with administrative staff (receptionists) and clinical providers (doctors and nurses), attention to comfort, provision of pre- and postsurgery care information, overall experience, and patient characteristics. The field test included 4,179 sampled patients from a total of 36 facilities (18 HOPDs and 18 ASCs) located across the United States.
The field test was implemented as a mixed-mode design (i.e., an initial mailed questionnaire followed by a telephone follow-up of nonrespondents to the mail survey) allowing us to test procedures for both mail and telephone survey administration (via computer-assisted telephone interview [CATI]) in English and Spanish. The data collection period spanned 6 weeks. The first phase (mail) lasted three weeks and the second phase (telephone follow-up to nonrespondents) lasted 3 weeks.
Of the 4,179 sample patients, 1,863 responded to the survey resulting in an overall (adjusted) response rate of 45.61%. Of those respondents who completed the survey, 30.4% responded by mail and 13.8% responded by telephone.
Field Test Analysis. The core of the field test analysis was a psychometric analysis (including tests of reliability and validity) of the survey items and proposed reporting composites. The goal of such an analysis was to assess the measurement properties of the proposed instrument and sub-domain composites created from item subsets, to ensure that the information reported from any future administrations of the survey was well-defined. Such careful definition will prevent data distortion or misinformation if they are publicly reported.
Final Survey. Based on the field test findings, the survey instrument was revised. Twelve questions were removed from the 49-item instrument used for the field test. The final Outpatient and Ambulatory Surgery CAHPS (OAS CAHPS) Survey has 37 items.
OAS CAHPS 2015 Mode Experiment. As part of a previous information collection request in 2015, CMS received OMB approval to conduct a randomized mode experiment with a sample of patients receiving outpatient surgeries or procedures. The mode experiment was designed to determine whether mode of administration, nonresponse, or patient-mix factors affect OAS CAHPS Survey scores.
CMS’ contractor, RTI, conducted the OAS CAHPS mode experiment as a one-time (cross-sectional) survey of patients who received care from sampled HOPDs and ASCs. A total of 70 facilities (38 HOPDs and 32 ASCs) participated in the mode experiment. From these facilities, RTI randomly selected 13,576 patients and assigned one of the three survey modes (mail-only, telephone-only, and mixed-mode). RTI constructed the sample frame for the mode experiment using patient information provided by the participating HOPDs/ASCs. To ensure that the patients selected into the sample were representative of the broad range of surgery procedures, RTI sorted the patient sampling frame by gender, age group, and procedure type, prior to selecting the patient sample.
Data collection activities for the mode experiment were conducted between September and December 2015. Data collection for each sample month began approximately 21 days after the sample month closed and ended within a 6-week period after the survey was initiated. The overall response rate (for all three modes) was 39%. The response rate for the mail-only mode was 37%, the telephone-only response rate was 34%, and the mixed-mode response rate was 50%.
The 2015 mode experiment findings produced estimates for patient-mix adjustments that were necessary to publically report comparative information. The following six patient-mix adjustors were found to be significant:
Overall Health Status
Overall Mental Health Status
Age Group
Education Attainment
Spoken English Proficiency (How Well Respondent Speaks English)
Surgery Category
The 2015 mode experiment results showed that nonresponse adjustments were not needed because the nonresponse adjusted weights did not add any explanatory power beyond that provided by the six patient-mix adjustors. The survey mode did not have significant impact on survey estimates, therefore mode effect adjustment was not needed for OAS CAHPS Survey.
Comparative results from the OAS CAHPS mode experiment were not publicly reported; the results were used to determine the need for adjustments for data collected during national implementation beginning in January 2016. Participating facilities received their own individual survey results compared to the aggregate data from the mode experiment.
Voluntary Participation for the OAS CAHPS Survey. As part of a previous information collection request in 2015, CMS received OMB approval for the National Implementation of the OAS CAHPS on a voluntary basis for HOPDs and ASCs that choose to participate. Voluntary participation began in 2016.
Non-substantive change Mode Experiment. CMS is seeking review and approval to conduct a mode experiment in 2018 to determine the impact of adding Web as a data collection mode for the national implementation of OAS CAHPS.
Synopsis of changes to requirements and burden for National Implementation. In response to public comments to the CY 2017 OPPS/ASC Final Rule (81 FR 79777), CMS has decided to continued voluntary implementation of the OAS CAHPS Survey throughout 2018. CMS continues to believe that the voluntary national implementation of the survey will provide valuable information moving forward. During the continued voluntary reporting period, CMS plans to conduct analyses of the data and plan for any necessary modifications to the survey instrument or CMS systems that would reduce the burden to patients and facilities. CMS continues to believe that the OAS CAHPS Survey measures address an area of care that is not adequately addressed in any current measure set and that the OAS CAHPS Survey will be useful to assess aspects of care where the patient is the best or only source of information. We have updated the burden estimates in section A.12 to reflect the change from required participation linked to reimbursement to continued voluntary participation in 2018.
Synopsis of changes to burden for Mode Experiment. Also in response to public comments to the CY 2017 OPPS/ASC Final Rule (82 FR 59216), CMS is proposing to test collecting data using Web surveys to explore whether hospitals and ASCs receive reliable email addresses from patients, whether there is adequate access to the internet across all types of patients and whether adding this new mode would introduce bias in the survey process due to nonresponse or mode of administration. The test is designed to compare the instrument and protocols using five mode conditions: mail-only, telephone-only, Web-only, Web with mail follow-up, and Web with telephone follow-up. We have updated the burden estimates in section A.12 to reflect the burden for facilities that voluntarily participate in the mode experiment.
The Centers for Medicare & Medicaid Services (CMS) is requesting clearance from the Office of Management and Budget (OMB) to implement nationally the Consumer Assessment of Healthcare Providers and Systems Outpatient and Ambulatory Surgery Survey (OAS CAHPS) to measure patients’ experience of care with outpatient and ambulatory surgery centers under Contract Number HHSM-500-2014-00426G. Under this information collection request, CMS is also seeking review and approval for a mode experiment that will be conducted to determine the impact of using three different data collection modes in the national implementation of OAS CAHPS.
While there is no requirement at the present to implement and report OAS CAHPS Survey data, CMS established a voluntary reporting program in which ASCs and HOPDs can choose to report their survey data to CMS and also choose whether or not to publicly report data.
In recent years, the number of ASCs has increased considerably. According to the Medicare Payment Advisory Commission’s 2014 Report to Congress on Medicare Payment Policy (MedPac, 2014), the number of Medicare-certified ASCs has grown by 12% from 2007 through 2012, increasing from 4,798 in 2007 to 5,357 by the end of 2012. That same report also shows that Medicare payments to ASCs have increased by 24% over that same time period, from $2.9 billion in 2007 to $3.6 billion in 2012. On the other hand, on average, Medicare spending on outpatient services grew from $29 billion in 2006 to $46 billion in 2011, which equates to 7.8% (MedPac, 2014). The Ambulatory Surgery Center Association (ASCA) also notes an increased volume of ASCs and HOPDs from approximately 5 million and 9.5 million (respectively) in 2005 to 6 million and 11 million (respectively) in 2010 (ASCA, 2011). Considering the growing number of ASCs and the increasing Medicare expenditures for outpatient surgical services from ASCs and HOPDs, the implementation of OAS CAHPS will provide CMS with much-needed statistically valid data from the patient perspective to inform quality improvement and comparative consumer information about specific facilities.
This OMB submission is in support of the national implementation of OAS CAHPS on a voluntary basis for HOPDs and ASCs. Starting in 2016, HOPD and ASC facilities are invited to participate in the national implementation of OAS CAHPS by contracting with a CMS-approved, independent third-party survey vendor to implement the survey on their behalf and to submit the OAS CAHPS data to CMS.
Mode Experiment
This OMB submission is also in support of an OAS CAHPS Survey mode experiment to test five mode conditions: mail-only, telephone-only, Web-only, Web with mail follow-up, and Web with telephone follow-up.
As discussed above, CMS’s contractor conducted a field test of OAS CAHPS in 2014 to test the reliability and validity of the survey items and implementation procedures. The field test was conducted in both English and Spanish. After reviewing field test results with the TEP, OAS CAHPS was finalized as a 37-item survey instrument. The final versions of the OAS CAHPS survey can be found in Attachments A (Mail Questionnaire) and B (CATI Questionnaire). Attachment D1 shows the cover letter and reminder letter for the mail survey).
The OAS CAHPS instrument contains questions about the patient’s experience with the check-in process, facility environment, communication with administrative staff (receptionists) and clinical providers (doctors and nurses), attention to comfort, pain control, provision of pre-and postsurgery care information, overall experience, and patient characteristics. The patient characteristic questions (within the “About You” section) were designed to comply with the U.S. Office of Minority Health’s requirements on data collection standards for race, sex, ethnicity, primary language, and disability status. Patients will also be asked to provide overall ratings of the outpatient surgery facility.
For national implementation, we will follow a similar model to other existing CAHPS Surveys in that we will require that facilities contract with a third-party, CMS-approved survey vendor to administer OAS CAHPS. Because data from the national implementation of OAS CAHPS will be used to produce comparative results, and because the national implementation of the survey will be conducted by multiple independent CMS- approved survey vendors, it is important that all vendors administer the survey using the same survey administration protocols and specifications. Therefore, CMS-approved survey vendors conducting OAS CAHPS on behalf of the HOPDs and ASCs will be required to use survey administration specifications developed by CMS. The HOPDs and ASCs will be able to choose a vendor to administer the survey using one of three data collection modes: mail-only, telephone-only, and mixed mode (mail survey with telephone follow-up of nonrespondents).
CMS will begin publicly reporting comparative results from OAS CAHPS after each facility has conducted data collection for 12 months. OAS CAHPS measures will enable consumers to make more informed decisions when choosing an outpatient surgery facility, aid facilities in their quality improvement efforts, and will help CMS monitor the performance of outpatient surgery facilities.
The information collected in the national implementation of OAS CAHPS will be used for the following purposes:
To provide a source of information from which selected measures can be publicly reported to beneficiaries to help them make informed decisions for outpatient surgery facility selection;
To aid facilities with their internal quality improvement efforts and external benchmarking with other facilities; and
To provide CMS with information for monitoring and public reporting purposes.
For the mode experiment, CMS plans to use information from this mode experiment to determine whether additional mode of administration (i.e. Web data collection) should be included in the current national implementation of OAS CAHPS protocols.
The national implementation of OAS CAHPS is designed to allow third-party, CMS-approved survey vendors to administer OAS CAHPS using mail-only, telephone-only, or mixed-mode (mail with telephone follow-up). Experience from previous CAHPS surveys shows that mail, telephone, and mail with telephone follow-up data collection modes work well for respondents, vendors, and health care providers. CMS is currently investigating the feasibility of implementing the OAS CAHPS Survey using a web surveys mode. If CMS determines that patient e‑mail address information is available through HOPDs and ASCs and response rates to a web survey are comparable to the traditional modes, an additional mode option may be incorporated into the current survey protocols.
The CMS-approved survey vendors who administer the survey during the national implementation will use an electronic data collection or CATI system if they administer a telephone-only or mixed-mode survey.
Mode Experiment
CMS will investigate the feasibility of implementing the OAS CAHPS Survey using Web surveys. If CMS determines that patient e‑mail address information is readily available from HOPDs and ASCs and response rates to a web survey are comparable to the traditional modes, an additional mode option may be incorporated into the current survey protocols.
As with a Computer Assisted Telephone Interview (CATI) system, which was used to administer the telephone-only mode during the mode experiment, a Web administration offers numerous advantages, including the following:
costs less than in-person data collection;
allows for a shorter data collection period;
reduces item nonresponse because the system controls the flow of the interview and complex routing;
increases data quality by allowing consistency and data range checks on respondent answers;
creates a centralization of process/quality control; and
reduces post-interview processing time and costs.
OAS CAHPS was designed to collect information that is fundamentally different from other CAHPS or patient experience of care surveys. CMS is not aware of any existing validated survey instrument where the unit of analysis is the hospital outpatient surgery department or ambulatory surgery facility, and the focus of the survey is on patient-reported experience of care. The information collected through this survey will therefore not duplicate any other effort and is not obtainable from any other source.
Many HOPDs and ASCs are already carrying out their own patient experience of care surveys. These diverse surveys do not allow for comparisons across outpatient surgical facilities. Making comparative performance information available to the public can help consumers make more informed choices when selecting an outpatient surgery facility and can create incentives for facilities to improve care they provide. OAS CAHPS will provide a standardized tool for collecting such information, comparisons across all facilities to enable consumers to compare facilities.
The current version of OAS CAHPS consists of a core set of questions followed by the mandated “About You” questions. In addition, outpatient surgery facilities may include their own facility- specific questions to the existing OAS CAHPS as long as these appear after the core survey questions.
Mode Experiment
Given that the mode experiment will run concurrently with voluntary national implementation of the OAS CAHPS, CMS will work to ensure that patients selected for the mode experiment are not included in the national implementation sample or other surveys.
Survey respondents are patients who have received care from a hospital-based outpatient surgery center or independently owned ASC. The survey should not impact small businesses or other small entities.
The national implementation of OAS CAHPS on a regular basis will allow for the collection of data about patients’ experience with outpatient surgical care at different points in time during a calendar year. Regular implementation will also allow sampled patients to assess their experience at the facility soon after their surgery or procedure is performed. Participating facilities will be asked to provide a sample frame consisting of patients who received at least one surgery or procedure during the sample month to their survey vendor on a monthly basis. Vendors will, in turn, be required to initiate the data collection from patients within 3 weeks after the sample month closes. Respondent burden is increased and the recall factor becomes a problem if patients are asked to recall their care experiences after longer lapses of time. Monthly sampling and administering the survey within 3 weeks after the close of the sample month will reduce the amount of time between outpatient care event and survey. Respondent recall will be enhanced, thus improving the quality of survey data and results. For this reason, CMS does not believe that a less frequent data collection period will result in the most accurate and complete data for public reporting and quality monitoring purposes. Although data collection will be completed by vendors on a monthly basis, data will be submitted on a quarterly basis.
There are no special circumstances with this information collection request.
The 60-day notice published in the Federal Register on April 16, 2018 (83 FR 16362). Comments were received.
For the survey development, CMS’s contractor convened a 10-member TEP and obtained guidance and input from the TEP on the sample design and survey administration specifications for both the national implementation and the 2015 mode experiment of OAS CAHPS. The TEP members consulted represented the following organizations:
Anesthesia Quality Institute;
ASC Quality Collaboration;
University of North Carolina at Chapel Hill;
Cleveland Clinic Health System;
Carilion Clinic Orthopedics;
National Center for Health Statistics;
HONOReform;
Ohio State Government;
The Joint Commission; and
Trinity Surgery Center.
No payments or gifts will be provided to respondents.
Individuals contacted as part of this data collection will be assured of the confidentiality of their replies under 42 U.S.C. 1306, 20 CFR 401 and 422, 5 U.S.C. 552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974), and OMB Circular A-130. The participant will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose. However, in instances where respondent identity is needed, the information collection will fully comply with all aspects of the Privacy Act.
Concern for the confidentiality and protection of respondents’ rights is critically important on any patient experience of care survey. Some patients might not be willing to participate in the survey for fear of retribution from the facility, Medicare, Medicaid, or other payer. There is also a concern that some patients might respond to the survey but might respond in a way that does not reflect their actual experiences with outpatient surgical care. Therefore, assurances of confidentiality are critically important with this patient population.
OAS CAHPS patients will be more willing to participate if an outside organization administers the survey. In addition, the HOPDs and ASCs will be asked not to discuss OAS CAHPS with their patients, and especially in any way that might influence the patients’ decision to participate in the survey or their responses to the survey. The cover letter included with the mail survey questionnaire sent to sample patients will encourage patients to call the survey vendor’s toll-free telephone number if they have any questions about the survey.
For the national implementation, survey vendors approved to conduct OAS CAHPS for participating facilities will be required to have systems and methods in place to protect the identity of sampled patients and the confidential nature of the data that they provide. The survey vendors will submit only de-identified survey data to RTI for analysis.
OAS CAHPS vendors will be required to include the following assurances of confidentiality in communications with sample patients:
the purposes of the survey and how survey results will be used;
participation in OAS CAHPS is voluntary;
the information they provide is protected by the Federal Privacy Act of 1974 (and that all project staff have signed affidavits of confidentiality and are prohibited by law from using survey information for anything other than this research study);
their survey responses will never be linked to their name or other identifying information;
all respondents’ survey responses will be reported in aggregate, no facility will see their individual answers;
they can skip or refuse to answer any question they do not feel comfortable with; and
their participation in the study will not affect the outpatient care or Medicare benefits they currently receive or expect to receive in the future.
During the mode experiment, HOPDs and ASCs will provide RTI with an electronic file containing information about patients who received an outpatient surgery or procedure in a designated month. The information will include personally identifiable information (name, address, gender, date of birth, email address, and telephone number). The file will also include protected health information used for survey eligibility determination and patient-mix analysis, including CPT-4 codes (indicator of types of surgeries or procedures performed), date of surgery/procedure, and facility name. Social Security numbers will not be provided.
RTI will include an assurance of confidentiality of the data to all participants. For those participating in the mail or Web survey, the cover letter will include an assurance of confidentiality. The introductory script for those interviewed by telephone will contain an assurance of confidentiality. Those patients included in the Web with mail follow-up data collection could receive the assurance of confidentiality twice—in the mail cover letter, and in the Web survey invitation should the mail interview be necessary. (Attachments A, B, and C show the mail, telephone and Web versions of the survey. Attachment D shows the survey cover letters and e-mail communications.) The three survey versions (mail, telephone and Web) are the same except that the CATI version includes an introductory script that covers the information provided in the lead letter and email, and the CATI version does not include (Q36 and Q37) that collect information on whether the patient needed help to respond to questions.
RTI understands the privacy and confidentiality concerns regarding access to OAS CAHPS data. All RTI staff members who will have access to patient information will be required to sign and abide by the terms of a nondisclosure agreement, where they agree to protect the identity of patients included in the mode experiment and the data they provide. RTI has redundant security protocols to protect data and computer systems. Servers are maintained in climate-controlled environments, with restricted access. A firewall stands between the internal systems and the Internet, requiring authentication of all users requesting access. User identification and passwords are unique and changed on a regular basis. Full backups are conducted on a weekly basis, with incremental backups performed nightly. Copies of backup materials are stored offsite in a secure location in case of system failure.
As data are collected and assembled into databases for analysis and interpretation, RTI incorporates a number of database security safeguards to protect data from accidental or intentional access and disclosure threats. RTI’s data collection and storage security measures include the following:
Maintenance of all servers in RTI’s environmentally controlled Computer Center, where computers are located in a center constructed of masonry with an automatically locking steel door that is locked at all times; fire protection is provided by a halon system with all servers having an Uninterruptible Power Supply.
User ID and password authentication to access all systems. Where appropriate, systems are configured to support the use of Digital Security Certificates for additional user authentication.
Encrypted transmission of data.
Use of Transport Layer Security, the successor technology to Secure Socket Layer for encryption of data across the Internet.
Connection to the Internet by an Internet firewall via a high-speed T2 (6.2 MBs) line. In the event of a failure, a T1 (1.544 MBs) backup will automatically provide uninterrupted Internet connectivity. Subscription to virus-protection services from McAfee VirusScan with automated update of virus signature files on all computers.
Redundant servers with automatic switchover to ensure 24/7 availability.
Daily incremental backups of all data files, with full backups created weekly.
Offsite storage of data backups.
Information collected in this survey is not considered to be of a sensitive nature. Questions in the survey are confined to respondent interactions and experiences with their outpatient surgery facility.
Wage Estimates
Individuals To derive average costs for individuals we used data from the U.S. Bureau of Labor Statistics’ May 2017 National Occupational Employment and Wage Estimates for our salary estimate (www.bls.gov/oes/current/oes_nat.htm). We believe that the burden will be addressed under All Occupations (occupation code 00-0000) at $24.34/hr since the group of individual respondents varies widely from working and nonworking individuals and by respondent age, location, years of employment, and educational attainment, etc.
Unlike our private sector adjustment to the respondent hourly wage (see below), we are not adjusting this figure for fringe benefits and overhead since the individuals’ activities would occur outside the scope of their employment.
Private Sector To derive average costs for HHAs, we used data from the U.S. Bureau of Labor Statistics’ May 2017 National Occupational Employment and Wage Estimates for all salary estimates (www.bls.gov/oes/current/oes_nat.htm). We believe that the burden will be addressed by a database administrator (occupation code 15-1141) at $42.81/hr. As indicated below we are adjusting our employee hourly wage estimate by a factor of 100 percent to $85.62/hr.
The 100 percent adjustments are rough estimates, both because fringe benefits and overhead costs vary significantly from employer to employer, and because methods of estimating these costs vary widely from study to study. Nonetheless, there is no practical alternative and we believe that doubling the hourly wage to estimate total cost is a reasonably accurate estimation method.
Information Collection Requirements and Burden Estimates
Individuals We continue to estimate that it takes approximately 8 minutes (0.13 hours) to complete the survey. Estimated annualized burden hours and costs to the respondent for the national implementation are shown in Exhibits A‑1 and A‑2. These estimates assume that 9,363 HOPDs and ASCs (the universe of Medicare-certified facilities). The estimated count of Medicare-certified ASCs is assumed to be 5,357.1 The estimated count of HOPDs is assumed to be 4,006.2 Due to the determination that participation will remain voluntary in 2018, the expectation is that 28% of the eligible HOPDs and ASCs will choose to sponsor the OAS CAHPS Survey. Thus the total number of participating HOPDs and ASCs for 2018 estimated to be 2,104 (1,124 HOPDs and 980 ASCs). Each participating facility will have 300 patients sampled that will complete the survey, thus the total number of expected respondents for voluntary national implementation is 631,200.
Mode Experiment
Estimated annualized burden to the respondent for the mode experiment are shown in Exhibits A‑1 and A‑2.
Exhibit A-1. Estimated Time (Annual)
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Hours per Response |
Total Burden Hours |
OAS CAHPS National Implementation (mail-only, telephone-only, and mixed-mode) |
631,200 |
1 |
0.13 |
82,056 |
OAS CAHPS Mode Experiment (mail-only, telephone-only, and mixed-mode) |
7,850 |
1 |
0.13 |
1,020.5 |
Total |
639,050 |
1 |
0.13 |
83,076.5 |
Exhibit A-2. Estimated Cost (Annual)
Form Name |
Number of Respondents |
Total Burden Hours |
Average Hourly Wage Rate |
Total Cost Burden |
OAS CAHPS National Implementation (mail-only, telephone-only, and mixed-mode) |
631,200 |
82,056 |
$24.34/hr |
$1,997,243 |
OAS CAHPS Mode Experiment (mail-only, telephone-only, Web-only, Web with mail, and Web with telephone ) |
7,850 |
1,020.5 |
$24.34/hr |
$24,839 |
Total |
639,050 |
83,076.5 |
$24.34/hr |
$2,022,082 |
Facilities The costs to ASC and HOPD facilities for the national implementation will be determined by the selected data collection mode (mail, telephone, or mixed mode) and by the number of sample patients included in the facility sample.
At this time, there is no mandated HOPD and ASC participation in OAS CAHPS; therefore, this survey will be administered by facilities on a voluntary basis. The burden associated with this is the time and effort put forth by the HOPDs and ASCs to submit the OAS CAHPS patient files to their approved OAS CAHPS survey vendor. Each facility participating in OAS CAHPS must prepare and submit to its survey vendor a file containing patient data on patients served the preceding month that will be used by the survey vendor to select the sample and field the survey. Preparing this file (essentially the sampling frame) for most HOPDs and ASCs can vary by the required level of effort. The data elements needed on the sample frame will be kept at a minimum to reduce the burden on all participating facilities. The burden associated with this survey administration is the time and effort put forth by the facility to prepare and submit the file containing patient data on patients. The survey instrument and procedures for completing the instrument are designed to minimize burden on all respondents. No significant burden is anticipated for small facilities beyond providing their contracted vendor with sample files of patients served. We have determined that the provision of the files will take 34 hours for each HOPD/ASC annually. The one-time start-up activities for facilities include contracting with an approved survey vendor and authorize the vendor on the OAS CAHPS website.
CMS believes that the 34 hours of labor, which the HOPD/ASC will need to provide patient records annually, can be conducted by a database administrator at $85.62/hr.
Assuming 2,104 facilities voluntarily participate in the OAS CAHPS, in Exhibit A‑3a, we have summarized the estimated time for the facilities to prepare the patient records. In Exhibit A‑3b, we have summarized the estimated cost to the facilities for preparing the patient records.
Mode Experiment
Each facility participating in the OAS CAHPS mode experiment must prepare and submit a file containing patient data on patients served the preceding month. RTI will use this file to select the sample and field the survey. Preparing this file (essentially the sampling frame) for most HOPDs and ASCs can vary by the required level of effort. The data elements needed on the sample frame will be kept at a minimum to reduce the burden on all participating facilities. The burden associated with this survey administration is the time and effort put forth by the facility to prepare and submit the file containing patient data on patients. The survey instrument and procedures for completing the instrument are designed to minimize burden on all respondents. We have determined that the provision of the files will take 8.5 hours for each HOPD/ASC.
CMS believes that the 8.5 hours of labor, which the HOPD/ASC will need to provide patient records can be conducted by a database administrator at $85.62/hr.
Assuming 70 facilities participate in the OAS CAHPS Mode Experiment, in Exhibit A‑3a, we have summarized the estimated time for facilities to prepare the patient records. In Exhibit A‑3b, we have summarized the estimated cost to prepare those records.
Exhibit A‑3a. Estimated Time for Facilities to Prepare Patient Records
Form Name |
Number of Respondents |
Number of Responses Per Respondent |
Average Hours per Response |
Total Burden Hours |
HOPDs/ASCs Patient Records for National Implementation |
2,104 |
1 |
34 |
71,536 |
HOPDs/ASCs Patient Records for Mode Experiment |
70 |
1 |
8.5 |
595 |
Total |
2,174 |
|
Varies |
72,131 |
Exhibit A‑3b. Estimated Cost for Facilities to Prepare Patient Records
Form Name |
Number of Respondents |
Total Burden Hours |
Average Cost to Contract |
Total Cost Burden |
HOPDs/ASCs Patient Records for National Implementation |
2,104 |
71,536 |
$85.62/hr |
$6,124,912 |
HOPDs/ASCs Patient Records for Mode Experiment |
70 |
595 |
$85.62/hr |
$50,944 |
Total |
2,174 |
72,131 |
$85.62/hr |
$6,175,856 |
Burden Summary
Respondent Type |
Respondents |
Total Responses (per year) |
Time per Response (hr) |
Total Time (hr) |
Labor Rate ($/hr) |
Total Cost ($) |
Patients |
639,050 |
639,050 |
0.13 |
83,076.5 |
24.34 |
2,022,082 |
Hospitals |
2,174 |
2,174 |
varies |
72,131 |
85.62 |
6,175,856 |
TOTAL |
641,224
|
641,224 |
varies |
155,208 |
varies |
8,197,938 |
Information Collection Instruments and Instruction/Guidance Documents
Attachment A – OAS CAHPS (Mail Survey)
Attachment B -- OAS CAHPS (Telephone Script)
Attachment C -- OAS CAHPS (Web Survey Screenshots)
Attachment D1 – Cover Letters
Attachment D2 – Letter Invitation for Web Survey
Attachment D3 – E-mail Invitation for Web Survey (1st Contact)
Attachment D4 – E-mail Invitation for Web Survey (Reminder)
The only cost is that for the time of the respondent. There is no anticipated recordkeeping burden because respondents are not required to keep a copy of the survey.
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. For national implementation, we have determined that there is an annual-time cost to the HOPDs/ASCs to secure the services of approved OAS CAHPS survey vendors to conduct OAS CAHPS on their behalf. In Exhibit A‑4, we have summarized the estimated cost burden to the facilities for securing the services of a survey vendor. Assuming 2,104 facilities voluntarily participate in the OAS CAHPS, at the estimated cost of $4,000 for contract costs, then the total cost is estimated to be $8,416,000.
Exhibit A‑4. Estimated Cost Burden to Facilities for Survey Vendors for National Implementation
Form Name |
Number of Respondents |
Total Burden Hours |
Average Cost to Contract |
Total Cost Burden |
HOPDs/ASCs contracting with approved OAS CAHPS Survey Vendors |
2,104 |
NA |
$4,000 |
$8,416,000 |
Total |
2,104 |
NA |
$4,000 |
$8,416,000 |
The annual cost to the government for the OAS CAHPS contractor to coordinate national implementation going forward has been updated to $1,737,341 from $1,339,243. This change reflects an updated contract year that includes additional reporting and analysis of the voluntary data in advance of public reporting. This revision is independent of the change in burden to the respondents and patients.
The total cost for the contractor to oversee the survey administration procedures is $1,737,341 over a period of 12 months. These costs include implementing the survey administration procedures; technical assistance for vendors and vendor oversight; and vendor training program including vendor approval process. Other activities include maintaining a data submission infrastructure, survey management tool, and project website; preparing public reporting formats.
At this time, there are no changes planned for the response time for the survey itself, however, due to the delay in implementing required participation linked to reimbursement, we estimate that the number of hospitals and ambulatory surgery centers that choose to voluntarily participate in 2018 will be reduced.
For the Mode Experiment, we project an increase of 3,140 responses and 409 hours (3,140 responses x 0.13 hr/response). For the National Implementation, we project a decrease of -2,177,700 responses and -283,101 hours (-2,177,700 responses x 0.13 hr/response). For Patient Records, we project a decrease of -7,289 responses and -249,611 hours (see section 12). Overall we project a decrease of -2,181,849 responses (3,140 -2,177,700 -7,289) and -532,303 hours (409 -283,101 -249,611).
OAS CAHPS is part of the CMS goal to share as much data as possible with the public about our Medicare-approved HOPDs and ASCs, by providing valid quality data to the public. Data collection for the voluntary national implementation of OAS CAHPS began January 1, 2016 for the HOPDs and ASCs that chose to participate in the survey. Sampling and data collection are conducted on a monthly basis by survey vendors working under contract with the participating HOPDs and ASCs. Patients who meet survey eligibility criteria (are 18 years or older, received an outpatient surgery or diagnostic procedure in the designated sampled month at a participating HOPD or ASC, and are discharged to home) are eligible for inclusion in OAS CAHPS.
CMS publishes results from the national implementation of OAS CAHPS after each participating outpatient surgery facility has submitted data for 12 consecutive sample months. Survey vendors submit data to CMS’ OAS CAHPS Data Center (maintained and operated by CMS’ OAS CAHPS contractor) by an established data submission deadline for each calendar year quarter. The OAS CAHPS results that are publicly reported reflect one year’s worth of data. In each quarterly data submission, we adjust the survey results patient mix. Prior to public reporting each quarter, we provide preview reports to all participating HOPDs and ASCs so that they see their own survey data that will be publicly reported on www.data.medicare.gov and they may send comments to us if something looks incorrect in the data. The results are posted on http://www.data.Medicare.gov/. The public reports show corresponding State and National averages so people can assess how the HOPDs’ and ASCs’ data compare with the State and National averaged OAS CAHPS data. In the future, we plan to post summary data on the CMS Compare site.
In order for patients to make objective and meaningful comparisons between outpatient surgery facilities, methods and adjustments must be put into place to account for significant sources of bias outside the control of the facilities. Known sources of bias include data collection mode and variability in patient-mix and response propensity across patients within outpatient surgery facilities. As part of this information collection request, CMS is requesting approval to conduct a randomized mode experiment with a sample of patients receiving outpatient surgeries or procedures to determine whether adding a Web data collection will impact patients’ responses to the survey based on data collection mode. In addition, data from the mode experiment will be used to determine which, if any, patient characteristics affect the ratings and reported experience of the healthcare encounter. If needed, during the national implementation of OAS CAHPS, CMS will develop models to statistically adjust survey results before comparative results are publicly reported. Comparative results from the OAS CAHPS mode experiment will not be publicly reported.
CMS is working with RTI to recruit approximately 70 outpatient surgery centers (35 HOPDs and 35 ASCs) to participate in the mode experiment to test a new Web mode of survey administration. The mode experiment will involve up to 23,312 patients across the five survey modes (mail-only, telephone-only, Web-only Web with mail follow-up and Web with telephone follow-up). To ensure that the patients selected into the sample are representative of the broad range of surgery procedures, we will sort the patient sampling frame by gender, age group, and procedure type, prior to selecting the patient sample.
The mode experiment will be a one-time (cross-sectional) survey of a sample of patients 18 years old and older who received outpatient surgery or a procedure from a HOPD or ASC facility during the sample month. Data collection for the mode experiment will take place in the fall of 2018 or as soon as possible after OMB clearance is received. Data collection for mail mode, telephone mode, Web mode, Web with mail mode, and Web with telephone mode conditions will be implemented concurrently for each sample month. Each monthly sample will be fielded over a 6-week data collection period. Data collection for the mode experiment will extend through 2019.
Data collection for each mode will consist of the following:
Mail-only Mode: All OAS CAHPS patients included in the mail-only sample will be sent a first package consisting of a cover letter, the questionnaire, and a pre-addressed, postage-paid return envelope. A second mailing containing a questionnaire and cover letter will be mailed to all sample patients who do not respond to the first mailing.
Telephone-only Mode: In this mode, all sample patients will be contacted via telephone by professional telephone interviewers who will be trained on OAS CAHPS administration procedures, including procedures for working with patients. Telephone interviewers will be trained on the appropriate response to common questions and concerns that patients may have about survey participation. A maximum of 5 telephone contact attempts per patient will be implemented to complete the survey.
Web-only Mode: All sampled patients included in the Web mode data collection sample will receive e-mail invitations when e-mail addresses are available, and all will receive mail invitations to the Web survey. Three additional reminders/invitations will be sent over the course of data collection.
Web with Mail Follow-up Mode: All sampled patients included in this mixed-mode data collection sample will start as the Web mode, with e-mail and mail invitations to the Web survey. A second round of invitations will be sent two weeks later. Nonrespondents to these invitations will be mailed a paper questionnaire with a cover letter, followed by a final e-mail reminder.
Web with Telephone Follow-up Mode: All sampled patients included in this mixed-mode data collection sample will start as the Web mode, with e-mail and mail invitations to the Web survey. Nonrespondents to these invitations will be contacted via telephone. Nonrespondents to the telephone contact will receive a final e-mail reminder.
The protocol for each mode and the number of weeks for each data collection activity are shown in Exhibit A‑5.
Exhibit A-5. Mode Experiment Data Collection Schedule by Mode Condition
|
Mail-only |
Telephone- only |
Web-only |
Web with Mail Follow-up |
Web with Telephone Follow-up |
Day 1 |
Mail initial questionnaire with cover letter to sample members |
Telephone interviewing for the entire data collection period |
E-mail invitation if address is available, mail Web survey invitation |
E-mail invitation if address is available, mail Web survey invitation |
E-mail invitation if address is available, mail Web survey invitation |
Day 14 |
|
|
Second e-mail invitation, second mailing of a Web survey invitation |
Second e-mail invitation, second mailing of a Web survey invitation |
Second e-mail invitation, second mailing of a Web survey invitation |
Day 21 |
Mail second questionnaire and letter to all sample members who do not respond to first mailing |
|
Third mailing of a Web survey invitation and e-mail invitation |
Mail questionnaire with cover letter to all sample members who do not respond to Web mode |
Begin telephone follow-up to all sample members who do not respond to the Web mode |
Day 28 |
|
|
Fourth mailing of a Web survey invitation |
|
|
Day 35 |
|
|
E-mail reminder with invitation |
E-mail reminder with invitation |
E-mail reminder with invitation |
Day 42 |
End |
End |
End |
End |
End |
Data from the mode experiment will be analyzed to assess the impact of nonresponse and to determine whether patients rate their outpatient surgical care differently by data collection mode. Mode experiment data will also be analyzed to determine which, if any, patient characteristics affect patients’ ratings of and reported experience with their outpatient care. CMS will use data from the mode experiment to develop models that will be used to statistically adjust survey results from the national implementation to account for factors that are beyond the control of the outpatient surgery facilities. Results from the mode experiment will not be publicly reported. CMS’ OAS CAHPS contractor will prepare a technical report describing the mode experiment results and recommendations for adjusting survey results for nonresponse, mode, and patient-mix.
Based on data collected for the mode experiment, RTI will conduct analysis to predict responses and ratings as a function of the mode used to administer the survey and, additionally, will estimate the potential effects of patient characteristics outside the control of the ASCs and HOPDs.
The regression models and variable formulations will be empirically driven. Each regression model in the mode experiment analysis will estimate a dependent variable, which will be either one of the responses or a composite of responses. The responses to the survey items of greatest interest usually have ordinal responses. Answer structures are typically yes/no, yes definitely/yes somewhat/no, or a global rating (0, 2…10). There is no constant linear or ratio relationship inherent in these responses. Two is not necessarily twice as good as 1, and 4 is not twice as good as 2. Such responses may be rigorously modeled using logit methods concerning the probability of selection. However, if the data approximate normality and are not strongly clustered at the extreme ends of the distributions, linear regression with numeric values 1–4 or 0–10 can be used successfully.
Generally, the linear forms of the estimation models will be:
Dependent variable = sum of (coefficients*patient characteristic indicators) + sum of (coefficients*mode indicators) + sum of (coefficients*hospital/ASC indicators).
When there are categorical patient characteristics, such as age groups, one group is omitted from the set of categories included in the model. That group is a reference category to which the effect of related categories is scaled. The facility variables will capture the facility–specific effects in order to isolate the effects of mode and other characteristics.
Each item under consideration for reporting will be tested for inclusion in the appropriate adjustment formula. The outcome of the process will be to determine the mode coefficients to be used as adjusters in the implementation phase and the patient characteristic variables that will be used as adjusters in that phase. The actual coefficients on patient characteristics used in the implementation adjustments will be re-estimated as part of that analysis.
We anticipate requesting data for analysis variables from three sources for the mode experiment analysis: the Facilities (ASCs and HOPDs), the survey itself, and the RTI data collection systems. Exhibit A-6 contains a list of potential variables for analysis.
Exhibit A-6. List of Potential Variables for Data Analysis
Data Variables |
From ASC or HOPD |
From Survey Respondent |
From Survey System |
Date of birth |
X |
|
|
Gender |
X |
|
|
Medical Record Number |
X |
|
|
Facility type (ASC or HOPD, single-specialty or multi-specialty) |
X |
|
|
Size of facility (patient volume) |
X |
|
|
Ownership type |
X |
|
|
Urban/Rural |
X |
|
|
Type of surgery or procedure |
X |
|
|
Overall health status |
|
X |
|
Mental health status |
|
X |
|
Language |
|
X |
|
Education level |
|
X |
|
Did someone help respondent with survey |
|
X |
|
Mode of survey administration |
|
|
X |
Interview completion date |
|
|
X |
Facility identifier |
|
|
X |
The dependent variables to be analyzed include all survey items (or will be derived from survey items). The independent variables may come from the survey as self-reported characteristics or may have another source (such as enrollment or other administrative data). Because the survey will be fielded to patients associated with all payers, there is no single source of data that can be used for the mode analysis. ASCs and HOPDs will need to provide data from the patient record to their survey vendor (during the national implementation) or to RTI (for agencies participating in the mode experiment). We recognize the importance of minimizing facility burden, and the mode experiment will be used to reduce the amount of auxiliary data collected in the national implementation.
CMS does not seek this exemption.
There are no exceptions taken to item 19 of OMB Form 83-1.
1 Medicare Payment Advisory Commission’s 2014 Report to Congress on Medicare Payment Policy (MedPac, 2014)
2 Federal Register Volume 79, Number 217 (Monday, November 10, 2014) http://www.gpo.gov/fdsys/pkg/FR-2014-11-10/html/2014-26146.htm
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | cannada |
| File Modified | 0000-00-00 |
| File Created | 2021-01-20 |