The Centers for Medicare & Medicaid Services (CMS) is requesting clearance from the Office of Management and Budget (OMB) to implement a national field test of the draft Outpatient and Ambulatory Surgery Experience of Care Survey to assist CMS in finalizing the survey instrument and implementation procedures and in making a decision about possible implementation of the survey on a national scale. This work will be implemented under Contract Number HHSM-500-2012-00087G.
Since 1995, the Agency for Healthcare Research and Quality (AHRQ) and its Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Consortium, in conjunction with the Centers for Medicare & Medicaid Services (CMS), have developed standardized CAHPS Surveys and tools for a variety of patient populations, including commercially insured ambulatory patients, patients whose care is covered by Medicare and Medicaid, dialysis patients, home health patients, hospital inpatients, dental patients, and patients who receive behavioral health care and services. The purpose of the CAHPS family of surveys is to collect data about patients’ assessment and rating of the care they receive from their health care provider or health care system.
In 2006, CMS began implementing the Hospital CAHPS (HCAHPS) Survey, which collects data about hospital inpatients’ rating of and experience with hospital inpatient care. CMS began publicly reporting HCAHPS Survey results on the Hospital Compare link on the Medicare.gov Web site in 2008. The HCAHPS Survey, however, includes data from samples of patients who receive inpatient hospital care. It does not include patients who received outpatient surgical care from hospital-based outpatient surgical departments (HOSDs), nor does it include patients who receive outpatient surgery from independently owned, freestanding ambulatory surgical centers (ASCs). In 2009, AHRQ reviewed and approved a Surgical Care CAHPS Survey, the development of which was sponsored by the American College of Surgeons and the Surgical Quality Alliance. However, that survey, which is applicable to patients who receive surgery in inpatient and outpatient settings, focuses on care provided by a specific surgeon, not on the care received at the facility level. An instrument measuring patient experience at the HOSD/ASC facility level would include the care, communication and consideration provided by all physicians, nurses, anesthesiologists, other health care providers and office staff that a patient might encounter during an outpatient surgery visit, as well as other topics including facility cleanliness, wait times, privacy, post-surgical care coordination and follow-up, and patient reported outcomes.
CMS began looking at Medicare beneficiaries’ experience of care with outpatient surgery via the Patient Satisfaction with Outpatient Survey: A National Survey of Medicare Beneficiaries conducted by the U.S. Office of the Inspector General in 1989. However, until recent years, the focus has been on patients’ experience with other types of health care. While some HOSDs and ASCs are conducting their own patient experience of care surveys with patients who receive outpatient surgery, to date there is no one standardized survey instrument in use to assess patient experiences with outpatient surgical care in HOSDs or ASCs.
In December 2011 there were approximately 5, 5,300 Medicare-certified ASCs and approximately 4,000 HOSDs. According to the Medicare Payment Advisory Commission’s (MedPAC’s) Report to Congress on Medicare Payment Policy1, in 2009 ASCs served more than 3.3 million Medicare fee-for-service patients. In 2006, there were 34.7 million ambulatory surgical visits, of which 19.9 million were performed in hospital-based outpatient surgery departments and 14.9 million were performed in freestanding ASCs.2 Because of advances in technology and medical care, the number of ASCs and the number of outpatient visits have increased significantly in the last two decades. As part of its focus on healthcare quality, HHS has been incorporating patient experience into its quality initiatives in different settings. ASCs and HOSDs serve millions of patients annually, and understanding patient experiences in these settings will be a part of measuring and improving the quality of services provided in these settings.
CMS believes that it will be helpful to have a single standardized survey instrument whose results can be compared across facilities, rather than having separate individual facility-sponsored surveys. The rigorous survey development process undertaken to create the finalized instrument also ensures independence, should the instrument eventually be adopted on a national level.
CMS’s contractor conducted a literature review on outpatient surgery surveys and issues that might inform the development of the survey or protocols for a national implementation. The purpose of the literature review was to obtain information about patients’ assessment of their outpatient surgery care—that is, the dimensions of domains of care that may be of interest to consumers when choosing an outpatient surgery center and any issues that may impact the development of the survey. In addition, the literature review was used to identify other issues that might affect the development of the survey, including sampling approaches, data collection, reporting, and quality improvement issues.
The contractor convened a meeting with a Technical Expert Panel (TEP) comprising individuals from industry, professional associations affiliated with the outpatient surgery industry and academia in February 2013. The purpose of the meeting was to discuss the goal of the HOSD/ASC survey and to understand how to encourage facilities to participate in the field test
On January 25, 2013, CMS published a Federal Register Notice soliciting the submission of survey domains and topic areas in the public domain measuring outpatient surgery patients’ experience of care. The notice of request for measures closed on March 26, 2013. CMS’s contractor reviewed all of the responses for their relevance for inclusion in this survey. Submitted items were entered into a comprehensive database, allowing comparisons across domains and topic areas.
We are seeking approval to conduct a field test of the Outpatient and Ambulatory Surgery Experience of Care Survey to test both the instrument and data collection procedures. The core of our analysis will be a psychometric analysis (including tests of reliability and validity) of the survey items and proposed reporting composites. (See Attachment 1 for a copy of the Outpatient and Ambulatory Surgery Experience of Care Survey questionnaire.) Survey participants will include patients that have had a recent (i.e., in the previous month) outpatient surgery. Patients that have had a recent diagnostic procedure such as a colonoscopy are also eligible. The survey questionnaire contains questions about the check-in process, facility environment, patient’s experience communicating with administrative staff (receptionists) and clinical providers (doctors and nurses), attention to comfort, pain control, provision of pre-and post-surgery care information, overall experience as well as patient characteristics.
The field test will be implemented as a quasi mixed-mode design (i.e., an initial mailed questionnaire followed by a telephone follow-up of non-respondents to the mail survey), allowing us to test procedures for both mail and telephone survey administration. This design will also allow us to identify issues that might arise as a result of combining these two modes. The field test is designed to collect data from patients who received outpatient procedures or surgery from one of the anticipated 18 HOSDs and 18 ASCs recruited to provide patient samples for this field test. We plan to complete interviews with 64 respondents from each of these 36 HOSDs and ASCs (totaling 2,304 completed interviews). We will program the Computer Assisted Telephone Interview (CATI) and develop the hard-copy questionnaire in both English and Spanish.
The survey data collection process will consist of an initial mailing of a cover letter and the questionnaire to all sampled patients; sampled patients who do not respond to the mail survey within 21 days will be assigned for telephone follow-up. (See Attachment 2 for a copy of the Outpatient and Ambulatory Surgery Experience of Care Survey cover letter. See Attachment 3 for a copy of the introductory script for the telephone follow-up.) The data collection period will be six weeks in duration, which means that telephone follow-up activities will last for three weeks.
The information collected through the field test will be used by CMS to inform the development of a public domain, standardized survey instrument and data collection procedures. Looking toward the survey development specifically, the data collected in this survey effort will be used to conduct a rigorous psychometric analysis of the survey content. The goal of such an analysis is to assess the measurement properties of the proposed instrument and sub-domain composites created from item subsets, to assure the information reported from any future administrations of the survey is well-defined. Such careful definition will prevent data distortion or misinformation if they are publicly reported. Data collection procedures will also be fine-tuned during this field test.
As indicated above, the field test will be conducted as a quasi- mixed mode design, with an initial mailed questionnaire followed by a telephone follow-up of non-respondents to the mail survey. The telephone mode will utilize a CATI system to reduce respondent burden. There are numerous advantages to administering a telephone interview using a CATI system, including the following:
costs less than in-person data collection;
allows for a shorter data collection period;
allows for less item nonresponse because the system controls the flow of the interview and complex routing;
increases data quality by allowing consistency and data range checks on respondent answers;
creates a centralization of process/quality control; and
reduces post-interview processing time and costs.
CMS is not aware of any existing standardized survey instrument where the unit of analysis is the hospital outpatient surgery department or ambulatory surgery facility, and the focus of the survey is on patient-reported experience of care. The information collected through this survey will therefore not duplicate any other effort and is not obtainable from any other source.
Survey respondents are patients who have received care from a hospital-based outpatient surgery center or independently owned ambulatory surgery center. The survey should not impact small businesses or other small entities.
This Supporting Statement requests clearance for a one-time data collection.
There are no special circumstances with this information collection request.
The 60-day Federal Register notice published on October 4, 2013 (78 FR 61848). Comments on the 60-day notice were received and we have revised the survey based on these changes. These changes include: changing “surgery center” to “facility;” revising questions and section headings to reflect that patients could have received information either before or after their procedure, and clarifying terminology describing staff at facilities. We also made some minor wording changes to further clarify questions. Below is a list of specific changes made in response to public comments:
Substituting “facility” for “surgery center”; (multiple questions);
Changing “Did anyone from the surgery center…” to “Did your doctor or anyone from the facility…” (multiple questions)
Changing “Doctors, nurses and other surgery center staff…” to “Doctors, nurses and other staff….” (multiple questions)
Change in Q 11: From “Did doctors, nurses and other surgery center staff make sure you were comfortable?” to “Did doctors, nurses and other staff make sure you were as comfortable as possible?
Change in Q 13: From “Did doctors, nurses or other surgery center staff answer all of your questions?” to “Did doctors, nurses or other staff answer your questions?”
Change in Q 22: From “Did anyone from the surgery center give you information about ways to control pain? To “Did your doctor or anyone from the facility give you information about what to do if you had pain as a result of your procedure”
Change in Q 23:Form “At any time after leaving the surgery center, did you have pain related to your procedure” to “At any time after leaving the facility, did you have pain as a result of your procedure?”
Change in Q 31: From “Signs of infection include fever, swelling, heat, drainage or redness. Before you left, did your doctor or anyone from the facility give you information about what to do if you had signs of infection?” to “Possible signs of infection include fever, swelling, heat, drainage or redness. Before you left, did your doctor or anyone from the facility give you information about what to do if you had possible signs of infection?”
Modification of the survey to reflect that information could have been communicated either before or after the procedure. This modification included:
Inserting Section III Communications About Your Procedure before Question 12;
Changing the wording of Question 12 from “Before your procedure, did you have any questions for the doctors, nurses or other surgery staff?” to “Did you have any questions for the doctors, nurses or other staff?”
Removing Section III After Your Procedure and following instructions;
Reversing the order of Questions 20 and 21;
Inserting Section IV Your Recovery after the new Question 20, and renumbering the two following section headings accordingly; and
Changing the wording of the new Question 20 from “Before you left the surgery center, did the staff prepare you for what to expect during your recovery?” to “Did your doctor or anyone from the facility prepare you for what to expect during your recovery?”
No payments or gifts will be provided to respondents.
Individuals participating in the field test will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c); 42 U.S.C. 1306, 20 CFR 401 and 4225 U.S.C.552a (Privacy Act of 1974), and OMB Circular No.A-130. The participant will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose. However, in instances where respondent identity is needed, the information collection will fully comply with all aspects of the Privacy Act.
Information collected in this survey is not considered to be of a sensitive nature. Questions in the survey are confined to respondent interactions and experiences with their outpatient surgery facility.
There is no cost to respondents other than approximately 10 minutes of their time to complete the survey. The Bureau of Labor Statistics reported that the average hourly wage for civilian workers in the United States in 2010 (U.S. Department of Labor, 2012) was $21.29. An estimate of $22 per hour allows for inflation and represents a conservative estimate of the wages of respondents. Estimated annualized burden hours and costs to the respondent for the field test are shown in Exhibits 1 and 2. The annual total cost burden is estimated to be $8,448.
The estimated 10-minute length of the survey (.17 hours) is based on timing tests. A total of 6 timing tests were conducted.
Estimated annualized burden hours and costs for the field test are shown in Exhibits 1 and 2.
Exhibit 1. Estimated Annualized Burden Hours
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Minutes per Response |
Total Burden Hours |
Outpatient and Ambulatory Surgery Experience of Care Survey (mail with telephone follow-up mode) |
2,304 |
1 |
10 |
384 |
Exhibit 2. Estimated Annualized Cost Burden
Form Name |
Number of Respondents |
Total Burden Hours |
Average Hourly Wage Rate* |
Total Cost Burden |
Outpatient and Ambulatory Surgery Experience of Care Survey (mail with telephone follow-up mode) |
2,304 |
384 |
$22.00 |
$8,448 |
* Based on average wages, “National Compensation Survey: Occupational Earnings in the United States, 2010,” U.S. Department of Labor, Bureau of Labor Statistics (http://www.bls.gov/ncs/home.htm).
The only cost is that for the time of the respondent. There is no anticipated recordkeeping burden because respondents are not required to keep a copy of the survey.
The total cost for the development and administration of the Outpatient and Ambulatory Surgery Experience of Care Survey will be $873,946 for cost of development of systems, protocols, and materials for conducting the field test data collection and analysis activities.
This is a new data collection of information. As discussed in section A.8 above, we have revised the survey based on public comments received in response to the 60-day Federal Register Notice. Since these changes were editorial and did not affect the content of the instrument, our original estimate of burden hours and costs is unchanged.
The Outpatient and Ambulatory Surgery Experience of Care Survey field test will be a one-time (cross-sectional) survey of a sample of patients 18 years old and older who received outpatient surgery or a procedure from a HOSD or ASC facility. The field test sample will consist of 4,190 outpatient surgery patients. Data collection will take place in the spring of 2014 or as soon as possible after OMB clearance is received. The data collection will take 6 weeks.
We anticipate that the analysis plan will include analyses needed to refine the survey instrument and those to support improved sampling, implementation, and data collection processes. Such analyses fall into the following fundamental categories: weighting, psychometric analysis and composite development.
(1) Psychometric Evaluation. Analyses will include evaluation of both individual items as well as composites. While composite measures for this survey cannot be identified until psychometric testing has been completed, composite measures typical CAHPS survey include composites such as Communication with Nurses, Communication with Doctors, Responsiveness of Hospital Staff, Pain Management, Communication about Medicines, and Cleanliness of Hospital Environment. Individual items will be analyzed to report item missing data and item distribution (including ceiling and floor effects) including response variance. Composite item sets will be analyzed using factor analysis and item response theory analysis to assess dimensionality, discriminability, dimensional coverage, and sub-group response biases. Internal consistency statistics will also be calculated. Finally, checks on composite validity will be performed.
(2) Field Test Estimates. Overall facility-level estimates for item and composite means with standard deviations will be provided as well. Estimates will also be provided by facility type and region.
(3) Weighting. Analyses will include the calculation of (a) Sampling weights to reflect the probability that each patient is selected for the survey; (b) nonresponse weights to reflect the probability that a sampled patient responds to the survey; and (c) poststratification weights to make the characteristics of the respondent sample more similar to the overall population.
Publication of Results: CMS may confidentially share facility-level estimates with facility administrators for quality improvement purposes. However, facility-level data from this survey will not be made publicly available to Medicare beneficiaries or the general public.
CMS does not seek this exemption.
There are no exceptions taken to item 19 of OMB Form 83-1.
1 Report to Congress, Medicare Payment Policy, Medicare Payment Advisory Commission, Chapter 5, Page 100, March 2011.
2 Ambulatory Surgery in the United States, 2006, U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Health Statistics Reports, Number 11, January 28, 2009-Revised September 4, 2009.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | cannada |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |