CMS-10500 - Supporting_Statement_B

Supporting Statement Part B:

Collection of Information Employing Statistical Methods

Consumer Assessment of Healthcare Providers and Systems Outpatient and Ambulatory

Surgery Survey (OAS CAHPS)

CMS-10500

The Centers for Medicare & Medicaid Services (CMS) is requesting clearance from the

Office of Management and Budget (OMB) to conduct the Outpatient and Ambulatory Surgery CAHPS (OAS CAHPS) Survey. The purpose of the OAS CAHPS Survey is to measure patients’ experience of care with hospital outpatient and ambulatory surgery centers in the United States. This information collection request seeks OMB approval for continued national implementation of the OAS CAHPS Survey with the addition of two new mixed modes of survey administration: web with mail follow-up and web with telephone follow-up, in addition to the three currently approved modes: mail-only, telephone-only, and mixed mode (mail with telephone follow-up). Voluntary national implementation, which began in 2016, will continue with Hospital Outpatient

Departments (HOPDs) and Ambulatory Surgery Centers (ASCs) contracting with an independent CMS-approved survey vendor to conduct the survey. The sampling plan for the national implementation of OAS CAHPS is described below.

B.1 Potential Respondent Universe and Sample Selection Method

The OAS CAHPS respondent universe is patients 18 years old and older who received an outpatient surgery or a procedure in a HOPD or ASC and who are not discharged to hospice or admitted as an inpatient in the hospital.

For the national implementation, sample selection is single-stage. ASCs and HOPDs that choose to participate work directly with a survey vendor of their choice. The vendors are responsible for selecting adequate patient sample needed to complete 300 complete surveys for each outpatient surgery facility annually.

B.1.1 Sampling Patients

For national implementation, outpatient surgery facilities assemble a sampling frame of their patients who received a surgery during the previous month. Each HOPD or ASC facility will submit a monthly file containing patient information for all eligible patients to its contracted survey vendor. To be selected for the OAS CAHPS sample, patients must meet the following eligibility requirements:

• have had an outpatient surgery or procedure from the facility in the prior calendar month;

• the surgery or procedure is included in the surgical codes which are being studied by OAS CAHPS (see B.1.2a);

• were at least 18 years old when they received their outpatient surgery or procedure;

• were not included in the sample for pre-specified period; and

• were not discharged to hospice or to a hospital for an inpatient stay.

B.1.2 Sampling Specifics

For the national implementation of OAS CAHPS, each participating facility sends to its contracted survey vendor patient sample frames containing information about each patient who received an outpatient surgery or procedure during the sample month. The survey vendor removes patients who do not meet survey eligibility requirements and then draws a random sample of the remaining patients.

Survey vendors working under contract with OAS CAHPS are instructed to use a reliable program to generate a random patient sample. CMS recommends that survey vendors use the free program RATSTATS, available from the DHHS, Office of Inspector General website, or some other validated sample selection programs such as SAS to select the sample. The recommended sampling procedure is simple random sampling, but stratified systematic sampling, disproportionate and proportional stratified sampling may be allowed subject to CMS’s approval.

A minimum of 300 completed surveys annually is the target for each participating outpatient facility. If a facility patient volume is too small to yield 300 completed surveys per year, a census is surveyed. The 300 completed surveys needed for analysis is derived from the formula for the precision of a proportion with the estimate at 0.5, the confidence interval of about +/- 0.05, and a confidence level of 95%. The number of patients needed to be selected each month to yield a minimum of 300 completed surveys per year will ultimately be determined by each facility and its survey vendor. Exhibit B-1 shows a general guide on the sample size for different mode.

Exhibit B-1. Expected Annual Sample Size to Achieve 300 Completed Surveys

Mode	Sample Size	Number of Respondents	Response Rate (%)
Mail Only	857	300	35
Telephone Only	938	300	32
Mail + Telephone Mixed Mode	750	300	40
Web + Mail Mixed Mode	770	300	39
Web + Telephone Mixed Mode	857	300	35

CMS recommends that before initiating data collection for the first time (for a client facility), the survey vendor acquires from the client facility sample frame information for 3 or 6 months prior to the first sample month of administration. These test files are used to determine an appropriate sampling rate to use for implementation. Sampling rates should be based on the number of patients who meet survey eligibility criteria in the frames of months 2 through 6. The frame of month 1 does not have any patients who are ineligible for the survey because they were not previously sampled.

B.2 Information Collection Procedures

Beginning in 2022, CMS plans to have five modes of survey administration allowed during the national implementation to give facilities options in how they would like to administer the survey, based on their goals and resources. These five modes are described below: • Mail-only Mode

• Mailing of the questionnaire and cover letter to all sampled patients.

• Second mailing of the questionnaire with a cover letter to sampled patients who do not respond to the first mailing within 3 weeks after the first questionnaire package is mailed.

• Telephone-only Mode

  • A maximum of five telephone contact attempts per sampled patient to complete the survey.

• Mixed Mode (Mail with Telephone Follow-up)

  • Mailing of the questionnaire and cover letter to all sampled patients.

  • Telephone follow-up with all sampled patients who do not respond to the questionnaire mailing. A maximum of five telephone contact attempts per sampled patient will be made to complete the survey.

• Web with Mail Follow-up Mode

  • Email invitations and send mail web survey invitation to all sampled patients

  • After two weeks, send a second email invitation and mail a second web survey invitation to all sampled patients who do not respond to the first invitation.

  • At the start of the third week, send a mail questionnaire with cover letter to all sampled patients who do not respond to the second invitation.

  • At the start of the fifth week, send a third email reminder to all sampled patients who do not respond to the mail questionnaire.

• Web with Telephone Follow-up Mode

  • Email invitations to all sampled patients.

  • After two weeks, send a second email invitation to all sampled patients who do not respond to the first invitation.

  • At the start of the third week, start telephone contact with all sampled patients who do not respond to the second invitation.

  • At the start of the fifth week, send a third email reminder to all sampled patients who do not respond to the telephone follow-up.

Data collection for each sampled patient should be initiated no later than 3 weeks after the close of the sample month. Once data collection begins, it must be completed within 6 weeks.

Survey vendors who wish to become approved to conduct the national implementation of

OAS CAHPS on behalf of outpatient facilities must complete the OAS CAHPS vendor training, which provides detailed guidance on the protocols and guidelines for all aspect of survey implementation, from sample selection to data collection and data submission.

B.3 Methods to Maximize Response Rate

To reduce nonresponse bias, every effort will be made to maximize the patient response rate while retaining the voluntary nature of the survey. RTI estimates achieving a response rate of approximately 35% for mail only, 32% for telephone only, 40% for mail with telephone mixed mode, 39% for web with mail mixed mode, and 35% for web with telephone mixed mode based on the 2019 mode experiment and voluntary national implementation data when available.

The questionnaire mailing includes a personalized cover letter containing information about the survey, including sponsorship and objectives, a description of how survey results are used, and the name and toll-free telephone number of a survey staff member that sampled patients can contact if they have questions or need additional information about the survey. Mailings also include a statement that assures patients that their survey responses will not be linked to their names or any other information that can identify them.

For the mail-only mode and web with mail follow-up, survey vendors use best practices in survey materials to enhance response rates. These best practices include using a simple font no smaller than 10-point size in the survey cover letters, allowing ample white space between questions in the questionnaire, avoiding a format that displays the questions as a matrix, using a unique sample identification number on the questionnaire rather than printing the sample member’s name, and displaying the OMB number and expiration date on the questionnaire. The second mailing for the mail mode is expected to increase the response rate, as is the telephone follow-up portion of the mixed-modes of implementation.

For the telephone-only mode and follow-up of mail or web survey nonrespondents for mixed-modes, survey vendors make up to five attempts to reach each sample patient, with those attempts varying by day of the week and time of day. Telephone interviewers are trained on how to answer questions that are most frequently asked by sample patients and to address any concerns that they may have about participating in the survey.

For the web modes, the instrument will be designed to be optimized for both desktop/laptop computers and for mobile devices. The web survey design will be Section 508compliant. Up to four e-mail and mail invitations will be sent to maximize response rates without burdening sample members with survey requests.

For the web with mail follow-up mode, a maximum of three e-mail and three mail contacts will be made.

For the web with telephone follow-up mode, a maximum of three e-mail and five telephone contacts will be made.

B.4 Tests of Procedures

During the 2019 mode experiment the following analyses were conducted:

• Analyses of individual survey items to assess missing data and item distributions;

• Hypothesis testing to detect differences in key variables between modes;

• Psychometric analysis of the composite and global measures;

• Analysis of response rates by prevalence or absence of email address for contacting sample members;

• Coverage bias analysis to determine if the web modes of administration introduced any biases compared to the previously approved modes; and

• The analysis of individual items and the hypothesis testing to form the basis for constructing an adjustor to be used for telephone and mixed-mode surveys.

RTI conducted regression analyses for key survey outcomes, including individual rating questions or composite measures, to determine the patient-mix adjustors which are necessary for reporting of the national survey results. RTI evaluated whether the ranking of outpatient facilities differed for adjusted and unadjusted results.

B.5 Statistical Consultation and Independent Review

This sampling and statistical plan was prepared by RTI International. The primary statistical design was provided by Patrick Chen of RTI International. Mr. Chen can be reached by telephone at (919) 541-6309 or by e-mail at [email protected].

REFERENCES

U.S. Bureau of Labor Statistics. (2019). May 2019 National Occupational Employment and Wage Estimates United States. U.S. Department of Labor, Bureau of Labor Statistics. Retrieved from http://www.bls.gov/oes/current/oes_nat.htm#00-0000. Based on average wages.

18