OMB Supporting Statement B rev 4-25-13 CLEAN

Supporting Statement B for Emergency Room Patient Experiences with Care Survey

Contract Number: HHSM-500-2012-00059G

January 15, 2013

Prepared for CMS

Suzanne Rotwein, Project Officer

RAND Corporation

1776 Main Street

P.O. Box 2138

Santa Monica, CA 90407-2138

TABLE OF ContentS

B. Collection of Information Employing Statistical Methods 4

B1. Respondent Universe and Respondent Selection 4

B2. Data Collection Procedures 7

B3. Response Rates and Non-Response 8

B4. Tests of Procedures or Methods 8

B5. Statistical and Data Collection Consultants 8

ATTACHMENTS 9

SUPPORTING STATEMENT

Emergency department patient experiences with care survey

B. Collection of Information Employing Statistical Methods

B1. Respondent Universe and Respondent Selection

Field test data collection will occur in 2013.

Selecting and Recruiting Emergency Departments

While freestanding emergency departments and urgent care centers have become more common in recent years, they tend to see a lower-acuity population (with fewer ambulance transports and admissions), provide different services, and have faster patient throughput – all of which could affect patient experiences with care and therefore require different survey approaches. As a result, we are planning a fully hospital-based sample.

We plan to sample 12 hospitals, targeting 300 completed surveys in each hospital. In order to enable an examination of both mode effect and comparison of stand-alone vs. supplemental administration, each hospital emergency room included in the sample must see both (a) at least 160 patients per month who are admitted to the hospital through the emergency room and (b) at least 240 patients per month who are discharged to the community. With approximately 13 percent of all emergency room visits resulting in an admission, this will eliminate the smallest one-third of hospitals in the U.S. from the potential sample. Smaller hospitals cannot be accommodated while maintaining the field period CMS requires, since adequate time is available to sample from only three months of emergency room visits in each hospital. Hospitals will be selected based on size (3 groups: medium (14,000-24,999 emergency room visits per year), large (25,000-49,999) and extra-large (50,000+ visits per year)) and geographic region (4 groups: northeast, south, midwest and west), recruiting 2 to 3 hospitals in each category to ensure that we will have one hospital participating in the field test in each of the twelve resulting categories (large-northeast, large-south, etc., with two to three hospitals recruited per category). These size requirements will ensure a mix of academic and community hospitals, as well as emergency rooms with varying levels of acuity and trauma designations (i.e., since extra-large hospitals are disproportionately likely to be academic Level I trauma centers with high acuity).

We will recruit two to three hospitals per category to result in field test participation by 1 hospital per category. Employing the model used to recruit hospitals for participation prior Hospital CAHPS data collection, we will conduct initial outreach to hospitals to secure an initial agreement of participation. Once hospitals have initially agreed to participation, CMS’ contractor will follow-up to discuss details of data transmission requirements and secure fully executed business associate agreements/data use agreements.

Sampling Emergency Department Visits

In each hospital, we will sample 1200 emergency room patients over the course of a three-month period, including an oversample of those admitted to the hospital. Patients who are sampled for one of their emergency room visits during the field test will not be eligible for sampling based on additional emergency room visits during the field test period. The sample will include 480 admitted patients and 720 discharged to the community. Given that admitted patients are expected to constitute 13% of all eligible patients, they would be sampled at approximate 4.5 times the rate as patents discharged into the community [(480/.13)/(720/.87)].

In each group, we assume 25 percent of patients will be deemed ineligible (see eligibility criteria described below), resulting in 360 eligible admitted and 540 eligible discharged patients.

Half of those admitted (n=180, randomly selected) will receive a stand-alone survey, divided into three equal randomly selected groups receiving mail only, telephone only, or combined mail/telephone survey administration (groups A, B, C); the other half will receive the survey as a supplement to HCAHPS with the same random division into three modes (groups D, E, F). Those discharged to the community will be randomized into three equal groups with the same division into three modes (groups G, H, I). Assuming a 33 percent response rate (typical for HCAHPS), this will result in 20 completes per hospital and 240 completes in the full field test in each of the groups A through F, and 60 completes per hospital/720 in the full field test in each of the groups G through I. This is a target number per hospital of 180 completed surveys from those discharged to the community and 120 completed surveys from those admitted to the hospital. With 12 hospitals, this is a total target of 2,160 completed surveys from those discharged to the community and 1,440 completed surveys from those admitted to the hospital.

The proposed design will allow 80% power to detect differences of 5.5 to 7.1 percent in response rate when comparing survey modes, survey forms, or patients admitted vs. those discharged to the community in 2-sided tests, alpha=0.05 (see Table 1). Similarly, when comparing responses to CAHPS items, we can detect small-to-very-small differences (Cohen’s d=0.11 to 0.15 SD); differences smaller than those shown in Table 1 are likely too small to be of substantive interest or to warrant changes in survey design.

Table 1. Power Analysis of Proposed Sample Design

Comparison	Detectable difference in response rate	Detectable difference in CAHPS measure (standard deviations)
Stand-alone arm vs. HCAHPS supplement arm within the admitted group	7.1 percent	0.15
Discharged to community arm vs. one admitted arm (either stand-alone or Hospital CAHPS supplement)	5.6 percent	0.12
Any two modes of survey administration	5.5 percent	0.11

Determining Patient Eligibility

Patient eligibility criteria have been determined in consultation with CMS and with input from the Technical Expert Panel, and are largely based on Hospital CAHPS eligibility criteria. In general, all adult patients are eligible for inclusion in the sampling universe, with the exception of the following ineligible groups:

• Patients under the age of 18

• Patients with a primary mental health or substance use diagnosis

• Patients who were discharged to hospice care, nursing homes, and skilled nursing facilities

• Patients who were transferred to another hospital

• Patients who died in the ED or who were admitted to the hospital from the ED and died during the inpatient stay

• Patients who request that they not be contacted (those who sign “no publicity” requests while hospitalize or otherwise directly request not to be contacted)

• Court/Law enforcement patients (i.e., prisoners)

• Patients with a foreign (Non-US or US Territory address) home address

• Patients who are excluded because of state regulations that place further restrictions on which patients may be contacted after discharge

• Homeless patients

• Patients who left without being seen and did not receive a billing code

• Patients who are sampled by the hospital for Hospital CAHPS survey administration regarding their inpatient experiences.

Identification of patients for exclusion will be based on hospital administrative data.

B2. Data Collection Procedures

There will be three modes of data collection, with sampled individuals randomized to the three modes:

1. Mail only: We will mail an initial letter, survey, and a business reply envelope. See Attachments D, E, and F for copies of the initial letter of the survey. All non-responders will be sent a second mailing three weeks after the initial mailing.

2. Telephone only: We will make five attempts to reach each sampled individual over a six week maximum period using computer assisted telephone interviewing (CATI). See Attachment G for the phone script for these calls. Calls will be made only between the hours of 9:00 am and 9:00 pm respondent local time (unless a respondent specifically requests a callback outside of this range).

3. Mail with telephone follow up: Individuals will receive an initial mailing mailed simultaneous to the mail only group and containing an initial letter, survey, and business reply envelope. For sampled individuals randomized to this mode who have not responded by mail by a cutoff date three weeks after the first survey mailing, we will begin attempts to complete surveys by phone using CATI. As with the telephone only mode, we will make five attempts to reach each sampled individual. These attempts will be made over a period of three weeks such that the entire field period is no longer than six weeks.

Procedures for all modes parallel mail and telephone procedures recommended for Hospital CAHPS data collection.

B3. Response Rates and Non-Response

We anticipate a response rate of 33 percent, based on recent experience with Hospital CAHPS surveys. We will employ multiple mail and phone contacts to minimize non-response. We will also plan for survey and item non response analysis. We will compute these statistics overall, and separately by mode of administration (mail; telephone; mixed) and discharge vs. stand-alone vs. supplement administration.

B4. Tests of Procedures or Methods

This data collection effort includes

1. A test of mode of data collection: mail-only; telephone-only; and mail with telephone follow up. This mode experiment is described above. Procedures for all modes parallel mail or telephone procedures recommended for Hospital CAHPS data collection.

2. A test of stand-alone vs. supplemental administration of emergency room survey items as part of Hospital CAHPS for those patients admitted to the hospital. This involves the inclusion of a small number of emergency room survey items appended to the standard Hospital CAHPS instrument, and an assessment of the equivalence of responses between the two modes of administration.

3. A test of the newly designed emergency room patient experiences with care survey, which is being fielded for the first time under this data collection effort.

B5. Statistical and Data Collection Consultants

The survey, sampling approach, and data collection procedures were designed by the RAND Corporation under the leadership of:

Robin Weinick, Ph.D.

RAND Corporation

1200 South Hayes Street

Arlington, VA 22202-5050

Kirsten Becker, MS

RAND Corporation

1776 Main Street

Santa Monica, CA 90407

ATTACHMENTS

Attachment A: Emergency room patient experiences with care survey – discharged to community

Attachment B: Emergency room patient experiences with care survey –admitted to hospital (stand-alone)

Attachment C: Emergency room patient experiences with care survey –admitted to hospital (Hospital CAHPS supplement)

Attachment D: Initial Letter for Emergency room patient experiences with care survey – discharged to community

Attachment E: Initial Letter for Emergency room patient experiences with care survey –admitted to hospital (stand-alone)

Attachment F: Initial Letter for Emergency room patient experiences with care survey –admitted to hospital (Hospital CAHPS supplement)

Attachment G: Emergency room patient experiences survey telephone script – all versions