CMS-10450 Supporting Statement B 2012-10-19

Supporting Statement B for CAHPS Survey for Physician Quality Reporting

Contract Number: HHSM-500-2005-00028I T0007

October 19, 2012

Prepared for CMS

Michael Wroblewski, Project Officer

RAND Corporation

1776 Main Street

P.O. Box 2138

Santa Monica, CA 90407-2138

TABLE OF ContentS

SUPPORTING STATEMENT 4

B. Collection of Information Employing Statistical Methods 4

B1. Respondent Universe and Respondent Selection 4

B2. Data Collection Procedures 5

B3. Response Rates and Non-Response 5

B4. Tests of Procedures or Methods 5

B5. Statistical and Data Collection Consultants 6

SUPPORTING STATEMENT

CAHPS Survey for Physician Quality REPORTING

B. Collection of Information Employing Statistical Methods

B1. Respondent Universe and Respondent Selection

National implementation of the survey will occur in 2014 and 2015 with an annualized total of 425 group practices. Samples for survey implementation will be drawn under an equal-probability design within each group practice. Specifically, we will draw samples by systematic sampling with a random start within a specified sort order using a skip interval calculated to attain the desired sample size. This gives the variance-reducing benefits of implicit stratification without requiring detailed stratification information for design or risking failure if such information proves to be inaccurate. We will select the sorting variables after reviewing all variables contained in the sampling frame; the general principle would be to sort by variables that might be associated with responses (for multiple items), especially those that are not likely to be in the patient-mix model. (An ideal example would be a utilization measure such as date of last visit, but we are not assuming availability of any particular variable.)

A target number of 276 completed questionnaires for each group practice evaluated was chosen to obtain a desired level of interunit reliability (IUR) for survey measures. The IUR is defined as 1−V/(V+²), where V is the variance of the estimate for a specific unit and ² is the between-unit variance of population means. Previous analyses have found that IUR exceeding 0.85 is desirable to attain reasonable precision, while in the Medicare CAHPS analysis, IUR=0.75 is regarded as a minimal standard.

One complication in developing sample sizes for practice-level assessment is that the number of physicians per medical practice is highly variable, and this affects between-site variation, which is a combination of a practice level component (A) and a physician component, which for a single physician we might call (B). With n physicians in a site, the combined variation of the means is A+B/n; intuitively, a small practice is most influenced by the particular style of individual physicians, while in a large practice individual characteristics tend to average out and only a smaller organizational variation remains. Using the guideline employed by NCQA and some other organizations for singleton physicians of 45 completes per medical home or group, and assuming that a large practice of 200 physicians looks like an ACO and requires about 300 completes. Under these assumptions we project the following numbers of completes as necessary to obtain a reliability of 0.75:

# Physicians in group	1	2	3	4	5	10	20	50	100	200
Sample	45	79	105	125	142	195	239	276	292	300

For a national implementation we understand that the participating practices will have an average of 50 physicians per group practice. Thus, we have assumed an average of 276 completes per entity (physician practice), and a completion rate of 45 percent. This will require an average of 613 patients per physician group.

B2. Data Collection Procedures

Data collection will be initiated with the mailing of a pre-notification letter signed by CMS’s Privacy Officer. The primary mode of data collection will be a mail survey; the first survey mailing will occur one week after the pre-notification letter. Two weeks after the initial survey mailing, beneficiaries will be mailed a follow-up survey packet with a modified cover letter signed by the CMS privacy officer (see attachment 2 for a copy survey letters). We will use computer assisted interviewing (CATI) as a secondary or non-response mode to be implemented approximately eight weeks after the initial survey mailing.

B3. Response Rates and Non-Response

We anticipate a response rate of 45 percent, based on recent experience with surveys of Medicare beneficiaries. We will employ multiple mail contacts and multiple modes (mail and CATI) to minimize non-response.

B4. Tests of Procedures or Methods

No tests of procedures or methods will be undertaken as part of this data collection.

B5. Statistical and Data Collection Consultants

The survey, sampling approach, and data collection procedures were designed by the RAND Corporation under the leadership of:

Robin Weinick, Ph.D. Julie Brown

RAND Corporation RAND Corporation

1200 South Hayes Street 1776 Main Street

Arlington, VA 22202-5050 Santa Monica, CA 90407

Data will be collected by a survey vendor, to be determined.