S4

Addendum to Supporting Statement A for CAHPS Survey for Physician Quality Reporting

The text below provides additional detail on the weighting and adjustment that will used to account for non-response and patient characteristics not under the control of the PRQS practices.

Sampling and Nonresponse Weights. Appropriate weighting of survey responses serves several purposes: (a) Sampling weights reflect the probability that each beneficiary is selected for the survey; and (b) nonresponse weights reflect the probability that a sampled beneficiary responds to the survey. Thus, calculation of appropriate weights is essential for creating valid estimates. Sampling weights are the ratio of eligible to sampled beneficiaries in each PQRS practice, and are calculated during the sample design process. That is, sampling weights adjust for any oversampling or stratified sampling that will be used to sample beneficiaries for survey administration such as oversampling high utilizers of care. Model-based, nonresponse weights reflect the inverse probability that an individual would respond to the survey. The nonresponse model describes factors associated with nonresponse and aims to (1) make the sample representative of assigned beneficiaries and correct for biases due to associations of nonresponse with variables in the sampling frame, (2) produce weights that make weighted distributions of these variables for respondents match unbiased estimates of the appropriate population distributions, and (3) inform future field procedures and sample design. These individual-level weights will be calculated using loglinear regression models; in consultation with CMS, beneficiary characteristics to be considered in the model will include those variables that are available for all sampled beneficiaries including age, low income status and Medicaid dual eligibility. Using weighting to account for sample design and nonresponse ensures that the adjusted estimates obtained from survey responses accurately reflect the total population eligible for sampling.

Case-mix Adjustment. Certain respondent characteristics, such as age, education, and health status, are not under the control of the entities whose performance is being assessed but are nonetheless related to beneficiaries’ survey responses. To ensure that comparisons between entities reflect differences in performance rather than differences in case mix, it is necessary to adjust for such respondent characteristics when calculating performance scores. In consultation with CMS, we will develop mixed effect regression models to appropriately adjust performance measures for differences in case mix across PQRS practices. These models would include fixed effects for case-mix adjustors, such as self-reported health, age, and education.