UPD1 CMS Response - final

Medicare Health Outcomes Survey (HOS) and Supporting Regulations at 42 CFR 422.152 (CMS Number CMS-10203)

OMB Number 0938-0701

CMS response to OMB Question, Comments and Concerns

1. Forward the reports regarding cognitive testing (the Bill Rogers reports).

The reports regarding cognitive testing and “Bill Rogers’ report” are two separate and independent reports. Discussion on cognitive testing was included: Supporting Statement – Part B, 2. Data collection procedures, A (i). The reports are included as Attachment 13 (English) and Attachment 14 (Spanish and Chinese). Bill Rogers’ report has to do with sample size design and is Attachment 11.

2. Supporting Statement Part A: Question 10 (Confidentiality) – Please include language that identifies the System of Records Notice(s) (SORN) associated with this data collection activity. Chris confirmed that it is the HPMS SORN.

Additional elaboration is found in Supporting Statement – Part A, 10. Confidentiality.

The System of Records Notices associated with this data collection effort are as follows:

1) Health Plan Management System (HPMS) [August 12, 1998 (Volume 63, Number 155) Pages 43187-43190] [Attachment 9]

2) Health Plan Management System (HPMS) – Notice of a Modified or Altered System of Records [January 14, 2008 (Volume 73, Number 9) Pages 2257-2263] [Attachment 10]

All respondent related material contains the following Privacy Statement: All information that would permit identification of any person who completes this survey is protected by the Privacy Act and the Health Insurance Portability and Accountability Act (HIPAA).  This information will be used only for purposes permitted by law and will not be disclosed or released for any other reason. If you have any questions or want to know more about the study, please call [vendor name] at [toll-free number].

3. Forward a drafted response to OMB’s proposal to add burden hours to the package that is currently under review to account for future cognitive testing efforts over the next three years. OMB is not looking for clear yes or no, but rather they simply wanted to avail you of the options you have to possible expand your reach with respect to the cognitive testing and potentially your ability to collect better information. OMB wants to know they you will at least consider the option, if not now, then in the future.

After discussing the option posed by OMB during the teleconference regarding the inclusion of burden hours to support future cognitive testing of survey, our survey design team recommended declining the increase. It was felt that such measures were not needed at this time nor in the foreseeable future. The team prefers design testing that uses a one-on-one “interview style” approach rather than a panel approach. While it is necessary to use a number of interviews in testing the validity of any change in question design, our team’s experience shows that inflating the number of respondents has a diminishing return value.

4. Survey Instrument – Page 1: Please revise the confidentiality language in box. OMB will not accept it if it refers to the information being confidential.

Attached are the updated HOS and HOS-M English, Spanish and Chinese versions for approval (HOS Attachment 4a, 4b and 4c) (HOS-M Attachment 6a, 6b and 6c).

Please refer to issue 2 for the updated language.

5. Supporting Statement Part B: Minimal Detectable Affects from year 1 to year 2– please expound on the current narrative to paint a clearer picture for OMB.

Additional elaboration is found in Supporting Statement – Part B, 1. Description of sample selection and universe, B (i): The survey protocol is designed to achieve a 70 percent response rate for the baseline survey. The targeted response rate for the follow-up survey is 80 percent. The initial sample size and targeted baseline and follow-up response rates are designed to achieve an analytic sample size with adequate statistical power to detect significant variation between health plans on physical and mental health outcomes. With an analyzable sample size of 500 completed surveys, we can distinguish plans that differ by 2 points with 90% power.

6. Supporting Statement Part B: Please add more supporting language to the sample size justification. Also, please add the justification the Chris discussed regarding the statistical power of the study.

Additional elaboration is found in Supporting Statement – Part B, 1. Description of sample selection and universe, B (ii): Due to variations in health plan population size, three sampling approaches are used. These approaches were developed and recommended by an independent statistical panel of experts. A report on MCO survey sample design [Attachment 11] was used to determine the following approach.

7. Supporting Statement Part B – Page 3: Please add clarifying language to the data collection protocol. As part of the revised section on the data collection protocol, please add additional detail about the mail phase efforts.

Additional elaboration is found in Supporting Statement – Part B, 2. Data collection procedures

Below is a timeline of survey field operations to help illustrate pertinent tasks and timeframes. This is followed by a more in depth discussion of the various steps.

Task	Time Frame
Send first questionnaire with cover letter to the respondent.	0 Days
A survey vendor may elect to initiate CATI for members with an invalid or undeliverable mailing address.	0-83 Days
Send a reminder/thank-you postcard to non-respondents.	4-10 Days
Send a second questionnaire with cover letter to non-respondents.	28 Days
Send a second reminder/thank-you postcard to non-respondents.	32-38 Days
Initiate CATI for non-respondents and members who return a blank or incomplete mail survey.	56 Days
Initiate systematic contact for all non-respondents and members who return a blank or incomplete mail survey so that at least telephone calls are attempted.	56-84 Days

(i) Mail phase.

This is the first method used to conduct the survey. The mail component of the survey uses standardized questionnaires, cover letters and postcards. To ensure comparability of results, the survey vendors must follow strict adherence to the established survey protocol, specifications manual and the quality assurance plan.

Because of the challenge in corresponding with a diverse population, it is necessary to support material in multiple languages. The three most common languages encountered with the Medicare population are English, Spanish and Chinese. Both the standard and modified instruments (Attachments 4 and 6, respectively) plus associated supporting correspondence (Attachments 5 and 7) are available in each of the three identified languages. Comprehensive cognitive testing of all three language versions was conducted to insure uniform comparability. (Attachment 13 – English and Attachment 14 Spanish and Chinese cognitive test reports)

The Pre-notification Postcard and Letter for First Questionnaire contain English and Spanish text. The Letter for First Questionnaire is double-sided. One side of the letter contains English text and the other side contains Spanish text. The Spanish text invites Spanish-speaking members to request a Spanish version of the questionnaire by calling the survey vendor’s toll-free customer support number.

Completed questionnaires can be manually key-entered into the computer, or optically scanned. To ensure quality for key-entered data, two separate data entry specialists must independently key answers for each questionnaire. A comparison of the separate entries identifies data entry errors that need adjudication by a supervisor.

(ii) Telephone phase.

Following the mail portion of the process, survey vendors identify members who did not respond to the mail survey and members who returned a blank or incomplete mail questionnaire (a questionnaire with less than 80 percent of questions complete). These members are eligible for telephone interviews.

The telephone component uses a standardized CATI script and specific design specification. The survey vendor is responsible for programming the scripts and specifications into its existing CATI software. To ensure the comparability of survey results, the survey vendor cannot change the wording of survey questions, the response categories or order of the questions.

The survey vendor attempts to contact non-respondents by telephone so that at least six telephone calls are attempted at different times of day, on different days of the week and in different weeks.

The survey vendor establishes training programs for all personnel involved in the telephone phase of the process. It establishes quality control procedures and monitors staff performance to ensure the integrity of the telephone interviewing process. The survey vendor monitors 10 percent of CATI interviews to evaluate the quality of interviewing and provides feedback and additional training as necessary.

8. Supporting Statement Part B – Non-response bias analysis: Please forward the report Chris referenced on the conference call.

Additional elaboration is found in Supporting Statement – Part B, 1. Description of sample selection and universe, C: To determine what level of non-response bias exists in the HOS, the issue was examined for both physical component summary scores and mental component summary scores for the Medicare fee-for-services HOS. Both response propensity weighting and imputation for non-respondents were used in the analysis. The findings of study supported that survey non-response to the FFS HOS does not adversely affect estimates average health status. [Attachment 12]

9. Survey Instrument – Page 58: Please remove the “Other race” option from the response selections.

Attached are the updated HOS and HOS-M English, Spanish and Chinese versions for approval (HOS Attachment 4a, 4b and 4c) (HOS-M Attachment 6a, 6b and 6c).