CMS-10445 Supporting Statement B

CMS-10445 Supporting Statement B.docx

Medicare Advantage Quality Bonus Payment Demonstration

OMB: 0938-1195

Document [docx]

Download: docx | pdf

Supporting Statement B

COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

Respondent Universe and Sampling Methods

After removing any plans that are considered ineligible (such as 1876 cost plans or those not currently active), it is anticipated that approximately 550 plans will comprise the survey sample. We plan to survey all eligible MA plans and so will not sample from this population. The mail survey will be completed by about 440 (or 80 percent) of these plans. Contact information for MA plans will come from CMS’s contract and plan contact databases maintained in the HPMS.

For the case studies, the research team will select up to 10 MAOs, using data from the HPMS, the MAO survey, and the environmental scan, to reflect diversity across the following types of characteristics:

Contract type
Basic plan structure (tightly managed group model HMO vs. looser HMO or PPO)
Geography and payment cohort
Beneficiary population
Star rating/quality scores, especially improvement in ratings
Corporate ownership

Procedures for the Collection of Information

The survey will be conducted with all MA plans under contract with CMS in 2012. In order to identify a knowledgeable respondent and gain commitment to participate from the MA contract representative/survey respondent, letters will be mailed to the president or CEO of each sampled health plan (see Appendix B). The letter will explain the purpose of the study and request that a survey respondent be designated. A contact name, phone number, and email address will be included in the letter to send back the survey respondent information. The ideal respondent will be a senior staff member who is most knowledgeable about the quality improvement activities conducted by the MA plan. Telephone interviewers will then contact the person to briefly introduce the survey, ask for a commitment to return the questionnaire quickly, and collect contact information needed to mail out the survey materials (see Appendix C). Each MA plan respondent will then receive a cover letter on CMS stationary, a questionnaire, a fact sheet containing detailed information about the study, and a postage-paid return envelope. This material will be sent via FedEx because of the overnight delivery feature and the guarantee that the material will reach the correct person. When requested by a survey respondent, a Microsoft Word version of the survey will be sent in an email attachment or by fax. Shortly after questionnaire delivery, respondents will be called to confirm that the materials have been received, respond to any questions or concerns, and emphasize the need for a timely response (see Appendix D). Two weeks after the initial mailing, we will make telephone calls to all nonrespondents to encourage them to complete and return the mail questionnaire. We will also give them the option of completing the survey by telephone. Reminder calls will continue weekly throughout the field period.

For the case studies, the team will begin outreach to the selected MAOs using a recruitment package that includes a description of the project, site visit process, types of staff members we would like to speak with, range of proposed dates, and time commitment required. We also propose to include an “endorsement” letter – a brief explanation of the project on CMS letterhead encouraging participation and assuring that participation does not impact their contracts. As appropriate, the team will also enlist the support of expert/stakeholder interviewees to lend further credence to research activities in the eyes of those identified for the case studies. Finally, it will be important to convey to the organizations that contract-specific data will only be shared in an aggregate format and that the team will take steps to mask the feedback of the participating organizations. The cover letter will address these issues and convey the willingness of the team to work with the plan to address any other privacy/confidentiality issues they may have.

Unless an appropriate contact has already been identified, the recruitment package will be sent to the president or CEO of the health plan. We will then follow up with the staff member to whom we are referred to schedule specific dates and times for site visits, depending on the availability of the MAO’s representatives.

The site visit team will consist of three project team members (two senior staff and one junior staff). Under the direction of the task leader, a research associate will work with each site to schedule the interviews at a convenient time. Each site visit will include interviews with key administrators such as chief executives, operating officers, financial, medical, and quality improvement officers, and other key employees leading performance improvement efforts connected to quality scores. By meeting with multiple people within the case study sites, the site visit team will be able to better understand the environment within which each site is operating and their history in terms of quality improvement and star ratings. We will schedule three-day site visits to ensure flexibility in accommodating the schedules of the various health plan staff members that will be interviewed, and to minimize the need for follow-up telephone calls post site visit.

Methods to Maximize Response Rates and Deal with Nonresponse

Response rates in the mail survey of MA plans will be maximized in a number of ways. We will mail introductory letters on CMS stationery and follow up with telephone calls to determine whether selected MA plans are eligible for the survey. During these calls, we will also identify the person most knowledgeable about the MA plan; we will then mail the survey to the designated survey respondent. The cover letter, which will be personalized and printed on CMS letterhead, will include contact information for the CMS project officer and a toll-free number at Mathematica to call for questions or to request a copy of the questionnaire. The letter will describe the evaluation and the purpose of the mail survey, and will provide instructions. It will indicate that the survey is voluntary and give the estimated time for completion.

Follow-up telephone calls by trained interviewers (during which plans can complete the survey) extend our strategy. We will send the initial questionnaire by mail and place a follow-up call if the respondent has not returned the questionnaire within two weeks. We will continue to place follow-up calls weekly throughout the field period to encourage response. We will offer multiple-mode options (mail, telephone, email, and fax). Interviewer training materials will emphasize tips for dealing with nonresponse. The questionnaire is relatively short and has only a few open-ended response categories. There are clear instructions on the first page. We considered making the survey available on the web but concluded that it was not cost effective to expend resources to program a web survey instrument when we expect to complete surveys with roughly 440 MA plans.

The response rate for the mail survey will be calculated as the number of MA plans that complete the questionnaire (either by mail or by telephone) divided by the total number of MA plans that were deemed eligible and mailed a questionnaire (all unique MA plans). Because we know the universe of approved, unique MA plans or contracts, the denominator of the response rate does not include ineligible plans or plans whose eligibility is unknown. Response rate calculations are based on standards established by the American Association for Public Opinion Research.

Based on previous surveys with similar populations, we anticipate achieving a minimum response rate of 80 percent on the survey. For non-respondents, we will construct a profile based on characteristics of the plans drawn from the HPMS and data collected through the pre-survey screening outreach.

To encourage participation in the case studies, the team will begin outreach to the selected MAOs using a recruitment package that includes a description of the project, site visit process, types of staff members we would like to speak with, range of proposed dates, and time commitment required. We also propose to include an “endorsement” letter – a brief explanation of the project on CMS letterhead encouraging participation and assuring that participation does not impact their contracts. As appropriate, the team will also enlist the support of expert/stakeholder interviewees to lend further credence to research activities in the eyes of those identified for the case studies. Finally, it will be important to convey to the organizations that contract-specific data will only be shared in an aggregate format and that the team will take steps to mask the feedback of the participating organizations. The cover letter will address these issues and convey the willingness of the team to work with the plan to address any other privacy/confidentiality issues they may have.

To facilitate MAO participation, the research team will travel to the MAO’s offices to conduct the interviews and schedule the interviews for dates and times that are convenient for the MAO representatives.

Tests of Procedures or Methods to Be Undertaken

Up to nine MA plans will be selected to pretest the survey instrument. Plans that have a range of quality scores will be selected for the pretest so that the entire questionnaire can be efficiently tested. The pretest will identify any items that are burdensome or difficult to respond to, and these items will be removed or revised accordingly. An average response time will be estimated from the pretest surveys.