CMS-10534.Generic Clearance Submission

Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback” (OMB Control Number: 0938-1185)

TITLE OF INFORMATION COLLECTION:

Reporting for the Health Insurance Marketplace and Qualified Health Plan Enrollee Experience Surveys: Health Insurance Marketplace Consumer Experience Surveys Project

PURPOSE:

Sections 1313 and 1321(a) of the Affordable Care Act provide the Secretary with general authority to establish standards and regulations related to Exchanges, Qualified Health Plans (QHPs), and other components of title I of the Affordable Care Act. In §155.1200(b)(3), State-based Marketplaces are directed to submit performance monitoring data including information on consumer satisfaction on an annual basis. Section 1311(c)(4) directs the Secretary to establish an enrollee satisfaction system that would evaluate the level of enrollee satisfaction of members in QHPs offered through an Exchange, for each QHP with more than 500 enrollees in the previous year, and provide this information to consumers on Marketplace websites.

In response to this legislation, the Centers for Medicare & Medicaid Services is developing and conducting surveys for Marketplace users and QHP enrollees. OMB approval for the psychometric test component of the Marketplace survey and the QHP Enrollee Experience survey (QHP Enrollee Survey) was obtained on February 12, 2014; OMB Control Number for the surveys is 0938-1221.

This fast track request is to conduct cognitive testing of draft reporting displays—including—for consumers, health plans, Marketplaces, and CMS. Users frequently interpret information very differently than experts expect and in ways that cannot be anticipated. This is the fundamental premise of all cognitive testing, including testing of survey items, educational materials, and Web sites.

We will conduct cognitive testing of the quality improvement reports of the Marketplace Survey results, the quality improvement reports of the QHP Enrollee Survey, and the public reports of QHP Enrollee Survey results. The reports will include language for the composites (e.g., application process, getting needed care) and measures. We will conduct the testing with four key audiences: consumers, small business employers, health plan quality mangers, and Marketplace regulators. One-on-one testing will assess the usability and understanding of measure labels and potential reporting displays for the Marketplace Survey results and QHP Enrollee Survey results. Using principles of effective display—and, in collaboration with consumer testing feedback for the public reporting of the federal Quality Rating System (QRS)—we will develop reporting mockups of data and explanatory language for the composites and overall-rating items used in the psychometric-test version of the surveys. Through one-on-one testing, interviewers will use think-aloud techniques—asking the participant to process what they are seeing aloud with interviewers—and direct probing to evaluate participants’:

• Understanding of the measures and composites: What information do users expect here? Do users want less, more, or the same amount of data? Can users understand and correctly interpret the information here? Do the measures that make up the composites make sense to users? What text changes may be needed?

• Understanding of potential reporting displays: Can users understand and correctly interpret and make a decision using the data displays as shown? What text changes may be needed?

  • For Marketplace regulators the focus of testing will be on results that summarize information from the QHP and Marketplace surveys.

  • For consumers, small business employers, and health plans, testing of QHP Enrollee Survey results will be done in alignment with consumer testing feedback for the public reporting of the QRS and other potential state-based reporting efforts.

We will conduct three rounds of testing for each type of report: quality improvement reports of the Marketplace Survey results, the quality improvement reports of the QHP Enrollee Survey, and the public reports of QHP Enrollee Survey results. The results from the first two rounds of testing will inform refinements to the language and displays to be tested in subsequent rounds of testing. The third and final round of testing will validate and verify labels and potential reporting displays that will support consumer, Marketplace regulator, and health plan quality manager use and decision-making based on the data.

DESCRIPTION OF RESPONDENTS:

A total of 90 respondents from four stakeholder groups will be included across 6 rounds of testing—three rounds for the Marketplace Survey results and three rounds for the QHP Enrollee Survey. Stakeholder groups include: consumers, small business employers, health plan quality mangers, and Marketplace regulators.

Consumers

Fifteen consumer respondents across three rounds of testing will be included in this study for public reports of the QHP Enrollee Survey results. Respondents for the consumer testing will include those who purchased a health insurance plan through their local Marketplace during the 2013–2014 open enrollment period as they have recent experience with the process of selecting health plans in this new environment. We will also recruit respondents who are uninsured, but eligible for health insurance through the Marketplace as a way of including consumers who may be selecting a health plan in the near future.¹

We will recruit respondents using a market research firm. Consumers will be screened to include a mix of participants from each of four target populations:

1. Young and healthy

• Must be between the age of 18 and 34.

• Must not have been diagnosed (now or ever) with any chronic condition

2. Healthy and 35+

• Must be between the ages of 35 and 64.

• Must not have been diagnosed (now or ever) with any chronic health condition

3. Adults with chronic conditions

• Must currently be diagnosed with a chronic condition such as asthma, diabetes, hypertension, etc.

4. Adults with low income

• Must have an income below 250% of the federal poverty guideline, but not enrolled in Medicaid.

In addition we will recruit for a mix of respondents across gender, race, ethnicity, household income, education, eligibility for subsidies, and family size. See Attachment B for consumer respondent recruitment screener. Screening criteria may be revised as needed based on recruitment success.

We will conduct usability testing with five consumer respondents to test the public reports of QHP Enrollee Survey results. After the first round of testing, two additional rounds with five consumer respondents each to test refinements to the display will be conducted for a total of 15 consumer respondents to the public reports of QHP Enrollee Survey results.

Small business employers

Across three rounds of testing, fifteen small business employers with less than 200 covered lives will participate in testing for this effort of the public reports of QHP Enrollee Survey results. We will recruit respondents using a market research firm with the following characteristics:

• Benefit managers or executives who are responsible for identifying or selecting health plans for their organization

• Less than 200 covered lives or 100 employees

• Geographic diversity (e.g., northeast, southwest)

• Industry diversity (e.g., manufacturing, information technology)

Telephone interviews lasting 60 minutes will be conducted one-on-one.

Health plan quality managers

Thirty health plan quality managers will participate in testing for this effort across six rounds of testing of the QHP Enrollee Survey results—15 will review the public reports of QHP Enrollee Survey results and 15 will review the quality improvement QHP Enrollee survey results. We will recruit respondents using a market research firm and/or referrals with the following characteristics:

• Work for a QHP that exists in the Marketplace

• Working in quality improvement or management for at least one year

• Interested in providing input on health quality measures and how best to present the information

• Responsible for designing consumer materials, as possible.

Telephone interviews lasting 60 minutes will be conducted one-on-one.

Marketplace regulators

Thirty Marketplace regulator respondents across three rounds of testing for the quality improvement Marketplace Survey results and quality improvement QHP Enrollee Survey results will be included in this study. AIR will gather a list of referrals from CMS, AIR, and others’ contacts of Marketplace regulators supporting the Federally-facilitated Marketplace within CMS and State-based Marketplaces for 60-minute one-on-one telephone interviews. Since the regulators will be identified by referral, there are no screening criteria. However, referrals must at a minimum work on or with the Marketplace(s) from a regulation capacity for at least 6 months. In addition we will recruit for a mix of respondents across federal and State-based Marketplaces and geographic diversity.

We will conduct testing with five Marketplace regulator respondents to test the Marketplace Survey. After the first round of testing, two additional rounds with five Marketplace regulator respondents each to test refinements to the display will be conducted for a total of 15 Marketplace regulator respondents to Marketplace Survey results. Similarly, three rounds of testing of the QHP Enrollee Survey results will be conducted with a total of 15 respondents.

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [ ] Customer Satisfaction Survey

[ ] Usability Testing (e.g., Website or Software [ ] Small Discussion Group

[ ] Focus Group [X] Other: Cognitive Testing (e.g., review of paper-based materials and language)__________________

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: ______Nidhi Singh Shah and Kathleen Jack____________

To assist review, please provide answers to the following question:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [ X ] Yes [ ] No

2. If Yes, will any information that is collected be included in records that are subject to the Privacy Act of 1974? [ ] Yes [ X ] No

3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [] Yes [X] No

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [X] Yes [ ] No

Marketplace regulators and health plan managers will not be provided an incentive as they have a vested interest in supporting this work. The Agency will provide remuneration to consumers and small business employers for several reasons:

• Respondents will be recruited for specific characteristics that are related to the subject matter (e.g., questions would only be relevant to those eligible for the Marketplaces). Since this is a very specific subject matter, it may be difficult to recruit eligible respondents and remuneration helps to attract a greater number of potential respondents.

• Consumer respondents will be asked to travel to the laboratory testing site/interview site, which may involve transportation and parking expenses. The remuneration is intended to help offset some of these costs. Recent experience recruiting this population for another Health and Human Services agency required remuneration at a higher rate to obtain and secure participation.

• Small business employers will be asked to take time away from their primary business activity to support work that may or may not have a direct benefit to them. As such, remuneration for their time may help produce better quality, more diverse respondents.

We will plan to provide incentives to consumers of $100 for 90-minute, in-person interviews in Washington, DC. Due to the location of testing, the incentive is higher than it might otherwise be in other areas of the country. We will provide small business employers that are benefit managers $100 for a 60-minute telephone interview and executives a $200 incentive. As necessary, we will consider providing an incentive to health plan quality managers.

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time*	Burden
Consumers	15	100 minutes	25 hours
Small business employers	15	65 minutes	16.25 hours
Marketplace regulators	30	65 minutes	32.5 hours
Health plan managers	30	65 minutes	32.5 hours
Totals	90	--	106.25hours
* Participation time includes screener, discussion, and consent.

FEDERAL COST: The estimated annual cost to the Federal government is $ 426,041.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions: N/A

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe? [ ] Yes [ ] No

If the answer is yes, please provide a description of both below (or attach the sampling plan)? If the answer is no, please provide a description of how you plan to identify your potential group of respondents and how you will select them?

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[ ] Web-based or other forms of Social Media

[ ] Telephone

[ ] In-person

[ ] Mail

[ ] Other, Explain

2. Will interviewers or facilitators be used? [ ] Yes [ ] No

Please make sure that all instruments, instructions, and scripts are submitted with the request.

Attachment A – Recruitment screeners and related communications for each audience

Attachment B – Consent forms for each audience

Attachment C – Reporting cognitive interview guides for each audience

Instructions for completing Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback”

TITLE OF INFORMATION COLLECTION: Provide the name of the collection that is the subject of the request. (e.g., Comment card for soliciting feedback on xxxx)

PURPOSE: Provide a brief description of the purpose of this collection and how it will be used. If this is part of a larger study or effort, please include this in your explanation.

DESCRIPTION OF RESPONDENTS: Provide a brief description of the targeted group or groups for this collection of information. These groups must have experience with the program.

TYPE OF COLLECTION: Check one box. If you are requesting approval of other instruments under the generic, you must complete a form for each instrument.

CERTIFICATION: Please read the certification carefully. If you incorrectly certify, the collection will be returned as improperly submitted or it will be disapproved.

Personally Identifiable Information: Provide answers to the questions. Note: Agencies should only collect PII to the extent necessary, and they should only retain PII for the period of time that is necessary to achieve a specific objective.

Gifts or Payments: If you answer yes to the question, please describe the incentive and provide a justification for the amount.

BURDEN HOURS:

Category of Respondents: Identify who you expect the respondents to be in terms of the following categories: (1) Individuals or Households;(2) Private Sector; (3) State, local, or tribal governments; or (4) Federal Government. Only one type of respondent can be selected per row.

No. of Respondents: Provide an estimate of the Number of respondents.

Participation Time: Provide an estimate of the amount of time required for a respondent to participate (e.g. fill out a survey or participate in a focus group)

Burden: Provide the Annual burden hours: Multiply the Number of responses and the participation time and divide by 60.

FEDERAL COST: Provide an estimate of the annual cost to the Federal government.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents: Please provide a description of how you plan to identify your potential group of respondents and how you will select them. If the answer is yes, to the first question, you may provide the sampling plan in an attachment.

Administration of the Instrument: Identify how the information will be collected. More than one box may be checked. Indicate whether there will be interviewers (e.g. for surveys) or facilitators (e.g., for focus groups) used.

Submit all instruments, instructions, and scripts are submitted with the request.

1 To be eligible for health coverage through the Marketplace, respondents: must live in the United States, must be a U.S. citizen or national (or be lawfully present).

8