CMS-10726 QHP Survey Focus Group Cog Testing Fast Track_Generic Clearance 01242020-revised-CLEAN

Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback” (OMB Control Number: XXXX-YYYY)

TITLE OF INFORMATION COLLECTION:

Survey Refinement for the Qualified Health Plan Enrollee Experience Survey

PURPOSE:

Section 1311(c)(4) of the Patient Protection and Affordable Care Act (PPACA) directs the Secretary of the Department of Health & Human Services (HHS) to establish an enrollee satisfaction survey to assess enrollee satisfaction with each Qualified Health Plan (QHP) offered through the Health Insurance Exchanges (Exchanges) (also known to consumers as Health Insurance Marketplaces)¹ and Small Business Health Options Program (SHOP) with more than 500 enrollees in the prior year. Additionally, Section 1311(c)(3) of the PPACA directs the Secretary to develop a quality rating for each QHP offered through an Exchange.

Based on this authority, CMS issued a regulation in May 2014 to establish standards and requirements related to QHP issuer data collection and public reporting of quality rating information in every Exchange.² As a condition of certification and participation in the Exchanges, CMS requires that QHP issuers submit QHP Enrollee Survey response data and QRS clinical measure data for their respective QHPs offered through an Exchange in accordance with CMS guidelines.³ Exchanges are also required to display QHP quality rating information on their respective websites.⁴ Please refer to the 2020 QRS and QHP Enrollee Survey Technical Guidance for all relevant statutory and regulatory citations for the QRS and QHP Enrollee Survey.⁵

CMS has not implemented significant changes to the survey since initial testing in 2016. The survey was developed in 2012 before there was an Exchange population with whom to test materials. Instead, CMS conducted cognitive testing with a proxy population, low-income consumers of the Massachusetts Health Connector. After collecting enrollee survey data for four years, CMS (and its contractor, Booz Allen Hamilton (Booz Allen)) better understands the Exchange population’s demographics and the QHP Enrollee Survey’s performance with this intended population. The Exchange population consists of consumers who purchase health care coverage through a QHP issuer on the Exchanges; once enrolled in a QHP these consumers become ‘enrollees’. The term consumer includes both those who are eligible to purchase health care coverage and those who are already enrolled in health care coverage purchased on the Exchanges.

In an effort to refine and potentially improve the survey, based on stakeholder feedback and prior analyses of survey results, CMS (via its contractor) will conduct focus groups and cognitive testing interviews with consumers to gather information for refinements; the details of these activities are described in this Generic Clearance information collection request. CMS expects refinements to address four primary topics: (1) refinements to reduce respondent burden, (2) refinements to reduce QHP issuer (issuer) burden (3) refinements that result in more actionable information to issuers (i.e. QHP issuers seeking to conduct quality improvement based upon enrollee reported experience), and (4) refinements to improve reliability of the survey results. Surveys in general, and CAHPS surveys in particular, have seen annual declines in response rates.⁶ A review of the national level QHP Enrollee Survey Quality Improvement (QI) Reports illustrate the falling response rates of the QHP Enrollee Survey.

The planned focus groups and cognitive testing interviews intend to inform revisions by making questions easier for respondents to understand, which will likely increase response rates while reducing missingness and improving reliability. Reducing survey length and improving the wording are both well-established methods for reducing respondent ^7,8,9,10

CMS will conduct focus groups with consumers to gain their perspectives on what matters to them in terms of health plan quality, which factors are important to them in selecting a health plan, and how health plans can support their health and well-being. Focus groups are the appropriate approach as these topics are underexplored in general, but particularly in the context of Exchange consumers. The focus group format will allow us to identify the range of consumer opinions and perspectives. Similarly, the team will conduct a focus group with QHP issuer representatives to understand their perspectives regarding the value the QHP Enrollee Survey adds to their quality improvement efforts and how survey results are, or are not, used.

In addition to two focus groups with consumers and one focus group with QHP issuer representatives, CMS will conduct cognitive testing interviews with consumers regarding revised questions to test question wording as well as word choice and phrasing in accompanying materials like survey emails and letters. This testing will help improve survey respondents’ understanding of questions and assure that respondent answers are in line with the questions’ intent. We will conduct cognitive testing on questions that are either a rewording of an existing question, a reordering of response options for existing questions, or on new questions for which there is a strong literature to support their use and wording. The team will conduct two rounds of cognitive testing interviews during which we will test the draft wording, revise based on results of testing, and then hold another round of testing to assure efficacy.

CMS will compile the results of the focus groups and cognitive testing interviews and based upon results and findings of the focus groups and cognitive testing interviews, CMS may draft updates to the survey and accompanying materials for the 2022 QHP Enrollee Survey.

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [ ] Customer Satisfaction Survey

[ ] Usability Testing (e.g., Website or Software [ ] Small Discussion Group

[X] Focus Group [X] Other: Cognitive Testing (review of paper-based materials and language)

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: __________________Nidhi Singh Shah

To assist review, please provide answers to the following question:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [ ] Yes [X] No

2. If Yes, will any information that is collected be included in records that are subject to the Privacy Act of 1974? [ ] Yes [ X ] No

3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [] Yes [X] No

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [X] Yes [ ] No

Booz Allen will offer consumer participants a $75 incentive. Upon participation in either a focus group or cognitive testing interview, the participant will be sent an e-gift card for $75.

BURDEN HOURS¹¹

Category of Respondent	No. of Respondents	Participation Time*	Burden
Consumers (Cognitive Testing)	20 (2 rounds)	120 minutes	40 hours
Consumers (Focus Group)	12 (2 groups)	150 minutes	30 hours
Issuer Representatives (Focus Group)	8	150 minutes	20 hours
Totals	40	--	90 hours
* Participation time includes screener, discussion, and consent. Maximum number of hours is estimated.

FEDERAL COST: The estimated total cost to the Federal government for conducting the research covered in this request will be approximately $257,986 in contract costs including labor hours, materials and supplies, overhead, general and administrative costs, and fees.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe? [X] Yes [ ] No

If the answer is yes, please provide a description of both below (or attach the sampling plan)? If the answer is no, please provide a description of how you plan to identify your potential group of respondents and how you will select them?

Description of potential group of respondents and selection process:

Consumer Recruitment

CMS (via its contractor Booz Allen) intends to recruit approximately six to eight consumers for each of the two consumer focus groups and ten to twelve consumers for each round of cognitive testing interviews totaling to 32 to 40 consumers for all refinement activities. For both the consumer focus groups and cognitive testing we intend to recruit through multiple channels, including recruitment materials disseminated by select TEP members (such as our consumer advocates), outreach to organizations working to get Americans insured, local Navigator groups, stakeholder organizations like America's Health Insurance Plans (AHIP), other consumer advocacy organizations, and through social media like local Facebook pages. Booz Allen will use a dedicated email address ([email protected]) for all recruitment activities. . (See Attachment C for detailed protocol.)

Booz Allen will screen potential focus group potential participants to meet one of the two following criteria:

1. They must have purchased a health insurance plan through the Exchange during the 2018 or 2019 open enrollment period. Because we are particularly interested in enrollees who have recent experience with the process of selecting health plans in this new environment, preference will be given to those who purchased coverage during the 2019 (for plan year 2020) open enrollment period due to full public display of plan quality and member experience ratings. Or,

2. Consumers who are uninsured, but eligible for health insurance through the Exchange, as a way of including potential consumers who might purchase coverage on an Exchange in the future. 

Additionally, participants will be stratified by income level and screened into one of the three health status categories in the tables below. One consumer focus group will consist of participants with income at or below 250% of the federal poverty level (FPL) and one consumer focus group will consist of participants with income above 250% of the federal poverty level. All information on the screener is self-reported. CMS and its contractor aim for diverse participation and stratification based upon FPL; focus groups will be stratified based upon information provided and as needed to reach the minimum number of participants.

Those interested in participating must complete an online screening form. We have included the recruitment screeners for all groups as an attachment to this request (Attachment A).

Issuer Recruitment

We intend to hold one issuer focus group with approximately six to eight participants recruited through the following channels: email and telephone outreach using Healthcare Organization Questionnaire (HOQ) contact information, stakeholder groups like AHIP, and CMS’ issuer newsletters. To be eligible to participate in the issuer focus group, the participant must: 

1. Work for a QHP currently operating on the Exchange;

2. Have job responsibilities in QI or management, direct customer service, or appeals management for the past year;

3. Be interested in providing input on health care quality measures.

We have included the recruitment screeners for all groups as an attachment to this request (Attachment A).

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[X] Web-based or other forms of Social Media

[X] Telephone

[X] In-person

[ ] Mail

[ ] Other, Explain

Booz Allen will be conducting consumer focus groups in person in either the Washington, DC or Denver, CO metro areas. The issuer focus group will be virtual via WebEx^® video conference. The cognitive testing interviews will be conducted via telephone.

2. Will interviewers or facilitators be used? [X] Yes [ ] No

Please make sure that all instruments, instructions, and scripts are submitted with the request.

Attachment A – Recruitment screeners and related communications for each audience

Attachment B – Consent forms for each audience

Attachment C – Protocols for focus groups and cognitive testing for each audience

1 Unless the context indicates otherwise, the term “Exchanges” refers to the Federally-facilitated Exchanges (FFEs) (inclusive of FFEs where the State performs plan management functions and the State-based Exchanges (SBEs) (inclusive of State-based Exchanges on the Federal Platform [SBE-FPs]).

2 Patient Protection and Affordable Care Act; Exchange and Insurance Market Standards for 2015 and Beyond, Final Rule, 79 Fed. Reg. 30240 at 30352 (May 27, 2014), 45 C.F.R. §§ 156.1120 and 156.1125.

3 45 C.F.R. §§ 156.200(b)(5),(h); 156.1120; and 156.1125.

4 45 C.F.R. §§ 155.1400 and 155.1405.

5 The Quality Rating System and Qualified Health Plan Enrollee Experience Survey: Technical Guidance for 2020 is available on the Marketplace Quality Initiatives Website. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityInitiativesGenInfo/ACA-MQI/ACA-MQI-Landing-Page

6 Advances in Survey Methodology: Maximizing Response Rates and the Representativeness of CAHPS Data. Research meeting, Rockville, Maryland 2018. https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance/survey-methods-research/summary-research-meeting.pdf

7 Dillman D, Smyth J, Melani L. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. Hoboken, New Jersey: John Wiley & Sons, Inc; 2014.

8 Sahlqvist S. Effect of questionnaire length, personalization and reminder type on response rate to a complex postal survey: randomized controlled trials. BMC Med Res Methodol. 2011;11:62 http://www.biomedcentral.com/1471-2288/11/62

9

10 Rolstad S. Response Burden and Questionnaire Length: Is Shorter Better? A Review and Meta-analysis. Value in Health. 2011;14:1101-1108

11 Cost burdens for consumers are calculated based on the average hourly wage from the Bureau of Labor Statistics (https://www.bls.gov/web/empsit/ceseesummary.htm)

and cost burdens for issuer representatives are based on the average hourly wage for business operations specialists from the Bureau of Labor Statistics (https://www.bls.gov/oes/current/oes131199.htm)

5