CMS-10597.Supporting Statement Part A

CMS-10597.Supporting Statement Part A.doc

CMS Healthcare.gov Site Wide Online Survey

OMB: 0938-1322

Document [doc]

Download: doc | pdf

SUPPORTING STATEMENT – PART A SUPPORTING STATEMENT FOR

PAPERWORK REDUCTION ACT SUBMISSIONS

HEALTHCARE.GOV SITE-WIDE ONLINE SURVEY

Introduction

CMS is requesting clearance for a survey among users of HealthCare.gov. The purpose of the survey is to gain an understanding of user experience, comprehension, and satisfaction with using the Federal Health Insurance Marketplace website established by the Affordable Care Act. The Marketplace provides coverage to uninsured Americans, as well as those already enrolled in Marketplace health insurance. One of the ways to purchase Marketplace insurance is through the online tools on HealthCare.gov. We have developed a survey to be administered to consumers while they are using the website. This survey is part of a continuing data collection program mandated by the ACA. It is designed to support the program goal to provide tools and information to help consumers to successfully find health insurance that they may not otherwise qualify for or find. Monitoring usability and the user experience through this ongoing survey provides the website developers with valuable information for use in continuous improvement of the website. The website survey is part of a larger research program to inform the development and enhancement of web tools for CMS programs such as the Health Insurance Marketplace.

A. Background

On March 23, 2010, the President signed into law the Patient Protection and Affordable Care Act (P.L. 111-148). On March 30, 2010, the Health Care and Education Reconciliation Act of 2010 (P.L. 111-152) was signed into law. The two laws are collectively referred to as the Affordable Care Act. The Affordable Care Act created new competitive private health insurance markets – called Marketplaces. The Marketplaces, which opened October 1, 2013, help individuals and small employers shop for, select, and enroll in high quality, affordable private health insurance plans that fit their needs at competitive prices. By providing a place for one-stop shopping, Marketplaces make purchasing health insurance easier and more understandable and will put greater control and more choice in the hands of individuals and small businesses.

The overall goal of the research mandated by the ACA is to provide consumer-based insights to inform the development and enhancement of online consumer tools on the Health Insurance Marketplace website HealthCare.gov. The overall research program is focused on conducting qualitative and quantitative research to support the implementation of the Health Insurance Marketplace with a focus on consumer experience of the web portal. The requested data collection is quantitative research.

The Marketplace portal provides information on health insurance options for a variety of consumer audiences. Such information must be publicly available, easily understandable, and relevant to the decision making process, in order to be effective. The health insurance information must be accessible and useful to the intended audiences, and ultimately effective in decision making by consumers. The form and context in which information is provided as well as the extent to which it can be tailored to needs of the audience are important determinants of communication effectiveness.

In addition to providing information, the Marketplace website also provides tools, such as the Application, Plan Compare, and Enrollment modules, that enable consumers to purchase private health insurance online using HealthCare.gov. Decision support content and resources are also provided to help consumers choose a health plan. The website survey will monitor the usability and effectiveness of the information, tools and decision support on a continuous basis.

The quantitative website survey will provide CMS with data that reflects the consumers’ perspective on interactions with the HealthCare.gov web site. Such information provides an important source of feedback that can be used to improve the existing portal and make the information more actionable, thus enhancing the users experience. Relatively continuous monitoring of this feedback during peak times of access will help alert our team to specific areas where problems may be occurring so that corrective actions may be taken

The following information provides policy context for research related to the Health Insurance Marketplace.

CMS has been entrusted with the responsibility for implementing many provisions of The Patient Protection and Affordability Care Act, H.R. 3590 (ACA) and the Health Care and Education Reconciliation Act, H.R. 4872 (HCERA).

The ACA sets forth certain requirements for the Health Insurance Marketplace for qualified individuals and qualified small businesses and their employees. Functions of the Marketplaces include development and maintenance of a website that facilitates the determination of eligibility for advance premium tax credits and cost-sharing reductions, as well as eligibility for Medicaid, CHIP, and Qualified Health Plans (QHP).The website must also present the consumer with standardized comparative information on qualified health plans.

Web-based consumer tools for the Marketplace are supported by sophisticated back-end products and processes, including a Federal data hub for verification purposes, an insurer (“issuer”) database, a health plan rating engine, and a rating/eligibility engine that pulls together rating information and business rules to provide real-time quotes to prospective enrollees.

Consumer web-based tools on both the Individual and SHOP Marketplaces must operate in the context of other information required to be presented on the website, including information on how to access other available resources for assistance (e.g., “Find Local Help”) and standardized educational information about health insurance concepts, information about the health insurance Marketplace, and content on subsidies, exemptions, penalties, and taxes.

Purpose of the proposed customer service survey

^{This brief survey has
been desi}^gⁿ^{ed
to focus on several key areas thought to be central to successfully
shopping for and enrolling in an insurance plan that provides a good
fit to the needs of the user. It will}^b^e^c^o^ndu^c^t^{ed
on a continuous basis and aggregated weekly. It}^will^gi^v^e^C^M^S^the
abi^lⁱ^t^y^t^o^{learn
where the website is effective and identify opportunities for
improvement in other areas. CMS recognizes that different types of
users will have different needs and that satisfaction with the site
may be related to these characteristics. Thus, the survey asks for
details on the specific needs and objectives of the consumer at the
time of the visit as well as basic consumer characteristics that can
be used to classify visitors into categories that reflect distinct
user profiles.}

The survey will provide information that can enhance our understanding of various aspects of the HealthCare.gov portal experience, such as the amount of time users spend using the website, ease of finding needed information, ease of performing each of several steps required to enroll for coverage (creating an account, completing an application, shopping and comparing health plans, and enrolling in a plan). It also examines perceptions such as confidence in ability to choose a Marketplace health plan and overall satisfaction with the website. Characteristics of respondents are collected as part of the survey to enable CMS to conduct in depth data analysis, such as comparison of the user experience between new enrollees and re-enrollees.

Weekly cross-sectional samples are designed to allow CMS to:

 Continually examine user experience in terms of ease of use

 Continually discover where improvements to the information or tools are required to improve the customer experience

Continually assess the impact of changes made to formatting or presentation of materials on the site

 Continually measure the satisfaction and confidence of website users

 Continually compare how experience and satisfaction vary by customer characteristics

However, the study design is not intended to support a temporal trends analysis, nor will it be used to conduct a summative evaluation of CMS’ communication effort.

It should be noted that this work is largely undertaken in the "context of discovery" as opposed to the "context of justification" (c.f. Reichenbach^¹). Formal statistical hypothesis testing will be secondary to developing a better understanding of the current experience of Marketplace website users.

Through a random intercept of website users, the survey sample will be generally representative of the population of audiences/customers that are served by the online Health Insurance Marketplace. CMS uses statistical tools often applied in more rigorous contexts to ensure that sample sizes will be adequate to detect differences and changes over time. Yet, formal null hypothesis-based statistical tests play a relatively minor role in CMS’ strategy. CMS intends to apply flexible continuous quality assessment modeling approaches to characterize the interaction of users with the web site materials. These models examine the effects of various aspects of the web site on selected outcomes related to consumer perception of the site. The approach allows one to assess the extent to which the site is effective as well as the specific features of the site that are most important to a specific group of users. This can help guide cost-effective quality improvement for the site over time.

This type of analysis will take priority over the more traditional survey-analytic approaches that has been applied in our other work (such as chi-square/log linear analysis for contingency tables), but this focus is due to our desire to use these data for continuous quality improvement rather than program evaluation purposes. Throughout, our focus in this effort remains on improving our understanding of the users’ experience in making this important healthcare-related decision.

The main goal of the survey will be to examine performance and effectiveness of the website in enabling users to successfully obtain health insurance online. The survey is designed and organized into several sections: Overall Experience, Looking for/reading information, Creating a HealthCare.gov Account, Viewing plans and costs, Starting or completing the Application, Shopping and comparing health plans, Enrolling in a plan, Overall satisfaction, and About you (demographics and other characteristics). CMS will not use the results of this survey to develop prevalence estimates (e.g., of the reason for lack of insurance). Rather these data will inform decisions regarding specific priorities for improvement of HealthCare.gov.

B. Justification

1. Need and Legal Basis

^This^w^o^rk^c^oⁿ^tri^bu^t^es^t^o^C^M^S’^ef^f^o^rts^t^o^ac^hⁱ^e^v^e^the^m^a^nd^at^e^s^o^f^t^h^e^P^atⁱ^ent^P^r^o^t^e^cti^oⁿ^and

Affordable Care Act of 2010. The law includes provisions to provide an online Marketplace that offers clear information and tools to enable consumers to shop for and make informed decisions about health insurance. It also requires CMS to conduct consumer research aimed at informing the building and enhancement of the website.

The work requested in this clearance allows CMS to gather information to develop and continually improve its website. Without this research, it will be difficult for CMS to gauge the extent to which online information and tools are being received in a way that encourages informed choices, as mandated in the legislation. Untested online tools and materials can have unintended consequences, such as when untested content or materials lead to misunderstandings resulting in failure of an individual to get insurance they qualify for or to purchase an inappropriate plan for their situation.

2. Information Users

^The^Cen^t^ers^f^o^r^M^edⁱ^ca^r^e^aⁿ^d^M^edⁱ^caid^Se^r^vⁱ^c^es^will^u^se^{the
data collected in this quantitative survey}^t^oⁱ^m^p^r^o^v^e^p^r^o^g^r^a^moperations. The information collected will be useful and minimally burdensome for the public as required by the Paper Reduction Act.

3. Use of Information Technology

The HealthCare.gov user survey will be conducted using on-line methods. ^{It does not require a signature from the respondent.}

4. Duplication of Efforts

^{This i}ⁿ^f^o^r^m^atⁱ^oⁿ^c^o^lle^c^ti^oⁿ^d^o^esⁿ^o^t^dup^lica^t^e^aⁿ^y^o^t^h^er^ef^f^o^rt,^and^t^h^eⁱⁿ^f^o^r^m^atⁱ^oⁿ^ca^nno^t^b^e obtained from any other source.

5. Small Businesses

^{This
survey does not affect small businesses or their employees.}

6. Less Frequent Collection

^Theⁱⁿ^f^o^r^m^atⁱ^oⁿ^w^ill
be^c^o^l^l^ec^t^{ed
monthly among a small number of website users}^t^o^al^l^o^w^the
A^g^ency^t^o^s^t^a^y^a^ware^o^f^{website
effectiveness}. Less frequent data collection will not support the need for feedback for use in continuous improvement.

7. Special Circumstances

^There^a^r^eⁿ^o^special^circ^u^m^stances^associated^with^thⁱ^s
in^f^o^r^m^atⁱ^oⁿ^c^o^llec^tⁱ^oⁿ^pa^c^kage.

8. Federal Register/Outside Consultation

^The⁶⁰^-^d^ay^Federal^Regis^te^r^noti^c^e^f^o^r
t^h^e^c^o^llec^tⁱ^oⁿ^p^ub^li^s^h^ed^oⁿ^{December 14, 2015 (80 FR 77352). The 30-day Federal Register notice
published on October 14, 2016 (81 FR 71101). No comments were
received.}

9. Payments/Gifts to Respondents

The consumer online survey will not provide incentives to respondents.

10. Confidentiality

^Respondents^t^o^t^h^e^sur^v^ey^t^oo^l^will^b^e^informed^{that
their responses will be kept private to the extent permitted by law,
under} the Privacy Act of 1974, as amended (45 CFR 5b).

Individual results will not be reported, in adherence to Section 903(d) of the Public Health Service Act (42 USC 299 a-1[c]) as follows: All information obtained will be reported in aggregate. No information will be published or released in other forms if the person who supplied the information or who is described in it is identifiable unless such person has consented (as determined under regulations of the Secretary) to its publication or release.

11. Sensitive Questions

^{There are no sensitive
questions. Demographic questions asked are optional.}

12. Burden Estimates (Hours & Wages)

^The^tab^l^e^s
be^l^o^w^p^r^es^eⁿ^t^t^h^e^es^tⁱ^m^a^t^{ed
annual b}^u^r^d^en^oⁿ^r^e^search
p^a^rtici^p^aⁿ^t^s.

	^{Open Enrollment}	^Off-Season	^{Burden Hours per Year}
^{Respondent Quota}	^1000/week	^50/week
^{Survey in minutes}	⁴	⁴
^{Total minutes per week}	⁴⁰⁰⁰	²⁰⁰
^{Total weeks}	¹²	⁴⁰
^{Total hours during open enrollment weeks}	⁸⁰⁰	¹³³	⁹³³

13. Capital Costs

^No^ca^p^ital^c^o^sts^are^as^s^o^cia^t^ed
with^t^h^is
iⁿ^f^o^r^m^a^tⁱ^oⁿ^c^o^llec^tⁱ^oⁿ^r^eq^u^es^t.

14. Cost to Federal Government

^The^e^s^ti^m^a^t^e^d^c^o^s^t^t^o^t^h^e^g^o^v^ern^m^{ent
is related to programming of the survey by an existing contractor,
data output to CMS, administrative monitoring of the survey results
and reporting results internally to CMS.}

15. Changes to Burden

^{This is a new collection.}

16. Publication/Tabulation Dates

^R^e^s^ul^t^s^f^r^o^m^t^h^e^aⁿ^a^l^y^sⁱ^s^o^f^t^h^e^s^e^d^a^t^a^w^il^l^b^e^p^r^e^s^eⁿ^t^e^dⁱⁿ^r^e^p^o^r^t^s^aⁿ^d^b^rⁱ^e^fing^s^f^o^r^seⁿⁱ^o^r^C^M^S

Management and others involved in the development and management of the HealthCare.gov website. There are no publication dates. There is no plan to disseminate the results publicly.

17. Expiration Date

^{The survey will display
the expiration date.}

18. Certification Statement

^The^p^r^o^p^o^sed
d^a^ta^c^o^ll^e^ctⁱ^oⁿ^d^o^esⁿ^o^tⁱⁿ^vo^l^v^e^aⁿ^y^e^x^cepti^oⁿ^s^t^o^t^h^e^ce^r^tificatⁱ^oⁿ^s^t^at^e^m^eⁿ^t

identified in OMB Form 83-I.

1 Reichenbach, H. (1938). Experience and prediction. An analysis of the foundations and the structure of knowledge. (Chicago: University of Chicago Press).

File Type	application/msword
Author	CMS
Last Modified By	WILLIAM PARHAM
File Modified	2016-10-14
File Created	2016-10-14