CMS-10458_Supporting_Statement_Part_A Extension 6-24-16_updated

Supporting Statement Part A: Individual and SHOP Marketplace

Customer Service Surveys (CMS-1048; OMB control number 0938-1203)

CMS is requesting clearance for two surveys to aid in understanding levels of awareness and customer service needs associated with the Health Insurance Marketplace established by the Affordable Care Act. Because the Marketplace will provide coverage to the almost 50 million uninsured in the United States through individual and small employer programs, we have developed one survey to be administered to

individual consumers most likely to use the Marketplace and another to be administered to small employers most likely to use the Small Business Health Options portion of the Marketplace. This request is part of broader data collection effort designed to support the program goal to improve customer satisfaction for people and business that likely eligible for health insurance coverage via the Marketplace.

A. Background

A1. Broad goals of the Marketplace outreach and education research program

^{The broad and overarching goals of CMS’ Office of Communications Marketplace Outreach and}

Education Research program are listed below. It should be noted that the surveys presented in this request for clearance only partially address the first and second points, and will not be used in the context of the third and fourth bullets.

 Provide evidence to guide recommendations for development and continuous improvement of outreach and education materials related to the Health Insurance Marketplace.

 Examine the environmental context, including current levels of awareness, knowledge, and perceptions about the Health Insurance Marketplace, within which CMS outreach and education information will be received by our target audiences.

 Understand how the target audiences change and shift as the Health Insurance Marketplace is rolled out, implemented, and utilized.

 Evaluate the effectiveness of CMS’ Marketplace outreach and education efforts.

To accomplish these goals, CMS will engage in a series of research tasks, including collecting primary qualitative and quantitative research from Marketplace target audiences; analyzing available public and proprietary data; and conducting scans of the environment to assess relevant Marketplace materials, information, and opinions being shared from sources outside of CMS. A request for generic clearance covering other projects that will address these four goals of the program (as well as other social marketing tasks) will be submitted to OMB in the future to support the primary data collection goals of the program. CMS is requesting clearance of two brief surveys designed to collect customer awareness and needs assessment information on a quarterly basis.

A2. Purpose of customer service surveys

^{These brief surveys, designed to be conducted quarterly, will give CMS the ability to obtain a rough}

indication of the types of outreach and marketing that will be needed to enhance awareness of and knowledge about the Marketplace for individual and business customers. CMS’ biggest customer service need is likely to be providing sufficient education so consumers: a) can take advantage of the Marketplace and b) know how to access CMS’ customer service channels. The surveys will provide information on media use, concept awareness, and conceptual or content areas where education for customer service delivery can be improved. Awareness and knowledge gaps are likely to change over

time based not only on effectiveness of CMS’ marketing efforts, but also of those of State, local, private sector, and nongovernmental organizations.

Quarterly cross-sectional samples are designed to allow CMS to:

 Continually examine awareness of the Marketplace with the goal of helping identify types of outreach and education efforts that might benefit customers.

 Continually assess where knowledge gaps exist so that CMS can determine where additional information resources might be expect to improve customer understanding and use of the Marketplace.

 Continually determine whether customers have appropriate expectations about their interactions with and use of the Marketplace.

 Continually examine how awareness and knowledge vary by customer segment, so that communications can be created that best target and reach the right people with the right information.

However, the study design will not support a temporal trends analysis, and cannot be used to conduct a summative evaluation of CMS’ communication effort.

It should be noted that this work is largely undertaken in the "context of discovery" as opposed to the "context of justification" (c.f. Reichenbach¹). Formal statistical hypothesis testing will be secondary to developing a better understanding of the potential Marketplace customers.

CMS has some suggested initial values for audience segments that may be important, defined in part by demographics and cognitive styles, and has attempted to include items that will allow exploration of these dimensions as they interact with the environment. Steps are being taken to ensure the survey samples are representative of the audiences/customers that are expected to be served by the emerging Health Insurance Marketplace, and CMS will be exploring the relationships between the dimensions defined by responses to these sets of questions and ongoing outreach and education activities. CMS has used tools often applied in more rigorous contexts to ensure that sample sizes will be adequate to detect the strongest of these relationships but development of understanding of this dynamic is in the very early stages. Formal null-hypothesis based statistical tests play a relatively minor role in CMS’ strategy. CMS intends to apply flexible statistical models (e.g., generalized linear models for multinomial, ordinal, and count variables) to explore these relationships, as well as the more traditional approaches (such as chi-square/log linear analysis for contingency tables), but the goal remains on improving our understanding of the consumers that we are seeking to serve.

The general approach to developing these surveys is similar to that taken in surveys of consumers by organizations such as Kaiser Family Foundation (who have posted information on the demographic characteristics of the uninsured and those in the non-group market that can help us assess the representativeness of our samples), Pew (who have been tracking trends in internet use as a function of key demographic audience segments), and Robert Wood Johnson Foundation (who, among other health and social marketing related topics, have conducted research with individuals who are likely to be eligible for Medicaid through the Medicaid expansion to uncover salient messages for raising awareness and knowledge among those audiences), as well as longstanding CMS-administered surveys (for example: National Medicare Education Community (NMEC) Survey of Medicare, administered from 1998-2008; Medicare Current Beneficiary Survey (MCBS), administered 1991 to present; and National Survey of Low Income Parents [CHIP]) that all aim to provide a means for understanding customer needs in a social marketing context). The items and scales used in the Marketplace surveys are derived from or directly taken from these sources, as they have been effective in data collection efforts with similar foci. Additionally cognitive testing was performed with individuals who would be target participants in the surveys to ensure that question intent and response options are not confusing or frustrating to potential participants

CMS is also including questions that will allow assignment of individual respondents from these surveys into psychographic grouping based on a unique consumer segmentation approach. This approach appreciates that healthcare consumers are not monolithic. KRC, a marketing research firm, has conducted extensive cross-generational research and found that socio-behavioral factors such as health status, importance of information, current health activity and security/stress around healthcare issues, along with demographics such as age and income are indicators of engagement with healthcare issues including prevention activities, attitudes toward healthcare quality, and decisions related to health insurance (Mark Richards, Senior VP, KRC, personal communication, 2011). They have created a unique psychobehavioral consumer segmentation approach which sorts the U.S. adult population into six distinct healthcare consumer groups. Each of the six segments (Informed, Healthy & Educated; Sick, Active & Worried; Mature & Secure; Healthy & Young; Passive & Skeptical; and Vulnerable & Unengaged) have unique characteristics, behaviors, and attitudes related to healthcare and respond to different marketing approaches. Analysis of a recent nationally representative survey U.S. adult population found that 92% of the uninsured fell into one of three of these segments. A 10-item battery that includes demographic as well as attitudinal dimensions can reliably assign consumers to the appropriate consumer segment. CMS has included these 10 items in the individual consumer survey because CMS believes that insights from this audience segmentation approach will result in a better understanding of the Health Insurance Marketplace audience and help to guide refinement of Marketplace-specific communications.

The main goal of the surveys will be to examine awareness and knowledge about the Health Insurance Marketplace, as measured by items in the “awareness” and “knowledge” topic sections; perceptions about obtaining health insurance, as measured in the “current/past behaviors and perceptions” topic section; and intent to purchase insurance and use the Health Insurance Marketplace, as measured by items in the “intent” topic section of the survey. To better develop and revise messages that appropriately target subsegments of uninsured individuals and those who purchase insurance in the individual market, questions measuring insurance situation, past behaviors, internet use, and demographic and psychographic characteristics allowing for detailed segmentation will also be included. CMS will not use the results of this survey to develop prevalence estimates (e.g., of the reason for lack of insurance). Rather these data will inform decisions regarding what information to message, how and where to present that information, and how to prioritize the types of messages being delivered to reach those most in need. The tables below provide details of the items included in the surveys for which clearance is requested. These tables provide the topics, questions, potential analyses, and conclusions to be drawn by topic.

Individual Marketplace Survey

Topic	Questions	Analyses	Conclusions/Actions
Screening Criteria	DG2: Age DGM1: Decision maker ING1: Insurance status ING2A: Insurance status ING2B: Insurance status	Primarily used to screen in appropriate respondents: Individuals between age 18-64 who are involved in making decisions about their own health insurance and are currently uninsured (N=625) or pay for their own individual health insurance plans	Individuals will be included/excluded in the survey implementation based on responses to these questions.

Topic	Questions	Analyses	Conclusions/Actions
		without employer or government assistance (N=275).
Insurance status	ING1: Insurance status	Comparisons on several variables will be made between uninsured and individual insured using Chi square, z-test for proportions for independent groups, T-test for means for independent groups, and regression analyses.	Observed differences will be used to inform decisions regarding message framing, placement, and prioritization to best target these audiences.
Insurance situation Satisfaction item and scale based on CMS NMEC and MCBS items.	ING3: Length of time uninsured ING4: Reason for being uninsured ISG1. Satisfaction with health insurance situation	Open ends will be categorically coded and frequencies will be run on all responses. Comparisons on several variables will to examine how insurance situation factors impact awareness, knowledge, and behavioral intent. Chi square, z- test, T-test, and regression analyses will be used.	Comparisons will enable examination of how variables such as length of time being uninsured and level of satisfaction with current situation as it pertains to health insurance status so that education can be crafted for individuals across the range of situations. This will also assist knowledge of extent to which these variables impact awareness of HIM messaging.
CMS segmentation Based on KRC items and scoring algorithm.	DG2: Age BG1A. Information seeking behaviors ATG1: Healthcare planning HSG2: Chronic conditions DG10: Income	These items will be used to create composite scores that segment each respondent into consumer audience segments: CMS expects that these will fall predominantly into the 3 segments that have been most commonly reported among the uninsured: Healthy and Young; Sick Active and Worried; Passive and Unengaged. Comparisons on several variables will be made to examine segmentation-specific similarities and differences. These will be treated as categorical grouping variables in the analysis.	It is likely that the uninsured in these groups will share some common behavioral and attitudinal characteristics that relate to being uninsured, but they may also differ in important aspects that will be essential to consider in our outreach, education, and social marketing efforts.
Current/past behaviors and perceptions	BG1B: Consideration and action toward obtaining health insurance in past 90 days PG1: Perceived difficulty in choosing a new plan ISG2: Sources of information about health insurance	Frequencies will be run and potential for composite “action” score will be determined using behavior items. Comparisons on awareness, knowledge, intent will be made between active/inactive. Chi square and regression will primarily be used. Perceived difficulty may mediate or moderate actions, frequencies will be obtained and potential med/mod effects will be examined on awareness, knowledge and intent. Open end sources will be coded and frequencies obtained – these will be examined as a function of knowledge of HIM.	Differences in awareness, knowledge, and intent by level prior behavioral action will assist in determining how outreach and education should be prioritized to reach active vs. inactive individuals.
Awareness	ISG1: Website awareness open	CMS will examine unaided and aided awareness and will examine	Examining unaided awareness will provide top-of-mind measure for

Topic	Questions	Analyses	Conclusions/Actions
	AWG1A: Attention to information about health insurance AWG2A-AWG5: Awareness of HIM AWG1B: Attention to healthcare law news AWG2, BG5: heard of/visited Healthcare.gov	awareness as a function of insurance status, situation, segmentation, and current/prior behaviors. Unaided open ends will be coded, frequencies on dichotomous and interval variables will be obtained. Chi square analyses will be done to compare groups with multichotomous regression approaches used as appropriate. Additionally, intent and knowledge as a function of awareness will be measured.	CMS. Aided awareness will assist in determining whether outreach and education messaging and materials is penetrating to the point of increasing recognition. Results will guide prioritization, targeting efforts, and development and revision to messages with an aim of heightening awareness among the target audiences.
Knowledge	KG7: Knowledge of HIM features and benefits	CMS will examine knowledge of key features and benefits of the HIM that are planned to be included in outreach and education materials. Frequencies and Chi square analyses will be obtained overall, as a function of insurance status, situation, segmentation, and current/prior behaviors. Additionally, intent as a function of knowledge will be examined using regression analyses.	Examining knowledge will provide insight into whether key features and benefits of the HIM are understood by the target audience. If knowledge of key features is low, CMS will revise outreach and education materials to enhance knowledge.
Intent	GR8: Intent to use and desire to learn more about HIM BG8: Intent to use once details shared	Frequencies will be obtained and intent will be examined as a function of insurance status, situation, and segmentation via Chi square analyses. Regression analyses will be used to examine intent as a function of knowledge and awareness.	In order for outreach to be salient it must not only raise awareness and knowledge, but it must also be compelling. CMS will use the results of this portion of the survey to improve materials so that they promote behavioral intent to use the HIM among the target audiences.
Internet use	IUG4-IUG15: Use of Internet and devices	Frequencies will be obtained.	Because the primary means for using HIM will be the Internet, it is important to get a sense of extent to which the target audiences are using the Internet on computers and other devices.
Demographics	DG1-DG10	Frequencies will be obtained. Data will be weighted for nonresponse as needed. Some comparisons will be made using Chi square on awareness, knowledge, and intent variables.	The demographics will be used to describe and characterize the survey respondents.

SHOP Marketplace Survey

Topic	Questions	Analyses	Conclusions/Actions
Screening Criteria	SBO2-NBO1	Primarily used to screen in appropriate respondents: Individuals who are the primary decision makers about health insurance for businesses that have	Individuals will be included/excluded in the survey implementation based on responses to these questions.

Topic	Questions	Analyses	Conclusions/Actions
		between two and 49 employees.
Insurance decisions	ING3A-INBO2: Length of time offering/not offering health insurance, portion of health insurance covered, use of brokers	Open ends will be coded and frequencies obtained on all variables. Comparisons between businesses that do/do not offer health insurance will be made using Chi square analysis.	Observed differences will be used to inform decisions regarding message framing, placement, and prioritization to best target business decision makers.
Current/past behaviors and perceptions	BG1B: Consideration and action toward obtaining health insurance in past 90 days PG1: Perceived difficulty in choosing a new plan ATG2: Perceived impact of healthcare law on business ISG2: Sources of information about health insurance	Frequencies will be run and potential for composite “action” score will be determined using behavior items. Comparisons on awareness, knowledge, intent will be made between active/inactive. Chi square and regression will primarily be used. Perceived difficulty may be related to actions, frequencies will be obtained and potential med/mod effects will be examined on awareness, knowledge and intent. Open end sources will be coded and frequencies obtained – these will be examined as a function of knowledge of SHOP.	Differences in awareness, knowledge, and intent by level prior behavioral action will assist in determining how outreach and education should be prioritized to reach active vs. inactive business decision makers.
Awareness	ISG2: Website awareness open AWG1A: Attention to information about health insurance AWG2A-AWG5: Awareness of SHOP AWG1B: Attention to healthcare law news AWG2, BG5: heard of/visited Healthcare.gov	CMS will examine unaided and aided awareness and will examine awareness as a function of insurance decisions and current/prior behaviors. Unaided open ends will be coded, frequencies on dichotomous and interval variables will be obtained. Chi square analyses will be done to compare groups. Additionally, intent and knowledge as a function of awareness will be measured.	Examining unaided awareness will provide top-of-mind measure for CMS. Aided awareness will assist in determining whether outreach and education messaging and materials is penetrating to the point of increasing recognition. Results will guide prioritization, targeting efforts, and development and revision to messages with an aim of heightening awareness among the target audiences.
Knowledge	KG7: Knowledge of SHOP features and benefits	CMS will examine knowledge of key features and benefits of the SHOP that are planned to be included in outreach and education materials. Frequencies and Chi square analyses will be obtained overall, as a function of insurance decisions, and current/prior behaviors. Additionally, intent as a function of knowledge will be examined using regression and propensity analyses.	Examining knowledge will allow CMS to understand what key features and benefits of the SHOP are understood by the target audience. If knowledge of key features is low, CMS will revise outreach and education materials to enhance knowledge.
Intent	GR8: Intent to use and desire to learn more about SHOP BG8: Intent to use once details shared	Frequencies will be obtained and intent will be examined as a function of insurance status, situation, and segmentation via Chi square analyses. Regression and propensity analyses will be used to examine intent as a function of	In order for outreach to be salient, it must not only raise awareness and knowledge, but it must also be compelling. CMS will use the results of this portion of the survey to improve materials so that they promote behavioral intent to use

Topic	Questions	Analyses	Conclusions/Actions
		knowledge and awareness.	the SHOP among the target audiences.
Internet use	IUG4-IUG15: Use of Internet and devices	Frequencies will be obtained.	Because the primary means for using SHOP will be the Internet, it is important to get a sense of extent to which the target audiences are using the Internet on computers and other devices.
Business Characteristics and Demographics	SBO3-DG10: Business information ING1-DG10: Individual demographics	Frequencies will be obtained. Some comparisons will be made using Chi square on awareness, knowledge, and intent variables.	The demographics will be used to describe and characterize the survey respondents.

The sample design is sufficient to provide quarterly snapshots of knowledge gaps, but as noted above is neither sufficient to allow temporal trends analysis nor to allow extrapolations of awareness to the universe of potential Marketplace users. A few of the major limitations include: small sample sizes (particularly for those with insurance that is not employer subsidized), low expected response rate (approximately 20% - which are expected to be typical for telephone survey data collection efforts), only English speakers and Spanish speakers will be included, and perhaps most importantly, we expect the composition of the “uninsured” subpopulation to change over time. The Spanish survey for individual consumers will be directly transcreated from the finalized English individual consumer survey instrument and will be submitted for clearance under a non-substantive change submission to OMB.

Why does this need to happen now?

^{CMS efforts related to education and customer service began in January 2013. Other efforts, from} unaffiliated groups are also beginning to appear, and this activity is expected to accelerate. A delay in implementing this work could delay CMS’ ability to counteract any misimpressions that develop, thereby hampering consumers’ ability to find and choose the most appropriate health insurance options for themselves and their families.

The overall goal is to maximize the extent to which consumers have access to useful sources of Marketplace program information in a form that can help them make the most of their benefits and options.

B. Justification

1. Need and Legal Basis:

^{This work contributes to CMS’ efforts to achieve the mandates of the Patient Protection and}

Affordable Care Act of 2010. The law includes provisions to communicate health and health care information clearly; promote prevention; provide patient-centered care; assure equity and cultural competence; and deliver high-quality care. All of these general goals can be enhanced through timely consumer research.

The work requested in this clearance allows CMS to develop and continually improve upcoming information products and marketing campaigns which promote the goals of legislation related to health literacy, cultural sensitivity and effective use of Marketplace program benefits. Without this research, it will be difficult for CMS to gauge the extent to which information and education around health insurance benefits, options, and other related information is being received in a way that encourages informed choices, as mandated in the legislation. There are also less obvious costs associated with waste of communication resources and lost opportunities if messages and materials are not perceived as relevant, are not clearly understood, or do not lead to the appropriate consumer behavior. Untested messages can also have unintended consequences such as when untested content or materials lead to misunderstandings resulting in project failure or loss of program credibility.

2. Information Users:

^{The Centers for Medicare and Medicaid Services will use this information to improve program}

operations. The information collected will be useful and minimally burdensome for the public as required by the Paper Reduction Act.

3. Use of Information Technology:

The SHOP survey will be conducted using on-line methods. Work undertaken as part of this collection can help inform whether IT-based methods for data collection can be expanded to more general consumer audiences in the future. More generally, ^{measuring and helping to improve salience of CMS outreach, education, and communications} efforts using information technology and emerging communication technology channels is a key part of our work and is reflected in assessments of overall perception of CMS programs and in consumer response to specific communication activities that use IT-based approaches.

4. Duplication Efforts:

^{This information collection does not duplicate any other effort, and the information cannot be}

obtained from any other source.

5. Small Business:

^{Programs that affect small business owners and employees of small businesses are included in the}

legislative mandate, so small businesses will be included in this data collection. CMS is mindful of the need to minimize burden on small businesses. For example, the small business survey will be conducted online so that small business owners may take them when time permits and at their leisure.

6. Less Frequent Collection:

^{The information will be collected quarterly to allow the Agency to stay aware of informational}

needs. Less frequent data collection will not support this initiative.

7. Special Circumstances:

^{There are no special circumstances with this information collection package.}

8. Federal Register/Outside Consultation:

^{The 60-day Federal Register notice for the collection published on April 22, 2016 (81 FR 23705). No comments were received.}

9. Payments/Gifts to Respondents:

The consumer survey will not use incentives, but the on-line SHOP component of the survey employs a respondent panel that offers a small incentive to those who complete the survey (“points” in an rewards program valued at approximately $4.00 per completed survey).

10. Confidentiality:

^{Respondents to the survey tool will be informed that their responses will be kept private to the extent permitted by law, under} the Privacy Act of 1974, as amended (45 CFR 5b).

Individual results will not be reported, in adherence to Section 903(d) of the Public Health Service Act

(42 USC 299 a-1[c]) as follows: All information obtained will be reported in aggregate. No information will be published or released in other forms if the person who supplied the information or who is described in it is identifiable unless such person has consented (as determined under regulations of the Secretary) to its publication or release.

11. Sensitive Questions:

^{The main issues addressed in this work deal with how individuals seek new information, how they}

use information, and how they make decisions about their health care and CMS program participation. These are typically not considered sensitive areas. However, there is no requirement to answer any question.

12. Burden Estimates (Hours & Wages):

^{The tables below present the estimated burden on research participants. The hourly burden is}

broken into the screener, which will be used to select survey participants and the survey, which will be administered to those who pass through the screening process. The individual survey screener is designed so that the complete survey will be administered only to individuals who are primary healthcare decision-makers who are currently uninsured or are currently purchasing health insurance through the individual insurance market. The business screener is designed so that the complete survey is administered only to small business owners or decision makers of businesses with fewer than 50 full time employees, who are primary decision makers about health insurance for their businesses.

It is anticipated that for each of the four waves, the screeners will be administered to 4,500 English speaking individuals and that 900 (20% of those screened) will complete the survey. For each wave of the Spanish language survey, 3,000 will be screened and 600 will complete the survey. For each wave of the business survey, 750 will be screened and 300 (40% of those screened due to precision added when inviting from exiting panel) will complete the survey. The tables below represent the burden for fielding of each screener and survey.

Table 1: Burden for four waves of Individual and SHOP Marketplace studies

Instrument	Method	Number of Respondents per Survey Wave	Hours per Response	Total Respondent Hours per Survey Wave	Number of Survey Waves	Total Respondent Hours	Total Cost
Screener: Individual, English	Telephone	4500	0.03	135	4	540	$143,500
Survey: Individual, English	Telephone	900	0.20	180	4	720	$430,500
Screener: Individual, Spanish	Telephone	3000	0.03	100	4	400	$143,500
Survey: Individual, Spanish	Telephone	600	0.20	120	4	480	$430,500
Screener: Small Business	Online	750	0.03	25	4	100	$72,360
Survey: Small Business	Online	300	0.20	60	4	240	$423,640

13. Capital Costs:

^{No capital costs are associated with this information collection request.}

14. Cost to Federal Government:

^{The estimated cost to the government for conducting the research covered in this request will be} approximately $1,644,000 per year in contract costs including labor hours, materials and supplies, overhead, general and administrative costs, access to the sampling frames, and fees. These cost estimates are based on competitive bids provided by our contractor and our experience in conducting this research.

15. Changes to Burden:

^{There are no changes in burden for this collection.}

16. Publication Tabulation Dates:

^{Results from the analysis of these data will be presented in reports and briefings for senior CMS}

Management and others involved in the development of CMS’s communication strategy. There are no publication dates. There is no plan to disseminate publically the results.

17. Expiration Date:

^{No exemption is sought.}

18. Exceptions to Certification for Paperwork Reduction Act Submissions

^{The proposed data collection does not involve any exceptions to the certification statement}

identified in line 19 of OMB form 83-I.

1 Reichenbach, H. (1938). Experience and prediction. An analysis of the foundations and the structure of knowledge. (Chicago: University of Chicago Press).

^{Supporting Statement A: Individual and SHOP Marketplace Customer Service Surveys}

2