Form #10 Form #10 Experimental Website

List of Tables

Table 1 Steps of the Health Plan Choice Experiment 2

Table 2 ARMS of the Helath Plan Choice Experiment 2

List of Figures

Figure 1 Tutorial Page 1 2

Figure 2 Tutorial Page 2 2

Figure 3 Tutorial Page 3 2

Figure 4 Tutorial Page 4 2

Figure 5 Start Page of the Web Site 2

Figure 6 Data Selection Screen ARM #4 2

Figure 7 Performance Overview Screen 2

Figure 8 Performance Overview ARM #2 2

Figure 9 Service Quality Tab 2

Figure 10 Getting Medical Care Tab 2

Figure 11 Keeping Enrollees Healthy Tab 2

Figure 12 Helping Sick Enrollees Get Better Tab 2

Figure 13 Enrollee Reviews Tab: Complaints 2

Figure 14 Enrollee Reviews Tab: Comments 2

Figure 15 Plan Choice Confirmation 19

Appendix A: Data Selection Screens for all Six ARMS of the Experiment 20

Health Plan Choice Experiment(s)

Introduction

The Health Plan Choice Experiment consists of a series of steps in which respondents are transferred from the Knowledge Networks Web site to the HealthPlanGuide Web site and then back to Knowledge Networks. Table 1 summarizes the steps of the experiment.

Table 1 Steps of the Health Plan Choice Experiment

Step 1	Respondent views the Knowledge Networks introduction to the experiment
	
Step 2	Respondent completes the pre-test questionnaire on Knowledge Networks’ Web site
	
Step 3	Respondent is transferred to HealthPlanGuide Tutorial and views the 4-page tutorial
	
Step 4	Respondent uses HealthPlanGuide Web site to “select” a health plan
	Start screen displayed; then respondent is assigned/sent to one of six ARMS 
	ARM 1	ARM 2	ARM 3	ARM 4	ARM 5	ARM 6
	Control	Experimental
	
Step 5	Respondent is transferred back to the Knowledge Networks Web site
	
Step 6	Respondent completes the post-test questionnaire on the Knowledge Networks Web site
	
Step 7	Respondent views conclusion and thank you on the Knowledge Networks Web site

The Health Choice Experiment consists of six ARMS: one of which is a control ARM and five of which are experimental ARMS. (See Table 2.) The ARMS differ based on the types of information displayed to the respondents and on the number of health plans listed on the HealthPlanGuide Web site.

As many as four types of information will be shown depending upon the ARM involved. The information types are as follows:

• Performance Overview: Summary of the information available in the ARM. (Shown to all, but with varying content.)

• Service Quality: Survey data using CAHPS items; fabricated CAHPS data.

• Treatment Quality: Fabricated HEDIS-like data

• Enrollee Reviews: Mimics volunteered enrollee comments about health plans

combined with fabricated rates of enrollee complaints.

ARMS numbered 1-3 present different sets of information to respondents. ARM 4 presents the same information as ARM 3 but offers a larger choice set – the number of health plans listed doubles from 4(ARM 3) to 8 (ARM 4). ARMs 5 and 6 presents the same information as two previous arms (ARMs 2 and 4 respectively), but having first primed negative emotions in respondents to assess how anxiety alters consumers’ propensity to process complex information sets and choice options.

The ARMS are described in detail in Table 2.

Table 2 ARMS of the Health Plan Choice Experiment

ARM Numbers	Tabs				Choice Set Size (Number of Health Plans)
	Performance Overview	Service Quality (CAHPS scores)	Treatment Quality (HEDIS scores)	Enrollee Reviews (Volunteered comments and complaint rates)
Control
1					4
Experimental
2					4
3					4
4					8
Experimental, Emotional Priming
5					4
6					8

About This Document

This document shows example screenshots for one of the ARMS of the experiment. The selected example is ARM #4 because it presents the largest amount of data. (ARM 4 is highlighted in Table 2.) All of the screenshots come from Steps 3 and 4 of the experiment.

To see the HealthPlanGuide Web site, please go to: http://www.healthplanguide.org/site/intro/.

User name: wowza

Password: cahps15

The experiment opens with an introduction from Knowledge Networks followed by a Pre-test Questionnaire. Upon completion of the Pre-test Questionnaire, respondents are provided with an introduction to the HealthPlanGuide tutorial pages and transferred to the HealthPlanGuide Web site. The respondents view the tutorial pages and then enter the main HealthPlanGuide site.

Tutorial Page 1

(Phase 3)

Figure 1 is the first tutorial screen respondents will see. It is intended to explain what they will find on the HealthPlanGuide web site and to familiarize them with the labels the site will use. When respondents are finished with this page, they will click “Continue” at the lower right hand side of the page.

Figure 1 Tutorial Page 1

Tutorial Page 2

(Phase 3)

Figure 2 is the second screen of the tutorial. It is intended to introduce respondents to the various things they can do while they are on the HealthPlanGuide Web site. When respondents are finished with this page, they will click “Continue” at the lower right hand side of the page.

Figure 2 Tutorial Page 2

Tutorial Page 3

(Phase 3)

Figure 3 shows a screenshot of the third page of the tutorial. It introduces the respondent to the sorting and filtering functions available on the site. When respondents are finished with this page, they will click “Continue” at the lower right hand side of the page.

Figure 3 Tutorial Page 3

Tutorial Page 4

(Phase 3)

Figure 4 shows the fourth and final screen in the tutorial. It introduces the respondent to how they will “choose” a health plan. When respondents are finished with this page, they will click “Continue” at the lower right hand side of the page. This will bring the respondent to the Start page of the HealthPlanGuide Web site.

Figure 4 Tutorial Page 4

HealthPlanGuide Start Screen

(Phase 4)

Figure 5 is the Start screen for the HealthPlanGuide web site. All respondents will see this screen. All respondents will enter their zip code, and select the number of miles they are willing to travel. This screen is intended to increase the feeling of reality on the site. However, the data and options subsequently presented do not vary with the choices respondents make on this screen.

When respondents have finished making choices, they hit the submit button in the lower right hand corner of the screen.

Figure 5 Start Page of the Web Site

Data Selection Screens

(Phase 4)

Figure 6 shows what happens after respondents hit the “submit” button on the previous screen. The respondent is taken to one of six Data Selection screens, depending upon the ARM of the experiment in which they are participating. The screenshot shown below displays the Data Selection screen respondents will see if they are in ARM #4, the most complex ARM of the experiment. All the Data Selection Screens are shown in Appendix A.

Tabs displayed on the Data Selection screen for ARM #4 are:

• Performance Overview

• Service Quality (CAHPS scores)

• Treatment Quality (HEDIS scores)

• Enrollee Reviews (Patient comments and complaints)

Respondents will select among the tabs. The next few pages will show what is displayed when the respondent selects each tab. Respondents may select any of the tabs in any order and can display only one tab at a time. They must select at least one tab in order to choose a health plan.

Figure 6 Data Selection Screen ARM #4

Performance Overview Screen

(Phase 4)

Figure 7 is a screenshot of what respondents see when they select the Performance Overview Tab. The screen is scrollable and includes a total of eight plans for ARMs 4 and 6, four plans for all other ARMs. The plans are initially listed in alphabetical order, but they can be sorted, highlighted, or filtered by the respondents.

For ARM #4, the Performance Overview provides a summary of the measures of Service Quality (CAHPS), Treatment Quality (HEDIS) and Enrollee Reviews (Comments and Complaints) for four health plans. (See Figure 7.)

Figure 7 Performance Overview Screen

This page is scrollable

Only a portion of the page is shown here.

For ARMS with less data, the Performance Overview page will include summaries of the data that is included that ARM. For example, the Performance Overview for ARM #2 will include summaries of the Service Quality and Treatment Quality data, but no Enrollee Reviews information will appear. (See Figure 8.)

This page is scrollable. Health plans initially appear in alphabetical order, but can be sorted, highlighted, or filtered by respondents.

Figure 8 Performance Overview ARM #2

This page is scrollable

Only a portion of the page is shown here.

Service Quality Screen #1

(Phase 4)

Figure 9 displays a screenshot of what the respondents will see when the Customer Service tab is chosen. The Customer Service tab presents CAHPS data attributed to individual health plans. The data shown for individual plans will not vary across ARMS; however, more data is presented for ARMS 4 and 6 because these ARMS list eight plans rather than the four plans in ARMS 1-3 and 5 will list 12 physicians. The page is scrollable. Plans are initially listed in alphabetical order, but can be sorted, highlighted or filtered by the respondent.

Figure 9 Customer Service Tab

This page is scrollable

Only a portion of the page is shown here.

Service Quality Screen #2

(Phase 4)

Figure 910 displays a screenshot of what the respondents will see when the Getting Medical Care tab is chosen. The Getting Medical Care tab presents CAHPS data attributed to individual health plans. The data shown for individual plan will not vary across ARMS. The page is scrollable. Health plans are initially listed in alphabetical order, but can be sorted, highlighted or filtered by the respondent.

Figure 10 Getting Medical Care Tab

This page is scrollable

Only a portion of the page is shown here

Treatment Quality Screen #1

(Phase 4)

Figure 101 displays a screenshot of what the respondents will see when the Keeping Enrollees Healthy tab is chosen. The data shown on this screen is intended to mimic HEDIS data. Data for individual plans will not vary across ARMS. However, for ARMS 1-3 and 5, four plans will be listed, whereas for ARMS 4 and 6 eight plans will be listed. The page is scrollable. Health plans are initially listed in alphabetical order, but can be sorted, highlighted or filtered by the respondent.

Figure 101 Keeping Enrollees Healthy Tab

This page is scrollable

Only a portion of the page is shown here.

Treatment Quality Screen #2

(Phase 4)

Figure 102 displays a screenshot of what the respondents will see when the Helping Sick Enrollees Get Better tab is chosen. The data shown on this screen is intended to mimic HEDIS data. Data for individual health plans will not vary across ARMS, but the number of plans presented does, in the manner described above. The page is scrollable. Physicians are initially listed in alphabetical order, but can be sorted, highlighted or filtered by the respondent.

Figure 112 Helping Sick Enrollees Get Better Tab

This page is scrollable

Only a portion of the page is shown here.

Enrollee Reviews Screen #1

(Phase 4)

Figure 123 displays a screenshot of what the respondents will see when the Enrollee Reviews tab is chosen. The data shown for individual plans is the same for all ARMS in which it is displayed, though again the number of plans varies across the different ARMs. The page is scrollable. .

The first Enrollee Reviews screen reports on the number of complaints (hypothetical data) files against each health plan at the relevant state agency, Data for two different years (2007 and 2008) are presented to users.

Figure 123 Enrollee Reviews Tab: Complaints

This page is scrollable

Only a portion of the page is shown here.

Enrollee Reviews Screen #2

(Phase 4)

Figure 124 displays a screenshot of what the respondents will see when the Enrollee Reviews tab is chosen. The data shown for individual plans is the same for all ARMS in which it is displayed, though again the number of plans varies across the different ARMs. The page is scrollable. .

The second Enrollee Reviews are comments volunteered by patients. They were constructed in part from actual patient comments found on web sites. These comments were modified and masked so that real names or identifying information is removed. The other source of comments is a set of fabricated comments created by the researchers using words and phrases with known emotional valence.

Figure 134 Enrollee Reviews Tab: Comments

This page is scrollable

Only a portion of the page is shown here.

Plan Choice Confirmation Screen

(Phase 4)

Figure 145 shows a screenshot of the screen that is displayed once a respondent has chosen a health plan. The respondent is asked to confirm their choice. If respondents confirm their choices, they are returned to the Knowledge Networks system to fill out the Post-Test questionnaire. If respondents do not confirm their choices, they are returned to viewing health plans.

Respondents must choose a health plan before they can go on the Post Test questionnaire. However, they may choose a plan at any point in the experiment after they land on a tab. Respondents do not have to view all of tabs before selecting a health plan. Indeed, they may pick any tab, review the plans only with the information available through that tab, and make their choice at that point. Or they may look at other tabs if they wish.

Figure 145 Plan Choice Confirmation

Appendix A: Data Selection Screens for all Six ARMS of the Experiment

Arm #1 Tabs Displayed:

• Performance Overview

• Service Quality (CAHPS scores)

ARM Number	Title	Description
1	Control Arm	CAHPS Scores Only [Two Drill-Down Pages/Tab, 4 plans Listed]

Arm #2 and Arm #5 Tabs Displayed:

• Performance Overview

• Service Quality (CAHPS scores)

• Treatment Quality (HEDIS scores)

ARM Number	Title	Description
2 5	Experimental Arm 1 Experimental Arm 4	CAHPS Plus HEDIS Scores [Four Drill-Down Pages/Tabs, 4 plans Listed] CAHPS Plus HEDIS Scores [Four Drill-Down Pages/Tabs, 4 plans Listed] Emotional Priming

Arms #3, #4, and #6 Tabs Displayed:

• Performance Overview

• Service Quality (CAHPS scores)

• Treatment Quality (HEDIS scores)

• Patient Reviews (Patient Comments)

ARM Number	Title	Description
3 4 6	Experimental Arm 2 Experimental Arm 3 Experimental Arm 5	CAHPS Plus HEDIS Scores Plus Comments [Five Drill-Down Pages/Tabs, 4 Health Plans Listed] CAHPS Plus HEDIS Scores Plus Comments [Five Drill-Down Pages/Tabs, 8 Health Plans Listed] CAHPS Plus HEDIS Scores Plus Comments [Five Drill-Down Pages/Tabs, 8 Health Plans Listed] Emotional Priming

23 of 23