CMS-10710. [Medicare Research GenIC#13-TRIPTK] Discussion Guide_UX Task & Talk

Status: Original and derived artifacts are available for this document.

Download: docx | pdf | html

Medicare Research
Discussion Guide 
UX ‘Task & Talks’

OMB Control Number: 0938-1382
GenIC #13 - TRIPTK


Overview (for Internal Teams only)

This session observes how participants naturally use Medicare.gov by asking them to complete simple tasks while thinking aloud, revealing points of confusion and confidence as they occur in real time. The goal is to understand how different users actually move through the website so we can identify practical, evidence-based improvements to the interface and content.

The estimated participation burden for this interview is 60 minutes, which includes all discussion and optional tasks. The structure and activities in this guide follow PRA-aligned principles, emphasizing low burden, voluntary participation, and plain-language clarity for all respondents. Note that Moderator notes & review guidance is in blue for clear instructions. 

Methodology 
UX “Task & Talk” — task-based evaluation. Note that tasks are optional & can default to verbal discussion as needed. 
Moderator(s)
TRIPTK (Michael, Etienne, CX team) 
Total across 3 markets
n36, mix of dyads (beneficiary & caregiver) & solo participation
Format
60 min virtual session

The conversation will flow as follows: 

Opening Remarks 
(5 min / 0-0:05)
Set expectations, confirm comfort, and align on the purpose of the session.
General Medicare Discussion
(10 min / 0:05 – 0:15)
Understand baseline experiences, perceptions, and habits related to Medicare and Medicare.gov.
Freeform Exploration
(15 min / 0:15 – 0:30)
Observe natural usage and hear real-time impressions as participants explore the website.
Task-led Exploration
(30 min / 0:25 – 0:55)
Guide participants through key tasks to surface challenges, confusion, and opportunities for improvement.
Wrap-Up + Close
(5 min / 0:55 – 0:60)
Capture final reflections and summarize the overall experience.




Opening Remarks
(5 min / 0-0:05)

Welcome participants, go over guidelines for the discussion, and share a privacy statement.

●  Hello!  My name is <MODERATOR> and I talk to people about their experiences using different websites & services. Today we’ll focus on Medicare.gov  – and your experience using it.
 
Moderator to exchange pleasantries / learn names of participants, etc
 
● Great! I want to remind you this is an open discussion. I’m looking to learn from you, not test you.                                           There are no “wrong” or “right” answers. So please, be open and direct – don’t worry about hurting my feelings or offending me. I work for an independent market research firm, we’ve been hired to do this research on behalf of Medicare.  Your participation and answers will have no impact on your personal Medicare account or experience. I’m here for one purpose: to understand your unique points of view. You’re my expert today. And if you want to stop or take a break at any time, please tell me.

● Participation is completely voluntary. You may skip any question or choose to participate verbally. Please do not share any personal identifiers such as full names, Social Security Numbers, Medicare Numbers, addresses, or medical record numbers.

● Before we dive deeper, I’d like to read this disclosure statement: According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-1382 GenIC #12. This information collection is being conducted to help improve informational notices for people with Medicare and the results will be used to ensure that people with Medicare understand important information about their plans. The time required to complete this information collection is estimated to average less than 60 minutes in total, including the time to hear instructions, ask and answer questions and complete any written or verbal tasks. This information collection is voluntary, and CMS will keep the information private to the extent provided by law. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

●  This discussion is to be used for research purposes only and is being recorded to aid in analysis and reporting across many such discussions. The information you provide will not be associated with you individually and will be aggregated with other participants. Do we have your permission to record our conversation and to use the information you provide in this discussion for research purposes? [Get verbal yes from all participants]

●  Do you have any questions before we get started?


General Medicare Discussion
(10 min / 0:05 – 0:15)

Get an understanding of participants’ general POV on Medicare and the website  
Throughout this entire discussion, we will be collecting information about:
    • Network Mapping (how does Participant interact with .gov, who might help them & how)
    • Navigation Style (how they move throughout the .gov and any DIY hacks)
    • Friction (where and why do they experience dissonance, functionally and emotionally)
    • Emotions (how do they feel throughout this journey)

All below Q&A will be probed with ‘Wh-‘ words as appropriate.
(Who, What, Where, Why, When, How)

●  For most of our time on this call, we’re going to be exploring the Medicare.gov website. 
But first, I’d love to learn a bit more about your overall Medicare experience.

● How long have you been on Medicare?
● Which plan are you on?
● How did you go about selecting that plan?
● Is it the same one you’ve had for the duration of your time?
   (if switched) What prompted you to switch?
   (if same) Tell me more, why have you stayed with this plan?

(Pivoting from general conversation to .gov)

●  Before we tour the website, can you tell me what your impressions are of it?
    • How did you find out about the website?
    • When would you visit the website?
    • Is this on your computer or your phone or your tablet? Do you use an app?
    • Can you tell me about a recent time where the website worked well for you?
    • What about a time when it didn’t work as you hoped or expected?
    • What did you do then? (Listen for how they troubleshoot)
    • Is it you who goes to the website? Does anyone else help you? (listen for family, friends etc..)
 

Freeform Exploration
(15 min / 0:15 – 0:30)

Get an understanding of participants’ initial POV on the website once we’re all looking at it together  
This section is a user-directed website journey & narration to find patterns and probe on key issues
 
●  OK – let’s dive into the website. For the rest of our time together, I want you to vocalize your thoughts, feelings and (any) frustrations as we explore the Medicare.gov website. No detail is too small.   
 
Moderator to get Respondent share their screen.
As Respondent explores the website, pay attention to who else they say plays a role in their Medicare experience, and where they intersect with their user journey (Caregivers, brokers, family, doctors, pharmacists, friends etc…)
 
●  What are your first impressions as you land on the homepage?
●  What stands out?
●  Does this look and feel like what you would expect from the Medicare website?
●  Is there anything you’d expect that you don’t know where to find it?
●  If you have any challenges with the website, what do you do?
    (call someone, search elsewhere, rage quit, try again, etc…)  

●  You said that when you go to this website, you typically do <answer from previous section>.                               
    Can you please walk and talk me through what you would do on a typical website visit
  	
Moderator to probe on Respondent commentary as they use the website.
Listen for stress and specific UX vexation & difficulty.
 
General Probes: 

●  What’s working well for you?
●  What feels like ‘a win’ and/or gives you confidence?
●  What’s surprising or frustrating?
●  What would you like it to do instead?
●  Are there any examples of other websites that do this well?

 If they say ‘I don’t really go here’ or aren’t able to self-direct and use the website, then proceed directly
 to guided task walkthroughs.


Task-led Exploration
(30 min / 0:25 – 0:55)

Explore key pages, features and functions to probe further and capture the user experience
 
If any of the below questions/sections have not yet been addressed organically during the User-Directed portion, Moderator will ask the Respondent to do certain tasks 
 
●  OK - if you were looking to figure out plan costs, how would you do that on this website? 

Moderator to listen as Respondent narrates their journey. If they cannot find the correct page, Moderator to guide them to the Learn what Medicare costs page, and then continue questions.

    • Have you been to this page before?
    • What do you think of it?
    • Does this page provide you with the info that you need?
    • What might you change, if anything?

 Let’s do <specific task> and talk about your impressions.

    • What do you think about this?
    • Did it work as expected?
    • What information do you wish were more clear?
    • What made this task easy or difficult?
    • What would you change, if anything?
 
Moderator to repeat the above questions pulling from  the following tasks & sections, based upon what Moderator has learned from the interview about what might be most interesting to the Respondent.

    • If you needed to compare and contrast plans…Compare coverage options
    • If you wanted to find specific drug plans…Find health & drug plans
    • If you need to find a doctor… Find care providers
    • If you wanted to understand how Medicare connects with other insurance…
How Medicare works with other coverage
    • If you were to file a claim or deal with a complaint…Claims, Appeals & Complaints
 

Wrap-Up & Close
(5 min / 0:55 – 0:60)

Conclude the interview.
●  Okay we are just about done. Do you have any final thoughts for me?
●  What 3 words would you use to describe this website experience?!
●  If you had a magic wand – what would you change about the Medicare website experience?
    (If their answer is about rules, coverage, money etc.. steer them back toward the .gov and/or general    
    digital experience for their response) 
●  We really enjoyed chatting with you and greatly appreciate all of your time. Thank you very much!
●  I’ll let our recruiting partner know we had a great session, and they’ll take care of your gratuity.



PRA Disclosure Statement
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-1382 GenIC #12. This information collection is being conducted to help improve informational notices for people with Medicare and the results will be used to ensure that people with Medicare understand important information about their plans. The time required to complete this information collection is estimated to average less than 60 minutes in total, including the time to hear instructions, ask and answer questions and complete any written or verbal tasks. This information collection is voluntary, and CMS will keep the information private to the extent provided by law. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.