Department of Labor
Employment and Benefits Security Administration
Customer Interviews
Discussion Guide
11/21/2022
Purpose: To explore and understand the customer awareness and satisfaction levels with EBSA and the information and resources needed; to explore the knowledge, perceptions, beliefs, intentions, barriers, motivators, and behaviors of customers; to gather insights on information and preferences on website content and features that will increase their familiarity, satisfaction, and desired behaviors related to engaging with EBSA
Methodology: CMRignite will screen interested customers to ensure they meet the characteristics of the target audience. We will conduct 10-15 web UX interviews in English or Spanish using a virtual platform, using a screen share format, and following a think-out-loud protocol. The protocol includes asking participants to share their thoughts and reactions out loud and explain them as the moderator shows EBSA resources and information on the website.
Target Audience: Multicultural workforce and their beneficiaries in the United States (customers of EBSA)
Age range: 18-67 years
Has employer-based health insurance and/or an employer-based retirement plan OR is a beneficiary of an individual who has them
Works at a company with more than 100 employees
Does not have coverage through programs such as Medicare, Medicaid, Indian Health Services, Veterans Affairs Health Care, or TRICARE
Ensure diversity in race/ethnicity, geographic location, income levels, ability levels (including visual disabilities), and those who lack access to resources
Assumptions:
The customers will have a basic level of understanding of technology and will be able to follow along as the interviewer takes them through the website, and answer questions about their experience.
The customers will consent to the interview being audio recorded. Transcripts will be prepared from the audio recordings for qualitative analysis.
____________________________________________________________________________
Introduction
Thank you for participating in this call. I’m conducting interviews on behalf of the Department of Labor (DOL) Employment Benefits Security Administration (EBSA). EBSA has the mission of ensuring the security of the retirement, health, and other workplace-related benefits of American workers, retirees and their families with job-based retirement plans, health plans, and other welfare benefit plans.
For the next 30-40 minutes, we’ll be walking through the Department of Labor Employment Benefits Security Administration website, and you will be sharing your thoughts and feedback about the website. This interview is focused on what you like and don’t like about the website, what information is helpful or is needed, and any suggestions you have for improvement. Our questions will focus on how you experience the website. We will go through different parts of the website together, and I will ask for your thoughts about some of the pages on the site.
Our conversation will be audio recorded and notes will be prepared for analysis; only our team will have access to the recordings. We’ll write a report based on all the interviews that we do with people like yourself, but individual participants will not be identified. The report will be prepared by our agency, CMRignite, to help inform improvements that may be made to the website. The report will be used for internal use only.
[Obtain verbal consent to record]
[Show the paragraph below on the screen]
The OMB control number for this collection is 1225-0093 and expires on 02/29/2024. According to the Paperwork Reduction Act of 1995, no person is required to respond to a collection of information unless such collection displays a valid OMB control number. Collection of this information is authorized by OMB. The obligation to respond to this collection is voluntary. We estimate it takes about 40 minutes to complete.
Participant Introduction
[The intent of these questions is to gather background info about the interviewees and develop rapport]
A few questions about you to get started…
Where do you work? hobbies, interests, etc.?
Have you visited any government websites recently? Which ones and what were the websites like?
Have you heard about the Department of Labor (DOL) Employment Benefits Security Administration (EBSA) before this call? What do you know/think about it? How did you hear about it?
[Interviewer shares screen with participant and pulls up the DOL EBSA website and puts the link in the chat for the interviewee to access the website]
[Interviewer scrolls through the website showing the interviewee the overall layout first for the landing page and then explains that they will be visiting six pages in total (home page, about us page, ask EBSA, resources including outreach events, MHPAEA, Surprise Billing), spending a minute or two on each page and describing what can be viewed from a content and format perspective]
[Ask if participant has visited the website; if not frame these questions hypothetically] Why would you visit this site/what would be your reason for visiting? What would you usually come to this site for? What would you do when you visit this site? If you can, please walk me through where you would go on this site (what you click on, what you would look for, etc.)?
What comes to mind first when looking at the home page/landing page of the website? What do you think about the look and feel of this page? Is there anything that stands out? Is there anything you would change (navigation, layout, colors, design, content, etc.)? What topics grab your attention? Where would you go first?
Okay now, we’re going to visit the About Us tab. [Briefly explain what’s on the page.] What information would you expect to find on this page?
The page has information about the history of the organization, what the organization does, the ERISA Advisory Council, the mission of DOL EBSA, the State All Payer Claims Databases Advisory Committee, and a page with a map to find your regional office nearest you.
What stands out to you? What do you think about the look and feel of the page? [Probe: layout, design, content] What would you find useful about this page? Where would you go first? Is there a particular link that you would visit more often? Why, or why not? Is there any additional links/information that you would like to see on any of these pages? What information do you want to know about DOL EBSA?
Okay now, we’re going to visit the Ask EBSA. What information would you expect to find on this page? [Briefly explain what’s on the page.]
The Ask EBSA page has information about programs and services, can provide answers to your questions, and assist you if you believe you have been denied a retirement, health, disability, or other ERISA - [a federal law that protects people who have job-based retirement and health plans] - employee benefit.
What stands out to you? What do you think about the look and feel of the page? [Probe: layout, design, content] What have you (or would you find) found useful about this page? Where would you go first? Is there a particular link that you would visit more often? Why, or why not? Is there any additional links/information that you would like to see on any of these pages? Could you see yourself using this page to contact a benefits advisor? Why or why not? What factors might make you more or less likely to submitting a form or calling EBSA?
Now, I’m going to ask you about the resources page. [Before visiting the page] What information would you expect to find on this page?
[The resources page has links to fact sheets, FAQ’s, reports, forms, seminars, webcasts, and more]
Is this something you could find yourself looking at? What stands out to you (i.e., colors, design, content)? Is there a particular link you would like to go to now? Is this information helpful? Is there any additional information you would like included on this page?
Now, I’m going to ask you about the page with information regarding the Mental Health Parity and Addiction Equity Act. [Before visiting the page] Have you heard of this before? What information would you expect to find on this page?
[The Mental Health Parity and Addiction Equity Act states that most health insurance providers must cover mental health and substance use disorder benefits the same way they cover physical health benefits]
What stands out to you? What do you think about the look and feel of the page? [Probe: layout, colors, design, content] What would you find useful about this page? Where would you go first? Is there a particular link that you would visit more often? Why, or why not? Is there any additional links/information that you would like to see on any of these pages? Is this page something you think you would share with a friend/family member?
Okay now, we’re going to visit the Surprise Billing page. What information would you expect to find on this page? [Briefly explain what’s on the page.]
[The Surprise Billing page has more information and explanation of benefits including requirements related to surprise billing, provider nondiscrimination, group health plan service providers, prescription, and drug health care spending, etc.]
What stands out to you? What do you think about the look and feel of the page? [Probe: layout, colors, design, content] What would you find useful about this page? Where would you go first? Is there a particular link that you would visit more often? Why, or why not? Is there any additional links/information that you would like to see on any of these pages? Is this page something you think you would share with a friend/family member?
Are there other pages that you would visit on this site? Where would you click? What do you like about these pages? Do you find the information useful? Is there information you would like to see on these pages about job-based health insurance and retirement benefits?
[Ask if participant is bilingual or speaks multiple languages] Does this website make sense to you as an [insert language] speaker? Do you have any challenges accessing the information and resources available? Do you think this page would make sense to someone you know that may not understand English as well? What website tools or resources should be available to those that don’t understand English as well? Did you see the language translations button? If not, where would you put a button for language translations?
[Ask if participant has a visual impairment, or someone they know has visual impairment] Is this website easy to read and understand for you or someone with a visual impairment? What is challenging about this website? Are there certain pages that are more/less difficult? What could make this page easier to read? [Probe on colors, font size, images, etc.]
What words would you use to describe this website? Please explain.
Where would you look for information online about your rights related to job-based health insurance and retirement benefits? [Probe social media, websites, etc.]
Where would you prefer to see information about DOL EBSA? [Probe about advertising out-of-home, digital, social media, and your job/HR department, etc.]
[The following questions will only be asked if time permits]
Are there any factors that would make you more likely to visit the website?
What factors make you less likely to visit the website?
What other websites are useful for people that are looking for info about your rights related to job-based health insurance and retirement benefits? What other websites do you like because they are organized/formatted well?
Is there any other feedback you have regarding the DOL EBSA website that you’d like to share?
Closing
Thank you so much for participating in this interview with us today. As a reminder, your information will be kept confidential and will not be shared with anyone outside our research team. You will receive a $75 gift card for your participation. Can you confirm the preferred email address you would like to receive this gift card?
November
2022
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Sebastian Becerra |
| File Modified | 0000-00-00 |
| File Created | 2023-09-11 |