WISS RG3 Usability Testing - OLA

Usability Testing Moderator's Guide for IRS Online Accounts

Introduction

Welcome to the Usability Testing session for the IRS Online Accounts platform. This guide will provide you with the structure and tools necessary to conduct a successful testing session. Our objective today is to gather actionable insights that will help improve user experience on the OLA platform.

This guide will walk you through each step of the process. Remember to assure participants about the confidentiality of their responses and to obtain consent for any recordings.

Preparation

To prepare for the session, ensure you have a quiet, well-lit room, a computer with internet access, and any necessary equipment. This will depend on whether the focus groups are held on an IRS campus or over Teams. Make sure to familiarize yourself with the OLA platform, focusing on its key features and navigation.

Set up the testing environment to resemble a typical user setting or make sure that the users have access to OLA and you are sharing your screen in Teams. You should also be recording and a notetaker should be present.

Conducting the Test

Greet the participant warmly, explaining the purpose and duration of the test. Read the script below. Make sure they understand there are no right or wrong answers and that they should act as they normally would while using the OLA.

Introduce the think-aloud protocol, encouraging the participant to verbalize their thoughts, actions, and feelings as they navigate through the tasks

READ: My name is [Your Name] and I'm here from Pacific Consulting Group on behalf of the Internal Revenue Service (IRS). I will be taking notes of our session and recording our meeting. Also on the line is another analyst who will be taking notes to ensure accuracy. If you are uncomfortable with a recording, please let us know.

The Internal Revenue Service (IRS) is working to improve its digital service through Online Accounts. The Online Account (OLA) web application is a digital services platform provided by the Internal Revenue Services (IRS). This research project seeks to understand user behavior, experiences, and expectations when interacting with the OLA platform The study aims to identify areas of improvement to enhance user experiences. We encourage you to think-aloud and verbalize your thoughts, actions, and feelings as you navigate through the tasks. There are no wrong answers and feel free to ask questions.

Ground Rules

Before we continue, I would like for you to listen to and acknowledge the following statements about this information collection.

• For the IRS to speak with the public, we are required to have approval from the Office of Management and Budget. Their approval number for this project is 1545-2290. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to the: IRS Special Services Section, 1111 Constitution Avenue, NW, SE:W:CAR:MP:T:M:S - Room 6129, Washington, DC 20224.

• You understand that the information and recording is for research purposes. All the data the IRS collects will be kept private to the extent allowed by law., nor will it appear in any written reports or publications.

• Your participation is voluntary, and you don’t have to answer every question.

• There are no right or wrong answers. If you have any areas of confusion, comments, or questions during our discussion today, please let me know.

• This session should last about 60 minutes.

• Lastly, you understand and consent to the use of the audio recording and screen sharing.

Task Scenarios

The following tasks are designed to simulate typical user interactions with the OLA. For each task, guide the participant through the process, and use the probing questions to gain deeper insights into their experience.

Section 1: OLA Usage Drivers

1. Tell us about the first time you used OLA and what prompted you to do so?

   1. How did you hear or learn about OLA?

   2. On a scale 1-5, with 1 indicating very difficult and 5 very easy, how easy or difficult was it to set up your online account?

2. How often do you login to OLA?

3. [Ask taxpayer to login to OLA] From the IRS homepage, can you navigate to OLA and login?

   1. [Observe how they login from the IRS homepage http://www.irs.gov]?

   2. On a scale 1-5, with 1 indicating very difficult and 5 very easy, how easy or diffult was it to navigate the OLA hompage and get to where you wanted to go?

Section 2: OLA Design and Usability Feedback

1. [Participant should now be on the OLA homepage.] Tell us what you find most useful.

   1. Tell us more about what was most useful (Elaborate).

2. Tell us what you findleast useful on the current homepage.

   1. Tell us more about what was least useful (Elaborate).

3. On a scale of 1-5, from very poor to very good, how would you rate the presentation of information on OLA?

   1. What information would you keep?

   2. What information would you remove?

   3. What are additional features that you would like to see?

   4. Is there any other tax related information that you would like to see here?

4. [Task – participant should be on the OLA homepage] Please locate your monthly installment agreement payment reminder notice (AKA CP 521) Walk us through your process of finding that notice.

   1. Once you have located your CP 521, walk us through what you would do next?

   2. On a scale of 1-5, with 1 indicating very difficult and 5 very easy, how easy or difficult was it to locate your CP 521 notice?

   3. Navigate now to the page where you would make a payment.

   4. On a scale of 1-5, with 1 indicating very difficult and 5 very easy, how easy was it to make a payment?

   5. Now navigate to your payment plan options. On a scale of 1-5, with 1 indicating very difficult and 5 very easy, how easy was it to change your payment plan? Section 3: Communications and Taxpayer Expectations

1. [Task – participant should be on the OLA hompage] Navigate to where you can modify your notification settings.

   1. On a scale of 1-5, with 1 indicating very difficult and 5 very easy, how easy was it to find your notification settings?

   2. What would you like to receive notifications about?

      1. How often would you like to receive notifications

   3. How would you prefer to receive notifications?

      1. Moderator Note: Ask question for each piece of information the participant expressed receiving a notification.

      2. Probe: Email, text, mail, etc.

2. Demonstrate how you would set your preferred way of receiving notifications.

   1. What improvements can be made here?

   2. What are some examples of how other financial instituons treat similar notifications?

      1. Do you prefer how other financial institutions use notifications?

Section 4: Privacy and Security

1. On a scale of 1-5 (1 least confidence, 5 being most confident), how much confidence do you have in the IRS to protect the privacy and security of your data?

   1. Tell me more about your response?

   2. Are there certain types of information you are hesitant to share through OLA?

Section 5: Debriefing

1. Reflect on your testing experience. What do you expect from a self-service tool?

2. On a scale of 1-5, rate how well OLA matched your expectations.

   1. Tell me more about your response.

3. What is the one change or improvement you would most like to see in OLA?

   1. What could the IRS learn from other similar websites/services you've used with good self-service?

   2. What tax-related information might you find outside of OLA that you wish was included?

4. Please, share any final thoughts or suggestions about OLA that you haven't expressed yet.

Conclusion

• Summarize key points discussed.

• Reiterate the importance of their contributions.

• Discuss the next steps and how their feedback will be used.

• Thank participants for their time and valuable insights.