Website Usability Study Interview

Interview #_____

FSA TO6 MOBILE DEVICE/PC PHASE II USABILITY STUDY

INTERVIEW GUIDE

FINAL VERSION 08/04/14

SAMPLE RECORD: 1 BORROWER IN REPAYMENT

2. CURRENT COLLEGE /ADULT STUDENT

DEVICE USED:

1. Android tablet with 7-10” screen

2. Android phone with a 4.5-5.5” screen

3. iPhone 4s

4. iPhone 5 or 5s

5. iPad 3^rd generation

6. iPad mini

7. Laptop computer

8. Personal Computer (PC)

I. INTRODUCTION

The U.S. Department of Education’s office of Federal Student Aid seeks to continuously improve its interactions with those who seek information and guidance on the FSA website regarding education financing issues and options. To that end we are conducting a brief interview to assess our performance and provide you with an opportunity to share any ideas or feedback you may have to help us better serve you.

 

The interview is conducted by Market Connections, a Virginia-based professional market research company. Your responses will remain confidential to FSA; we hope this will allow you to feel free to be candid in your feedback to us.

As mentioned, the purpose of this interview is to talk about a website operated by the Department of Education’s office of Federal Student Aid, and get your impressions. We will soon go online and I’ll ask you to use [your [RECONFIRM MOBILE DEVICE] /the computer in front of you] to review some information and complete a number of tasks.

It is very important that you talk out loud through the steps you take while on the web site. Tell us what you are doing and why, and the feelings you have, including any problems you experience, and also any areas where things are going particularly well. We are interested primarily in your personal views on the content, terminology, the layout and detail of information, and how the task-completion process unfolds for you, along with any other insights you may have.

Again, don’t tell me what you think other people might say about the site – I want to know what you think.

This will take about 45 minutes.

We are recording so we can prepare a report, and I have colleagues observing in another room. As part of the recording process, we have a few cameras set up so that we can see what you see on [your device/the PC].

To reiterate, your comments will be treated as confidential. The information gained from this study will be used to improve FSA’s website to make it more intuitive and user-friendly. Do you have any questions? 0:03

II. INITIAL IMPRESSIONS OF THE SITE

When we invited you here, you told us you were: [READ APPROPRIATE STATEMENT]

1. [BORROWER IN REPAYMENT] A college student or graduate currently repaying one or more of your student loans.

2. [CURRENT COLLEGE/ADULT STUDENT] A current undergrad or graduate student. Are you a returning student repaying any student loans at this time? YES__ NO ___

1. Understanding your current situation, what topic areas or information would be most helpful or relevant to you personally if you were to visit a website titled “Student Aid.gov”? What specifically would you want to look for?

The site you are about to review contains a variety of information. You’ll be accessing the Test site instead of the live site. Please type in https://test.ise.ed.gov and take a few minutes now to get familiar with the home page. Also, feel free to explore any of the links you see; you do not necessarily need to stay on this page. I’m interested in where you’d go next. Please remember to talk aloud through your actions so we can know what you’re thinking as you explore the homepage. [INTERVIEWER: ALLOW THE RESPONDENT TO EXPLORE THE HOMEPAGE FOR A FEW MINUTES.]

2. Thinking about the home page itself, please tell me how you would grade it from A to F in terms of the ease of finding information on your particular areas of interest, where F is “poor” and A is “excellent.”

GRADE (A-F) ______

1. What makes you say that?

2. To what degree does the homepage give you an idea of what to expect when looking through the site?

3. Prior to our interview, had you ever visited this site previously? IF YES:

1. For what purpose?

2. Were you able to accomplish what you set out to do?

4. Had you ever specifically visited this site using your mobile device? For what purpose?

   1. IF YES: Are there any specific challenges or advantages to viewing and navigating this site with your mobile device versus on a PC or laptop? What exactly?

3. Given what you have seen so far, are you more likely to want to continue looking through the site, or are you more likely to close the site?

1. What makes you say that? 0:13

III. USABILITY TASKS

1. Understanding that you are a [current college student not yet repaying your loans/student loan borrower in repayment] where would you go for information on this topic?

Task 1: Other FSA Websites:

2. Where would you go to find other websites provided by Federal Student Aid? (IF NEEDED: https://test.ise.ed.gov/login and then click on the other tab)

3. What is the purpose of this type of information?

4. Is it helpful to you?

5. What do you think of the location of this information on the studentaid.gov website? (PROBE: Is it where you would have expected to find it? IF NOT: Where would you place a link to this information?)

________________________________________________________________________

Task 2: Site Log In: https://test.ise.ed.gov/login

6. Where would you go to log in to the site?

7. What kind of information would you expect to have access to, once you have logged in?

Now please touch/click the heading, “Log In to My Federal Student Aid”. (IF NEEDED: See the navigation bar at the top-right of the screen and then touch/click “Log In”)

For the purposes of today’s discussion, and to protect your privacy, I won’t ask you fill in your actual information. Instead, please type in the following information [TO BE PROVIDED BY FSA].

8. What’s the first thing you said to yourself when touching/clicking “Log In”?

Task 3: My Student Aid Homepage: https://test.ise.ed.gov/my-student-aid

9. Now that the login procedure has taken you to the “My Federal Student Aid” homepage, please take a minute to look it over, and give me your overall impression of this page.

1. What is its purpose?

2. What are your feelings about the layout and the way the information is organized and presented on-screen?

10. How useful is it to you as a [borrower in repayment/student not yet repaying loans]?

1. What makes you say that? 0:21

2. Based on what you have seen so far, is it possible to make a payment through the site?

3. Where would you go specifically to make a payment? (We’ll get back to that issue in a few minutes)

11. There are two ways to view this page: the version that lays out the details in a table format, and also through graphs.

1. Where would you go to change the way to view the page? (IF HAVING DIFFICULTY: touch/click the icon)

2. What are the bars communicating to you? (IF HAVING DIFFICULTY: red bar is principal, gray is interest; green bars are grant amount)

12. Given the [device/PC] you are using to view the information which format – table or graph – do you prefer to view this page?

1. What’s the reason behind your preference? (PROBE: format/layout, readability given the size of the screen)

13. There is also an option to show all of the information, or collapse it. How would you go about doing that? (IF HAVING DIFFICULTY: touch/click blue bars)

1. How useful or helpful is that feature?

14. Now, on this page, find the heading called “Your Program Enrollment”. Take a moment to read the statement below that heading, and tell me what you think about it. (IF DIFFICULTY LOCATING: The statement begins, “The length of your program of study impacts your eligibility…”)

1. Is this statement easy or difficult to understand?

2. IF DIFFICULT TO UNDERSTAND: What is it about the statement that makes it difficult to understand? (PROBE IF NECESSARY: terminology, sentence structure, length, level of detail)

15. Now find the section titled “Common Questions”, and take a minute to look it over. To what extent is this section valuable to you?

1. What leads you to that conclusion?

2. Is this where you would have expected to find this information? (IF NOT: where do you suggest this information be placed?) 0:32

1. Now find the section titled “Glossary” and take a minute to look it over. To what extent is this section valuable to you?

1. What leads you to that conclusion?

17. Now find the section titled “Quick Links” and take a minute to look it over. To what extent is this section valuable to you?

1. What leads you to that conclusion?

18. Now find the “Contact Us” section. To what extent is this section helpful to you?

1. What leads you to that conclusion?

Task 4: Loan Detail page https://test.ise.ed.gov/my-student-aid/loan-detail

19. How would you find more information about a specific loan?

20. a. What is this information telling you?

1. Describe how you would go about making a payment. (PROBE AS NEEDED)

2. To what degree is the information easy, or difficult, to understand? What specifically?

3. Is there anything about this page that would cause you to stop or give up the process of making a payment? What exactly? (PROBE: the amount or type of information or instructions, missing information, or the way the information appears on the page)

1. IF NOT ADDRESSED ABOVE: Is it an issue of not being able to understand the information, or is it because there is not enough information to guide you?

2. IF NOT ADDRESSED ABOVE: What could be done make the process more understandable or intuitive for you?

Task 5: Grant Detail page https://test.ise.ed.gov/my-student-aid/grant-detail

21. Now how would you find more information about your grants? (IF NEEDED: Click on the name of the grant).

22. What is this information telling you?

1. To what degree is the information easy, or difficult, to understand?

2. What could be done make it easier to understand?

0:42

23. Now that you’ve reviewed several aspects of the studentaid.gov website and the My Federal Student Aid information, at any point during our discussion would you have given up looking on the site because of the way the information was formatted or laid out?

24. Realistically, would you have performed any of these tasks on your mobile device?

1. Which ones?

2. Were there any tasks you definitely would not have tried with your mobile device?

25. Do you have any final advice for the Office of Federal Student Aid to consider, in order to provide a better website experience and to provide information and guidance for you with your student loans?

1. What would you like to see added to StudentAid.gov in the future?

That concludes our interview. Thanks very much for your input. It was very helpful! 0:45

Market Connections, Inc. Proprietary and Company Confidential Page 6 of 6