B1 StudentAid.gov Mobile Device Usability Evaluation IDI Guide FINAL 11/27/12
Interview #_____
FSA MOBILE DEVICE USABILITY STUDY
FINAL VERSION 11/27/12
CURRENT COLLEGE /ADULT STUDENT
HIGH SCHOOL SENIOR
PARENT OF HS SENIOR
DEVICE USED:
Android Tablet
ANDROID PHONE
iPad
iphone
I. INTRODUCTION
The U.S. Department of Education’s office of Federal Student Aid seeks to continuously improve its interactions with those who seek information and guidance on the FSA website regarding education financing issues and options. To that end we are conducting a brief interview to assess our performance and provide you with an opportunity to share any ideas or feedback you may have to help us better serve you.
The interview is conducted by Market Connections, a Virginia-based professional market research company. Your responses will remain confidential to FSA; we hope this will allow you to feel free to be candid in your feedback to us.
As mentioned, the purpose of this interview is to talk about a website operated by the Department of Education’s office of Federal Student Aid, and get your impressions. We will soon go online and I’ll ask you to use your mobile device [RECONFIRM DEVICE USED] to review some information and complete a number of tasks.
It is very important that you talk through out loud the steps you take while on the web site. Tell us what you are doing and why, and the feelings you have, including any problems you experience and also any areas where things are going particularly well. We are interested primarily in your personal views on the content, terminology, the layout and detail of information, and how the task-completion process unfolds for you, along with any other insights you may have.
Again, don’t tell me what you think other people might say about the site – I want to know what you think.
This will take about 40 to 45 minutes.
We are recording so we can prepare a report, and I have colleagues observing in another room. As part of the recording process, we have a few cameras set up so that we can see what you see on your device.
To reiterate, your comments will be treated as confidential. The information gained from this study will be used to improve FSA’s website to make it more intuitive and user-friendly. Do you have any questions? 0:03
II. INITIAL IMPRESSIONS OF THE SITE
When we invited you here, you told us you were: [READ APPROPRIATE STATEMENT ]
[BORROWER IN REPAYMENT] An [undergrad/graduate] student with outstanding student loans.
[CURRENT COLLEGE/ADULT STUDENT] A current undergrad or graduate student. Are you a returning student repaying any student loans at this time? YES__ NO ___
[PARENT OF HIGH SCHOOL SENIOR] The parent of a high school senior intending to enter a two- or four- year college in the next year.
[HIGH SCHOOL SENIOR] A high school [senior/junior] who intends to enter a two- or four-year college in the [next year/next few years].
Understanding your current situation, what topic areas or information would be most helpful or relevant to you personally if you were to visit a website titled “Student Aid.gov”? What specifically would you want to look for?
The site you are about to review contains a variety of information. Please type in studentaid.gov and take a few minutes now to get familiar with the home page. Also, feel free to explore any of the links you see; you do not necessarily need to stay on this page. I’m interested in where you’d go next. Please remember to talk aloud through your actions so we can know what you’re thinking as you explore the homepage. [INTERVIEWER: ALLOW THE RESPONDENT TO EXPLORE THE HOMEPAGE FOR A FEW MINUTES.]
Thinking about the home page itself, please tell me how you would rate it on a scale of 1 to 10 in terms of the ease of finding information on your particular areas of interest, where 1 is “poor” and 10 is “excellent.”
Poor Excellent
1 2 3 4 5 6 7 8 9 10
What makes you say that?
So far, which features on the homepage have caught your attention? Please explain. [PROBE: specific features liked, disliked]
0:10
To what degree does the homepage give you an idea of what to expect when looking through the site?
When you first assessed the homepage, did you carefully read the text, skim it, or mostly ignore the text portion?
READ CAREFULLY____ SKIMMED ___ IGNORED___
IF SKIMMED/IGNORED: Please take few minutes to look over the text once more. WAIT TWO MINUTES AS THEY READ, THEN ASK: thinking about the text you read, in addition to any other information on the page, to what degree does the homepage give you an idea of what to expect on the site as a whole? How so?
Prior to our interview, had you ever visited this site previously? IF YES:
For what purpose?
Were you able to accomplish what you set out to do?
Had you ever specifically visited this site using your mobile device? For what purpose?
Are there any specific challenges or advantages to viewing and navigating this site with your mobile device versus on a PC or laptop? What exactly?
At this point, what suggestions, if any, do you have for improving the homepage to meet your needs?
IF MEGA MENU/BLACK NAVIGATION BAR MENTIONED, PROBE BRIEFLY, THEN ASK, “Is there anything else?”
Given what you have seen so far, are you more likely to want to continue looking through the site, or are you more likely to close the site?
What makes you say that?
0:20
III. USABILITY TASKS
TASKS: BORROWERS IN REPAYMENT 0:20
Understanding that you are a student loan borrower in repayment, where would you go for information on this topic?
Task 1: Review “Understanding Repayment” contents
Please touch the heading, “Understanding Repayment”. Studentaid.gov/repay-loans/understand. Looking at this information, what would you touch first to learn more?
What do you think of the contents covered in this section?
Is there too much information here to process, or not enough information to get a good sense of what you want to learn?
What about the amount of text to read? (Too much, too little, just right?)
What are your feelings about the terminology used – do you feel it was written with you in mind?
What are your feelings about the layout and organization of this information? (PROBE: Does it flow in a logical way?)
Is there anything you would change, or is it fine as is?
0:26
Task 2: Review Loan Forgiveness Options
Now look for information on loan forgiveness. [IF PARTICIPANT HAVING TROUBLE: touch “Repay Your Loans”.
What would you do next? [IF STILL HAVING DIFFICULTY: Touch “Forgiveness, Cancellation and Discharge” link. Then touch the link “Learn about forgiveness, cancellation or discharge”]
On this page, after scrolling down a bit, you’ll see a table. Take a moment to look it over. What is the purpose of this table?
Please tell me how easy or difficult it was to find this information, using a scale of 1 to 10, where 1 is “very difficult and 10 is “very easy”.
Very difficult to find Very easy to find
1 2 3 4 5 6 7 8 9 10
What suggestions, if any, do you have for improving the table? [PROBE IF NECESSARY: …in terms of the content, layout and design, terminology or any other factors?]
Task 3: Locate and use Monthly Loan Repayment Estimate Calculator 0:31
Touch the Repay Your Loans menu. Locate and use the repayment calculator to estimate the monthly payment amount for the Pay as You Earn option. You may use fictional loan amounts for this exercise. [IF PARTICIPANT IS HAVING TROUBLE LOCATING THE CALCULATOR: touch “Understanding Repayment”, then touch “Repayment Plans” and then scroll down to locate the Pay as You Earn option and the calculator for that option.]
Is this where you would have expected to find the calculator?
IF NOT: What needs to change to make the calculator easier to find?
IF NOT ALREADY ON THE “REPAYMENT PLANS” MENU: Now touch the “Repayment Plans” menu. Please locate a calculator. [IF PARTICIPANT IS STILL HAVING TROUBLE LOCATING A CALCULATOR: Scroll down to the table. You’ll see some Quick Links for different types of repayment plans, such as Standard Plan, Pay as you Earn and others. Touch the Pay as you Earn link. (IF NECESSARY: now scroll down to find the calculator to calculate your estimated loan payments for that repayment plan. You will see the “Calculate your Estimated loan Payments…” link)
Please tell me how you would rate the calculator itself on a scale of 1 to 10 where 1 is “not at all easy to use ” and 10 is “very easy to use.”
Not at all easy to use Very easy to use
1 2 3 4 5 6 7 8 9 10
Please tell me how you would rate the calculator itself on a scale of 1 to 10 where 1 is “not at all easy to understand the results” and 10 is “very easy to understand the results.”
Not at all easy to understand Very easy to understand
1 2 3 4 5 6 7 8 9 10
What makes you say that?
What feelings, if any, do you have in terms of locating the calculators? What exactly?
0:40
If you were on your own to look for the calculator, realistically, would you have continued to search for it or would you have abandoned the search and moved on to something else? What makes you say that?
What do you feel are the pros and/or cons of completing any of these tasks on your mobile device, versus doing so on a PC or laptop?
Realistically, would you have performed any these tasks with your mobile device? Which ones? Are there any you definitely would not have tried with your mobile device?
Now that you have reviewed several aspects of the studentaid.gov website, do you have any final advice for the Office of Federal Student Aid to consider, in order to make the site easier to use for those using a mobile device such as yours?
SKIP Q15 IF NO TIME
Having seen/used the calculator, and become acquainted with the repayment plan options, what would be your next step? [PROBE: How would you expect to go about making a payment?]
That concludes our interview. Thanks very much for your input. It was very helpful! 0:45
TASKS: CURRENT COLLEGE/ADULT STUDENTS 0:20
Understanding that you are a current college student straight out of high school, or a returning college student, you may or may not currently be repaying your student loans. In your case, are you repaying any student loans at this time? YES__ NO__
Task 1: Review “Understanding Repayment” contents
Please touch the heading, “Understanding Repayment”. Studentaid.gov/repay-loans/understand. Looking at this information, what would you touch first to learn more?
What do you think of the contents covered in this section?
Is there too much information here to process, or not enough information to get a good sense of what you want to learn?
What about the amount of text to read? (Too much, too little, just right?)
What are your feelings about the terminology used – do you feel it was written with you in mind?
What are your feelings about the layout and organization of this information? (PROBE: Does it flow in a logical way?)
Is there anything you would change, or is it fine as is?
0:26
Task 2: Review Overview of Direct Loan and FFEL Program Repayment Plans table
Now look for information on ways to choose a repayment plan. [IF PARTICIPANT HAVING TROUBLE: scroll down a bit and touch “Repayment Plans”.]
What would you do next? [IF STILL HAVING DIFFICULTY: Touch “Get details about repayment plans” link.]
On this page you see a table, “Overview of Direct Loan and FFEL Program Repayment Plans”. Take a moment to look it over. What is the purpose of this table?
Please tell me how you would rate the information in the table on a scale of 1 to 10, where 1 is “not at all relevant” and 10 is “very relevant” to you.
Not at all relevant Very relevant
1 2 3 4 5 6 7 8 9 10
Please tell me how you would rate the information in the table on a scale of 1 to 10, where 1 is “after reviewing the table I do not understand how to choose the best plan for me” and 10 is “I clearly understand how to choose the best plan for me ”.
Do not understand how to choose Clearly understand how to choose
1 2 3 4 5 6 7 8 9 10
Task 3: Locate and use Monthly Loan Repayment Estimate Calculator 0:31
Touch the Repay Your Loans menu. Locate and use the repayment calculator to estimate the monthly payment amount. You may use fictional loan amounts for this exercise. [IF PARTICIPANT IS HAVING TROUBLE LOCATING A CALCULATOR: touch “Understanding Repayment”, then touch “Repayment Plans” and then scroll down to locate the calculator.]
Is this where you would have expected to find the calculator?
IF NOT: What needs to change to make the calculator easier to find?
IF NOT ALREADY ON THE “REPAYMENT PLANS” MENU: Now touch the “Repayment Plans” menu. Please locate a calculator. [IF PARTICIPANT IS STILL HAVING TROUBLE LOCATING A CALCULATOR: Scroll down to the table. You’ll see some Quick Links for different types of repayment plans, such as Standard Plan, Graduated Repayment Plan and others. Touch any of those links, and then locate the link to calculate your estimated loan payments for that repayment plan. (IF NECESSARY: scroll down until you see “Calculate your Estimated loan Payments…” link)
Please tell me how you would rate the calculator itself on a scale of 1 to 10 where 1 is “not at all easy to use ” and 10 is “very easy to use.”
Not at all easy to use Very easy to use
1 2 3 4 5 6 7 8 9 10
Please tell me how you would rate the calculator itself on a scale of 1 to 10 where 1 is “not at all easy to understand the results” and 10 is “very easy to understand the results.”
Not at all easy to understand Very easy to understand
1 2 3 4 5 6 7 8 9 10
What makes you say that?
What feelings, if any, do you have with regard to finding the calculator? What exactly?
If you were on your own to look for the calculator, realistically, would you have continued to search for it or would you have abandoned the search and moved on to something else? What makes you say that?
What do you feel are the pros and/or cons of completing any of these tasks on your mobile device, versus doing so on a PC or laptop?
Realistically, would you have performed any these tasks with your mobile device? Which ones? Are there any you definitely would not have tried with your mobile device? 0:43
Now that you have reviewed several aspects of the studentaid.gov website, do you have any final advice for the Office of Federal Student Aid to consider in order to make the site easier to use for those using a mobile device such as yours?
SKIP Q15 IF NO TIME
Having seen/used the calculator, and become acquainted with the repayment plan options, what would be your next step? [PROBE: How would you expect to go about making a payment?]
That concludes our interview. Thanks very much for your input. It was very helpful! 0:45
TASKS: PARENTS OF HS SENIORS 0:20
Understanding that you are the parent of a high school senior planning to enter college in the next year, where on the site would you go to understand about applying for student aid?
Task 1: Review “FAFSA: Apply for Aid” contents
I want you to touch the menu, “FAFSA: Apply for Aid” (studentaid.gov/fafsa). Looking at this information, what would you touch first to learn more?
What do you think of the contents covered in this section?
Is there too much information here to process, or not enough information to get a good sense of what you want to learn?
What about the amount of text to read? (Too much, too little, just right?)
What are your feelings about the terminology used – do you feel it was written with you in mind?
What are your feelings about the layout and organization of this information? (PROBE: Does it flow in a logical way?)
Is there anything you would change, or is it fine as is?
0:25
Task 2: Review FAFSA Overview video
Please touch the video link titled “FAFSA Overview” and either during or after the video let me know what you think.
Thinking about the video, please tell me how you would rate it on a scale of 1 to 10 in terms of your ability to understand the information presented in the video, where 1 is “the video is not at all easy to understand ” and 10 is “very easy to understand.”
Not at all easy to understand Very easy to understand
1 2 3 4 5 6 7 8 9 10
What makes you say that?
SKIP Q5 IF NO TIME
Again thinking about the video, please tell me how you would rate it on a scale of 1 to 10 in terms of its effect on your confidence in being able to complete the student aid application process, where 1 is “I am not at all confident ” and 10 is “I am highly confident .”
Not at all confident Highly confident
1 2 3 4 5 6 7 8 9 10
What makes you say that? 0:40
What are your feelings regarding the length of the video when viewing on your mobile device? (PROBE: Is it too long, too short, just right?)
Please tell me how you would rate the ease of finding this video on a scale of 1 to 10, where 1 is “very difficult” and 10 is “very easy” to find.
Very difficult to find Very easy to find
1 2 3 4 5 6 7 8 9 10
What makes you say that?
When using your mobile device, what circumstances would lead you to view videos of this type?
What do you feel are the pros and/or cons of watching this video on your mobile device, versus doing so on a PC or laptop?
0:33
Task 3: Assess “Types of Aid” menu information
Thinking about this section in general, please tell me how you would rate it on a scale of 1 to 10 in terms of the ease with which you can find what you are looking for, where 1 is the information I want is “Very difficult to find” and 10 is “Very easy to find”.
Very difficult to find Very easy to understand
1 2 3 4 5 6 7 8 9 10
What makes you say that?
What are your feelings about the number of steps required to complete the form?
Also thinking about this section, please tell me how you would rate it on a scale of 1 to 10 in terms of your ability to understand the information presented, where 1 is “very difficult to understand” and 10 is “very easy to understand.”
Very difficult to understand Very easy to understand
1 2 3 4 5 6 7 8 9 10
What makes you say that?
What do you feel are the pros and/or cons of completing any of these tasks on your mobile device, versus doing so on a PC or laptop?
Realistically, would you have performed any these tasks with your mobile device? Which ones? Are there any you definitely would not have tried with your mobile device?
Now that you have reviewed several aspects of the studentaid.gov website, do you have any final advice for the Office of Federal Student Aid to consider, in order to make the site easier to use for those using a mobile device such as yours?
That concludes our interview. Thanks very much for your input. It was very helpful! 0:45
TASKS: HIGH SCHOOL SENIORS 0:20
Understanding that you are a high school [senior/junior] planning to enter college in the next [year/few years], where on the site would you go to understand more about college?
Task 1: Review “Prepare for College” contents
I want you to touch the heading, “Prepare for College” (studentaid.gov/prepare-for-college). Take a few minutes to look at this section.
What do you think of the contents covered in this section?
Is there too much information here to process, or not enough information to get a good sense of what you want to learn?
What about the amount of text to read? (Too much, too little, just right?)
What are your feelings about the terminology used – do you feel it was written with you in mind?
What are your feelings about the layout and organization of this information? (PROBE: Does it flow in a logical way?)
Is there anything you would change, or is it fine as is?
0:26
Task 2: Review “Preview the Financial Aid Process” infographic
Please locate and touch the graphic titled “Preview the Financial Aid Process” and let me know what you think. (IF INITIAL DIFFICULTY: under what main menu heading would you expect to find that information?
(IF STILL HAVING DIFFICULTY, DESCRIBE AS NEEDED: Touch “Prepare for College”, then touch “Checklists for Academic and Financial Preparation” link. Then scroll down to “Preview the Financial Aid Process”. This is the graphic I’d like you to review. Please take a few moments to look it over.
What do you feel are the pros and/or cons of finding this graphic on your mobile device, versus doing so on a PC or laptop? (PROBE: how about finding this information in general?)
What do you feel are the pros and/or cons of viewing this graphic on your mobile device, versus doing so on a PC or laptop?
Thinking about the graphic, please tell me how you personally would rate it on a scale of 1 to 10 in terms of the impact it has, if any, on encouraging your decision to go to college, where 1 is “no impact at all” and 10 is “very strong impact” on you.
No impact at all Very strong impact
1 2 3 4 5 6 7 8 9 10
What makes you say that?
(PROBE IF RATING OF 6-10): Does the impact come mainly from the content, or the way the content is presented in the graphic, or both?)
(PROBE IF RATING OF 1-5): What, if anything, would you suggest to increase the impact of this information?
0:32
Task 3: Review “Overview of the financial aid process” video
Of course, another aspect of going to college involves how to finance your college education. With that in mind, please go to the main menu heading labeled, “Types of Aid” and touch the video link titled “Overview of the Financial Aid Process” and either during or after the video, let me know what you think.
Please tell me how you would rate the ease of finding the video on a scale of 1 to 10, where 1 is “very difficult to find” and 10 is “very easy to find”.
Very difficult to find Very easy to find
1 2 3 4 5 6 7 8 9 10
What makes you say that?
Thinking about the video itself, please tell me how you would rate it on a scale of 1 to 10 in terms of your ability to understand the information presented in the video, where 1 is “not at all easy to understand ” and 10 is “very easy to understand.”
Not at all easy to understand Very easy to understand
1 2 3 4 5 6 7 8 9 10
What makes you say that?
Realistically, is this the type of information you would a) want to share and discuss together with your parents; b) watch on your own without your parents’ involvement; or c) forward to your parents to watch on their own without any other involvement on your part?
SKIP Q12 IF NO TIME
Still thinking about the video, please tell me how you would rate it on a scale of 1 to 10 in terms of its effect on your confidence in being able to complete the student aid application process, where 1 is “I am not at all confident ” and 10 is “I am highly confident .”
Not at all confident Highly confident
1 2 3 4 5 6 7 8 9 10
What makes you say that? 0:40
Would your rating change if you knew your parents will be actively involved in completing the student aid application process? What would be your new rating?
Is this the type of information you would share with your friends? How exactly would you do that?
What do you feel are the pros and/or cons of completing any of these tasks on your mobile device, versus doing so on a PC or laptop?
Realistically, would you have performed any these tasks with your mobile device? Which ones? Are there any you definitely would not have tried with your mobile device?
Now that you have reviewed several aspects of the studentaid.gov website, do you have any final advice for the Office of Federal Student Aid to consider in order to make the site easier to use for those using a mobile device such as yours?
That concludes our interview. Thanks very much for your input. It was very helpful! 0:45
| File Type | application/msword |
| File Title | FREDDIE MAC/LENDING TREE USABILITY STUDY |
| Author | Dave Glantz |
| Last Modified By | Authorised User |
| File Modified | 2012-11-27 |
| File Created | 2012-11-27 |