Download:
pdf |
pdfMODERATOR’S GUIDE
Spanish Ask CFPB UX Testing
August 9-11, 2016
Spanish Ask Moderator’s Guide | 1
Introduction
Moderator’s Guide for Spanish Ask
Thank you for participating in this study today. My name is _____, and I work with the User
Experience Team at Fors Marsh Group on behalf of the Consumer Financial Protection Bureau
(CFPB). Today we will be evaluating a website. I will ask you questions and give you tasks to complete
using the site. Your comments and feedback will help the team improve the site. I did not create the
site, so please do not feel like you have to hold back on your thoughts to be polite to me. We’re
interested in both your positive and negative reactions to the site. Difficulties you may run into reflect
the design of the site, not your skills or abilities. Please keep in mind that I am not testing you or your
knowledge. Rather you are helping us to see how we can improve this site.
The entire session will last about 60-90 minutes. In accordance with the Paperwork Reduction Act of
1995, this study has been approved by the Federal government’s Office of Management and Budget
(OMB) under OMB control number 3170-0024. Without this approval we could not conduct this
study. Do you have any questions so far?
Do you have any questions so far?
Let’s cover a couple things before we get started:
We are making a video recording of the computer screen during this session, but your name and
personal information will not be associated with the recordings or your responses.
I am interested in your thoughts and reactions as we proceed. This is important because I can
see what you are doing, but I don’t know why you are doing it. So I need your help. While you
are working, I would like you to think aloud. I would like you to:
• Tell me what you are thinking
• Describe the steps that you are taking.
• Tell me what you are doing, what you are going to do, and why.
• Tell me why you clicked on a link and what you expect to happen next.
• This will give me insight into what you’re thinking about as you’re using the site.
There are no right or wrong answers. Your comments and opinions will be kept private, and only used
and shared in combination with the feedback we get from other people in a way that does not
directly identify you. Participation in this study is voluntary. You may withdraw from participation at
any time. Any questions?
Ok, we are ready to begin. I would like you to start by asking you some background questions.
Spanish Ask Moderator’s Guide | 2
Section 1: Interview
1.1. You had indicated that you were recently seeking information on a question [fill in what
their question was about]. Tell me a little bit more about the question that you had.
1.2. Did you do any research to answer your question? What did you do to start the process?
1.3. What resources (e.g., websites) did you consult to try to answer your question? If so, what
was your experience like using those resources? Was there anything that you liked or
disliked about those resources? Was there any resource you would have liked to have
access to? What did you think about the quality of the information?
Section 2: Arriving from Organic Search
2.1. Let’s say you have your question about [Insert response from 1A] and you want to use the
internet to try to find an answer to it. Please use the internet to find an answer to your
question.
[Moderator –See if they go to Ask CFPB on their own. If not, bring them back to the search
results about their decision and add Ask CFPB to the search terms.]
Participant enters search terms and returns search results.
2.2. What are your impressions of the Ask CFPB entry? How does it compare to the others on
the page? Which one would you click on and why?
Participant arrives on Ask CFPB answer page.
2.3. What are your first impressions of this page?
2.4. What would you do next? Would you leave the site or stay?
2.5. Do you find the site to be trustworthy? Why or why not?
2.6. What is the purpose of this site?
2.7. Who do you think is sponsoring the site?
2.8.
What is important to you on this page?
Spanish Ask Moderator’s Guide | 3
[If they do not stay on the site] Why did you leave the website? Did you get the answer you
were looking for? What do you think about the quality of the answer? What is missing?
2.9.
Please take a few minutes to review this page. Let me know as you’re reading if you find
anything confusing or unclear.
2.10. What are your impressions of this page now that you’ve reviewed the information?
2.11. How confident are you that the website answered your question?
Not Confident
Slightly
Moderately
Very
Extremely
At All
Confident
Confident
Confident
Confident
1
2
3
4
5
2.12. Do you have any recommendations for how to improve the filtering process or the content
on this page?
Section 3: Navigating from one answer page to another
3.1.
Let’s say you want to find an answer to the other question we discussed about [Insert
other response from 1A]. Please use the site to find an answer to your question.
3.2.
What are your impressions of this page (search results page)?
3.3.
[Participant arrives on answer page] Please take a few moments to review this page. Let
me know as you’re reading if you find anything confusing or unclear.
3.4.
What are your impressions of this page now that you’ve reviewed the information?
3.5.
How confident are you that the website answered your question?
Not Confident
Slightly
Moderately
Very
Extremely
At All
Confident
Confident
Confident
Confident
1
2
3
4
5
3.6.
Do you have any recommendations for how to improve this page?
Spanish Ask Moderator’s Guide | 4
Section 4: Navigating to an answer page using related topics and filters
4.1.
Let’s say you had a related question about [Insert a related topic]. How would you find
related questions and answers about this topic?
4.2.
[Moderator: Provide participant with filtering criteria if it wasn’t used organically in their
search.] What if you were searching for information and you only wanted to look at
questions and answers about [Insert filter label]?
4.3.
How well did the filtering process work? Is it what you expected?
4.4.
Please take a few moments to review the first result. Let me know as you’re reading if you
find anything confusing or unclear.
4.5.
What are your impressions of this page now that you’ve reviewed the information?
4.6.
How confident are you that the website answered your question?
Not Confident
Slightly
Moderately
Very
Extremely
At All
Confident
Confident
Confident
Confident
1
2
3
4
5
4.7.
How was your experience with navigating to this page?
4.8.
Do you have any recommendations for how to improve the process of navigating to this
page or the content on this page? Did the filtering process work as expected?
Section 5: Additional Q&A Tasks
[Moderator: navigate participant to the Spanish Ask CFPB home page.]
5.1.
Let’s say you wanted to know the difference between an interest rate and APR. How would
you find an answer to your question using this site?
5.2.
Please take a few moments to review this page. Let me know as you’re reading if you find
anything confusing or unclear.
5.3.
What are your impressions of this page now that you’ve reviewed the information?
Spanish Ask Moderator’s Guide | 5
5.4.
How confident are you that the website answered your question?
Not Confident
Slightly
Moderately
Very
Extremely
At All
Confident
Confident
Confident
Confident
1
2
3
4
5
5.5.
How was your experience with navigating to this page?
5.6.
Do you have any recommendations for how to improve this page?
5.7.
Let’s say you want to explore similar questions to this one–let’s say you want to figure out
the differences between a fixed rate and variable rate loan. How would you find an answer
to your question?
5.8.
Please take a few moments to review this page. Let me know as you’re reading if you find
anything confusing or unclear.
5.9.
What are your impressions of this page now that you’ve reviewed the information?
5.10. How confident are you that the website answered your question?
Not Confident
Slightly
Moderately
Very
Extremely
At All
Confident
Confident
Confident
Confident
1
2
3
4
5
5.11. How was your experience with navigating to this page?
5.12. Do you have any recommendations for how to improve this page?
5.13. Let’s say you’ve been contacted a lot by a debt collector and you want to know if there’s a
limit to the number of times that the collector can contact you. How would you find an
answer to your question using this site?
5.14. Please take a few moments to review this page. Let me know as you’re reading if you find
anything confusing or unclear.
5.15. What are your impressions of this page now that you’ve reviewed the information?
Spanish Ask Moderator’s Guide | 6
5.16. How confident are you that the website answered your question?
Not Confident
Slightly
Moderately
Very
Extremely
At All
Confident
Confident
Confident
Confident
1
2
3
4
5
5.17. How was your experience with navigating to this page?
5.18. Do you have any recommendations for how to improve this page?
5.19. Let’s say you’ve been thinking about buying a car but you’re not sure how much you
should spend or borrow on the loan. How would you find an answer to your question using
this site?
5.20. Please take a few moments to review this page. Let me know as you’re reading if you find
anything confusing or unclear.
5.21. What are your impressions of this page now that you’ve reviewed the information?
5.22. How confident are you that the website answered your question?
Not Confident
Slightly
Moderately
Very
Extremely
At All
Confident
Confident
Confident
Confident
1
2
3
4
5
5.23. How was your experience with navigating to this page?
5.24. Do you have any recommendations for how to improve this page?
5.25. Let’s say you’re preparing to send money to a friend in another country and you’re not
sure if there are laws that protect this type of transaction. How would you find an answer
to your question using this site?
5.26. Please take a few moments to review this page. Let me know as you’re reading if you find
anything confusing or unclear.
5.27. What are your impressions of this page now that you’ve reviewed the information?
Spanish Ask Moderator’s Guide | 7
5.28. How confident are you that the website answered your question?
Not Confident
Slightly
Moderately
Very
Extremely
At All
Confident
Confident
Confident
Confident
1
2
3
4
5
5.29. How was your experience with navigating to this page?
5.30. Do you have any recommendations for how to improve this page?
Section 6: Other Tasks
6.1.
What could you do if you wanted to provide the CFPB with feedback about the helpfulness
of the answer?
6.2.
Was this what you were expecting? Would you expect to be able to provide more
feedback?
6.3.
What would you do if you wanted to share this information with a friend, family member,
or colleague?
6.4.
Would you want to print an answer page if you had a question? What could you do if you
wanted to print this page? Which method would you use? What would you expect the
printed page to look like? [Moderator: Instruct participant to click on print button.] What
would you do with the printed page? How would you use it?
Spanish Ask Moderator’s Guide | 8
Section 7: Satisfaction Questionnaire (System Usability Scale)
1. I think that I would like to use the Spanish Ask CFPB site frequently if I had a financial
question:
Strongly
Disagree
1
Disagree
Neutral
Agree
2
3
4
Strongly
Agree
5
2. I found the Spanish Ask CFPB site unnecessarily complex:
Strongly
Disagree
1
Disagree
Neutral
Agree
2
3
4
Strongly
Agree
5
3. I thought the Spanish Ask CFPB site was easy to use:
Strongly
Disagree
1
Disagree
Neutral
Agree
2
3
4
Strongly
Agree
5
4. I think that I would need the support of a technical person to be able to use the Spanish Ask
CFPB site:
Strongly
Disagree
1
Disagree
Neutral
Agree
2
3
4
Strongly
Agree
5
5. I found the various functions in the Spanish Ask CFPB site were well integrated with each
other:
Strongly
Disagree
1
Disagree
Neutral
Agree
2
3
4
Strongly
Agree
5
6. I thought there was too much inconsistency in the Spanish Ask CFPB site:
Spanish Ask Moderator’s Guide | 9
Strongly
Disagree
1
Disagree
Neutral
Agree
2
3
4
Strongly
Agree
5
7. I would imagine that most people would learn to use the Spanish Ask CFPB site very quickly:
Strongly
Disagree
1
Disagree
Neutral
Agree
2
3
4
Strongly
Agree
5
8. I found the Spanish Ask CFPB site very cumbersome to use:
Strongly
Disagree
1
Disagree
Neutral
Agree
2
3
4
Strongly
Agree
5
9. I felt very confident using the Spanish Ask CFPB site:
Strongly
Disagree
1
Disagree
Neutral
Agree
2
3
4
Strongly
Agree
5
10. I needed to learn a lot of things before I could get going with the Spanish Ask CFPB site:
Strongly
Disagree
1
Disagree
Neutral
Agree
2
3
4
Strongly
Agree
5
Net Promoter Score (NPS)
Please circle the numbers that most appropriately reflect your impressions about using this website.
On a scale of 1 to 10, with 10 being more likely, how likely is it that you would recommend this site
to a friend, colleague, or family member with similar goals in mind?
1
2
3
4
5
6
7
8
9
10
Spanish Ask Moderator’s Guide | 10
Section 8: Debriefing
8.1. What was your experience like using this site?
8.2. Let’s walk through your responses on the satisfaction questionnaire. [Follow-up to Q1 on
SUSQ] Do you think you’d use this site the next time you had a financial question? If you
had a financial question, do you think you would come to this site before using Google?
Why or why not?
8.3. What are your impressions of how search operated? Did it operate how you expected?
What are your impressions of the search results page?
8.4. What are your impressions of the amount of information you received on each of the
pages?
8.5. How well did the site answer your questions?
8.6. What are your impressions of how the “related topics” section worked? Did it work how you
expected it to? Did you notice that section at first? [If not] When did you notice it?
8.7. What are your impressions of the “similar questions” section? Did it work how you
expected it to? Did you notice that section at first? [If not] When did you notice it?
Section 9: Version Comparison (Bilingual Speakers)
9.1. Let’s go to the English version of this site.
9.2. Let’s say you wanted to know the difference between an interest rate and APR. How would
you find an answer to your question using this version of the site?
9.3. Please take a few moments to review this page.
9.4. What do you think about the differences between the two language versions? What do you
like and dislike about each?
Section 10: Closing
10.1. Do you have any additional recommendations for the designers of the site?
10.2. Any other feedback about the site that we did not get to cover today?
Spanish Ask Moderator’s Guide | 11
| File Type | application/pdf |
| Author | Edward Pierce |
| File Modified | 2016-07-21 |
| File Created | 2016-07-21 |