0704-UTRF_FINRED_Usability_Guide_11.29.22

OMB CONTROL NUMBER: 0704-UFTR

OMB EXPIRATION DATE: XX-XX-XXXX

AGENCY DISCLOSURE NOTICE

The public reporting burden for this collection of information, 0704-UFTR, is estimated to average 1.5 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding the burden estimate or burden reduction suggestions to the Department of Defense, Washington Headquarters Services, at [email protected]. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

Discussion Guide

Bowen – Financial Readiness

July 2022

Table of Contents

Introduction 2

Before We Begin 3

General Questions 4

Task #1. Initial Impressions of Homepage 5

Task #2. Free Exploration 6

Task #3. Knowing Your Benefits and Entitlements 6

Task #4. Managing Your Money 7

Task #5. Saving and Investing 8

Task #6. Planning for the Future 9

Task #7. Understanding Consumer Protections 10

Task #8. Tools and Additional Resources and Trending Topics 11

Task #9. Search 12

Task #10. Service Provider and Leaders Only 12

Task #11. Mobile 13

Exit Interview Questions 14

Introduction

Purpose of session and background, verbally covered with each user.

• Thank you for taking the time to meet with me today and participate in this study.

• My name is ________ and I work for Usability Sciences. We are an independent consulting company and we bring in people like you to look at various websites or applications to get your feedback. As you know, your participation in this research is voluntary and you are welcome to terminate this session at any time.

• This study is sponsored by the Department of Defense Office of Financial Readiness (after first use: DoD Office of Financial Readiness). Study staff will diligently protect your personal information and ensure that your opinions, ideas, concerns, etc. will not be identifiable as you. While the information in this study will be used only for research purposes, results and recommendations will be shared with government officials, however your identity will remain anonymous.

• The purpose of the research is to gather feedback and insights on a website specifically designed for the military community. I am interested in hearing what you like and dislike about the site and what you think could be done to make it more valuable and easier to use. Your feedback is very important, as it will be used to drive improvements to the site.

• [Brief participant on the lab set-up and session flow]

• Once we get started, I’ll give you a few tasks or scenarios and ask you to walk and talk me through them. These tasks will explore various areas of the site interface, and I’ll be gathering your feedback along the way.

• A few things to keep in mind during our session today:

  • I did not design what you will be looking at today, so whether your feedback is positive or negative, you can't hurt my feelings. I am completely neutral. Please be as honest as possible.

  • I am testing the site and not you; if you find something confusing or unclear on the site, it is important for me to know that.

  • While going through the process, please think out loud and let me know what you are thinking along the way. I can see what you are looking at, but I can't see what you're thinking.

• Do you have any questions for me before we begin?

Before We Begin

Background questions.

1. Talk to me a little about your financial journey. How knowledgeable would you say you are about things like budgeting, savings, investing, retirement planning, credit and debt management, etc.?

2. Have you done any online research on any of these or similar financial topics?

   1. If yes:

      • What type of information have you looked for?

      • Were you able to find what you needed?

      • What sites, resources, online tools have you used?

      • Were the resources/tools helpful? How?

      • Was there anything you remember being challenging or frustrating? What?

   2. If no:

      • Any particular reason why not?

      • Do you foresee yourself doing this type of research in the future?

      • If you were to look online for financial-type information, what, if anything, would be of interest to you? Why?

3. Have your financial needs or knowledge areas changed over time as you have been part of the military? How?

4. What are the top two or three features or content that would be most useful to you on a financial readiness site? Why?

5. When researching, what type of device do you usually use (desktop/laptop, tablet, mobile)?

General Questions

The following questions are examples of the typical questions that will be asked throughout testing as appropriate, to prompt discussion and insight from the user.

General Questions and Things to Observe on the various pages

• What are your initial thoughts?

• How does this compare to what you expected?

• What are your thoughts on the look and feel?

• What key information does this page convey to you?

• What are your thoughts on the images and graphics?

• Overall thoughts on the layout and organization?

• What area would you be interested in clicking? Why?

• Is there anything that is unclear to you? How can we fix it?

• Is there anything missing that you expected to see?

• What area of the page drew your attention first?

• Where would you go next?

• What do you expect to see next?

• Observe: Do users find the top-level navigation labels intuitive?

• Observe: Does user navigate via the left rail menu? Are the labels intuitive?

• Observe: Does user select the CTAs (Read More buttons, links, etc.) to navigate? The images?

• Observe: Does the user view any of the videos without prompting?

• Observe: Does the user ‘search’ without prompting?

Questions on selected FINRED Assets

• Is this what you expected?

• Is there value to this information?

• Is this something you would read if not here with me? Why or why not?

• Is there anything that is confusing or unexpected?

• Anything you expected to see that is not here?

• (If applicable) Thoughts on this being a blog posting vs a pdf vs a page within the site?

Questions on selected non-FINRED Assets (will spend minimal time on non-FINRED assets)

• Is this what you expected?

Task # Initial Impressions of Homepage

Go ahead and open the link to the site. www.finred.usalearning.gov

Take a moment to look over the homepage and share your initial thoughts. Feel free to scroll around, just don’t navigate away from the page quite yet.

Moderator Guide (optional questions):

• What are your initial thoughts?

• What does this page convey to you?

• Based on what you see here, who would you say the site is intended for?

• What might you do to get started? What would you look at first?

  • What areas are compelling?

  • Any that don’t interest you?

• Thoughts on the upper navigation and tabs at the top of the homepage? Are these resources you might click on?

• Thoughts on the ‘Find a Personal Financial Counselor’ tab? Is this something you would click on and explore?

• What about the ‘Financial Well-being Assessment’ tab? Something you would click on and explore?

• Thoughts on the upper navigation terms and organization, such as ‘About Us, Knowing Your Benefits and Entitlements, etc’? Are these terms how you would organize financial terms you are looking for? Are they something you could click on?

• Thoughts on the rotating banner of resources? Is this something you might click on? Would you prefer more static features vs ones that rotate through?

• Thoughts on what is listed under ‘News & Events’?

• What about the Facebook widget? Something you would ‘follow’?

• What about the FINRED Blog or Featured Resource widget? Is highlighted content like this beneficial? Noticeable?

  • What type of content might encourage you to ‘Read more’?

• The way it is presented, is it clear that the blog and featured resource is NOT part of social media?

• Thoughts on the overall look and feel of this page?

• Is there anything that is unclear to you? Anything unexpected or missing?

• Have you ever been to this site before? Ever hear of it/does it sound familiar?

• Observe: Does user scroll the page?

• Observe: Does he comment on the News & Events? Widgets?

• Observe: Does he select ‘Read Now’? Does he select any of the icons/graphics? Which?

• Observe: Does user comment on the social media widgets? Does he scroll thru the Facebook postings without prompting?

Task # Free Exploration

You said you would be interested in exploring {whatever area the user mentioned in Task #1}. What type of information would you expect to find there?

Go ahead and look around that area and talk to me about it.

Notes:

• Moderator will probe as applicable; see the following tasks for details related to the specific area of interest.

• Once the specific area is exhausted, the user will be asked what other areas of the site he wants to explore. Moderator will probe as applicable, ensuring all areas/tasks are covered by a sub-set of participants.

• If/when user has explored two areas of interest to him, he will be prompted to look at other areas of the site by being presented with one or more tasks (below).

Task # Knowing Your Benefits and Entitlements

Let’s say you heard that there are resources available to help service members pay for education and you want to learn more about this. How would you do that on this site?

Note: Based on scenario, user might select the asset(s) bolded below. However, moderator will not force user to select it.

Prompt users to provide feedback on:

• Education Benefits: Avoid These Costly Higher Education Traps (fact sheet) (graphic card feature)

Moderator Guide (Optional Questions):

• Before selecting ‘Knowing Your Benefits and Entitlements’, what type of information would you expect to find there?

Benefits and Entitlements

• How does this meet your expectations?

• Talk to me about some of the resources available. Are there any listed that would be of interest to you? Which? Why?

• Based on the name and description of each, do you have a good feel for the type of content that would be available? Is there too much or too little of a description?

• What about the left menu – thoughts on the items listed here? Do they belong together, within the Benefits and Entitlements section?

• Allow user to select resource of interest. Ask them to provide feedback on the content and layout.

Task # Managing Your Money

You are interested in: (assigned topics will vary among users)

• Monitoring your credit

• Learning about the latest changes to the tax laws

• Creating a budget

• Tracking your spending

• Learning your Financial Well-being

Is there any information on this site to help you? Where would you look?

Note: Based on scenario, user might select the asset(s) bolded below. However, moderator will not force user to select it.

Prompt users to provide feedback on:

• Pay and Savings: The Impact of Inflation on Financial Decision (graphic card features)

• Major Purchases: Know the True Cost of Your Loan (graphic card features)

• Tax Information: 8 Tips to Maximize Your Savings This Tax Season (graphic card features)

• Additional Information: Financial Well-being Assessment (graphic card features)

Moderator Guide (optional questions):

• Before selecting ‘Managing Your Money’, what type of information would you expect to find?

Managing Your Money

• How does this meet your expectations?

• Talk to me about some of the resources available. Are there any listed that would be of interest to you? Which? Why?

• Based on the name and description of each, do you have a good feel for the type of content that would be available? What do you think about the descriptions?

• What about the left menu – thoughts on the items listed here? Do they belong together, within the Managing Your Money section?

• Allow user to select resource of interest. Have them provide feedback on the content and layout.

• If user has not already viewed Financial Well-being Assessment ( listed under Additional Information), prompt them to select the graphic card feature and provide feedback.

Task # Saving and Investing

Is there any information available on this site to help you get a handle on things you need to start thing about?

or

You want to learn more about stocks and bonds. Is there any information available on this site to help you better understand investment opportunities?

Note: Based on scenario, user might select the asset(s) bolded below. However, moderator will not force user to select it.

Prompt users to provide feedback on:

• Retirement Savings: Thrift Savings Plan Early Distribution Considerations (graphic card feature)

• Investing: Investing Doesn’t Have to Be A Puzzle, Investing Microlearning (graphic card features)

Moderator Guide (optional questions):

• Before selecting ‘Savings and Investing’, what type of information would you expect to find there?

Savings and Investing

• How does this meet your expectations?

• Talk to me about some of the resources available. Are there any listed that would be of interest to you? Which? Why?

• Based on the name and description of each, do you have a good feel for the type of content that would be available? Is there too much or too little of a description?

• What about the left menu – thoughts on the items listed here? Do they belong together, within the Saving and Investing section?

• Allow user to select resource of interest. Ask them to provide feedback on the content and layout.

• If user has not already viewed Mobile App, Sen$e download links listed under Additional Resources), prompt him to select the asset and provide feedback.

Task # Planning for the Future

You have been reading that it is important to: (assigned topics will vary among users)

• Have a power of attorney

• Have some sort of plan to help you stay on track with your finances and you want to learn more.

Is there any information on this site to help you? Where would you look?

Note: Based on scenario, user might select the asset(s) bolded below. However, moderator will not force user to select it.

Prompt users to provide feedback on:

• Estate Planning: How Powers of Attorney Serve the Military article (graphic card featured article)

• Readiness: A Roadmap for Your Financial Future infographic (or any of the Roadmap infographics)

• Insurance: Planning Ahead With the Survivor Benefit Plan infographic (graphic card feature)

Moderator Guide (optional questions):

• Before selecting ‘Planning for the Future’, what type of information would you expect to find there?

Planning for the Future

• How does this meet your expectations?

• Talk to me about some of the resources available. Are there any listed that would be of interest to you? Which? Why?

• Based on the name and description of each, do you have a good feel for the type of content that would be available? Is there too much or too little of a description?

• What about the left menu – thoughts on the items listed here? Do they belong together, within the Planning for the Future section?

• Allow user to select resource of interest. Ask them to provide feedback on the content and layout.

• If user has not already viewed the ‘featured’ assets listed under Additional Resources (DoD Career Ready or Planning for the Future Additional Resoruces), prompt him to select an asset and provide feedback.

Task # Understanding Consumer Protections

Let’s say you want to learn: (assigned topics will vary among users)

• If there are any special protections for service members when it comes to dealing with debt collectors

• What your rights are under the Fair Credit Reporting Act

• About the Truth-in-Lending Act

How might you approach that on this site? Where would you go?

Note: Based on scenario, user might select the asset(s) bolded below. However, moderator will not force user to select it.

Prompt users to provide feedback on:

• Consumer Credit and Protections: Protect Your Finances During PCS With Servicemembers Civil Relief Act infographic (graphic card feature)

• Consumer Credit and Protection: Savvy Buying Tips for Military Members (graphic card feature pdf)

Moderator Guide (optional questions):

• Before selecting ‘Understanding Consumer Protections’, what type of information would you expect to find there?

Consumer Credit and Protections

• How does this meet your expectations?

• Talk to me about some of the resources available. Are there any listed that would be of interest to you? Which? Why?

• Based on the name and description of each, do you have a good feel for the type of content that would be available? Is there too much or too little of a description?

• What about the left menu – thoughts on the items listed here? Do they belong together, within the Consumer Credit and Protections section?

• Allow user to select resource of interest. Ask them to provide feedback on the content and layout.

• If user has not already viewed the 3 ‘featured’ assets listed under Additional Resources (Road to Financial Readiness, It Pays to Visit a Personal Financial Counselor, Understanding Consumer Protections Additional Resources landing page), prompt him to select an asset and provide feedback.

Task # Tools and Additional Resources and Trending Topics

What would you expect from these sections from the website?

Note: Based on scenario, user might select the asset(s) or landing pages bolded below. However, moderator will not force user to select it.

Moderator Guide (optional questions):

• What would you expect to see or do from the ‘Tools and Additional Resources’ section of the website?

  • How would you use it?

  • Is there anything missing you expect to see here?

  • Is there anything you do not want to see here that should be removed?

• Allow user to select landing page of interest. Ask them to provide feedback on the content and layout.

  • Allow user to select resource of interest. Ask them to provide feedback on the content and layout.

• What would you expect to see or do from the ‘Trending Topics’ section of the website?

  • How would you use it?

  • Is there anything missing you expect to see here?

  • Is there anything you do not want to see here that should be removed?

• How does using this drop-down compare to using other pages on the website like ‘Managing Your Money’?

• If you wanted information on improving your emergency fund, what would you do?

• Allow user to select landing page of interest. Ask them to provide feedback on the content and layout.

  • Allow user to select resource of interest. Ask them to provide feedback on the content and layout.

• If they don’t select ‘MISSION: Dollars and Cents blog’, prompt them to click and explore the landing page.

  • Is this what you expected?

  • What are your thoughts on having this type of content on here?

  • How likely would you be to read the content here? How would you expect to get it?

  • How does the layout of this page compare with the layout of another page on the website for example, ‘Managing Your Money’?

Task # Search

If user has not used Search without prompting, ask him to search for a topic of interest, for example ‘Retirement Planning’, ‘Education Assistance’, ‘Financial Advisor’, ‘Debt’ or ‘Credit’, ‘Continuation Pay’.

Moderator Guide (optional questions):

Search Results

• Talk to me about the search results. Is this what you were expecting?

• In your own words, what are you looking at? What does this info mean?

  • Does user notice the ‘Sort by’?

  • Does user understand that some of the results link to the FINRED site and others link to external sites?

  • Does user notice there may be multiple pages of results? Can he scroll to other pages?

  • Does user comment on the option of searching on Google?

• Thoughts on the displayed description? Is this enough information to help you decide whether you want to select the link or not?

Task # Service Provider and Leaders Only

Note: This section is for Service Providers or service members in leadership roles

Moderator Guide (optional questions):

• Have you looked or used the Service Provider and Leader section?

  • If so, why? What were you looking for? Did you find it helpful?

    • If no, what did you expect to see there?

• Take a look and tell me what you think of what is there?

  • Thoughts on the content and resources provided?

  • How does this compare with what you expected?

  • How likely are you to use any of these resources? Why, why not?

• How would you use the main Service Provider and Leader landing page?

  • What are your thoughts on the rotating banner?

  • How does this compare to other websites you go for information?

• Allow user to select landing page of interest. Ask them to provide feedback on the content and layout.

  • Allow user to select resource of interest. Ask them to provide feedback on the content and layout.

Task # Mobile

Time permitting, users will be asked to browse the mobile site and provide general feedback on the look, images, navigation.

Exit Interview Questions

Time permitting, these questions will be asked at the end of testing.

1. What are your overall thoughts on what you saw today? Any specific likes or dislikes?

2. How does your initial impression of the site compare to your final impression?

   1. Before we looked at the site, you said you were interested in learning about {x, y, z}. How does what you saw compare to what you hoped to find?

   2. Was any of the content particularly motivating to act on? What type of action would you take?

3. Were there any features or content you expected to find that were not available? What?

4. Any content you were surprised to see? What?

5. What about the overall navigation and organization of the site?

   1. Is there anything jumping out as being out-of-place?

6. What about the terminology and the language? What type of changes need to be made?

7. Is this a site you would return to again, why/why not and if so, for what?

8. Would you encourage others to visit this site?

9. Would you be more apt to visit a financial site geared for members of the military, such as this one, or for geared for the general public? Why?

10. If you could change two things about the site, what would they be?

11. Can you think of anything we have not talked about that would be helpful for my client to know as they are making modifications to the site?

12. We greatly appreciate your time and participation today. As a thank you for your time, you will be receiving $150. Incentives will be sent in the form of a Usability Sciences’ company check by the United States Postal Service. We will be mailing out your check within 3 business days of your participation today. If for some reason you do not receive the check or have issues, please reach out to your contact from the recruiting firm (the person who scheduled you for the session).

This discussion guide ensures that all testing objectives are covered. We will also strive to observe natural usage patterns on the site. Therefore, tasks, flow, and timing will be adapted to meet user profiles and new testing needs.