MHV Redesign - Final

VA Generic Fast-Track Clearance Request Complete - MHV Redesign - Final.pdf

Generic Clearance for the Collection of Qualitative Feedback on Agency Service Delivery (NCA, VBA, VHA)

MHV Redesign - Final

OMB: 2900-0770

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 2900-0770 can be found here:

Document [pdf]

Download: pdf | pdf

Request for Approval under the “Generic Clearance for the Collection of
Routine Customer Feedback” (OMB Control Number: 2900-0770)
TITLE OF INFORMATION COLLECTION:
My HealtheVet (MHV) Website Redesign Veteran and Family Caregiver Focus Group

PURPOSE:

Human Factors Engineering (HFE) is currently in the pre-planning stages of a
series of 3-4 focus groups (25-30 total participants). The purpose of these focus
groups will be to gather feedback for the My HealtheVet website redesign effort.
Based on the results of these sessions, a proposed high-level navigation and user
interface (UI) design will be prepared. Additional sessions, either virtual or face-toface, will define and validate increasingly detailed navigation and UI designs.

DESCRIPTION OF RESPONDENTS:
The focus groups will include a total of 25-30 participants. Each focus group will
include approximately 8-9 veterans and caregivers, recruited by the My HealtheVet
project team. Recruited participants should represent a broad sampling of MHV
users (i.e., varying age groups, genders, technology skills, use of VA services,
medical conditions, etc.).
While a majority of the participants should be current MHV users, a small
number of individuals should be non-users of MHV (not more than ¼ of a total
group). Additionally, cognitively impaired participants (with PTSD and/or TBI)
should be included to ensure the dashboard design works well for them (not more
than ¼ of a total group).
Demographics will be collected as part of the recruiting effort in order to save
time during the session for more meaningful activities. These demographics will
include age, gender, technology use, mobile use, DS Logon account type (if any),
current MHV user, tasks performed on MHV, and perceived barriers to technology
use. This information will be collected via an online questionnaire to minimize
burden on the MHV coordinators.

TYPE OF COLLECTION: (Check one)
[ ] Customer Comment Card/Complaint Form
[ ] Usability Testing (e.g., Website or Software
[X] Focus Group

[ ] Customer Satisfaction Survey
[ ] Small Discussion Group
[ ] Other: ______________________

CERTIFICATION:
I certify the following to be true:
1. The collection is voluntary.
2. The collection is low-burden for respondents and low-cost for the Federal Government.
3. The collection is non-controversial and does not raise issues of concern to other federal
agencies.
4. The results are not intended to be disseminated to the public.
5. Information gathered will not be used for the purpose of substantially informing influential
policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have
experience with the program or may have experience with the program in the future.
Name:_Nancy R. Wilck ______________________________________________
To assist review, please provide answers to the following question:
Personally Identifiable Information:
1. Is personally identifiable information (PII) collected? [X] Yes [ ] No
2. If Yes, will any information that is collected be included in records that are subject to the
Privacy Act of 1974? [ ] Yes [X] No
3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [ ] Yes [ ] No
Gifts or Payments:
Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to
participants? [ ] Yes [X] No
BURDEN HOURS

Category of Respondent

No. of
Respondents

Participation Time
( × minutes =)

Burden
(÷ 60 =)

27
27
27

x 5 = 135
x 180 = 4860
x 185 = 4995

2.25 hours
81.00 hours
83.25 hours

Individuals – Demographic Survey
Individuals – Focus Group Session
Totals

FEDERAL COST: The estimated annual cost to the Federal government is $17,484.
If you are conducting a focus group, survey, or plan to employ statistical methods, please
provide answers to the following questions:
The selection of your targeted respondents
1. Do you have a customer list or something similar that defines the universe of potential
respondents and do you have a sampling plan for selecting from this universe?
[ ] Yes [X] No
If the answer is yes, please provide a description of both below (or attach the sampling plan). If
the answer is no, please provide a description of how you plan to identify your potential group of
respondents and how you will select them?
The MHV Coordinators at the following VA Medical Centers have been contacted and
asked to recruit 8-10 focus group participants.
• Washington, DC
• Baltimore, MD
• Richmond, VA
• Martinsburg, WV
• Wilkes-Barre, PA
Depending on the success of this recruitment effort, specific VA Medical Centers will
be selected as the sites of the focus group sessions.

Administration of the Instrument
1. How will you collect the information? (Check all that apply)
[ ] Web-based or other forms of Social Media
[ ] Telephone
[X] In-person. MHV Redesign Moderator Guide and MHV Redesign Participant
Activity Book, both attached.

[ ] Mail
[X] Other, Explain. A web-based demographic survey will be sent via email to
each selected focus group participant prior to the in-person focus group
sessions.

2. Will interviewers or facilitators be used? [X] Yes [ ] No

Supporting Statement for Paperwork Reduction Act Generic Information
Collection Submissions for “Generic Clearance for the Collection of
Qualitative Feedback on Agency Service Delivery”
A.

JUSTIFICATION

Circumstances Making the Collection of Information Necessary

The U.S. Department of Veterans Affairs Office of Informatics and Analytics (OIA) and the Human
Factors Engineering office (HFE) seek to obtain OMB approval of a generic clearance to collect
qualitative feedback from Veterans and family caregivers on a proposed redesign of the My HealtheVet
(MHV) website.
2.

Purpose and Use of the Information Collection

This collection of information, in the form of a customer survey and Veteran Focus Group sessions, is
necessary to help the MHV program ensure that its users have an effective, efficient, and satisfying
experience. This feedback will provide insights into customer satisfaction and focus attention on areas
where communication, training or changes in the MHV website can improve delivery of products and
services.
3.

Consideration Given to Information Technology

Initial demographic survey information will be collected from respondents in the form of an email
message and an online (web based) survey response.
4.

Duplication of Information

No similar data are gathered or maintained by the OIA or HFE or are available from other sources
known to these agencies.
5.

Reducing the Burden on Small Entities

There will be no small business involvement in this information collection event.
6.

Consequences of Not Conducting Collection

Without end user input into the redesign of the My HealtheVet program and website, we cannot be
assured that the resultant software product will meet the needs of Veterans and family caregivers.
7.

Special Circumstances

There are no special circumstances. The information collected will be voluntary and will not be used for
statistical purposes.
8.

Consultations with Persons outside the Agency

No public comments were solicited or revived.
9.

Payment or Gift

No payment has been authorized for respondents.
10.

Confidentiality

Participants and their feedback will be identified using unique IDs that are in no way related to their
personally identifiable information. During the session, the facilitator will explain the process of data
collection and assure the participants of their anonymity. The facilitator will also gather permission for
recording.
11.

Sensitive Nature

No questions will be asked that are of a personal or sensitive nature.
12.

Burden of Information Collection

The instruments used to collect information will result in a one-time burden of hours totaling
approximately 83.25 hours as described in the table below.

Category of Respondent
Individuals – Demographic Survey
Individuals – Focus Group Session
Totals

13.

No. of
Respondents

Participation Time
( × minutes =)

Burden
(÷ 60 =)

27
27
27

x 5 = 135
x 180 = 4860
x 185 = 4995

2.25 hours
81.00 hours
83.25 hours

Costs to Respondents

No costs are anticipated to the respondents.
14.

Costs to Federal Government

The anticipated cost to the Federal Government is approximately $17,484 annually. These costs
include preparation, execution and travel costs for government and contract staff.
15.

Reason for Change

Not applicable. This is a new request for a generic clearance.
16.

Tabulation of Results, Schedule, and Analysis Plans

There are no plans to publish the information collected on any of the forms.
17.

Display of OMB Approval Date

We are not seeking approval to omit the expiration date for the OMB approval from our Focus Group
instruments.
18.

Exceptions to Certification for Paperwork Reduction Act Submissions

There are no such exceptions.
B.

Collection of Information employing Statistical Methods

This collection of information does not employ statistical methods.

Focus Group: My HealtheVet Redesign

Moderator Guide
Veterans Health Administration
Human Factors Engineering
September 2014

Focus Groups: My HealtheVet Redesign
Moderator Guide

Table of Contents
How to Use this Moderator Guide .......................................................................................................................... 3
Session Timing Aide ............................................................................................................................................ 3
Room Set Up & Greeting Participants ................................................................................................................ 4
Session Script .......................................................................................................................................................... 5
Introduction (5 minutes) ..................................................................................................................................... 5
Ice Breaker: Get it Off the Chest (10 minutes) ................................................................................................... 7
Activity #1 – Card Sort: Primary Features & Content (25 minutes) .................................................................... 8
Activity #2 – First Impressions: Opening Screen (15 minutes) ......................................................................... 10
Activity #3 – Show Me: New Login (15 minutes) .............................................................................................. 11
Activity #4 – Walkthrough: View Lab Results in Blue Button (20 minutes) ...................................................... 13
Break ................................................................................................................................................................. 14
Activity #5 – First Impressions: Home/Dashboard (20 minutes) ...................................................................... 15
Activity #6 – Walkthrough: Print Upcoming Appointments (20 minutes) ........................................................ 16
Activity #7 – Design Inspection: Prescription Refill List (20 minutes) .............................................................. 17
Follow Up Questions from HFE (15 minutes) TIME PERMITTING ..................................................................... 18
Closing (10 minutes) ......................................................................................................................................... 18
Demographics Questionnaire ............................................................................................................................... 19
Closing Survey ....................................................................................................................................................... 21

Focus Groups: My HealtheVet Redesign
Moderator Guide

How to Use this Moderator Guide
The purpose of document is to provide the moderator with the script, tools and materials necessary to
successfully complete the facilitation of the focus groups to gather input for My HealtheVet overarching portal
design and functionality/features improvement.

Session Timing Aide
The following table is provided as a timing guide in order to ensure that all activities can be completed in the
total time allotted (180 minutes; 3 hours total, with a 10 minute break).
Activity Start Times

Time

Item

On
Hour

On Half
Hour

5 min

:00

:30

Introduction

10 min

:05

:35

Ice Breaker - Get it Off the Chest

25 min

:15

:45

Activity #1 - Card Sort: Primary Features & Content

15 min

:40

:10

Activity #2 - First Impressions: Opening Screen

15 min

:55

:25

Activity #3 - Show Me: New Login

20 min

:10

:40

Activity #4 - Walkthrough: View Lab Results in Blue Button

10 min

:30

:00

Break

20 min

:40

:10

Activity #5 - First Impressions: Home/Dashboard

20 min

:00

:30

Activity #6 - Walkthrough: Print Upcoming Appointments

20 min

:20

:50

Activity #7 - Design Inspection: Prescription Refill List

10 min

:40

:10

Follow Up Questions (open for questions from HFE; this also acts as a session “buffer” in
the event that any one portion runs long)

10 min

:50

:20

Closing (includes feedback/participation surveys and time for participants to complete their
book, if necessary)
Table 1. Recommended Session Agenda & Timing Aid

Focus Groups: My HealtheVet Redesign
Moderator Guide

Room Set Up & Greeting Participants
If possible, set the room up with desks/tables in a “u” shaped fashion such that participants can easily see and
speak to each other. Seat the moderator near the front of the room, with the participants, not in the center or
near the back, and leave an empty seat next to the moderator for the “Show Me” activities.
Observers/recorders should not be seated in the “u” with the participants and moderator, but should position
themselves at desks/tables in an unobtrusive location outside the “u”. Set up should also include connecting
and testing the projector, determining where to project (if no screen is provided) and locating light switches so
lights can be dimmed when needed.
As each participant arrives:
1. Greet the participant.
2. Check them off of the roll call sheet.
a. If the participant has not completed the demographic survey, provide them with a survey. Mark
it with their participant ID and ask them to have a seat to complete the survey and return it to
you before proceeding.
3. Provide the participant with a name card, a pen and an Activity Book. On the Activity Book cover, write
the participant’s ID number for future identification of their individual feedback provided in the book.
4. Ask the participant to please write only their first name on both sides of the name card and display it at
the front of their desk space when they have a seat.
a. Provide a marker for writing names.
5. Ask the participant to select a seat and make themselves comfortable.

Focus Groups: My HealtheVet Redesign
Moderator Guide

Session Script
Introduction (5 minutes)
The session today will focus on VA’s patient portal website, also known as My HealtheVet. My HealtheVet was
designed for Veterans, active duty Service Members, their dependents and caregivers. It helps you partner
with your health care team by providing you information and tools to make informed decisions and manage
your health care. As you may know, My HealtheVet has been around for about ten (10) years. The VA is
preparing to make quite a few revisions and enhancements to it, and has been gathering feedback from
Veterans like you to inform the changes. We have reviewed the feedback gathered to date and have prepared
some ideas to address a number of the concerns that we have heard. Our session today will focus on gathering
your feedback on how well these new designs address Veteran needs.
Many of you probably already use My HealtheVet. If you don’t, that’s ok too. We would like to thank you for
participating today. We rely heavily on volunteers such as you to make the VA’s systems better for Veterans.
We have representatives from VA’s Human Factors Engineering office who will be observing today and taking
notes. At the end of the session, time permitting, I may open up the floor for follow up questions from them.
But, until then, you will be interacting only with each other and me.
The session today will last no longer than 3 hours. We will have a 10 minute break in the middle of the session
to use the restroom and check messages. Until then, we would appreciate if you would avoid using your
mobile devices, if you have any, and place them on silent.
The session today is broken down into a series of quick activities discussing various aspects of the proposed
redesign for My HealtheVet. You each have a “Participant Activity Book” to follow along and provide your
individual feedback. The session will move quickly and will require your undivided attention to each activity.
So, please do not work ahead of the group.
After the activities, we may wrap up with questions from HFE regarding your feedback on the topics discussed
today. We will close the session with a questionnaire to gather your feedback on the session.
Your participation is completely voluntary and you may withdraw at any time. All of the information that you
provide will be kept confidential, and your name will not be associated with the feedback that you provide,
verbally or via your Activity Book. In order to compile your feedback with other participants in this study, we
would like to record the session.
Question: Does anyone object to recording?
OK, we will start recording now.

Focus Groups: My HealtheVet Redesign
Moderator Guide

Now that we are recording, for our records, I would like to confirm that we have your permission to record
your voice. Does anyone object to recording? Thank you.
Question: Does anyone have any questions before we get started?

Focus Groups: My HealtheVet Redesign
Moderator Guide

Ice Breaker: Get it Off the Chest (10 minutes)
We are going to start off with an ice breaker activity to get to know each other by sharing your interests in
participating today. Please flip to page 4 of your Activity Book for a worksheet on this exercise. In a moment, I
will call on you one at a time. When I call on you, please introduce yourself. Then, based on your thoughts
when you agreed to participate today, tell us about the most important feedback that you hoped to share
with the VA. Each of you will have only a couple of minutes to share. You can feel free to provide more details
to our team by writing those thoughts down in your Activity Book on page 4.
[Moderator: Call on each participant, by first name, in a “round robin” fashion and pose the question above.
Allow each participant 1-2 minutes to provide their answer.]

Activity Book Page 4: Ice Breaker

Focus Groups: My HealtheVet Redesign
Moderator Guide

Activity #1 – Card Sort: Primary Features & Content (25 minutes)
[Moderator: While you are reading the introduction to this activity, ask a team member to hand out the sets of
index cards and stack of sticky notes to each of the participants.]
We are interested in how you think about the content and features available on My HealtheVet. In this
activity, we will learn more about how you would organize and group these items.
One of my team members is providing each of you with a set of index cards and a small pad of sticky notes. On
each index card, a type of content or a feature is listed. I’d like to ask you to use the next 10 minutes to
separate the cards into groups that represent content and features that you would expect to appear together
on the My HealtheVet web site. When you are finished, use the sticky notes to label each of your groups and
then answer the question found on page 5 in your Activity Book. After about 15 minutes, we will discuss your
work as a group.

Activity Book Pages 5-6: Activity #1

Okay everyone; let’s talk about how you grouped the cards. We are specifically interested in your groups – not
each individual card, but how you bundled them together and for what reason. We are going to collect your
cards and labels, so we will be able to see all of the details.
Probing Questions: Who would like to share how they grouped and why? Did anyone group differently? Why?
[Moderator: For the sake of time, please avoid allowing participants to list out each grouping and the specific
cards in each group. Instead, encourage participants to discuss the groupings as a whole and their thought
process surrounding why they grouped certain items together.]
Focus Groups: My HealtheVet Redesign
Moderator Guide

Thank you for your Feedback. We are going to move on to Activity #2, which covers the opening screen of My
HealtheVet.
[Moderator: As you move on to the next activity, ask a team member to collect the index cards from each
participant. For each set, staple the cards together with the sticky note listing the label. Then place the set in
an envelope marked with the participant’s ID.]

Focus Groups: My HealtheVet Redesign
Moderator Guide

Activity #2 – First Impressions: Opening Screen (15 minutes)
We will take the next couple minutes to do an activity in your book and then discuss as a group. Please direct
your attention to your Activity Book. Flip to page 7 and use the next 2 minutes to complete Activity #2 on
your own. When you are finished, we will spend a few minutes discussing as a group.
[Moderator: Probe participants individually, as needed, to ensure that they were able to find the activity and
are working successfully in the activity book.]

Activity Book Pages 7-8: Activity #2

Okay everyone; let’s talk about your responses. [Moderator: Display the current design on the screen.]
Probing Question: Who would like to share their first impressions of the opening screen?
Probing Question: Does this design make you more or less likely to use My HealtheVet? If you are a current
user, how does it compare with the current design?
[Moderator: Be prepared to display the current design on the screen, if needed.]
Thank you for your Feedback. We are going to move on to Activity #3, which covers the Login features of My
HealtheVet.

Focus Groups: My HealtheVet Redesign
Moderator Guide

Activity #3 – Show Me: New Login (15 minutes)
In this activity, we are going to look at a prototype of a potential new design for the login features for My
HealtheVet. For this activity, I need a volunteer to sit here next to me and drive the prototype using my
computer. Any takers?
[Moderator: Identify a volunteer to interact with the prototype and ask them to relocate to the seat next to
you.]
Thank you for volunteering! You don’t need to do anything yet. In a moment, I’m going to ask you to use the
prototype to login to My HealtheVet. As you work, please “think out loud” as you are presented with the
screens. In other words, I’d like for you to give us all a “blow by blow” of what you are thinking as you work.
For example, explain why you are clicking on the elements that you are clicking on, why you are entering the
information that you enter and what is going through your head as you work. It may be a bit awkward at first
but it’s really very easy once you get used to it. Basically, we’d like any thought that comes into your head to
come out your mouth. For example, if I were driving to work I might say as I get in my car, “I’m opening the car
door and sitting down, my seat really hot from sitting in the sun…I should put it in the shade tomorrow. I’m
putting on my seat belt; I don’t want to get hurt if I’m in an accident. I’m turning the ignition. I’m almost out of
gas, I need to remember to get some tonight on my way home. I’m looking behind me before I reverse to
make sure no one’s in the alley; there are a lot of kids walking to school sometimes and that makes me
nervous.”
Question: Does that make sense?
All of you all will be able to see what [insert volunteer first name] is doing on the projector here. As [insert
volunteer first name] works, please feel free to make notes on how you would work differently by using page 9
in your Activity Book. But, please refrain from interrupting [him/her]. When [he/she] is done, we will discuss
your thoughts about this prototype as a group.
Question: Does anyone have any questions before [insert volunteer first name] gets started?

Focus Groups: My HealtheVet Redesign
Moderator Guide

Activity Book Pages 9-10: Activity #3

[Moderator: Address any questions and then ask the volunteer to begin. If Morae is being used, start the
recorder.]
Thank you for showing us how it’s done! Okay everyone; let’s talk about your thoughts.
Probing Question: Does anyone feel that they would have done anything differently from [insert volunteer
first name]?
Early in the login process, My HealtheVet displayed a set of options for different credentials that can be used.
I’m showing that part of the prototype on the projector now so that you can refer to it.
[Moderator: Display the dialog box with the credential options on the screen.]
Probing Question: Which login options seemed useful or applicable to you? Why do you think that the VA is
offering all of these different options?
Thank you for your Feedback. We are going to move on to Activity #4, which covers another feature of My
HealtheVet. After the next activity, we will take a 10 minute break.

Focus Groups: My HealtheVet Redesign
Moderator Guide

Activity #4 – Walkthrough: View Lab Results in Blue Button (20 minutes)
In this activity, we are going to walk through a prototype of a potential new design for part of the VA’s Blue
Button feature. Blue Button allows you to view, print or download all of the personal health information
currently available in your My HealtheVet account. This includes labs, appointments, prescriptions and other
medical records as well as DOD military service information. In this walkthrough, we are going to be focusing
on viewing and interpreting lab results.
I would like each of you to think about a situation when you have recently had lab work completed and you
were interested in viewing those results online. We are going to rotate through the room in a “round robin”
fashion as we walk through the prototype to view and interpret lab results. When I call on you, please explain
the action that you would take next based on whatever is currently displayed on the screen. For each action
and screen, we will discuss as a group. During this activity, please use the space provided on page 11 in your
Activity Book to provide us with your notes on how you would use this feature.

Activity Book Pages 11-12: Activity #4

[Moderator: Display the dashboard homepage on the screen. Begin with the person on your right and ask the
person what they would do to access their most recent lab results. Make the corresponding action based on
their response. Ask the probing questions below for each speaker and then proceed to the next person.]
Probing Question (for the speaker): Is this what you expected (to see/to happen)?
Probing Question (for the group): Did anyone expect something different? Would anyone have done anything
differently?

Focus Groups: My HealtheVet Redesign
Moderator Guide

Break
Thank you for your feedback. We are now going to take a 10 minute break.
The time is now [read time]. We will get started again promptly at [current time plus 10 minutes].
When we return, we are going to move on to Activity #5, where you will get to review and provide feedback
on the new dashboard style homepage for accessing all of your My HealtheVet features.

Focus Groups: My HealtheVet Redesign
Moderator Guide

Activity #5 – First Impressions: Home/Dashboard (20 minutes)
We will take the next couple minutes to do an activity in your book and then discuss as a group. Please direct
your attention to your Activity Book. Flip to page 13 and use the next 2 minutes to complete Activity #5 on
your own. When you are finished, we will spend a few minutes discussing as a group.
[Moderator: Probe participants individually, as needed, to ensure that they were able to find the activity and
are working successfully in the activity book.]

Activity Book Pages 13-14: Activity #5

Okay everyone; let’s talk about your responses. [Moderator: Display the design on the screen.]
Probing Question: Who would like to share their impressions of this design?
Probing Question: Does this design make you more or less likely to use My HealtheVet? If you are a current
user, how does it compare with the current design?
[Moderator: Be prepared to display the current design on the screen, if needed.]
Probing Question: If you are a current user, would this design make it easier or more difficult to access the
features that you use the most?
Thank you for your feedback. We are going to move on to Activity #6, which covers another feature of My
HealtheVet.
Focus Groups: My HealtheVet Redesign
Moderator Guide

Activity #6 – Walkthrough: Print Upcoming Appointments (20 minutes)
In this activity, we are going to do another walkthrough like the one we completed before the break. This
time, we are going to be looking at a prototype of a potential new design for part of the VA’s Appointment
features. We are going to be focusing on the ability to view and print past and upcoming appointments.
I would like each of you to think about a situation when you were interested in viewing or printing your
appointments using My HealtheVet. As before, we are going to rotate through the room in a “round robin”
fashion as we walk through the prototype to view and print appointments. When I call on you, please explain
the action that you would take next based on whatever is currently displayed on the screen. For each action
and screen, we will discuss as a group. During this activity, please use the space provided on page 15 in your
Activity Book to provide us with your notes on how you would use this feature.

Activity Book Pages 15-16: Activity #6

[Moderator: Display the dashboard homepage on the screen. Begin with the person to the right of the last
person to contribute to the lab results walkthrough. Ask the person what they would do to view their
appointments. Make the corresponding action based on their response. Ask the probing questions below for
each speaker and then proceed to the next person.]
Probing Question (for the speaker): Is this what you expected (to see/to happen)?
Probing Question (for the group): Did anyone expect something different? Would anyone have done anything
differently?
Thank you for your feedback. We are going to move on to Activity #7, which covers the last feature of My
HealtheVet that we will be looking at today.

Focus Groups: My HealtheVet Redesign
Moderator Guide

Activity #7 – Design Inspection: Prescription Refill List (20 minutes)
We will take the next couple minutes to do an activity in your book and then discuss as a group. Please direct
your attention to your Activity Book. Flip to page 17 and use the next 3 minutes to complete Activity #7 on
your own. When you are finished, we will spend a few minutes discussing as a group.
[Moderator: Probe participants individually, as needed, to ensure that they were able to find the activity and
are working successfully in the activity book.]

Activity Book Pages 17-18: Activity #7

Okay everyone; let’s talk about your responses. [Moderator: Display the design on the screen.]
Probing Question: Who would like to share their thoughts about this design?
[Moderator: Probe participants specifically regarding language and labels used on the screen (e.g., statuses;
VA meds and non-VA meds/self-entered).]
Probing Question: Does this design make you more or less likely to use My HealtheVet? If you currently use
this feature, how does it compare with the current design?
[Moderator: Be prepared to display the current design on the screen, if needed.]

Focus Groups: My HealtheVet Redesign
Moderator Guide

Follow Up Questions from HFE (15 minutes) TIME PERMITTING
That was our last activity. We will now open the floor for follow up questions from VA’s Human Factors
Engineering team.

Closing (10 minutes)
[Moderator: While reading the text below, ask a team member to pass out the feedback and participation
survey.]
That brings us to the end of the session. Thank you for your participation, candor and feedback. We are now
passing out a follow up survey with some specific questions on today’s panel as well as your experience
overall. We welcome any feedback on how we can make things better for future panels.
When you are finished with the survey, you may use any remaining time to provide any final feedback in your
Activity Book. A blank sheet is provided at the end of the book (on page 19) for this purpose. You may also feel
free to flip back to any other portion of the book to provide more feedback in any area. When you are
finished, please turn in your Activity Book to me before you leave.
Thank you and have a great week.

Focus Groups: My HealtheVet Redesign
Moderator Guide

Demographics Questionnaire
This questionnaire will be distributed by the recruiters via SurveyMonkey prior to the session. Those
participants who do not complete it in advance will be asked to complete it in paper form as part of the check
in process.
1. Gender:

□ Male
□ 18 – 39

□ Female
□ 40 – 59

2. Age Range:
□ 60 or older
3. Do you currently use VA healthcare services? □ Yes □ No
4. Are you a caregiver for a Veteran currently using VA healthcare service?

□ Yes □ No

5. If “Yes”, do you use your Veteran’s My HealtheVet account?

□
□
□

Yes
No, I have my own account.
No, my Veteran does not have a My HealtheVet account.

6. Do you have a My HealtheVet account?

□ Yes □ No

7. If “Yes”, what “level” of account do you have?

□
□
□
□
□

Basic

□
□
□
□
□
□

Appointment

□
□
□
□

Laptop or Desktop Computer

Advanced
Premium
I Don’t Know

Not Applicable (I don’t have an account.)
8. If “Yes”, do you use any of the following features? Please select all that apply.
Prescription Refills
Downloading Medical Records (Blue Button)
Secure Messaging
Other: ____________________________________________

Not Applicable (I don’t have an account.)
9. When you use the Internet, what device do you use most of the time?
Mobile Phone
Tablet Device
Gaming System

10. Do you use apps on a mobile device (e.g., smartphone or tablet)?
Focus Groups: My HealtheVet Redesign
Moderator Guide

□ Yes □ No
19

11. Do you use any apps provided by the VA?

□ Yes □ No

12. If “Yes”, which apps?
13. How would you describe your use of technology (in general)?

□
□
□
□
□

Beginner

□
□
□
□
□
□

Does not affect my use of technology

Advanced Beginner
Intermediate
Advanced Intermediate

Expert
14. If you are being treated for a cognitive disorder (including Post Traumatic Stress Disorder or Traumatic
Brain Injury), do you feel that your condition affects your ability to use technology (including web sites
and/or mobile apps)?
Mildly affects my use of technology
Affects my use of technology
Significantly affects my use of technology
Prevents my ability to use technology most of the time
Not Applicable (I am not being treated for a cognitive disorder or I prefer not to answer.)

15. Do you require any assistive technologies when using a computer?

□ Yes □ No

16. If “Yes”, please describe:

Focus Groups: My HealtheVet Redesign
Moderator Guide

Closing Survey
The following survey will be administered to the participants at the end of each session.
Thank you for your participation in our study. It was a pleasure to work with you, and we really appreciate
your time. In an effort to continually improve how studies are conducted by Human Factors Engineering, we
would like your feedback on the session carried out by the team today.
1. The Human Factors team sufficiently explained the purpose of the session and my role as a study
participant.

□
□
□
□
□

Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree

□
□
□
□

Agree
Neutral
Disagree
Strongly Disagree

□
□

Agree
Neutral

Focus Groups: My HealtheVet Redesign
Moderator Guide

□
□

Disagree
Strongly Disagree

□
□
□
□

Agree
Neutral
Disagree
Strongly Disagree

Any comments? ______________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
5. We are dedicated to providing you the best possible experience. If you have identified areas where we
need to improve, could we contact you to discuss further? This is completely voluntary. If you are
willing to be contacted, please include your contact information below. Thank you.
Name: _____________________________________________________
Email: ____________________________________________________
Phone: ____________________________________________________
6. Are you interested in participating in future studies to help the VA build better software products for
Veterans, Caregivers and Providers? For example, testing VA online tools such as mobile apps and
websites?
□ Yes. If “Yes”, please complete the remaining questions in this questionnaire.

□

Focus Groups: My HealtheVet Redesign
Moderator Guide

7. I am a:

□
□
□

Clinician
Veteran
Family Caregiver

8. Full name and Title: ____________________________________________________________
9. Email (if not provided above): ____________________________________________________
10. Phone (if not provided above): ___________________________________________________
11. Select your test location preference(s):
□ Remote (must have computer, high speed internet, phone and a quiet location for test session)

□

In person, on site (please specify VA Medical Facility): __________________________

Focus Groups: My HealtheVet Redesign
Moderator Guide

Focus Group: My HealtheVet Redesign

Participant Activity Book
Participant ID#: ______

Veterans Health Administration
Human Factors Engineering
September 2014

Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Table of Contents
How to Use this Activity Book ................................................................................................................................. 3
Ice Breaker: Get It Off Your Chest ........................................................................................................................... 4
Activity #1 ............................................................................................................................................................... 5
Activity #2 ............................................................................................................................................................... 7
Activity #3 ............................................................................................................................................................... 9
Activity #4 ............................................................................................................................................................. 11
Activity #5 ............................................................................................................................................................. 13
Activity #6 ............................................................................................................................................................. 15
Activity #7 ............................................................................................................................................................. 17
Any Additional Feedback ...................................................................................................................................... 19

Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

How to Use this Activity Book
This “Activity Book” is intended to be used by you during the focus group session. You will use it during the
session to follow along with the activities and provide your individual feedback. The session will move quickly
and will require your undivided attention to each activity. So, please do not work ahead of the group.
Because the session will move quickly, you can feel free to provide additional comments or feedback in the
book after the session before you turn it in to the moderator. A blank sheet is provided at the end of the book
(on page 19) for this purpose.
The VA’s Human Factors Engineering office is conducting this session today. It will focus on VA’s patient portal
website, known as My HealtheVet. My HealtheVet was designed for Veterans, active duty Service Members,
their dependents and caregivers. It helps you partner with your health care team by providing you information
and tools to make informed decisions and manage your health care. As you may know, My HealtheVet has
been around for about ten (10) years. The VA is preparing to make quite a few revisions and enhancements to
it, and has been gathering feedback from Veterans like you to inform the changes. We have reviewed the
feedback gathered to date and have prepared some ideas to address a number of the concerns that we have
heard. Our session today will focus on gathering your feedback on how well these new designs address
Veteran needs. HFE would like to thank you for participating today. We rely heavily on volunteers such as you
to make the VA’s systems better for Veterans.

Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Ice Breaker: Get It Off Your Chest
Task: When the moderator calls on you, introduce yourself to the group by providing your first name and
then answering the following question. You will have 1-2 minutes to share with the group. Use the space
below to share any other feedback that you are not able to say.

What is the most important feedback you hoped to share with the VA during the session
today?
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________

Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Activity #1
We are interested in how you think about the content and features available on My HealtheVet. In this
activity, we will learn more about how you would organize and group these items.

Task: The moderator will provide you with a set of index cards and a small pad of sticky notes. On each index
card, a type of content or feature is listed. Using the next 10 minutes, separate the cards into groups that
represent content and features that you would expect to appear together on the My HealtheVet web site.
When you are finished, use the sticky notes to label each of your groups and then answer the following
question. After about 15 minutes, we will discuss your work as a group.

Please list the labels for each of the groups that you created. For each, please explain why
you grouped your cards together in this fashion and why you chose the label:
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Use the space below to provide any additional feedback on this topic:
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________

Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Activity #2
The following graphic depicts a potential new design for the main screen for My HealtheVet:

Task: Review the screen design above and then answer the following questions. Spend no more than 2-3
minutes total. We will discuss your responses as a group, but are interested in your individual feedback.

Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

How does this design make you feel about My HealtheVet? Please list the first three
feelings, thoughts or impressions that come to mind:
1. ____________________________________________________________________________________
2. ____________________________________________________________________________________
3. ____________________________________________________________________________________

In your opinion, would this design make you more or less likely to use My HealtheVet?
More Likely
No Change
Less Likely

Activity #3
In this activity, we are going to look at a prototype of a potential new design for the login features for My
HealtheVet. For this activity, a volunteer will drive the prototype using the moderator’s computer. If you are
not the volunteer, please do not interrupt him/her as he/she works.

Task: Instead, please use the space below to keep your notes on how you would do things differently from
our volunteer. Please also answer the question provided below. When our volunteer is finished using the
prototype, we will discuss your thoughts about this prototype as a group.

What would you do differently than our volunteer?
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________

Which of the following sets of credentials (login options) would be most useful or
applicable to you? (Please select all that apply.)
My HealtheVet
DSLogin
PIV CARD
CAC Card
Norton Symantec
USAA
Federal Cloud
None of these. I suggest allowing me to use: __________________________________
Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Activity #4
In this activity, we are going to walk through a prototype of a potential new design for part of the VA’s Blue
Button feature. Blue Button allows you to view, print or download all of the personal health information
currently available in your My HealtheVet account. This includes labs, appointments, prescriptions and other
medical records as well as DOD military service information. In this walkthrough, we are going to be focusing
on viewing and interpreting lab results.

Task: We are going to rotate through the room as we walk through the prototype. When the moderator calls
on each person, the individual will contribute to the walkthrough by explaining the action that they would take
next. For each action and screen, we will discuss as a group. During this activity, please use the space below to
provide us with your notes on how you would use this feature.

How would you use this prototype?
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________

Do you currently use Blue Button on the My HealtheVet web site?
Yes
No
I don’t remember

Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Activity #5
The following graphic depicts a potential new design for the home page after you login to My HealtheVet:

On the graphic above, please mark the first three things you would interact with by
providing a number in a circle (e.g., , or ) with an arrow to the item. For each, please
explain how you expect that item to work:
1. ____________________________________________________________________________________
2. ____________________________________________________________________________________
3. ____________________________________________________________________________________

Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Is there anything missing from the design that you think should be added? If so, please
describe:
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________

In your opinion, would this design make you more or less likely to use My HealtheVet?
More Likely
No Change
Less Likely

In your opinion, would this design make it easier or more difficult to access the features
that you use the most (or would use the most, if you are not currently using My
HealtheVet)?
Yes, it would make it easier for me to access the features I use most.
It would not change the effort to access the features I use most.
No, it would make it harder for me to access the features I use most.

Activity #6
In this activity, we are going to walk through a prototype of a potential new design for part of the VA’s
Appointment features. We are going to be focusing on the ability to view and print past and upcoming
appointments.

Do you currently use appointment features on the My HealtheVet web site?
Yes
No
I don’t remember
Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Activity #7
The following graphic depicts a potential new design for viewing your prescriptions on My HealtheVet:

On the graphic above, please mark the first three things you find confusing by providing a
number in a circle (e.g., , and ). For each, please explain why you feel it is confusing:
1. ____________________________________________________________________________________
2. ____________________________________________________________________________________
3. ____________________________________________________________________________________
Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

Which prescription will expire first?
__________________________________________________________________________________________

What is the earliest date that you can order a refill of Simvastatin?
__________________________________________________________________________________________

Which prescription is ready for you to pick it up?
__________________________________________________________________________________________

When can you expect to receive the refill you ordered for Omeprazole?
__________________________________________________________________________________________

In your opinion, would this design make you more or less likely to use My HealtheVet?
More Likely
No Change
Less Likely

Any Additional Feedback
Because the session moved quickly, you can feel free to provide additional comments or feedback in the space
below (or in the corresponding areas with each activity) before you turn it in to the moderator.

Use the space below to provide any additional feedback or thoughts that you were not able
to share during the session today:
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
__________________________________________________________________________________________
Veteran Panel Focus Group: My HealtheVet & Accessing Mobile Apps
Participant Activity Book

File Type	application/pdf
File Title	DOCUMENTATION FOR THE GENERIC CLEARANCE
Author	558022
File Modified	2015-09-22
File Created	2014-08-04