2020 Common Data Elements Repository (CDER) Website Usability Testing (NLM)

OMB Control Number: 0925-0648
Expiration Date: 05/31/2021
Public reporting burden for this collection of information is estimated to average 60 minutes
per response, including the time for reviewing instructions, searching existing data sources,
gathering and maintaining the data needed, and completing and reviewing the collection of
information. An agency may not conduct or sponsor, and a person is not required to respond
to, a collection of information unless it displays a current valid OMB control number. Send
comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to NIH, Project Clearance Branch, 6705
Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0648). Do not return
the completed form to this address.
______________________________________________________________________________

Pre-test Questions:

Can you tell us about yourself and your experience with CDEs & the repository?
Do you use the NIH CDE Repository website regularly? If not, Which CDE websites do
you use and why?
a. How often do you visit these websites?
b. What do you like/dislike about these websites?
3. How do you usually prepare your clinical research forms?
4. Home page Questions
a) Having now seen the home page, what is your initial reaction? What is your opinion
of the design and content?
b) If you visited this page, which link would you start with? Why? What would you
expect?
c) Without clicking, describe the options you see? What do you think they are for? Any
more?
d) Is there anything that stands out? Why do you say this?
e) Is there anything you would have expected/wanted/needed to see on this page that
is not here?
f) Is there anything you like/dislike? Why do you say this?
g) Is there anything unclear? Why do you say this?
h) Do you think the sample content available is helpful and engaging? Why do you say
this?
1.
2.

Task based usability testing
Activity #1, Find CDE and Export
Goal/Output:
Inputs:
Steps:

Success
Criteria:
Notes:

Understand findability and learnability
CDE Name “Race”
1. Go to the NIH CDE R website (Starting Point)
2. Click on CDEs link in the header
3. Search for “race”
4. Click on the 3rd CDE “Race” with “LOINC” identifier
5. Click on “Export” button
6. Click on “JSON or XML” (Finish Point)
Exporting the “Race” CDE as in JSON or XML format

Scenario:
You are working on a new research project and you need to collect race data using LOINC
codes. Go to the NIH CDE Repository website and export the appropriate “Race” CDE.
Probes:
1. Where would you start to look for this information?
2. What keyword are you specifically looking for?
3. Upon finding the page, is this what you expected?
4. Is this where you expected this information to be?
5. What do you think about the page layout? Why do you say that?
6. How did you find this task? Was this an easy/difficult task? Why do you say this?
7. Can you suggest any improvements?

Activity #2, Find Form and Export
Goal/Output:
Inputs:
Steps:

Success
Criteria:
Notes:

• Understand findability and learnability
• Understand the ability to export forms for different formats
Form Name “Demographics”
1. Go to the NIH CDE R website (Starting Point)
2. Click on Forms link in the header
3. Search for “Demographics”
4. From the filters panel, click on “NINDS”
5. Select the form
6. Click on “Export” button
7. Select and click on “FHIR Questionnaire” (Finish Point)
Exporting the “Demographics” form as FHIR Questionnaire format

Scenario:
Your team is working on an existing clinical research project for the National Institute of
Neurological Disorders and Stroke (NINDS) that utilizes FHIR standards. You were tasked to find
an existing demographics form that has at least 30 questions. Go to the NIH CDE Repository
website and export an appropriate “Demographics” form.
Probes:
1. Where would you start to look for this information?
2. Upon selecting this page, what do you think of the layout and information on the page?
3. Is there anything unclear?
4. Have a read of the information. Do you think the copy is clear and easy to follow?
5. Is there enough information?
6. How did you find this task? Was this an easy/difficult task? Why do you say this?
7. Can you suggest any improvements?

Activity #3, Find CDE source
Goal/Output:
Inputs:
Steps:

Success
Criteria:
Notes:

Understand how the users evaluate the credibility of the CDEs
CDE “Gender Code”
1. Go to the NIH CDE R website (Starting Point)
2. Click on CDEs link in the header
3. Search for “Gender Code”
4. From the filters panel, click on “Qualified” from the registration
status group
5. From the filters panel, click on “NCI” from the Classification
group
6. From the table
a. The user can Identify the CDE source; or
b. Click on the CDE
i. From the General Details, the user can find the
CDE source
Finding and identifying the “caDSR: 2179640 v1” CDE source
(additional measures of success will be evaluated across all activities,
see PowerPoint document for additional measures of success)

Scenario:
Your team is working on clinical research for the National Cancer Institute (NCI), and you were
asked to find a qualified Gender Code Common Data Element (CDE). Go to the NIH CDE
Repository website, find a qualified Gender Code CDE, and identify the source of the CDE.
Probes:
1. Where would you start to look for this information?
2. What keyword are you specifically looking for?
3. Is this what you would expect? Why do you say this?
4. What do you think of the layout of this page?
5. Is this a useful tool? Is this something you are likely to use? Why do you say that?
6. Are there any other features would you expect or like to find?
7. How did you find this task? Was this an easy/difficult task? Why do you say this?
8. Can you suggest any improvements?

Activity #4, Find CDE Linked Forms
Goal/Output:
Inputs:
Steps:

Understand the use of research tools and scales
CDE “Feeling down, depressed, or hopeless in last 2 weeks” page
1. Go to the CDE page (Starting Point)
2. Click on CDEs “Linked Forms” button on the page

Success
Criteria:

Finding and Identifying how many forms are linked to the “Feeling
down, depressed, or hopeless in last 2 weeks” CDE.
Answer: 4 forms
(additional measures of success will be evaluated across all activities,
see Powerpoint document for additional measures of success)

Notes:

Scenario:
You are working on clinical research and you want to know if the “Feeling down, depressed, or
hopeless in last 2 weeks” CDE is used in other clinical research. How many forms use the
“Feeling down, depressed, or hopeless in last 2 weeks” CDE?

Probes:
1. Where would you start to look for this information?
2. What keyword are you specifically looking for?
3. Is this what you would expect? Why do you say this?
4. What do you think of the layout of this page?
5. Is this a useful tool? Is this something you are likely to use? Why do you say that?
6. Are there any other features would you expect or like to find?
7. How did you find this task? Was this an easy/difficult task? Why do you say this?
8. Can you suggest any improvements?

Activity #5, Find CDE Used By
Goal/Output:
Inputs:
Steps:

Better understand the navigation system and findability
CDE “Person Birth Date” page
1. Go to the CDE page (Starting Point)
2. Identify the organization name

Success
Criteria:

Finding and identifying the organization name for “Person Birth Date”
CDE.
Answer: NINR NCI

Notes:

Scenario:
You are examining the “Person Birth Date” common data element. Find the name(s) of the
organization(s) that used the CDE.
Probes:
1. Where would you start to look for this information?
2. Is there anything unclear or unexpected?
3. How did you find this task? Was this an easy/difficult task? Why do you say this?
4. Can you suggest any improvements?

Activity #6, Find “More like this” CDE
Goal/Output:
Inputs:
Steps:

Understand the navigation system and findability
CDE “Body Weight” page
1. Go to the CDE page (Starting Point)
2. Click on CDEs “More Like This” button on the page

Success
Criteria:
Notes:

Finding and identifying how many similar.
Answer: 10 CDEs

Scenario:
How many CDEs are similar to the “Body Weight” CDE?
Probes:
1. Where would you start to look for this information? Where would you expect it to be?
2. Having found this information, is it what you expected? Why do you say this?
3. Is there anything unclear or unexpected?
4. How did you find this task? Was this an easy/difficult task? Why do you say this?
5. Can you suggest any improvements?

Activity #7, Compare CDE
Goal/Output:
Inputs:
Steps:

Success
Criteria:
Notes:

• Understand how the end users interact and pick a specific CDE
• Understand usage of unique forms
No input
1. Go to the website home page (Starting Point)
2. Find 2 CDEs through the search page
a. Export the results
i. Compare the results
b. Compare the results on the page
3. Add the CDEs to board
a. Export the result
i. Compare the results
b. Compare the result on the page
Find the CDEs and compare them

Scenario:
You are interested in comparing the permissible values of two or more CDEs. Using the
website, compare two or more CDEs’ permissible values.
`
Probes:
1. Where would you start to look for this information? Where would you expect it to
be?
2. Having found this information, is it what you expected? Why do you say this?
3. Is there anything unclear or unexpected?
4. How did you find this task? Was this an easy/difficult task? Why do you say this?
5. Can you suggest any improvements?

Post-test Survey
1. System Usability Scale
Thinking about the NIH CDE Repository website, how much do you agree or disagree with
each of the following statements:
Questions
I think that I would like to use this
system frequently
I found the system unnecessarily
complex.
I thought the system was easy to use.
I think that I would need the support of
a technical person to be able to use this
system.
I found the various functions in this
system were well integrated.
I thought there was too much
inconsistency in this system.
I would imagine that most people would
learn to use this system very quickly.
I found the system very cumbersome to
use.
I felt very confident using the system.
I needed to learn a lot of things before I
could get going with this system.
Ask if they have any further comments

Strongly
Agree
Agree

Neutral

Disagree

Strongly
Disagree

Tasks
See Worksheet to develop tasks for testing on how to create tasks from the user stories.
Index

Starting point

Instruction to participants

What to observe

Focus
on
issue

Index

Starting point

Instruction to participants

What to observe

Focus
on
issue

T0

Interview

Please share with us what do you do. How much do
you already know about NLM CDE Repo?

T1

Browser starting
page

You are working on a new research and you need to
find a common data element to measure blood
pressure. You have been told to use the CDE
Repository to find one.
• What can you do on this site?

T2

CDE home →
Search vs
browsing

T3

CDE results →
Narrow down of
results

T4

CDE results



Website findability

If browsing or searching is used
Prompt participant to browse if
needed

2

What will you do to narrow down the results?

Filter usability

1

Which one will you choose? Why?

What do users think/do when
there are many identical or
similar elements?

1

Index

Starting point

Instruction to participants

What to observe

Focus
on
issue

Index

Starting point

Instruction to participants

What to observe

Focus
on
issue

T5

Single CDE
display

What information are displayed for this CDE ?

Tabbed display of a CDE page visibility

4

T6

CDE home

What will you do if you want to find how other
studies have ask people about blood pressure?

Visibility of the forms search visual and conceptual

3

T7

CDE home

Browsing or searching
Prompt participant to browse if
needed

2

T8

CDE form search
results

Which one of the forms will you choose? Why?

recognization of the source

1

T9

CDE form

What information are displayed for the form you've
chosen?

Tabbed display usability

4