Moderator Guide

Moderator’s Guide

USITC.gov

User Experience Interviews

23 April 2014

_Introduction

Thank you for participating in this interview today. My name is __, and I am a researcher with the Fors Marsh Group User Experience Team. Today you’ll be helping to evaluate the USITC website. The pages you will interact with are of the live site and contain the same information you would see if you accessed the site at work or home.

I will ask you questions and give you some tasks to complete using the website. Your comments and feedback will help the design team improve the site. I am a researcher and did not create the site, so please do not feel like you have to hold back on your thoughts to be polite to me. I’m interested in both your positive and negative reactions. Difficulties you may run into reflect issues with the design of the site, not your skills or abilities. Please keep in mind this is not a test of you or your knowledge, but it’s a test of the website. You are helping us to see how we can improve the site.

The entire session will last about 90 minutes. In accordance with the Papework Reduction Act of 1995, this study as been approved by the Federal government’s Office of Management and Budget (OMB) under OMB control number 3117-0222 and expires on 9/30/2017. Without this approval we could not conduct this study. Do you have any questions so far?

Let’s cover a few things before we get started.

• We are making a video recording of the computer screen during this session, but your name and personal information will not be associated with the recordings or your responses.

• I am interested in your thoughts and reactions as we proceed. This is important because I’ll be able to see how you’re using the website, but I won’t know why you are doing what you’re doing. So I need your help. While you are working, I would like you to think aloud. If you would please describe the steps you’re taking and what you’re thinking about as you are using the site.

• There are no right or wrong answers, and your comments and opinions will only be used in combination with the feedback that we get from other people.

• Have you had the opportunity to review the Privacy Act Statement?

• Do you have any questions?

• Ok, we are ready to begin. I would like you to start by asking you some background questions.

_{Questions and Tasks}

Section 1: Getting to know the participant

Goal: To get to know participants and explore their relevant past behavior.

1. You had indicated on the questionnaire you completed that you are in a [Insert role] role with your organization. Could you tell me about your responsibilities in this role that take you to the USITC website?

2. On the questionnaire you completed, you had indicated some of the reason(s) you’ve visited the USITC website in the past. Could you tell me about some of the things you come to the website to do?

3. What has your experience been like using the USITC website?

4. How would you describe the mission of USITC and the purpose for their site?

5. Do you use any other online websites or resources in conjunction with the USITC website to fulfill the job responsibilities we’ve discussed? [If so] What are those resources?

6. [If necessary] What has your experience been like using those other resources?

7. Is there any information that you’ve found on other websites that you would like to see included on the USITC site? [If so] What information would you like to see included?

Section 2: First impressions of the USITC website

Starting Page: USITC Homepage

Goal: To evaluate users’ first impressions.

Now let’s go to the USITC website. Please type in www.usitc.gov into your web browser.

1. What are the first three words that come to your mind when looking at this page?

2. What was the first thing that caught your eye?

3. What are your other first impressions?

Section 3: Feedback on the top navigation

Starting Page: USITC Homepage

Goal: To evaluate users’ expectations of the organization of information.

1. Without clicking anything, would you take a look at the top navigation menu, what would you expect to find under About USITC?

2. What would you expect to find under Popular Topics?

3. What would you expect to find under Press Room?

4. What would you expect to find under Import Injury?

5. What would you expect to find under Intellectual Property?

6. What would you expect to find under Industry/Economic Analysis?

7. What would you expect to find under Tariff Affairs?

Section 4: Free exploration of the USITC website

This section will vary in length depending on the participants’ frequency of use and variety of tasks they complete on the site.

Starting Page: USITC Homepage

Goal: To evaluate how users’ typically engage and interact with the site

I’m going to take a step back now and let you interact with the site.

1. You had discussed a little bit ago about what you do on the site. Please go ahead and do what you typically do when you come to the site as if I wasn’t here. Please remember to think aloud as you’re using the site.

2. [If necessary] You had also mentioned that you come to the site to [insert 2^nd task from Section 1]. Please go ahead and show me what you typically do using the site.

3. [If necessary] You had also mentioned that you come to the site to [insert 3^rd task from Section 1]

4. Is there anything else that you typically do on the site?

Section 5: Common Tasks

Audience: All participants will receive these tasks

Starting Page: USITC Homepage

Goal: To evaluate the overall usability, learnability, and usefulness of the USITC site by observing participants’ completing common tasks.

Let’s say you would like to learn about who the USITC is, what they do, and what their mission is.

1. Please show me where you would find information about the history of the USITC.

2. Please show me where you would find information about USITC’s mission.

3. Let’s say you want to find more information about Judge Sandra Lord. Please show me where you would find information about her.

4. Let’s say you want to find more information about Comissioner Scott Kieff. Please show me where you would find information about him.

5. Let’s say you wanted to find information about USITC’s strategic goals. Please show me where you would find that information.

6. Let’s say you have a few unanswered questions after using the site. Please show me what you would do if you wanted to send a question to USITC.

Section 6: Audience-Specific Tasks

Starting Page: USITC Homepage

Goal: To evaluate the overall usability, learnability, and usefulness of the USITC website by observing participants complete audience specific tasks.

6.1–6.3: Importers/Exporters/Manufacturers/Distributors/Customs Brokers

Scenario: [If participant is not from this audience segment] Let’s say you’re an import specialist, responsible for researching, reviewing, and assigning classifications of incoming products according to the Harmonized Tariff Schedule.

1. Let’s say you are trying to look up specific tariff information for electronic devices. Please go ahead and find tariff information for this type of product. [Moderator: stop participants once they reach the HTS as the HTS is not being evaluated.]

2. Let’s say you weren’t able to find the tariff information you were seeking. Please show me what you would do if you were seeking assistance in finding how the product is classified.

3. What would you do if you wanted to review a prior version of the HTS?

6.4–6.12: Corporations/Businesses/Small Businesses

Scenario: [If participant is not from this audience segment] Let’s say you’re a compliance administrator for a large corporation and you interact with the company’s overseas suppliers to ensure that all import procedures are followed.

4. Let’s say you are trying to look up specific tariff information for a silicon metal. Please go ahead and find tariff information for this type of metal. [Moderator: stop participants once they reach the HTS as the HTS is not being evaluated.]

5. Let’s say you weren’t able to find the tariff information you were seeking. Please show me what you would do if you had an unanswered question about how the product is classified.

4. What would you do if you wanted to review a prior version of the HTS?

4. Now let’s say you heard about a recent investigation about a manufacturer that is pricing their imports below fair market value . The investigation is relevant for your business. Please find information about the investigation into Heavy Forged Hand Tools from China.

4. Please find a federal register notice related to an ongoing investigation on this topic.

4. What would you do if you had a question about how antidumping and countervailing duty orders are imposed?

4. Let’s say you’ve heard about a patent infringement that is relevant for your company and you want to find out how these type of investigations are typically conducted. Please find information that explains the process.

4. Let’s say you want to review a complaint filed with USITC about semiconductor devices and semiconductor device packages. Please show me how you would find information about this complaint.

4. Let’s say you wanted to find a federal register notice about an investigation into the importation of certain stainless steel products. Please find information about that investigation.

Item 6.13–6.18: U.S. Government/Foreign Government

Scenario: [If participant is not from this audience segment] Let’s say you’re a congressional liason for ITC who works closely with Capitol Hill to revise legislation as well as clearance on many of the ITC commission publications.

13. Let’s say you need to get information about how patent infringement investigations are typically conducted. Please find information that explains the process.

13. Let’s say you want to review a complaint filed with USITC about semiconductor devices and semiconductor device packages. Please show me how you would find information about this complaint.

13. Let’s say you wanted to find a federal register notice about an investigation into the importation of certain stainless steel products. Please find information about that investigation.

13. Let’s say you wanted to review the 2015 Year in Trade publication from USITC. Please show me where you would find that publication.

13. As you’re reading the 2015 edition, you learn that you need to reference the 2010 Year in Trade publication. Please show me where you would find that information.

13. Let’s say you’re looking for information about trade shifts in U.S. merchandise in 2014. Please find this information.

6.19–6.23 : Law firm/Legal consultants

Scenario: [If participant is not from this audience segment.] Let’s say you’re a lawyer who specializes in fair trade compliance. You’ve recently been consulting for a large U.S. machinery manufacturer about compliance issues related to U.S. antidumping policies and countervailing duties.

13. Now let’s say you heard about a recent investigation about a manufacturer that is pricing their imports below fair market value. The investigation is relevant to your client’s case. Please find information about the investigation into Heavy Forged Hand Tools from China.

13. Please find a federal register notice related to an ongoing investigation on this topic.

13. Let’s say you are preparing for an upcoming hearing and you want to find more information the USITC hearing process. Please show me what you would do to find that information.

13. Let’s say you need to attend a hearing on September 29^th on aluminum but you’re not sure what time the hearing starts and where it’s location. Please find this information.

13. In preparation for this hearing, you want to review a news release about a study the USITC conducted on the aluminum industry and global aluminum trade. Please read the news release about this study.

6.24–6.26: Academic Researchers

Scenario: [If participant is not from this audience segment] Let’s say you’re a economics researcher at a large university working on a project about the global textile and apparel trade, particularly with Asian countries.

13. Let’s say you’re looking for details of China-U.S. trade over the past 10 years, specifically about how much product has been imported into the U.S. from China. Please find recent Commission publications related to this project.

13. Are there any Commission publications from a previous year that may be helpful?

13. Let’s say you’re now looking for information about trade shifts in U.S. merchandise in 2014. Please find this information.

6.27–6.32: Media

Scenario: [If participant is not from this audience segment.] Let’s say you’re a business journalist who writes for the Business & Trade section of a major newspaper.

13. In preparation for writing a story, you’d like to review a press release about possible modifications to the United States and Morocco’s Free Trade Agreement. Please find the press release so you can write your article.

13. You’re next writing a story on Cree Inc., an American manufacturer of

Lighting-class LED’s and its case against its suppliers for violations of trade laws pertaining to patent infringement and false advertising of light-emitting diode products. Please find information about the final verdict of this case so you can write your article.

13. Let’s say you’re writing another piece about an investigation on antidumping and countervailing duty regarding Cased Pencils from China. Please show me how you would find the necessary information to write your report.

13. Let’s say you are looking to attend a hearing related to the article you’re writing. Before you attend, you want to find more information about the USITC’s hearing process. Please show me what you would do to find that information.

13. Let’s say the hearing you plan to attend on the topic of aluminum is on September 29^th but you’re not sure what time the hearing starts and its location. Please find this information.

13. In preparation for this hearing, you want to review a news release about a study the USITC aluminum industry and global aluminum trade. Please read the news release about this study.

Section 7: Questionnaires

Goal: To receive a score on the usability and learnability of the website

System Usability Scale

Please circle the numbers that most appropriately reflect your impressions about using this website.

1. I think that I would like to use the USITC website frequently:

   Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
   1	2	3	4	5

2. I found the USITC website unnecessarily complex:

   Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
   1	2	3	4	5

3. I thought the USITC website was easy to use:

   Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
   1	2	3	4	5

4. I think that I would need the support of a technical person to be able to use the USITC website:

   Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
   1	2	3	4	5

5. I found the various functions in the USITC website were well integrated with each other:

   Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
   1	2	3	4	5

6. I thought there was too much inconsistency in the USITC website:

Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
1	2	3	4	5

7. I would imagine that most people would learn to use the USITC website very quickly:

   Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
   1	2	3	4	5

8. I found the USITC website very awkward to use:

   Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
   1	2	3	4	5

9. I felt very confident using the USITC website:

   Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
   1	2	3	4	5

10. I needed to learn a lot of things before I could get going with the USITC website:

Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
1	2	3	4	5

Net Promoter Score (NPS)

On a scale of 1 to 10, with 10 being more likely, how likely is it that you would recommend this site to a friend, colleague, or family member?

1

2

3

4

5

6

7

8

9

10

Section 8: Follow-up questions

1. What stands out to you about your experience using this site? What worked well; what did not work well? [Probe about observations made during the interview]

2. How was your experience with navigating to find the information you were looking for? How was your experience with using the links to navigate? How was your experience with using search to navigate?

3. What do you think about the way the information on the site was organized?

4. What do you think about the terms that are used on the site? [If necessary] Were there any terms that were confusing or unclear?

5. What information was most valuable to you? Why?

6. What do you think about the visual design of the site and the pages you visited?

7. What information on the site did you find to be most valuable? Did you learn any new information on the website that you were unaware of before, or that was a surprise to you?

8. Do you trust the information on this site? Why?

9. Was there anything missing from the site that you expected to see, or expected to have access to, but didn’t?

10. What type of electronic device do you typically use to read this kind of content (mobile, desktop, laptop, tablet)? Where would you be while reading?

11. Where would you go or what would you do after using this site?

12. [Follow up on satisfaction questionnaire responses as necessary.]

13. What did you like most about the site?

14. What did you dislike the most about the site?

15. If you could change one thing on the site, what would it be?

Section 9: Open Card Sort

For our next activity, I’ll ask you to sort various topics. The feedback you’ll provide will help organize the information on the site. This activity should take approximately 20 minutes.

Step 1. Take a look at the list of topics below. I’d like you to sort them into groups that make sense to you. There are no right or wrong answers. Just do whatever makes sense to you.

Step 2. Drag an item into the area to create your first group.

Step 3. At any pont you can create a label for the groups you have created.

Step 4. You can continue to add items to a group by dropping them on top. Please think aloud as you’re sorting the topics into the groups of your choosing.

Table 1. Topics for card sort

Information about employees (bios/roles/photos)	Seminars	Staff publications
Laws & regulations	Working Papers	International Economic Review
USITC history	Offices of the Secretary, Procurement, Inspector General, External Relations, Human Resources, Equal Employment Opportunity	Journal of International Commerce and Economics
Commission notices about intellectual property	Mission of the USITC	Executive Briefings on Trade
Harmonized Tariff Schedule	Complaints filed with the USITC	Commission notices about antidumping and countervailing
What is it like to work for the USITC?	Antidumping and countervailing investigations	Congressional bill reports
USITC news releases	Copyright infringment investigations	Industry trade and technology review
Operating hours of the USITC	Industry and economy investigations	Safeguard publications
Petitions	FAQs about antidumping and countervailing	Sunset review database
Research tools	Outcomes of past cases	Federal register notices
Tariff information	Information about the status of an ongoing investigation	Publications about antidumping and countervailing
Role of the USITC	Location of hearings	FAQs about intellectual property
Countries that participate in the USITC	Filing documents with the USITC	Intellectual property statistics
Branch of Government that oversees the USITC	Confidential business information when filing a complaint	Industry/economic analysis publications
Investigators at the USITC	Classification of products with tariffs	General factfinding commission notices
Patents and trademarks information	Special tariff rate codes	Tariff FAQs
Penalties for violating trade laws	Intellectual property publications

USTIC.gov User Experience Interviews  3