Usability testing

Survey Planning and Design Document

October, 2016

Recruitment Survey and Focus Group Participation for the WWW.USITC.GOV Usability Study

The U.S. International Trade Commission (USITC) has procured vendor services to conduct a usability study of the USITC’s public website (www.usitc.gov). The goal of this usability study is to obtain deliverable actionable user insights and design recommendations pertaining to the (a) information architecture, (b) layout, (c) navigation, (d) nomenclature, and (e) search functionality of the website. The usability study being conducted incorporates obtaining a usability focus group and interviewing actual users through face-to-face interaction and observations.

The USITC is seeking to collect information from the public in two forms to establish the focus group and to support this usability study: 1) a voluntary, web-based survey seeking information about potential recruits to serve as participants in the usability focus group, and 2) observation and interaction with those recruits who participate in the moderated study, either in-person or remotely, seeking analyzable information on how they naturally use and experience the website.

The recruitment survey will be issued from October XX, 2016 to October XX, 2016 to find 24 participants who fit into six defined audience segments of people who regularly visit the USITC’s website. The usability study focus group will take place between October XX, 2016 and November XX, 2016 either at the vendor’s laboratory in Arlington, VA or remotely via web-based access.

1. The Group/Study Population

The first population involved will be all members of the general public who visit the USITC’s public website. This population will be reached via a voluntary, web-based survey that is presented to a segment of users who visit the USITC’s website. The survey is presented to visitors through the incorporation of a commercial tool, Ethnio, into the USITC’s website which presents a screener (i.e., pop-up) asking users about their willingness to voluntarily participate in the usability focus group. Should the user indicate their willingness, they are asked to complete a survey whose questions are designed to determine if the respondent falls into one of the following six audience segments (i.e., categories of common USITC website users) being recruited for participation in the interview portion of the study:

1. Importers/exporters/manufacturers/distributors/customs brokers

2. Corporations/large and small business owners or representatives

3. Government

4. Law firms/consultants

5. Academic researchers/students, and

6. Media.

The survey responses will be categorized into those that fall into one of the six audience segments defined above and those that do not.

Ethnio has the capability to limit the number of times a user is presented with the screener to minimize the burden on website users. A cookie will be placed in a visitor’s browser configuration when they are asked to participate so most will only see the Ethnio screener once. We will also be able to adjust the display interval (percentage of page views to present the screener) as well as the timing interval (the delay from when the page loads to when the screener appears).

The second population will be the usability study and focus group participants who are selected from the respondents to the survey above. Respondents will be contacted to verify the information they provided in the survey is accurate and to determine if they are willing to participate in the study. Four adults (over the age of 18) will be selected in each of the six audience segments for a total of 20. The vendor will then schedule and conduct moderated usability interviews with the focus group participants either on-site at the vendor’s lab or through remote means. The interview session will last approximately 90 minutes each.

B. Survey Review & Field Testing

In September 2016, members of the USITC Web Team reviewed the vendor’s proposed recruitment survey and the usability study interview script with regard to scope and clarity of questions. Feedback was provided to the vendor and appropriate updates were made. The table below identifies individuals from OCIO who were participants for the review of the survey and script.

Name	Organization	Email Address
Allison Hazen	OCIO	[email protected]
Kathleen Rumsey	OCIO	[email protected]
Joel Moeller	OCIO	[email protected]

The same USITC staff also visited the vendor’s on-site laboratory to inspect and receive a demonstration of how the one-on-one interviews are conducted.

Field testing of external user was not conducted for either the recruitment survey or the interview script.

C. Reporting Burden and Projected Cost

The focus group interview session will take a minimum of 90 minutes, longer than a typical focus group session.  Additionally, focus group participants must be recruited quickly from several highly-specialized, hard-to-recruit participant categories (customs brokers, members of the media, business owners, lawyers, academic professors, and government officials).  For this reason, participants in the usability study focus group – whether attending in person or remotely -- will be offered a $75 gift card.

A lower amount would jeopardize being able to find willing participants among the specialized focus group participant categories.  It would also risk the agency incurring additional cost if the usability focus group study period (and resulting contract) needed to be extended to find enough recruits.

The reporting burden is estimated to be:

Survey & Recruitment

Total number of survey respondents: (No.) Not to exceed 51

Frequency of response: (No.) 1

Average completion time per survey: (hours) 0.05

Total burden: (hours) 2.5 hours

Total cost: (dollars) $170.85 (2.5 hours X $67/per hour)

Usability Study Group

Total number of survey respondents: (No.) Not to exceed 24

Frequency of response: (No.) 1

Average completion time per survey: (hours) 1.5

Total burden: (hours) 36 hours

Total cost: (dollars) $2,412 (30 hours X $67/per hour)

Note: The hourly cost estimate reflects the average USITC employee hourly cost.

3