Usability Test Protocol

ATTACHMENT D:

USABILITY TEST PROTOCOL

DRAFT E-Verify Public Website Usability Testing Moderator’s Guide

Revised April 5, 2013

1. Introduction

Thank you for taking the time to help us today. My name is ________ and I work for Westat, a research firm based in Rockville, Maryland. I am conducting the interview with you on behalf of U.S. Citizenship and Immigration Services (USCIS), to study the effectiveness of the E-Verify Program’s communication and outreach activities for [companies that are enrolled in the program/companies that are not currently enrolled in the program/workers who want to learn more about E-Verify]. In particular, we want to see how well the E-Verify website meets your needs, and how useful it is in helping you learn about the E-Verify Program. That is how you will help us today.

Today I’ll be asking you to use the E-Verify website to find different kinds of information that have been posted on the site and to share your reactions with me. Your feedback will help Citizenship and Immigration Services understand what it’s like to use the E-Verify website, so we can make it the most useful to future users of the site. I’ll be able to watch what you do on screen from my own computer and this will help me understand what it’s like for you to work with the site. This session will last approximately 75 minutes.

[Description of any observers present]

2. Consent

As a reminder, your participation is your choice, and you can skip any question or stop at any time. In all reports and presentations of this information, your statements will be grouped with those of other participants, and your name and your organization’s name will never be used.

Do you have any questions about your participation?

3. Previous E-V experience survey [for users]

1. If you needed information about immigration status and employment, would you go directly to the E-Verify web site, or would you go to a search engine like Google?

2. Generally, how often do you use the public E-Verify website? Less than once per month, 1 – 3 times per week, 4 – 6 times per week, more than 6 times per week?

3. Why do you usually access the website?

4. What were you looking for the last time you used the website?

5. Do you recall if you found what you were looking for?

6. How long did it take you to find this information? Was this a satisfactory amount of time?

7. How do you learn about changes to E-Verify? Have you ever visited the “What’s New” portion of the site? (if yes) Have you visited it more than once?”

8. Approximately, how long has your company been using E-Verify? How long have you personally been using E-Verify?

4. Scenarios/tasks

There are specific kinds of information I’m going to have you look for so we can cover different portions of the E-Verify site. Please work as you would if you were doing this on your own at work or at home, at a speed that’s comfortable for you. Remember that we’re testing the site, we’re not testing you. Our goals are to test how easy or hard it is to navigate on the website, and find the information you are looking for. As you’re looking for the information I asked you to find on the website, please say out loud what you are thinking, and tell me what you are looking at and clicking on as you do it. We call this a “think-aloud” procedure. I will be interested in hearing what you expect to see when you click on something, and what you like and don’t like about the site. Occasionally, after you've said or done something, I may ask you to tell me more, so that I can better understand how you interpret and act on the information you see.

You might have some questions as you work on a task. Please go ahead and ask them, because I’m interested in learning from you what isn’t clear or easy to understand. However, sometimes I may not be able to answer your questions, because I need to see how you use the site on your own. We’ll have some time at the end when I’ll ask for your feedback about your experiences using this site.

Do you have any questions before we get started?

User Information Search Tasks

1. Find a page that defines an E-Verify Employer Agent. Find an explanation for how an
   E-Verify Employer Agent is different from the way your company uses E-Verify.

2. Find a list of Webinars that are designed for employers. Which one, if any, would you be most interested in attending? What is most appealing about it?

3. Find a page that defines a tentative nonconfirmation (TNC) finding.

4. Find a page that specifies the two agencies from which workers can receive a TNC finding. Identify reasons listed on the website that explain why a worker might receive a TNC finding from each agency.

5. Suppose a worker at your company, whose native language was not English, received a TNC finding. Find where you can download TNC notices and referral letters in languages other than English.

6. Suppose your company moved to a new location. Find instructions on how to change your company’s address in your E-Verify account.

7. Suppose a new staff member is assigned the responsibility of helping to initiate E-Verify queries. Find instructions on how you can add this user to your company account.

8. Find the E-Verify newsletter.

9. Find and watch a video about creating a case in E-Verify. Does this video match your experience? In what ways?

10. The Employer Search Tool provides information about employers that have enrolled in E-Verify. Use the Employer Search Tool to find employers in your zip code. Is the information for your company correct?

11. Find a page with information about the Photo Matching tool. What is it? When is it used? What is compared during the Photo Matching process?

12. Find a page explaining the similarities and differences between E-Verify and the Form
    I-9. Identify one of several additional rules that apply to the Form I-9 if an employer is enrolled in E-Verify.

13. Find the percent of E-Verify cases that are immediately confirmed as work-authorized.

14. Find a page that explains the three-day rule. Identify the two reasons that are considered acceptable for employers failing to meet the three-day rule.

15. Suppose your company rehires a former worker. Find a page explaining whether that worker should be treated differently than first-time workers at your company, with regards to E-Verify. If applicable, find an explanation of how their treatment should differ.

16. Find a list of the types of support from USCIS that are available to employers with questions about using E-Verify. Identify what types of issues are addressed by each type of support.

17. Find a page explaining the E-Verify RIDE program. Determine in which states RIDE is used.

Nonuser Information Search Tasks

1. Suppose your company was considering enrolling in E-Verify. Find a page that specifies what decisions your company must make before enrolling in E-Verify.

2. Suppose your company decided to enroll in E-Verify. Find a page explaining what equipment your company would need.

3. Employers who enroll in E-Verify must sign a Memorandum of Understanding. Find the MOU on the E-Verify website.

4. Find a page explaining what employers must do, in addition to signing the MOU, before they are able to use E-Verify.

5. Find a list of Webinars that are designed for employers. Which one, if any, would you be most interested in attending? What is most appealing about it?

6. Find the E-Verify newsletter.

7. Find a page explaining the similarities and differences between E-Verify and the Form
   I-9. Identify at least one additional rule that applies to the Form I-9 if an employer is enrolled in E-Verify.

8. Find a page explaining at what point in the hiring process a worker should be entered into the E-Verify system.

9. Find a page that explains the three-day rule. Identify the two reasons that are considered acceptable for employers failing to meet the three-day rule.

10. Find and watch a video about enrolling in E-Verify. Does this video clearly explain the process? If not, what questions do you have?

11. Staff who use E-Verify can have different levels of access. Find a page that explains the different user roles that can be registered with E-Verify.

12. Find a page that describes a company’s legal responsibilities after signing up for
    E-Verify.

13. Find and watch a video about creating a case in E-Verify. Does this video clearly explain the process? If not, what questions do you have?

14. Find a page that defines a tentative nonconfirmation (TNC) finding.

15. Find a page that explains what a company must do if a worker receives a TNC finding.

16. Find a page that explains what a company must do if a worker decides not to contest a TNC finding.

17. Find the posters that companies using E-Verify are required to have displayed. Identify the languages in which these posters are available.

18. Find a page that defines an E-Verify Employer Agent.

19. The Employer Search Tool provides information about employers that have enrolled in E-Verify. Use the Employer Search Tool to find employers in your zip code.

20. Find the percent of E-Verify cases that are immediately confirmed as work-authorized.

Immigrant Rights Representatives Information Search Tasks

1. Find a page that defines a tentative nonconfirmation (TNC) finding.

2. Find a page that specifies the two agencies from which workers can receive a TNC finding. Identify reasons listed on the website that explain why a worker might receive a TNC finding from each agency.

3. Find a description of E-Verify Self Check. Identify who the program is intended for. Would you be inclined to use it? Why or why not?

4. Find a list of Webinars that are designed for workers. Which one, if any, would you be most interested in attending? What is most appealing about it?

5. Find a page explaining the similarities and differences between E-Verify and the Form
   I-9. Identify what additional rules apply to the Form I-9 if an employer is enrolled in E-Verify.

6. Find the E-Verify newsletter.

7. Find three E-Verify resources that are designed for workers. How do you think a worker would use them? What languages are they in? Are they easy to understand?

8. Find and watch a video that shows a worker’s options after receiving a TNC. Does this video clearly explain the process? If not, what questions do you have?

9. Find a video about worker rights and responsibilities and watch the first five minutes only. Does this video clearly explain the process? If not, what questions do you have?

10. Find a page that describes the rights and responsibilities workers have under E-Verify. Identify the languages in which this information is available.

11. The Employer Search Tool provides information about employers that have enrolled in E-Verify. Use the Employer Search Tool to find employers in your organization’s zip code. Identify some the reasons that a company enrolled in E-Verify may not be in your search results.

12. Suppose you had questions about employment-related discrimination. Find information for who you should contact with your questions.

13. Suppose a worker believed an employer was misusing E-Verify. Find information for who they should contact with their concerns.

14. Identify what immigration-related websites are linked to through the E-Verify website. Are there other sites that you think should be included?

15. Suppose a worker receives a TNC through the Social Security Administration (SSA). Find a page explaining how many days they have to contact the agency and what documents they should take with them when contesting.

16. Suppose a worker receives a TNC through Department of Homeland Security (DHS). Find a page explaining how many days they have to contact the agency and what documents they should take with them when contesting.

17. Suppose a worker receives a TNC but decides not to contest the finding. Find a description of what the worker can expect to happen next.

18. Find a page that identifies ways in which workers can reduce their chances of receiving a TNC notice based on their completion of the Form I-9.

19. Find the percentage of E-Verify cases that are immediately confirmed as work-authorized. Find the percentage of cases that are found to be not authorized to work.

5. Debriefing

User

1. What do you like most about the site and why?

2. Who do you think is the intended audience(s) for the website? Do you know where to look for information that is targeted to you, a company that uses E-Verify?

3. Is there anything you find particularly frustrating about using the site? What makes it frustrating? What do you think could improve your experience in searching for information on the website?

4. Would you return to the website to search for information? Why/why not?

Nonuser

1. What do you like most about the site?

2. What do you see as the purpose of the website? Is the purpose clear after a few seconds without much reading? How well do you think the web site achieves this purpose?

3. What, if anything, on the website encouraged you to want to look into using E-Verify?

   1. If nothing, what type of information, if not already on the website, would motivate you to use the Program? What action would you take, if any, as a result of reading this information?

   2. Is there anything on the website that would discourage you from further exploring the E-Verify website or from using E-Verify?

4. Now that you’ve had a chance to answer some questions about the site, how might you use the site in the future? (e.g., to look up general information about the program, to learn which companies use it, to find out about how to sign up?)

5. What, if any, changes would you most like to make to the site? Why?

Immigrant rights representatives

1. What do you like most about the site?

2. What do you see as the purpose of the website? Is the purpose clear after a few seconds without much reading? How well do you think the web site achieves this purpose?

3. Do you think workers will visit the website? Why or why not? If yes- how well does the website address those topics? Are there any important topics for workers that are missing from the site?

4. What, if any, changes would you most like to make to the site? Why?

5. [If used site prior to testing]: Has your impression of the site changed based on what you found? If yes, in what way(s)?

   1. What, if any, changes would you most like to make to the site? Why?

   2. Would you return to the website? Why/why not?

All respondents

	Strongly disagree	Somewhat disagree	Neither agree nor disagree	Somewhat agree	Strongly agree
The website is visually appealing.
It was easy to move from one page to another.
The overall organization of the site is easy to understand.
Individual pages are well designed.
Terminology used in this website is clear.
The content of the website met my expectations.
I would be likely to use this website in the future.
I was able to complete my tasks in a reasonable amount of time.
Overall, the website is easy to use.