Software Review Protocal

	Software Review Protocol
	March 27, 2014 (resubmitted March 31, 2014)
	Prepared for: U.S. Department of Homeland Security Washington, DC		Prepared by: Westat An Employee-Owned Research Corporation® 1600 Research Boulevard Rockville, Maryland 20850-3129 (301) 251-1500

Westat Draft Web Services Software Review Protocol

Revised March 27, 2014

Section A. E-Verify Browser Home Page

NOTE: The home page includes many navigation features and information for users to manage their various E-Verify activities. This also serves as the gateway to E-Verify; those who use the browser always start interfacing with E-Verify from here. This may be the area in which the Web Services software differs from the browser the most.

Overall Focus: Is Web Services software home page/management system as comprehensive and user-friendly as the E-Verify home page below? What’s different?

A1. Using the E-Verify browser home page above as the basis of comparison, please respond to the following questions for the Web Services (WS) software home page.

1a. (Navigation features)

The Web Services (WS) software home page includes:

( 1 ) Exactly the same navigation features

( 2 ) Additional navigation features (Specify):

( 3 ) Fewer navigation features (Specify features NOT included in WS home page):

1b. (Navigation features)

Does the WS software provide any of these navigation features outside of the home page?

( 1 ) Yes (Describe features and how they are accessed):

( 2 ) No

1c. (User-friendliness)

The WS home page appears to be:

( 1 ) Just as user-friendly

( 2 ) More user-friendly (Explain):

( 3 ) Less user-friendly (Explain):

1d. (E-Verify news)

The WS home page provides:

( 1 ) Major E-Verify news/updates on the same screen

( 2 ) A link to E-Verify news (Explain):

( 3 ) No link to E-Verify news

1e. (Types of case alerts)

The WS home page provides:

( 1 ) The same types of case alerts (SKIP TO 1g)

( 2 ) Different types (Explain): (SKIP TO 1g)

( 3 ) No case alerts (GO TO 1f)

[ANSWER 1f ONLY IF RESPONSE TO 1e IS “NO CASE ALERTS”]

1f. (Types of case alerts)

If there are no case alerts on the WS home page, the WS software:

( 1 ) Provides the same case alerts elsewhere (Specify how each screen is accessed):

( 2 ) Provides different case alerts elsewhere (Specify how they are different and how each screen is accessed):

( 3 ) Does not provide any case alerts

1g. (Types of resources)

The WS home page provides:

( 1 ) The same types of resources (SKIP TO A2)

( 2 ) Different types (Explain): (SKIP TO A2)

( 3 ) No resources (GO TO 1h)

[ANSWER 1h ONLY IF RESPONSE TO 1g IS “NO RESOURCES”]

1h. (Types of resources)

If there are no resources on the WS home page, the WS software:

( 1 ) Provides the same resources elsewhere (Specify how each screen is accessed):

( 2 ) Provides different resources elsewhere (Specify how they are different and how each screen is accessed):

( 3 ) Does not provide any resources

A2. Using the E-Verify browser’s View Cases and Search Cases screenshots below as the basis of comparison, please respond to the following questions about viewing and searching cases in the WS software.

View Cases
Search Cases

2a. (Case management- view cases)

The WS features for viewing cases are:

( 1 ) Located on the WS home page

( 2 ) Located elsewhere (Specify location and how it is accessed):

2b. Case management- view cases)

The WS features for viewing cases are:

( 1 ) The same as the E-Verify browser

( 2 ) Not the same (Explain what is different):

2c. (Functionality of view cases)

Viewing existing cases in WS software appears to:

( 1 ) Function in the same manner as the E-Verify browser

( 2 ) Function differently (Explain):

2d. (Case management- search cases)

The WS features for searching cases are:

( 1 ) Located on the WS home page

( 2 ) Located elsewhere (Specify location and how it is accessed):

2e. (Case management- search cases)

The WS features for searching cases are:

( 1 ) The same as the E-Verify browser

( 2 ) Not the same (Explain what is different):

2f. (Functionality of search cases)

Searching cases in WS software appears to:

( 1 ) Function in the same manner as the E-Verify browser

( 2 ) Function differently (Explain):

Section B. Data Input Section

NOTE: E-Verify browser requires users to enter Form I-9 information into the system.

Overall Focus: Is the data input section of the Web Services (WS) software as comprehensive as the E-Verify data input section below? What’s different?

Citizenship Status	Form I-9
Type of Document

B1. Using the E-Verify browser’s Data Input screenshots above as the basis of comparison, please respond to the following questions in the WS software.

1a. (Data input approach)

For data input, the Web Services (WS) software uses:

( 1 ) Exactly the same approach (SKIP TO 1c)

( 2 ) Electronic Form I-9

( 3 ) Other approach (Specify):

1b. (Data input approach)

For data input, the WS software seems to:

( 1 ) Link to other database outside of the software to extract/obtain the Form I-9 information

( 2 ) Cannot determine

1c. (Help/instruction for entering the information)

The WS software includes help/instruction on the following:

(Detailed comparison of each help/instruction screen is included in section E.)

( 1 ) Exactly the same fields

( 2 ) Additional fields (Specify fields NOT included in the browser):

( 3 ) Fewer fields (Specify fields NOT included in WS):

( 4 ) No help/instruction fields are provided

Section C. Initial Verification Process

NOTE: The E-Verify browser requires users to submit one case at a time.

Overall Focus: Is the initial verification process of the Web Services (WS) software as comprehensive as the E-Verify browser sections below? In E-Verify, the initial verification process refers to these three procedures: Pre-TNC check, Expired documentation, Photo matching process. What’s different?

C1. Using the E-Verify browser’s Initial Verification Process screenshots below as the basis of comparison, please respond to the following questions in the WS software.

1a. (Case submission)

For submitting cases, the Web Services (WS) software:

( 1 ) Has to submit one case at a time (CONTINUE TO 1b)

( 2 ) Has the capability to submit multiple cases at a time (SKIP TO 1c)

( 3 ) Cannot be determined (CONTINUE TO 1b)

Pre-TNC screen	Expired document screen	Photo matching screen

1b. (Prompts)

The screenshots above are from the E-Verify browser for Pre-TNC, Expired Document, and Photo matching. How does the software generate the above prompts:

( 1 ) Generates screens that contain exactly the same info

( 2 ) Generates screens that contain different info.

(Specify :)

( 3 ) Generates a different screen

(Specify :)

1c. (Prompts)

The screenshots above are from the E-Verify browser for Pre-TNC, Expired Document, and Photo matching. For the Web Services (WS) software that has the capability to submit multiple cases at a time, how does the software generate the above prompts:

( 1 ) Generates screens that contain exactly the same info for one case at a time

( 2 ) Generates screens that contain exactly the same info for multiple cases at a time

(Specify :)

( 3 ) Generates different screens

(Specify :)

Section D. Case Results

NOTE: The E-Verify browser provides the case results in three stages. Depending on the case, the initial finding is also the final (e.g., immediately work authorized cases), while others would have results in all three stages (e.g., contested TNC resulted in authorized or FNC). Regardless, they require the users/workers to take action.

Overall Focus: Is the case results portion of Web Services (WS) software as comprehensive as in the
E-Verify browser shown below? In E-Verify, the browser provides the case results in three stages – Initial, Interim, and Final. What’s different? It is interesting to see how these results are generated by the software that has capability to submit multiple cases.

D1. Using the E-Verify browser’s Case Results screenshots above as the basis of comparison, please respond to the following questions in the WS software.

Single cases only:

1a. (Case results)

The screen generated by the Web Services (WS) software to request a name review contains:

( 1 ) Exactly the same information

( 2 ) Additional information (Specify):

( 3 ) Less information (Specify information NOT included in WS):

Multiple cases only:

1b. (Case results)

The screen(s) generated by the Web Services (WS) software to request a name review contain(s):

( 1 ) Exactly the same information on one screen

( 2 ) Exactly the same information on multiple screens

( 3 ) Additional information (Specify):

( 4 ) Less information (Specify information NOT included in WS):

Single cases only:

1c. (Case results)

The screen generated by the Web Services (WS) software for a DHS verification in process contains:

( 1 ) Exactly the same information

( 2 ) Additional information (Specify):

( 3 ) Less information (Specify information NOT included in WS):

Multiple cases only:

1d. (Case results)

The screen(s) generated by the Web Services (WS) software for a DHS verification in process contain(s):

( 1 ) Exactly the same information on one screen

( 2 ) Exactly the same information on multiple screens

( 3 ) Additional information (Specify):

( 4 ) Less information (Specify information NOT included in WS):

Single cases only:

1e. (Case results)

The screen generated by the Web Services (WS) software for an SSA case in continuance contains:

( 1 ) Exactly the same information

( 2 ) Additional information (Specify):

( 3 ) Less information (Specify information NOT included in WS):

Multiple cases only:

1f. (Case results)

The screen(s) generated by the Web Services (WS) software for an SSA case in continuance contain(s):

( 1 ) Exactly the same information on one screen

( 2 ) Exactly the same information on multiple screens

( 3 ) Additional information (Specify):

( 4 ) Less information (Specify information NOT included in WS):

All Cases:

1g. (Case results)

The TNC process through the Web Services (WS) software has the user:

( 1 ) Transferred over to the E-Verify browser (SKIP TO SECTION E)

( 2 ) Continue through the WS software (CONTINUE TO 1h)

( 3 ) Other (Specify): (SKIP TO SECTION E)

Single cases only:

1h. (Case results)

The screens generated by the Web Services (WS) software for a DHS TNC contain:

( 1 ) Exactly the same information

( 2 ) Additional information (Specify):

( 3 ) Less information (Specify information NOT included in WS):

Multiple cases only:

1i. (Case results)

The screens generated by the Web Services (WS) software for a DHS TNC contain:

( 1 ) Exactly the same information for all cases on each screen

( 2 ) Exactly the same information for all cases on multiple screens

( 3 ) Additional information (Specify):

( 4 ) Less information (Specify information NOT included in WS):

Single cases only:

1j. (Case results)

The screens generated by the Web Services (WS) software for an SSA TNC contain:

( 1 ) Exactly the same information

( 2 ) Additional information (Specify):

( 3 ) Less information (Specify information NOT included in WS):

Multiple cases only:

1k. (Case results)

The screens generated by the Web Services (WS) software for an SSA TNC contain:

( 1 ) Exactly the same information for all cases on each screen

( 2 ) Exactly the same information for all cases on multiple screens

( 3 ) Additional information (Specify):

( 4 ) Less information (Specify information NOT included in WS):

All Cases:

1l. (Case results)

The TNC letters/notices through the Web Services (WS) software are:

( 1 ) The same as in the E-Verify browser

( 2 ) Different from those in the E-Verify browser (Explain):

Single cases only:

1m. (Case results)

The screen generated by the Web Services (WS) software for a DHS FNC contains:

( 1 ) Exactly the same information

( 2 ) Additional information (Specify):

( 3 ) Less information (Specify information NOT included in WS):

Multiple cases only:

1n. (Case results)

The screen(s) generated by the Web Services (WS) software for a DHS FNC contain(s):

( 1 ) Exactly the same information for all cases on one screen

( 2 ) Exactly the same information for all cases on multiple screens

( 3 ) Additional information (Specify):

( 4 ) Less information (Specify information NOT included in WS):

Single cases only:

1o. (Case results)

The screen generated by the Web Services (WS) software for an SSA FNC contains:

( 1 ) Exactly the same information

( 2 ) Additional information (Specify):

( 3 ) Less information (Specify information NOT included in WS):

Multiple cases only:

1p. (Case results)

The screen(s) generated by the Web Services (WS) software for an SSA FNC contain(s):

( 1 ) Exactly the same information for all cases on one screen

( 2 ) Exactly the same information for all cases on multiple screens

( 3 ) Additional information (Specify):

( 4 ) Less information (Specify information NOT included in WS):

Single cases only:

1q. (Case results)

The screen generated by the Web Services (WS) software for a DHS No Show contains:

( 1 ) Exactly the same information

( 2 ) Additional information (Specify):

( 3 ) Less information (Specify information NOT included in WS):

Multiple cases only:

1r. (Case results)

The screen(s) generated by the Web Services (WS) software for a DHS No Show contain(s):

( 1 ) Exactly the same information for all cases on one screen

( 2 ) Exactly the same information for all cases on multiple screens

( 3 ) Additional information (Specify):

( 4 ) Less information (Specify information NOT included in WS):

Single cases only:

1s. (Case results)

The screen generated by the Web Services (WS) software for closing a case contain:

( 1 ) Exactly the same information

( 2 ) Additional information (Specify):

( 3 ) Less information (Specify information NOT included in WS):

Multiple cases only:

1t. (Case results)

The screen(s) generated by the Web Services (WS) software for closing a case contain(s):

( 1 ) Exactly the same information for all cases on one screen

( 2 ) Exactly the same information for all cases on multiple screens

( 3 ) Additional information (Specify):

( 4 ) Less information (Specify information NOT included in WS):

Section E. Help Screen

Overall Focus: Are the help screens of the Web Services (WS) software as comprehensive as in the E-Verify browser shown below?

E1. Citizenship Status

1a. (Citizenship status)

Does the WS software have instruction/help for citizenship status?

( 1 ) Yes (GO TO 1b)

( 2 ) No (SKIP TO E2)

1b. (Citizenship status)

Does the help/instruction appear:

( 1 ) Through the current software

( 2 ) Through a link to a different site/location (Explain):

1c. (Citizenship status)

Does the WS software provide the same instruction/help as the E-Verify browser?

( 1 ) Yes

( 2 ) No (Specify):

E2. Various documents used for verification

2a. (Various documents used for verification)

Does the WS software have instruction/help for these documents?

( 1 ) Yes, for all of them (CONTINUE TO 2b)

( 2 ) Yes, for some of them (Specify): (CONTINUE TO 2b)

( 3 ) No (SKIP TO E3)

2b. (Various documents used for verification)

Does the help/instruction appear:

( 1 ) Through the current software

( 2 ) Through a link to a different site/location (Explain):

2c. (Various documents used for verification)

Does the WS software provide the same instruction/help as the E-Verify browser?

( 1 ) Yes

( 2 ) No (Specify):

Help for List B and List C Documents

Help for Driver’s license or ID card issued by a U.S. state or outlying possession

Help for ID card issued by a U.S. federal, state or local government agency

Help for Native American tribal document

Help for Social Security Card

Help for Employment authorization document issues by the U.S. Department of Homeland Security

Help for Arrival/Departure Record (Form I-94) with temporary I-551 Stamps or Refugee Admission Stamp (reciept)

E3. Data entry fields

3a. (Data entry fields)

Does the WS software have instruction/help for the data entry fields?

( 1 ) Yes, for all of them (CONTINUE TO 3b)

( 2 ) Yes, for some of them (Specify): (CONTINUE TO 3b)

( 3 ) No (SKIP TO E4)

3b. (Data entry fields)

Does the help/instruction appear:

( 1 ) Through the current software

( 2 ) Through a link to a different site/location (Explain):

3c. (Data entry fields)

Does the WS software provide the same instruction/help as the E-Verify browser?

( 1 ) Yes

( 2 ) No (Specify):

Help for entering Last Name (Complex Surnames)

Help for entering Driver’s License or ID Card Expiration Date

Help for entering Hire Date

Help for entering Employer Case ID

Help for U.S. Passport and Passport Card Number

Help for U.S. Passport and Passport Card Expiration Date

Help for Permanent Resident Card (Form I-551)

Help for Alien and I-94 Numbers

Help for Foreign Passport Number

Help for Visa Number

Help for Machine-Readable Immigrant Visa (MRIV)

Help for Employment Authorization Document (Form I-766)

E4. Three-day rule

4a. (Three-day rule)

Does the WS software have instruction/help for the three-day rule?

( 1 ) Yes (CONTINUE TO 4b)

( 2 ) No (SKIP TO E 5)

4b. (Three-day rule)

Does the help/instruction appear:

( 1 ) Through the current software

( 2 ) Through a link to a different site/location (Explain):

4c. (Three-day rule)

Does the WS software provide the same instruction/help as the E-Verify browser?

( 1 ) Yes

( 2 ) No (Specify):

E5. Closing a case

5a. (Closing a case)

Does the WS software have instruction/help for closing a case?

( 1 ) Yes (CONTINUE to 5b)

( 2 ) No (END)

5b. (Closing a case)

Does the help/instruction appear:

( 1 ) Through the current software

( 2 ) Through a link to a different site/location (Explain):

5c. (Closing a case)

Does the WS software provide the same instruction/help as the E-Verify browser?

( 1 ) Yes

( 2 ) No (Specify):