Pretest of Questionnaire

NATIONAL SURVEY OF E-VERIFY EMPLOYERS: 2015

Findings from the Pretest of Questionnaire

May 7, 2015 Updated 5-19-15

1. Background

During April 2015, Westat conducted cognitive focus groups of E-Verify employers to pretest the draft 2015 questionnaire for the Survey of E-Verify Employers. The focus groups each targeted one of the following five types of companies that use E-Verify: small companies, medium-sized companies, large companies, very large companies, and employment agencies.¹ Table 1 shows the distribution of focus group participants by key characteristics that were used for sampling. These characteristics include type of company, whether the company had a Tentative Nonconfirmation since September 9, 2013, whether it was mandated to use E-Verify, and whether it was in a high-risk industry (i.e., industries with historically high proportions of undocumented workers).

Table 1. Distribution of focus group participants by type of focus group

Characteristic	Number agreed to participate	Number participated in focus group
All companies	47	32
Type of company
Small companies	11	8
Medium-sized companies	11	8
Large companies	9	6
Very large companies	8	6
Employment agencies	8	4
Had TNC since 9/9/13
Yes	35	24
No	12	8
Mandated to use E-Verify
Yes	25	18
No	22	14
In high-risk industry
Yes	16	11
No	31	21

NOTE: Small employers are defined as having 0 to 99 employees, medium-sized companies have 100 to 999 employees, large companies have 1,000 to 4,999 employees, and very large companies have 5,000 or more employees.

The primary purpose of the pretest was to identify and correct any potential issues with new and modified survey items before full-scale implementation of the 2015 survey. Prior to the pretest, Westat established a working group of survey methodologists and USCIS content experts to identify new topics and assess the relevance, usefulness, and burden of each 2013 survey item in relation to the core goals of E-Verify and changes to the program. Westat incorporated feedback from two working group meetings into a pretest version of the survey. Cognitive focus groups were conducted with five broad types of employers to assess whether:

• New and substantially modified survey questions are clear;

• Questions convey the intended meaning;

• The information requested is available; and

• Questions require an excessive amount of time to complete.

Two versions of survey were tested to explore alternative versions for selected questions, including the placement of an open-ended question asking for suggestions for improving
E-Verify and the need for a “Not applicable” response option in three grid questions. Half of the focus group participants received the original version of the survey while the other half received a survey with modified versions of these questions.

The research team assigned to the study conducted the focus groups using WebEx, a Web hosting service for integrated teleconferencing.

2. Pretest Findings

This section presents findings from the focus group discussions by survey question. Survey questions in yellow highlights indicate that the findings have implications for changes to that question.

Question A6

(ALL COMPANIES)

A6. Which description below best fits your company?

Reason for testing question: The first response option to this question was changed from “A single location company” to “Companies with a single location,” and the second response option was changed from “Multiple location company” to “Companies with offices at multiple locations.” The primary focus of the pretest was interpretation of the second response option.

Issue(s)/Findings: Participants generally interpreted “Companies with offices at multiple locations” to mean having various branches of the company at different locations, having companies throughout the country, or having companies at different addresses. However, one participant said this meant the number of locations using E-Verify while another said it meant having more than one physical location. All agreed that if a company has all its buildings on one campus, it should be considered a single location.

Recommendation: Keep the current response options but replace “offices” with “offices or branches” in order to make the response options more comprehensive.

Question A14 in version 1 survey (same as question D21 in version 2 survey)

(ALL COMPANIES)

A14. Before we continue with the survey, we would like your comments or suggestions for improving E-Verify.

Reason for testing question: This item is the same as in the 2013 survey. The primary purpose of the pretest was to probe for an appropriate location in the 2015 survey. During the debrief of the 2013 survey report, it was suggested that placing this open-ended question at the end of the survey could have biased the comments that survey respondents wrote in the open-ended question. To explore this issue, we tested the placement of this item as question A14 in version 1 survey and as question D21 in version 2 survey. We also probed for how participants felt about having this item before the last question in version 2 survey.

Issue(s)/Findings: Participants generally expressed a preference for placing this open-ended item at the end of the survey.

• Almost all participants completing version 1 survey felt that it was better to place question A14 at the end of the survey. One participant said it was strange to see this question being asked so early because it would typically be placed at the end of the survey. Another said she answered all other questions in the survey and then returned to answer question A14. Others thought that placing the question at the end of the survey would give them a chance to reflect on their experiences as they respond to the other questions about using E-Verify, and this would help them answer the open-ended question about suggestions for improving the Program.

• Participants completing version 2 survey answered the open-ended question as D21, which came before the last question on the survey (i.e., D22 which asks respondents whether they support or oppose a list of changes that could be made to E-Verify). Almost all participants felt that it was appropriate to place the open-ended question towards the end of the survey. A few participants expressed a preference for the open-ended question to be placed at the very end of the survey (i.e., as the last question).

• Participants completing both versions of the survey were asked whether the placement of the question early or late in the survey would have affected the way they answered the question. Most participants said a different placement would not have affected their comments, and of these, a few said they have used E-Verify for a long time and know exactly what changes they would like to see.

Recommendation: Place this open-ended as D21 towards the end of the survey; that is, before the last question to address the concerns raised by participants in the USCIS briefing of the 2013 survey findings.

Question B4

(ALL CURRENT USERS)

B4. For each of the statements below, select the answer that best represents your company’s experience with E-Verify enrollment and start-up process.

1. The online enrollment process was easy to complete

2. The online enrollment process was too time-consuming

Reason for Testing Question: These items were revised from the 2013 survey. The previous items referred to the “registration process” instead of the “enrollment process.” The primary focus of the pretest was on participants’ interpretation of “enrollment” versus “registration.” A secondary focus was to explore the ability to answer the questions.

Issue(s)/Findings: Participants’ interpretations of the “enrollment process” were varied. Some participants thought it was the sign-up process, others thought it was the same as registration, and almost everyone thought it included the tutorial. One participant described the online enrollment process as a two-part process—enrollment of the company and then enrollment of the company’s E-Verify administrator.

A few respondents said it was difficult to answer these questions because the company enrolled in E-Verify a long time ago, the person that enrolled the company was no longer there, or the participant “enrolled herself in E-Verify” but not the company. When asked how much time is “time-consuming” for enrolling the company in E-Verify, some respondents said more than one hour while others said more than two hours.

Recommendations:

1. Pull out the two items from the grid and ask as a separate grid question (see new question B4 in post-pretest survey).

2. Provide a definition of enrollment in italics to read:

Enrollment refers to the initial process of signing up a company for E-Verify including signing the MOU, determining which access method to use, and providing company information. Enrollment does not include taking the E-Verify tutorial or using
E-Verify.

Question B5d (B6d in revised survey)

(ALL CURRENT USERS)

B5. For your company, how helpful are each of the following resources and features that are provided as part of the E-Verify system?

(d) E-Verify Quick Reference Guide

Reason for Testing Question: This is a new question. The primary focus of the pretest was to examine participants’ familiarity with the E-Verify Quick Reference Guide.

Issue(s)/Findings: About half of the participants were familiar with the E-Verify Quick Reference Guide, while the others were either unaware of this resource or had never used it.

Recommendation:

Keep the current question and response options. Participants have the choice of responding “Not aware of item” or “Never used item.”

Question B6 (B7 in revised survey)

(ALL CURRENT USERS)

B6. Thinking about system navigation and data entry issues, how user-friendly is the
E-Verify system?

Reason for Testing Question: The response options for this question have been revised from the 2013 survey; the original response options were: “Very user-friendly, Somewhat-user friendly, Not very user-friendly, Not at all user-friendly.” The current response options are “Very user-friendly, Moderately user-friendly, Slightly user-friendly, Not at all user-friendly.” The primary purpose of pretesting this item was to obtain participants’ opinions on whether “Moderately user-friendly” is different from “Slightly user-friendly.”

Issue(s)/Findings: Most of the participants agreed that “Moderately user-friendly” is different from “Slightly user-friendly.” For example, a few participants indicated that “Moderately user-friendly” is a bit better than “Slightly user-friendly.”

Recommendation: Keep the current question and response options.

Question B7 (B8 in revised survey)

(ALL CURRENT USERS)

B7. Have you personally ever tried calling the E-Verify Technical Help Desk
(800-741-5023) or the E-Verify Customer Service number (888-464-4218)?

Reason for Testing Question: The response options for this question have been revised from the 2013 survey to include a new response option—“No, we had problems but decided not to call.” The primary purpose of pretesting this item was to explore interpretation of this new response option.

Issue(s)/Findings: No respondent chose this response option. When asked about the condition under which employers may have problems in using E-Verify but decide not to call USCIS, participants reported that individuals may be frustrated with the situation or they may resolve the problem themselves or with the help of other colleagues.

Recommendation: Keep the current question and response options.

Questions C2, C3, C5, and C6: These questions relate to the reasons companies use
E-Verify and whether they would continue using E-Verify if it were not mandated.

Question C2

(ALL COMPANIES)

C2. Which of the following was the main reason your company agreed to participate in
E-Verify?

Reason for Testing Question: The question was revised to include a “Don’t know” response option. The primary focus of the pretest was on the use of this new response option.

Issue(s)/Findings: Participants generally found it easy to report a primary reason for using
E-Verify. Two participants who answered “Don’t know” to this question said it was because
E-Verify was set up at the company before they arrived and they have no way to get that information.

Recommendation: Keep current question, including “Don’t know” response option.

Question C3

(ALL CURRENT USERS)

C3. Please answer the following questions about your company’s current use of E-Verify

Reason for Testing Question: The question was revised to make it less wordy while retaining the same meaning. The primary focus of the pretest was on clarity of the question.

Issue(s)/Findings: Participants generally found this question to be clear and easy to answer, and they said that they only had to read it once to understand what it was asking. A few participants who answered “Don’t know” to this question said it was because they would have to do some research to find out, but that it would be pretty easy to get this information after looking into it. One respondent who answered “Don’t know” to C3b (whether company does business in state or locality that requires participation in E-Verify), indicated that her company used E-Verify because of federal contracts but she didn’t care about answering C3b because it didn’t matter whether the state or locality required participation.

Recommendation: Keep current question.

Question C5c

(CURRENT USERS REQUIRED TO USE E-VERIFY LIKELY TO CONTINUE)

C5. Why would you be likely to continue using E-Verify?

(c) To remain more competitive with other companies in our industry

Reason for Testing Question: This is a new question for mandated users who responded they would be “Very likely” or “Likely” to continue using E-Verify (C4). The primary focus of the pretest was to probe for participants’ understanding of the phrase “remain competitive.”

Issue(s)/Findings: Participants generally interpreted this question as intended, and employment agencies were more likely than other companies to indicate that using E-Verify has made their companies more competitive. For example, one participant from an employment agency said using E-Verify makes them a “stronger business” and promotes a “stronger partnership with clients.” Another participant who thought that all companies were required to use E-Verify didn’t see E-Verify use as a competitive factor but felt that it could make his company more competitive since their clients require that they use E-Verify.

Recommendation: Keep current question and response options.

Question C6d:

(CURRENT USERS REQUIRED TO USE E-VERIFY UNLIKELY TO CONTINUE)

C6. Why would you be unlikely to continue using E-Verify?

(d) The financial costs of using E-Verify outweigh the benefits of using it

Reason for Testing Question: This is a new question for mandated users who responded they would be “Unlikely” or “Very unlikely” to continue using E-Verify (C4). The primary focus of the pretest is to find out participants’ interpretation of item C6d compared to that of C6b
(E-Verify is burdensome to use).

Issue(s)/Findings: Participants generally thought that items C6b and C6d addressed different concerns. For example, one participant said that “burdensome” referred to how challenging or time consuming it is to use E-Verify, while the “financial costs” refer to the burden of affording staff.

Recommendation: Keep current question and response options.

Questions C9, C10, C11: These are open-ended follow-up questions to the following statements in question C8 about the impact that using E-Verify has had on the company.

• Qualified workers were difficult to recruit because E-Verify was used (question C8c)

• Using E-Verify created a competitive advantage for this company (question C8g)

• Using E-Verify caused this company to be less competitive (question C8h)

Most participants skipped the follow-up questions C9, C10, and C11 because they either disagreed or strongly disagreed with the statements in question 8.

Question C9

ANSWER ONLY IF E-VERIFY MADE IT DIFFICULT TO RECRUIT QUALIFIED WORKERS (C8c is ‘Strongly Agree’ OR ‘Agree’).

C9. How has using E-Verify made it harder for your company to recruit qualified workers?

Reason for Testing Question: The question is new. The primary focus of the pretest was interpretation of and responses to this open-ended question.

Issue(s)/Findings: Participants generally interpreted the question as intended; i.e., whether the use of E-Verify is discouraging qualified workers from applying for jobs at the company. For example, one participant said that the question is asking whether having the E-Verify poster up is scaring people away from applying, and several other participants agreed with her.

Recommendation: Keep current question.

Question C10

ANSWER ONLY IF USING E-VERIFY CREATED A COMPETITIVE ADVANTAGE FOR THIS COMPANY (C8g is ‘Strongly Agree’ OR ‘Agree’).

C10. How has using E-Verify created a competitive advantage for your company?

Reason for Testing Question: The question is new. The primary focus of the pretest was interpretation of and responses to this open-ended question.

Issue(s)/Findings: Participants generally interpreted the question as intended; i.e., whether the use of E-Verify has made the company more competitive. For example, interpretations included whether using E-Verify makes companies more competitive to a vendor or contractor, whether using E-Verify increases a company’s hiring power and attractiveness to potential applicants, and whether having E-Verify “provides a sense of tranquility to clients, vendors, and guests regarding whether their workforce is employment authorized.”

Recommendation: Keep current question.

Question C11

ANSWER ONLY IF USING E-VERIFY MADE THIS COMPANY LESS COMPETITIVE (C8h is ‘Strongly Agree’ OR ‘Agree’).

C11. How has using E-Verify caused your company to be less competitive?

Reason for Testing Question: The question is new. The primary focus of the pretest was interpretation of and responses to this open-ended question.

Issue(s)/Findings: Participants generally interpreted the question as intended; i.e., whether the use of E-Verify has caused the company to be less competitive. For example, one participant wrote that they ended up losing many skilled workers who went to work with competitors that did not use E-Verify.

Recommendation: Keep current question.

Questions C12g, C13, C14, C15: These questions relate to difficulties the company has had submitting information by the three-day deadline.

Question C12g and C12a

(ALL CURRENT USERS)

C12. Please consider each of the following statements related to E-Verify and select the response that best represents the experiences at your company.

(g) At times it is impossible to submit the information required by the deadline.

(a) It is impossible to fulfill all the company obligations required by the E-Verify verification process.

Reason for Testing Question: This question has been revised; the previous version read: “At times, the number of employees is so great that it is impossible to submit the information by the deadline.” The primary focus of the pretest was the participants’ interpretation of the question.

Issue(s)/Findings: Participants generally interpreted the question as intended; i.e., whether it was sometimes difficult for the company to meet the 3-day deadline for submitting Form I-9 information to E-Verify. For example, one participant who disagreed with the statement in her completed survey said that the three-day deadline was an adequate length of time. Another participant who agreed with the statement said that the three-day deadline is too strict.

Although the moderator questions focused on C12g above, one of the participants indicated she used the “Not applicable” response option for C12a: “It is impossible to fulfill all the company obligations required by the E-Verify verification process.” She responded “Not applicable” and said “Our company doesn’t really have obligations required, so the question didn’t really apply to anything. I don’t have a checklist that is required or forced at all to use this, so it didn’t really apply,” which indicates she interpreted this response in a way it was not intended.

Recommendations:

1. Keep current question for C12g.

2. Modify C12a to read: “It is impossible to fulfill all the requirements in the E-Verify verification process.”

Question C13

ANSWER ONLY IF AT TIMES IT IS IMPOSSIBLE TO SUBMIT INFORMATION BY THE DEADLINE (C12g is ‘Strongly Agree’ OR ‘Agree’).

C13. Which of these situations make it difficult for your company to meet the E-Verify deadline for submitting cases within 3 days of hire?

(Please choose ALL THAT APPLY)

Reason for Testing Question: This question is new. The primary focus of the pretest was on question interpretation and response options.

Issue(s)/Findings: Participants generally said that they found this item easy to answer and that they knew immediately that they could provide more than one response. However, one participant said that the first time through she did not realize that it was all that apply, but she couldn’t think of a way to make this clearer. Based on participants’ feedback, it appears that meeting the deadline was particularly difficult for small companies, companies that hire seasonal workers, and employment agencies.

Recommendation: Modify question into a grid question with response options of “Yes” and “No” for each item (similar to question D8). This would make the question structure similar to other grid questions in which participants are required to provide a response for each item.

Question C14

ANSWER ONLY IF AT TIMES IT IS IMPOSSIBLE TO SUBMIT INFORMATION BY THE DEADLINE (C12g is “Strongly Agree” OR “Agree”).

C14. Which ONE of these situations presents the most difficulty for your company to submit cases within 3 days of hire? (Please choose only one response)

Reason for Testing Question: This question is new. The primary focus of the pretest was on question interpretation and response options.

Issue(s)/Findings: Participants generally said that this question was clear and they had no trouble selecting only one response. However, a few participants said that they did not read the question correctly at first and ended up selecting multiple responses. One participant selected the same multiple responses as in the previous item, but she didn’t think there was anything that could be done to make the instruction clearer since it is was very explicit.

Recommendation: Keep question format since the web survey will be programmed to allow participants to select only one response.

Question C15

ANSWER ONLY IF AT TIMES IT IS IMPOSSIBLE TO SUBMIT INFORMATION BY THE DEADLINE (C12g is ‘Strongly Agree’ OR ‘Agree’).

C15. How many days would you like to have to submit this information?

Reason for Testing Question: This question is new. The primary focus of the pretest was on question interpretation and response options.

Issue(s)/Findings: Participants generally thought this question was clear and interpreted the question as intended; i.e., they said that “this information” was referring to the Form I-9 information entered into E-Verify. A few participants thought that three days were adequate, but most wanted five days or a week to submit this information.

Recommendation: Keep current question and response options.

Question C16 in version 1 survey (same as question C16b in version 2 survey) and question C16a in version 2 survey. These questions relate to companies’ hiring patterns and difficulties in using E-Verify for seasonal workers

Question C16

ANSWER ONLY IF E-VERIFY IS DIFFICULT TO USE FOR HIRING SEASONAL WORKERS (C12h is ‘Strongly Agree’ OR ‘Agree’).

C16. Which of the following best describes your company’s hiring pattern?

Reason for Testing Question: This is a new question. The primary focus of the pretest was question interpretation and response options.

Issue(s)/Findings: Many participants interpreted the question as intended and understood the difference between seasonal and year-round workers. However, there was some confusion as to how to classify full-time versus part-time workers. A few participants said that they considered “year-round” to mean full-time staff and “seasonal” to mean part-time workers. One participant stated that seasonal workers at her company are not allowed to be full time, but another participant said that they have many full-time seasonal workers at her company. One participant said that at her company some employees start out as seasonal workers but then work their way into full employment. When asked if seasonal workers could also be full-time, participants agreed and provided examples such as college students hired during the summer.

Recommendations:

1. Add the following italicized text to the question stem:

Seasonal workers could work full time or part time during limited times of the year.
Year-round workers could also work full time or part time throughout the year.

2. Ask this question of ALL current users and then ask question C16b as a follow-up to companies that hire seasonal workers.

Question C17 (C6b in version 2 survey)

ANSWER ONLY IF YOU INDICATED IN C16a THAT YOUR COMPANY HIRES ONLY SEASONAL WORKERS OR BOTH SEASONAL AND YEAR-ROUND WORKERS.

C16b. How easy is it to use E-Verify for seasonal workers?

Reason for Testing Question: This question is a modified version of question C12h; for version 2 survey, the item was pulled out of the grid and modified because the original question was not applicable to companies that did not hire seasonal workers. The primary focus of pretesting C16b in version 2 survey was question interpretation and response options.

Issue(s)/Findings: Participants that hired seasonal workers did not report any difficulty with answering this question in version 2 survey. Several participants agreed that it was difficult to use E-Verify to hire seasonal workers, especially if they are hired in large numbers and off site.

Recommendation: Add this question to the revised survey.

Question C18 (C19, C20, C21, C22 in revised survey)

(ALL CURRENT USERS)

C18. To what extent do you agree or disagree with the following statements about the overall costs of E-Verify to your company?

Reason for Testing Question: This is a new question; the two-item grid question in the pretest version of the 2015 survey was intended to replace several questions on E-Verify costs in the 2013 survey. The primary focus of the pretest was question interpretation, response options, and ability to answer.

Issue(s)/Findings: Participants generally found this question to be clear and said they only had to read it once to understand what it was asking. Most also said they were able to answer the question on their own and did not need to consult with anyone else at their company. However, a review of responses to the question on the completed surveys showed that everyone answered “Disagree.” This suggested that participants were answering the question in terms of “Yes/No” response options. After trying a couple of alternate versions of question C18 that did not work well, we found a set of 4 questions that worked well (see questions C19 through C22 in the revised survey). Overall, participants preferred this alternate form of C18 because they felt it was more self-explanatory, specific, and straight-forward. Most participants also agreed that it was helpful to include examples of the costs of setting up and maintaining E-Verify.

Recommendation: Replace current question C18 with four alternate questions (C19 through C22).

Questions C21 and C22 (C21 was dropped from the survey and C22 is C25 in revised survey)

(CURRENT USERS USING PHOTO MATCHING)

C21. To what extent do you agree or disagree that the use of Photo Matching reduced your responsibility to be certain that the person presenting the document is the right person?

Reason for Testing Question: This item is in a new location (it was pulled out of a previous grid). The primary focus of the pretest was interpretation of the question in its new structure and location.

Issue(s)/Findings: Participants had varying opinions as to how to interpret this question. Some thought the question was referring to the “ease of comparing the photo on the screen to the photocopy” while others thought it was easier to have the screen and copy of the picture ID to know they have the correct person. Other participants thought the question meant that they did not have as much responsibility in making sure the ID matches or that they were not responsible for the results of the verification. One participant said that the question is asking if they are using technology to their advantage and doing extra work to see if people are who they say they are.

Recommendations:

1. Drop question C21 from survey because participants vary widely in their interpretations of the question. This question was confusing and one that could easily result in respondents answering incorrectly because they did not read or interpret it carefully.

2. Modify question C22 (C25 in revised survey) which asks whether the company compares the photo provided in the E-Verify Photo Matching response to the picture on the document the worker provided. To capture more useful information on Photo Matching, provide several options about the frequency (i.e., always, often, sometimes, never) in which each action is taken.

Questions C24 and C25 (C27 and C28 in revised survey): These are follow-up questions to question C23 (C26 in revised survey) which asks whether using E-Verify has affected the company’s willingness to hire job applicants who appear to be foreign born.

Question C24 (C27 in revised survey)

(CURRENT USERS RESPONDING ‘MORE WILLING’ TO C23)

C24. Why is your company more willing to hire job applicants that appear to be foreign born?

Reason for Testing Question: This was a grid question in the 2013 survey. It is made open-ended in the 2015 survey because it was not applicable to most of the participants based on their answer to question C23 (C26 in revised survey). The primary focus of the pretest was interpretation and responses to open-ended question.

Issue(s)/Findings: As in the 2013 survey, this question was skipped by most participants. Participants answering the question generally interpreted it as intended, commenting that they feel more safe hiring now with E-Verify, particularly when their “name is on the line.”

Recommendation: Keep current question for the 2015 survey but consider dropping in future iterations.

Question C25 (C28 in revised survey)

(CURRENT USERS RESPONDING ‘LESS WILLING’ TO C23)

C25. Why is your company less willing to hire job applicants that appear to be foreign born?

Reason for Testing Question: This was a grid question in the 2013 survey. It is made open-ended in the 2015 survey because it was not applicable to most of the participants based on their answer to question C23. The primary focus of the pretest was interpretation and responses to open-ended question.

Issue(s)/Findings: No focus group participants answered that they were less willing to hire job applicants that appear to be foreign born, so this question did not apply.

Recommendation: Keep current question for the 2015 survey but consider dropping in future iterations.

Question D3, D4, and D5: New questions about how often workers provide email addresses on their Form I-9, how often employers submit this information E-Verify, and reasons for not always submitting this information

Question D3

(ALL CURRENT USERS)

D3. How often would you say workers provide email addresses on their Form I-9?

Reason for testing question: This is a new item. The primary focus of the pretest was question interpretation and ability to answer.

Issue(s)/Findings: Most participants said the question was easy to answer and they did not need to consult with anyone else before answering. However, one participant noted that she had to go back and look at the form for their single new hire so it might be difficult to answer the question if someone had to check the forms for large numbers of hires.

When asked to say more if they answered “Always” or “Often,” one participant said that the response category “Always” is “too strict because it would mean every single form.” (This is an optional field on the Form I-9.) One participant said that about 20 to 30 percent of new hires filled in email addresses, another said it was about 50 percent, and another said it was about 90 percent. One participant answered “Sometimes” because “she’s lucky if new hires know how to read, let alone have an email address or smart phone.”

Recommendation: Keep the current question but modify the response options to read

“Usually, Sometimes, Rarely, Never.”

Question D4:

ALL CURRENT USERS)

D4. If workers provide their email addresses on the Form I-9, how often do you submit that information to the E-Verify system when creating a case for the worker?

Reason for testing question: This is a new item. The primary focus of the pretest was question interpretation and ability to answer.

Issue(s)/Findings: All participants who responded said it was easy to answer the question. Many participants said they never submit the email because it is not a required field. When asked to explain why they don’t always submit this information, they responded that it was an optional field, it is time consuming, it is not mandatory (i.e., there is no asterisk beside this field in the
E-Verify system indicating it is required), or it is illegible. While one participant thought that submitting email addresses is important for the purpose of notifying workers “in case of a reject,” another wondered why email addresses were even needed since it was so time-consuming to submit to E-Verify. Others indicated that they do not encourage new hires to provide email addresses because of privacy concerns.

Recommendation: Keep the current question and the response options.

Question D5:

(CURRENT USERS THAT DO NOT ALWAYS SUBMIT WORKERS’ EMAIL ADDRESSES)

D5. Why don’t you always submit the workers’ email addresses to the E-Verify system?

Reason for testing question: This is a new question. The primary focus of the pretest was interpretation of and responses to the open ended question.

Issue(s)/Findings: All participants correctly interpreted the question and their responses reflected this understanding of the question. In many cases, they repeated comments they made for questions D3 and D4 about the field not being required or not knowing why it was needed. For example, one participant indicated that she would not share this information if this request came from the new hire; she also said they would be more likely to share this information if there was an explanation about its intended use.

Recommendation: Keep the current question.

Question D6, D7, and D8: These are new questions on the duplicate case alert on whether they had a duplicate case alert, how useful it was, and what they did after getting the alert.

Question D6

(ALL CURRENT USERS)

D6. E-Verify has implemented a new feature to alert employers when they are creating a duplicate case (i.e., when the social security number of the current case matches another recently submitted case).

How often have you had a duplicate case alert when creating a case in E-Verify?

Reason for testing question: This is a new item. The primary focus of the pretest was question interpretation and ability to answer.

Issue(s)/Findings: Participants who responded said the question was easy to answer and it was clear or they understood what a duplicate case was.

Recommendation: Keep the current question and response options.

Question D7

(CURRENT USERS THAT HAD A DUPLICATE CASE ALERT)

D7. Generally, how useful was the duplicate case alert?

Reason for testing question: This is a new item. The primary focus of the pretest was question interpretation and ability to answer

Issue(s)/Findings: Participants who responded said the question was easy/very easy to answer. When asked to say more about why they said the duplicate case alert was “Slightly useful” or “Not at all useful,” one participant explained that they hired at different locations and already know they may have duplicates; but it’s not useful to have the alert coming up after they’ve entered almost everything.

Recommendation: Keep the current question and the response options.

Question D8

(CURRENT USERS THAT HAD A DUPLICATE CASE ALERT)

D8. When you received a duplicate case alert, how did you respond to it?

Reason for testing question: This is a new question. The primary focus of the pretest was question interpretation and ability to answer

Issue(s)/Findings: Participants agreed that the statements were clear and that the directions were also clear, although one person said she had to read the directions more than once to realize that she needed to provide an answer for each statement. In answering the question, a few participants checked only the “Yes” response and left the “No” response blank for the remaining questions (this is a common issue in this type of grid question; the “No” is implied).

Recommendation: Keep the current question.

Questions D12, D13, and D14: These are new questions on the Further Action Notice (FAN) on whether participants used the FAN, how satisfied they were with this experience, and how often they followed a list of procedures in using the FAN.

Question D12

(CURRENT USERS THAT HAD A TNC)

D12. USCIS recently implemented the Further Action Notice to replace the Tentative Nonconfirmation notice.

Has your company used the Further Action Notice to process Tentative Nonconfirmations?

Reason for Testing Question: This is a new question. The primary focus of the pretest was on participants’ familiarity with the term Further Action Notice and their ability to answer the question.

Issue(s)/Findings: Participants that had a TNC since September 9, 2013 were familiar with the Further Action Notice and found the question easy to answer. However, about half of the participants from small companies did not know what the “FAN” was and some still referred to the notice as the TNC notice instead of the Further Action Notice.

Recommendation:

Modify the question stem to read: On September 9, 2013, USCIS replaced the Tentative Nonconfirmation notice with the Further Action Notice.

Has your company used the Further Action Notice to process Tentative Nonconfirmations?

Question D13

(CURRENT USERS THAT USED THE FAN)

D13. Generally, how satisfied were you with your experience in using the Further Action Notice?

Reason for Testing Question: This is a new question. The primary focus of the pretest was on participants’ ability to answer the question.

Issue(s)/Findings: Participants generally found the question easy to answer. One exception is that a participant found it a little difficult to decide if it was easy or not although she did respond being satisfied with using the FAN.

Recommendation:

Keep the current question and response options.

Question D14

(CURRENT USERS THAT USED THE FAN)

D14. How often does your company follow the procedure in using the Further Action Notice and Referral Date Confirmation?

Reason for Testing Question: This is a new question. The primary focus of the pretest was on participants’ interpretation of the question and their ability to answer it.

Issue(s)/Findings: Participants generally reported that the statements were clear, the question was easy to answer, and they did not have to consult with anyone in the company to answer it. However, when asked how comfortable they would be in responding sometimes or never if their company did not follow a procedure, participants reported that they would not feel comfortable choosing these responses. There was general agreement that they would never report “Never” and that individuals may be uneasy reporting “Sometimes,” even if this was the case. For example, one person said “If you know you should be doing these things, you’d check ‘Always’ for each category regardless.”

Feedback from participants suggests that the current question is leading because the text “follow the procedure” in the stem of the question encourages participants to report “Always” even if their company did not always do so. As noted by some participants, ALL of the statements reflect procedures that should be followed.

Recommendations:

• To make the question stem more neutral, modify it to read:

How often does your company do the following when using the Further Action Notice and Referral Date Confirmation?

• Modify the statements by adding a few distractors of nonconforming behavior to the statements.

  • Drop the original statement D14b, “We provide workers with the Further Action Notice in English.”

  • Add new statement, “We retain a copy of the signed Further Action Notice.”

  • Modify statement D14f to read: “We create a new case without closing the old one if the information on the Further Action Notice is incorrect.”

Question D21 in version 1 survey (D22 in revised survey)

(ALL COMPANIES)

D21. The following statements describe possible changes that could be made to
E-Verify procedures. Please select the answer that best describes your views for each of these possible changes.

f. Allowing employers that are not federal contractors to verify existing employees

Reason for testing question: This item has been revised from 2013; the previous version reads “Allowing all companies to verify existing employees.” The primary focus of the pretest was response item interpretation. A secondary focus was possible influence on participants’ responses to the open-ended question about changes to E-Verify procedures.

Issue(s)/Findings: Only a few participants responded to the question about what this item was saying and they interpreted it correctly. Additionally, none of the participants knew that federal contracts contain a rule allowing federal contractors to verify existing employers but they said that knowing this would not change their response to item D21f –the majority of which were “Strongly Support,” “Support,” or “No Opinion.”

Only one participant had made this suggestion in the open-ended question about possible changes to E-Verify procedures and none of the participants would have added it to their response for the open-ended question, no matter where the open-ended question was located in the survey.

Recommendation: Keep the current question and the response options.

“Not applicable” response options for questions B12, C12, and D17: Two versions of the questions were tested; version 1 survey included the “Not applicable” option while version 2 did not provide this response option.

Question B12 (B13 in revised survey)

(ALL COMPANIES)

B12. Have you ever had any of the following problems with the E-Verify Technical Help Desk or the E-Verify Customer Service number?

Reason for testing question: The primary purpose was to test whether the question needed a “Not applicable” response option. Version 1 survey included this response option while version 2 survey dropped the option.

Issue(s)/Findings: In general, participants did not see the need for a “Not applicable” response option for this question

Participants with version 1 survey did not select “Not applicable” for this question except for a few who should have skipped the question based on their answer to question B7. Note that the Web survey will be programmed to skip respondents who answered that they did not call the
E-Verify Help Desk or Customer Service number.

Participants with version 2 survey did not see a need for a “Not applicable” response option and no one selected this answer in their completed survey.

Recommendation: Drop the “Not applicable” response option for this question.

Question C12

(ALL COMPANIES)

C12. Please consider each of the following statements related to E-Verify and select the response that best represents the experiences at your company.

Reason for testing question: The primary purpose was to test whether the question needed a “Not applicable” response option.

Issue(s)/Findings: In general, participants did not see the need for a “Not applicable” response option for this question. The one exception is that respondents that do not hire seasonal workers needed to check “Not applicable.”

Most participants with version 1 survey checked “Not applicable” for C12h because they did not hire seasonal workers. Two participants selected this response option for item C12e because they did not have employees with long or difficult names and one participant selected the response option for C12f because they did not need frequent assistance from the Help Desk to use E-Verify. These participants said they would have checked disagree if the question did not include a “Not applicable” option. One participant misinterpreted question C12a and answered “Not applicable” for C12a because their company did not have obligations for E-Verify.

Participants with version 2 survey did not see a need for a “Not applicable” response option and no one selected this answer in their completed survey.

Recommendation: Drop the “Not applicable” response option for this question and modify C12a to make it clearer that the question is asking about E-Verify requirements (see earlier discussion of suggested changes to question C12).

Question C17

(CURRENT USERS THAT HAD A TNC

D17. Please consider each of the following statements related to Tentative Nonconfirmations received during employment verification using the E-Verify system. Select the answer that best represents the experiences of your company.

Reason for testing question: The primary purpose was to test whether the question needed a “Not applicable” response option.

Issue(s)/Findings: None of the participants with version 1 survey answered “Not applicable” for any of the items in question D17. In addition, participants with version 2 survey did not see a need to have this response option in the question.

Recommendation: Drop the “Not applicable” response option for this question.

Other questions with “Not applicable” response options C8, D1, D14, D15, and D19 in revised survey.

We reviewed participants’ completed surveys to examine whether they selected “Not applicable” for these questions in order to assess our decision to retain the response option in the pretest draft of the survey. This decision was based on an assumption that many of the statements in the grid questions reflect situations that may not occur for some companies. Based on a review of participants’ responses, we confirmed that it is necessary to retain a “Not applicable” response option for these questions.

1 Using a randomized approach (although not a statistically valid random sample), participants for each focus group were selected from the E-Verify Transaction Database to reflect whether their companies had Tentative Nonconfirmatons since September 9, 2013, whether they were required by state or federal mandates to use E-Verify, and whether they were in industries that historically employed a high proportion of undocumented workers.