NIST, ITL, Marking and Verifying Ballots Collection - Supporting Statement

OMB Control #0693-0043 Expiration Date: 03/31/2022

NIST Generic Clearance for Usability Data Collections

NIST – Marking and Verifying Ballots Before Casting: Requirements for Usability and Accessibility

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be surveyed and why the group is appropriate to survey.

In previous research efforts, National Institute of Standards and Technology (NIST) has investigated the use of voting systems in U.S. elections. In this effort, Center for Civic Design, under contract to NIST, will gather information from people registered to vote in the United States about how they mark and review their ballots. This is exploratory research through interviews and observations to understand the role of verification of the ballot in the voting process and what makes this process usable and accessible to voters.

Having previously researched, via literature and web searches, existing research data on the accuracy and efficiency of existing and prototype voting systems, we will now interview voters to learn more about their experiences in elections and their understanding of the role of verifying their ballot before casting it.

This questionnaire is a part of a research session in which we observe people using a voting system. The goal of this research is to better understand how to design voting systems and voting processes to support and encourage voters to review and verify their marked paper ballots after using an electronic ballot marking systems before casting. This information collection is being conducted by the Center for Civic Design, working for NIST under Contract GS-06F-0942Z, Order Number 1333ND18FNB770325. The voters will consist of 60 people with a range of voting experience and demographics. These voters are appropriate to fill this role due to their personal experiences and abilities.

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

This information collection was developed by the Center for Civic Design and NIST researchers. The voting history and demographics collected are based on extensive prior work developing test methods for the Voluntary Voting System Guidelines and testing voting systems for usability and accessibility.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

Voters will be recruited in two different ways:

1. For sessions that are scheduled in advance and for which participants are asked to participate at a specific time and date, an email will be sent to people who have participated in prior research. They will respond to the address in the email to indicate their willingness to participate. We will reply to confirm and select a date and time for the meeting.

2. The researchers will work with community centers and organizations to set up dates and times for meetings at locations where appropriate voters will be available. Notices will be posted inviting voters to participate. They will respond in person to indicate their willingness to participate.

In either case, the voting systems must be set up in a mocked-up polling place and are not easily portable. Participants will need to travel to these locations. Locations will include independent living centers, a centrally-located usability lab, and a centrally-located county clerk’s office.

The sample of 60 voters will be chosen based on a convenience sample. Researchers will take notes during the meeting to collect subject-matter expert responses. We are also offering an incentive payment in consideration of the time the use of the voting system and interview takes, the range of voter abilities needed for the usability testing, and the travel that is required. Participants recruited for pre-scheduled appointments where they must travel to the research location will be paid $75. Participants who are recruited at a location where they do not have to travel to a specific appointment will be paid $50. We ask for up to an hour of their time with careful attention to the questions and activities in the research session. More specifically, the incentive payment is needed to encourage participation because:

• The voting systems are ballot marking devices which are designed to be accessible for voters with disabilities. Our sample needs to include voters with a range of abilities, e.g., low literacy, low vision, and mobility and dexterity limitations. This population often have transportation burdens and are hard-to-find.

• Given the small number of participants for this qualitative study, an incentive will ensure that we get a range of individuals across this population to insure data quality.

• It would be time and cost-prohibitive to setup voting systems in the field for each participant.

• The survey is part of an exploratory cognitive/usability research protocol and compensation is typical for such studies. In particular, both NIST and Center for Civic Design have extensive experience with usability testing of voting systems and have been successful in recruiting participants using this range of compensation.

To qualify, participants must be registered voters and must speak English well enough to converse with the researchers and read the ballot information and instructions.

There will be a limited number of appointments or opportunities to participate at each location, filled on a first-come-first-served basis. The expected response rate is 25-30%. We will continue the invitations until the sample is complete.

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

The results will be qualitatively analyzed; the purpose of this information collection is to gather sufficient background knowledge about marking and verifying ballots in the U.S. to inform future work on voting system guidelines. There will be no direct generalization of the responses to a larger population; responses may lead to future research that will be generalized to the U.S.

3