Usability Benchmarks Voting System - Supporting Statement

0693.0043.Usability.Benchmarks.VotingSystems.SuppStmt.doc

NIST Generic Clearance for Usability Data Collections

Usability Benchmarks Voting System - Supporting Statement

OMB: 0693-0043

Document [doc]
Download: doc | pdf




DOC/NIST Usability Data Collection

Generic Clearance

OMB Control No. 0693-0043

Expiration Date: 10/31/2012

Usability Benchmarks for Voting Systems Research Project

  • Screener for Voting System Testing (Script)


  • Screener for Accessible Voting Machine Testing (Same Script)


  • Post-Test Questionnaire



  1. Explain who will be surveyed and why the group is appropriate to survey.


As part of a research study to confirm already-developed benchmarks and test methods for performance-based standards for the accessibility and usability of voting systems, the National Institute of Standards and Technology (NIST) intends to recruit individuals who are eligible to vote (United States citizens over 18 years old) to participant in user-based tests of voting machines. The voting machines are being evaluated to determine if voters with broader demographics (including voters with disabilities) are able to accurately enter and submit votes using the machines in a mock election. In a past study, NIST completed an analysis that proved the capability of the NIST usability performance test to detect differences in voter performance between machines. The Election Assistance Commission and its Technical Guidelines Development Committee has asked NIST to investigate the effect of a broader voter demographic on the usability performance testing. In the current study, NIST will recruit individuals across a broader age and education range and include some subjects with vision and dexterity disabilities.

The subjects will be recruiting using the screening questionnaire and script. If they are suitable, they will be scheduled for the usability test. When the subjects arrive for the usability test they well be given an informed consent and instructed to vote on a voting machine. Subjects will then be asked to fill out the post-test questionnaire. NIST hopes to confirm the reliability of the usability performance tests for voting systems with voters without disabilities. NIST will also explore the differences in usability performance, as determined by our test protocol of accessible voting systems designed for people with disabilities. NIST will recruit up to 1,000 individuals for these tests, approximately 60 subjects with disabilities.



  1. Explain how the survey was developed including consultation with interested parties, pretesting, and responses to suggestions for improvement.


The survey questionnaires that have been developed are based on standard templates used by NIST usability and accessibility tests and by most usability engineering professionals. Similar demographic and confidence/satisfaction questions have been piloted, and validated by usability groups studying various hardware and software systems. NIST has incorporated the feedback and suggestions from various team members based on NIST past studies and outside technical consultants and the surveys are designed based on best practices for usability test questionnaires.


  1. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.


Subjects will be screened as appropriate participants by determining that they are US citizens at least 18 years old and older (eligible voters, registered or not) who:


  • have no work experience in the election community, and no immediate family with work experience in the election community, and


  • have no work experience in the election community and no immediate family with work experience in an Internet or software development company or a usability or market research business/company

They will further be screened for specific gender, race, education, and age demographics, and, for testing accessible voting machines, various disabilities. It is critical for the repeatability and reliability of the study to have a very well-defined demographic for the testing of the machines with individuals who do not have disabilities. For the experiments with the accessible voting machines, NIST is not focused on these demographics but primarily with finding test subjects who are blind, low vision (using large print), low vision (using large print and audio), or who have limited dexterity as those are the factors that drive the performance of the accessible voting machines. After screening, subjects will be scheduled for a testing appointment. At the time of testing, all subjects will sign an informed consent form that fully explains the study. After each subject has given informed consent to participate in the research study, he or she will be given a paper (or audio instructions for visually impaired voters) with detailed instructions of who to vote for. Then the subject will use the voting machine to vote. After voting, subjects will be given the post-test questionnaire.


The survey will be administered on paper. The expected response rate is 100% since NIST will have the demographic data from the screening and recruiting process and the subjects will complete the survey as part of their participation in the mock election test. The subjects will be paid $50-$200 to participate in the study. Experience has shown, it is very difficult to recruit subjects with disabilities and they incur larger costs to travel to and from test sites. NIST use contractors and they deem this is a reasonable token of appreciation for a usability test.



  1. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.


The purpose of this research project is to create a standard, reliable method for measuring usability performance of voting systems, which will ultimately help to provide usable voting systems (machines) for U.S. citizens to use in voting. Once the method is standardized, the method will be used:


    1. To set a baseline benchmark measure of usability and accessibility for voting to be used as a requirement for voting machines.


    1. To compare results between tests conducted on different machines when executed by voting system test laboratories.



3


File Typeapplication/msword
File TitleDOC/NIST Usability Data Collection
AuthorYonder, Darla
Last Modified ByYonder, Darla
File Modified2012-06-13
File Created2012-06-13

© 2024 OMB.report | Privacy Policy