Explain who will be surveyed and why the group is appropriate to survey.
As part of a research study to confirm already-developed benchmarks and test methods for performance-based standards for the accessibility and usability of voting systems, NIST intends to recruit individuals who are eligible to vote (over 18 and US citizens) to participant in user-based tests of voting machines. The voting machines are being evaluated to determine if voters with broader demographics (including voters with disabilities) are able to accurately enter and submit votes using the machines in a mock election. In a past study, NIST completed an analysis that proved the capability of the NIST usability performance test to detect differences in voter performance between machines. The Election Assistance Commission and its Technical Guidelines Development Committee has asked NIST to investigate the effect of a broader voter demographic on the usability performance testing. In the current study, NIST will recruit individuals across a broader age and education range and include some subjects with vision and dexterity disabilities. NIST will give subjects a screening questionnaire, and, if they are suitable, they will be instructed to vote on a voting machine. Subjects will then be asked to fill out a post-test questionnaire. NIST hopes to confirm the reliability of the usability performance tests for voting systems with voters without disabilities. NIST will also explore the differences in usability performance, as determined by our test protocol, of accessible voting systems designed for people with disabilities. NIST will recruit up to 1000 individuals for these tests.
Explain how the survey was developed including consultation with interested parties, pretesting, and responses to suggestions for improvement.
The survey questionnaires that have been developed are based on standard templates used by NIST usability and accessibility tests and by most usability engineering professionals. Similar demographic and confidence/satisfaction questions have been piloted, and validated by usability groups studying various hardware and software systems. NIST has incorporated the feedback and suggestions from various team members based on NIST past studies and outside technical consultants and the surveys are designed based on best practices for usability test questionnaires.
Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.
Subjects will be screened as appropriate participants by determining that they are US citizens at least 18 years old and older (eligible voters, registered or not) who:
have no work experience in the election community, and no immediate family with work experience in the election community, and
have no work experience in the election community and no immediate family with work experience in an Internet or software development company or a usability or market research business/company
They will further be screened for specific gender, race, education, and age demographics, and, for testing accessible voting machines, various disabilities. It is critical for the repeatability and reliability of the study to have a very well-defined demographic for the testing of the machines with individuals who do not have disabilities. For the experiments with the accessible voting machines, NIST is not focused on these demographics but primarily with finding test subjects who are blind, low vision (using large print), low vision (using large print and audio), or who have limited dexterity as those are the factors that drive the performance of the accessible voting machines. After screening, subjects will be scheduled for a testing appointment. At the time of testing, all subjects will sign an informed consent form that fully explains the study. After each subject has given informed consent to participate in the research study, he or she will be given a paper (or audio instructions for visually impaired voters) with detailed instructions of who to vote for. Then the subject will use the voting machine to vote. After voting, subjects will be given the post test questionnaire. As part of the study, completion of the questionnaire is required. If subjects do not complete the questionnaire, their voting data will not be included in the study.
The survey will be administered on paper. The expected response rate is 100% since NIST will have the demographic data from the screening and recruiting process and the subjects will complete the survey as part of their participation in the mock election test. The subjects will be paid $50-$200 to participate in the study.
Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.
The purpose of this research project is to create a standard, reliable method for measuring usability performance of voting systems, which will ultimately help to provide usable voting systems (machines) for U.S. citizens to use in voting. Once the method is standardized, the method will be used:
To set a baseline benchmark measure of usability and accessibility for voting to be used as a requirement for voting machines
To compare results between tests conducted on different machines when executed by voting system test laboratories.
| File Type | application/msword |
| File Title | Paperwork Reduction Act |
| Last Modified By | Darla Yonder |
| File Modified | 2008-05-23 |
| File Created | 2008-05-23 |