Download:
pdf |
pdfTesting & Certification Program
Manual
Version 2.0
Effective MAy 31, 2015
United States Election Assistance Commission
1335 East West Highway, Suite 4300, Silver Spring, MD 20910
www.eac.gov
OMB Control # Pending
EAC Voting System Testing and Certification Program Manual, Version 2.0
The reporting requirements in this manual have been approved under the Paperwork Reduction Act of
1995, OMB Control No. 0000-0000. Persons are not required to respond to this collection of
information unless it displays a currently valid OMB number. Information gathered pursuant to this
document and its forms will be used solely to administer the EAC Testing and Certification Program.
This program is voluntary. Individuals who wish to participate in the program, however, must meet
its requirements. The estimated total annual hourly burden on the voting system manufacturing
industry and election officials is 105 hours. This estimate includes the time for reviewing the
instructions, gathering information and completing the prescribed forms. Send comments regarding
this burden estimate or any other aspect of this collection, including suggestions for reducing this
burden, to the U. S. Election Assistance Commission, Voting System Testing and Certification
Program, Office of the Program Director, 1335 East West Highway, Suite 4300, Silver Spring, MD
20910.
OMB Control Number PENDING
i
EAC Voting System Testing and Certification Program Manual, Version 2.0
Table of Contents
1.
Introduction .............................................................................................................. 1
2.
Manufacturer Registration ...................................................................................... 12
3.
When Voting Systems Must Be Submitted for Testing and Certification ................ 19
4.
Certification Testing and Technical Review ........................................................... 30
5.
Grant of Certification .............................................................................................. 44
6.
Denial of Certification ............................................................................................. 53
7.
Decertification ........................................................................................................ 60
8.
Quality Monitoring Program ................................................................................... 71
9.
Requests for Interpretations ................................................................................... 75
10. Release of Certification Program Information ........................................................ 80
Appendix A: Manufacturer Registration Application Form ............................................. 84
Appendix B: Application for Voting System Testing Form ............................................. 85
Appendix C: Voting System Anomaly Reporting Form ..... Error! Bookmark not defined.
Appendix D: Voting System Test Plan Outline .............................................................. 90
Appendix E: Voting System Modification Test Plan Outline………………………………93
Appendix F: Voting System Test Report Outline…………………………………………...95
OMB Control Number PENDING
ii
EAC Voting System Testing and Certification Program Manual, Version 2.0
OMB Control Number PENDING
iii
EAC Voting System Testing and Certification Program Manual, Version 2.0
1. Introduction
1.1. Background. In late 2002, Congress passed the Help America Vote Act of 2002 (HAVA), which
created the U.S. Election Assistance Commission (EAC) and vested it with the responsibility of
setting voting system standards and for providing for the testing and certification of voting
systems. This mandate represented the first time the Federal government provided for the
voluntary testing, certification and decertification of voting systems nationwide. In response
to this HAVA requirement, the EAC has developed the Federal Voting System Testing and
Certification Program (“Certification Program”).
1.2. Authority. HAVA requires that the EAC certify and decertify voting systems. Section 231(a)(1)
of HAVA specifically requires the EAC to “… provide for the certification, de-certification and
re-certification of voting system hardware and software by accredited laboratories.” The EAC
has the sole authority to grant certification or withdraw certification at the Federal level,
including the authority to grant, maintain, extend, suspend, and withdraw the right to retain
or use any certificates, marks, or other indicators of certification.
1.3. Scope. This Manual provides the procedural requirements of the EAC Voting System Testing
and Certification Program. Although participation in the program is voluntary, adherence to
the program’s procedural requirements is mandatory for participants. The procedural
requirements of this Manual supersede any prior voting system certification requirements
issued by the EAC.
1.4. Purpose. The primary purpose of the EAC’s Voting System Testing and Certification Program
Manual is to provide clear procedures to Manufacturers for the testing and certification of
voting systems to specified Federal standards consistent with the requirements of HAVA
Section 321(a)(1). The program, however, also serves to do the following:
1.4.1. Support State certification programs.
1.4.2. Support local election officials in the areas of acceptance testing and pre-election system
verification.
1.4.3. Increase quality control in voting system manufacturing.
1.4.4. Increase voter confidence in the use of voting systems.
1.5. Manual. This Manual is a comprehensive presentation of the EAC’s Voting System Testing
and Certification Program. It is intended to establish all of the program’s administrative
requirements.
1.5.1. Contents. The contents of the Manual serve as an overview to the program itself. The
Manual contains the following chapters:
OMB Control Number PENDING
1
EAC Voting System Testing and Certification Program Manual, Version 2.0
1.5.1.1. Manufacturer Registration. Under the program, a Manufacturer is required to
register with the EAC prior to participation. This registration provides the EAC
with needed information and requires the Manufacturer to agree to the
requirements of the Certification Program. This chapter sets out the requirements
and procedures for registration.
1.5.1.2. When Voting Systems Must Be Submitted for Testing and Certification. All systems
must be submitted consistent with this Manual before they may receive a
certification from the EAC. This chapter discusses the various circumstances that
require submission to obtain or maintain a certification.
1.5.1.3. Certification Testing and Review. Under this program, the testing and review
process requires the completion of an application, employment of an EACaccredited laboratory for system testing, and technical analysis of the laboratory
test report by the EAC. The result of this process is an Initial Decision on
Certification. This chapter discusses the required steps for voting system testing
and review.
1.5.1.4. Grant of Certification. If an Initial Decision to grant certification is made, the
Manufacturer must take additional steps before it may be issued a certification.
These steps require the Manufacturer to document the performance of a trusted
build (see definition at section 1.16), the deposit of software into a repository, and
the creation of system identification tools. This chapter outlines the action that a
Manufacturer must take to receive a certification and the Manufacturer’s postcertification responsibilities.
1.5.1.5. Denial of Certification. If an Initial Decision to deny certification is made, the
Manufacturer has certain rights and responsibilities under the program. This
chapter contains procedures for requesting reconsideration, opportunity to cure
defects, and appeal.
1.5.1.6. Decertification. Decertification is the process by which the EAC revokes a
certification it previously granted to a voting system. It is an important part of
the Certification Program because it serves to ensure that the requirements of the
program are followed and that certified voting systems fielded for use in Federal
elections maintain the same level of quality as those presented for testing. This
chapter sets procedures for Decertification and explains the Manufacturer’s
rights and responsibilities during that process.
1.5.1.7. Quality Monitoring Program. Under the Certification Program, the EAC will
implement a quality monitoring process that will help ensure that voting systems
certified by the EAC are the same systems sold by Manufacturers. The quality
OMB Control Number PENDING
2
EAC Voting System Testing and Certification Program Manual, Version 2.0
monitoring process is a mandatory part of the program and includes elements
such as fielded voting system review, anomaly reporting, and manufacturing site
visits. This chapter sets forth the requirements of the Quality Monitoring
Program.
1.5.1.8. Interpretation. An Interpretation is a means by which a registered Manufacturer
or Voting System Test Laboratory (VSTL) may seek clarification on a specific
Voluntary Voting System Guidelines (VVSG) standard. This chapter outlines the
policy, requirements, and procedures for requesting an Interpretation.
1.5.1.9. Trade Secrets, Confidential Commercial, and Personal Information. Federal law
protects certain types of information provided to the government from public
release. This chapter outlines the program’s policies, procedures, and
responsibilities associated with the public release of potentially protected
commercial information.
1.5.2. Maintenance and Revision. Version 2.0 of the Manual will continue to be improved and
expanded as experience and circumstances dictate. The Manual will be reviewed
periodically and updated to meet the needs of the EAC, Manufacturers, VSTLs, election
officials, and public policy. The EAC is responsible for revising this document. And all
revisions will be made consistent with Federal law. Substantive input from stakeholders
and the public will be sought whenever possible, at the discretion of the agency. Changes in
policy requiring immediate implementation will be noticed via policy memoranda and will
be issued to each registered Manufacturer. Changes, addendums, or updated versions will
also be posted to the EAC’s website at www.eac.gov.
1.6. Program Methodology. The EAC’s Voting System Testing and Certification Program is one
part of the overall conformity assessment process that includes companion efforts at State and
local levels.
1.6.1. Federal and State Roles. The process to ensure voting equipment meets the technical
requirements is a distributed, cooperative effort of Federal, State, and local officials in the
United States. Working with voting equipment manufacturers, each of these officials has a
unique responsibility for ensuring the equipment a voter uses on Election Day meets
specific requirements.
1.6.1.1. The EAC Program has primary responsibility for ensuring voting systems
submitted under this program meet Federal standards established for voting
systems.
1.6.1.2. State officials have responsibility for testing voting systems to ensure the system
will support the specific unique requirements of each individual State. States
may use EAC VSTLs to perform testing of voting systems to unique state
OMB Control Number PENDING
3
EAC Voting System Testing and Certification Program Manual, Version 2.0
standards while the systems are being tested to Federal standards. The EAC will
not, however, certify voting systems to state standards.
1.6.1.3. State or local officials are responsible for making the final purchase choice and
are responsible for deciding which system offers the best fit and value for their
specific State or local jurisdiction.
1.6.1.4. State or local officials are also responsible for performing acceptance testing to
ensure that the equipment delivered is identical to the equipment certified at the
federal and state levels, is fully operational, and meets the contractual
requirements of the purchase.
1.6.1.5. State or local officials should perform pre-election logic and accuracy testing
and post-election auditing to confirm equipment is operating properly and is
unmodified from its certified state.
1.6.2. Conformity Assessment, Generally. Conformity assessment is a system established to
ensure a product or service meets the applicable requirements. Many conformity
assessment systems exist to protect the quality and ensure compliance with standards of
products and services. All conformity assessment systems attempt to answer a variety of
questions:
1.6.2.1. What specifications are required of an acceptable system? For voting systems, the
EAC Voluntary Voting System Guidelines (VVSG), Notice of Clarification and
Request for Information address this issue. States and local jurisdictions also
have supplementing standards.
1.6.2.2. How are systems tested against required specifications? The EAC Voting System
Testing and Certification Program is a central element of the larger conformity
assessment system. The program, as set forth in this Manual, provides for the
testing and certification of voting systems to identified versions of the VVSG. The
Testing and Certification Program’s purpose is to verify voting systems meet
manufacturer specifications and the requirements of the VVSG.
1.6.2.3. Are the testing authorities qualified to make an accurate evaluation? The EAC
accredits VSTLs, after the National Institute of Standards and Technology (NIST)
National Voluntary Lab Accreditation Program (NVLAP) has reviewed their
technical competence and lab practices to ensure the test authorities are fully
qualified. Furthermore, EAC technical experts review all test plans and test
reports from accredited laboratories to ensure an accurate and complete
evaluation. Many States provide similar reviews of laboratory reports.
OMB Control Number PENDING
4
EAC Voting System Testing and Certification Program Manual, Version 2.0
1.6.2.4. Will Manufacturers deliver units within manufacturing tolerances equivalent to those
tested? The VVSG and this Manual require vendors to have appropriate change
management and quality control processes to control the quality and
configuration of their products. The Certification Program provides mechanisms
for the EAC to verify Manufacturer quality processes through field system
testing and manufacturing site visits. States have implemented policies for
acceptance of delivered units.
1.7. Program Personnel. All EAC personnel and contractors associated with this program are held
to the highest ethical standards. All agents of the EAC involved in the Certification Program
are subject to conflict-of-interest reporting and review, consistent with Federal law and
regulation.
1.8. Program Records. The EAC Program Director is responsible for maintaining accurate records
to demonstrate the testing and certification program procedures have been effectively fulfilled
and to ensure the traceability, repeatability, and reproducibility of testing and test report
review. All records will be maintained, managed, secured, stored, archived, and disposed of in
accordance with Federal law, Federal regulations, and procedures of the EAC.
1.9. Submission of Documents. Any documents submitted pursuant to the requirements of this
Manual shall be submitted:
1.9.1. Via secure e-mail, if sent electronically, or physical delivery of a compact disk or other
media deemed acceptable by the EAC, unless otherwise specified.
1.9.2. In a Microsoft Word or Adobe PDF file, formatted to protect the document from alteration.
1.9.3. With a proper signature when required by this manual. Documents requiring an authorized
signature may be signed with an electronic representation or image of the signature of an
authorized management representative and must meet any and all subsequent
requirements established by the Program Director regarding security.
1.9.4. By certified mail or similar means allowing for tracking, if sent via physical delivery, to
the following address:
Testing and Certification Program Director
U.S. Election Assistance Commission
1335 East West Highway, Suite 4300
Silver Spring, MD 20910
1.10. Receipt of Documents—Manufacturer. For purposes of this Manual, a document, notice,
or other communication is considered received by a Manufacturer upon one of the following:
OMB Control Number PENDING
5
EAC Voting System Testing and Certification Program Manual, Version 2.0
1.10.1. The actual, documented date the correspondence was received (either electronically or
physically) at the Manufacturer’s place of business; or
1.10.2. If no documentation of the actual delivery date exists, the date of constructive receipt for
the communication. For electronic correspondence, documents will be constructively
received the day after the date sent. For mail correspondence, the document will be
constructively received 3 days after the date sent.
1.10.3. The term receipt shall mean the date a document or correspondence arrives (either
electronically or physically) at the Manufacturer’s place of business. Arrival does not
require that an agent of the Manufacturer open, read, or review the correspondence.
1.11. Receipt of Documents—EAC. For purposes of this Manual, a document, notice, or other
communication is considered received by the EAC upon its physical or electronic arrival at the
agency. All documents received by the agency will be physically or electronically date
stamped and this stamp shall serve as the date of receipt. Documents received after the
regular business day (5:00 pm Eastern Standard Time), will be treated as if received on the
next business day.
1.12. EAC Response Timeframes. In recognition of the responsibilities and challenges facing
Manufacturers as they work to meet the requirements imposed by this program, State
certification programs, customers, State law and production schedules, the EAC will provide
timeframes for its response to significant program elements. This shall be done by providing
current metrics on the EAC’s website regarding the average EAC response time for (1)
approving Test Plans, (2) issuing Initial Decisions, and (3) issuing Certificates of Conformance.
1.13. Records Retention—Manufacturers. The Manufacturer is responsible for ensuring all
documents submitted to the EAC, or that otherwise serve as the basis for the certification of a
voting system, are retained. A copy of all such records shall be retained as long as a voting
system is offered for sale or supported by a Manufacturer and for 5 years thereafter.
1.14. Record Retention—EAC. The EAC shall retain all records associated with the certification
of a voting system as long as such system is fielded in a State or local election jurisdiction for
use in Federal elections. The records shall otherwise be retained or disposed of consistent with
Federal statutes and regulations.
1.15. Publication and Release of Documents. The EAC will release documents consistent with
the requirements of Federal law. It is EAC policy to make the certification process as open and
transparent as possible. Any documents (or portions thereof) submitted under this program
will be made available to the public unless specifically protected from release by law. The
primary means for making this information available is through the EAC website.
OMB Control Number PENDING
6
EAC Voting System Testing and Certification Program Manual, Version 2.0
1.16. Definitions. For purposes of this Manual, the terms listed below have the following
definitions.
Appeal. A formal process by which the EAC is petitioned to reconsider an Agency Decision.
Appeal Authority. The individual or individuals appointed to serve as the determination
authority on appeal.
Build Environment. The disk or other media that holds the source code, compiler, linker,
integrated development environments (IDE), and/or other necessary files for the compilation and
on which the compiler will store the resulting executable code.
Certificate of Conformance. The certificate issued by the EAC when a system has been found to
meet the requirements of the VVSG. The document conveys certification of a system.
Commercial Off-the-Shelf. Software, firmware, device or component that is used in the United
States by many different people or organizations for many different applications other than
certified voting systems and that is incorporated into the voting system with no manufacturer- or
application-specific modification.
Commission. The U.S. Election Assistance Commission, as an agency.
Commissioners. The serving commissioners of the U.S. Election Assistance Commission.
Component. An identifiable and discrete part of the larger voting system essential to the
operation of the voting system, and an immediate subset of the system to which it belongs.
Compiler. A compiler is a computer program that translates programs expressed in a high-level
language into machine language equivalents.
Days. The term days shall refer to calendar days, unless otherwise noted. When counting days,
for the purpose of submitting or receiving a document, the count shall begin on the first full
calendar day after the day the document was received.
De minimis change order. A de minimis change order is a change to a certified voting system’s
hardware, software, Technical Data Package (TDP), or data, the nature of which will not
materially alter the system’s reliability, functionality, capability, or operation. Any changes made
to a system under test will result in the manufacturer supplying a list and detailed description of
all changes.
Disk Image. An exact copy of the entire contents of a computer disk.
OMB Control Number PENDING
7
EAC Voting System Testing and Certification Program Manual, Version 2.0
Election Official. A State or local government employee who has as one of his or her primary
duties the management or administration of a Federal election.
Federal Election. Any primary, general, runoff, or special election in which a candidate for
Federal office (President, Senator, or Representative) appears on the ballot. In addition, for the
purposes of this Manual, the term will include any and all Pre-Election Testing and Post-Election
Testing and/or auditing done in conjunction with any primary, general, runoff, or special election
involving a candidate for Federal office.
Fielded Voting System. A voting system purchased or leased by a State or local government that
is being use in a Federal election.
File Signature. A file signature, sometimes called a HASH value, creates a value that is
computationally infeasible of being produced by two similar but different files. File signatures, a
set of files produced using HASH algorithm, are used to verify that files are unmodified from
their original version.
HASH Algorithm. An algorithm that maps a bit string of arbitrary length to a shorter, fixedlength bit string. (A HASH uniquely identifies a file similar to the way a fingerprint identifies an
individual. Likewise, as an individual cannot be recreated from his or her fingerprint, a file
cannot be recreated from a HASH. The HASH algorithm used primarily in the NIST National
Software Reference Library (NSRL), and this program, is the Secure HASH Algorithm (SHA-1)
specified in Federal Information Processing Standard (FIPS) 180-1.)
Installation Device. A device containing program files, software, and installation instructions for
installing an application (program) onto a computer. Examples of such devices include
installation disks, flash memory cards, and PCMCIA cards.
Integration Testing. The end-to-end testing of a full system configured for use in an election to
assure that all legitimate configurations meet applicable standards.
Lines of Code. Any executable statements, flow control statements, formatting (e.g. Blank lines)
and comments.
Linker. A computer program that takes one or more objects generated by compilers and
assembles them into a single executable program.
Manufacturer. The entity with ownership and control over a voting system submitted for
certification.
Mark of Conformance. A uniform notice permanently posted on a voting system signifying it is
EAC certified.
OMB Control Number PENDING
8
EAC Voting System Testing and Certification Program Manual, Version 2.0
Memorandum for the Record. A written statement drafted to document an event or finding,
without a specific addressee other than the pertinent file.
Modification. A modification is any change to a previously EAC‐certified voting system’s hardware,
software, or firmware that is not classified as a de minimis change order.
Proprietary Information. Commercial information or trade secrets protected from release under
the Freedom of Information Act and the Trade Secrets Act.
Sub-assembly. A major functional piece of equipment essential to the operational completeness
of a component of a voting system. Examples of major sub-assemblies for voting systems
include, but are not limited to:
• Printers
• Touch screen terminals
• Scanners/Tabulators
• Card readers
• Ballot boxes
• Keyboards
• CPUs
• Memory modules, USB drives, and other portable memory devices
• External data storage devices, external hard drives, etc.
• Motherboards, processor board and other PWB assemblies, when supplied separately
from a complete unit
System Identification Tools. Tools created by a Manufacturer of voting systems which allow
elections officials to verify that the hardware and software of systems purchased are identical to
the systems certified by the EAC.
Technical Reviewers. Experts in the area of voting system technology and conformity assessment
appointed by the EAC to provide expert guidance.
Testing and Certification Decision Authority. The EAC Executive Director or Acting Executive
Director.
Testing and Certification Program Director. The individual appointed by the EAC Executive
Director to administer and manage the Testing and Certification Program.
Trusted Build. A witnessed software build where source code is converted into machinereadable binary instructions (executable code) in a manner providing security measures which
help ensure that the executable code is a verifiable and faithful representation of the source code.
Voting System. The total combination of mechanical, electromechanical, and electronic
equipment (including the software, firmware, and documentation required to program, control,
OMB Control Number PENDING
9
EAC Voting System Testing and Certification Program Manual, Version 2.0
and support the equipment) used to define ballots; cast and count votes; report or display
election results; connect the voting system to the voter registration system; and maintain and
produce any audit trail information.
Voting System Test Laboratories. Independent testing laboratories accredited by the EAC to test
voting systems to EAC approved voting system standards. Each Voting System Test Laboratory
(VSTL) must be accredited by the National Voluntary Laboratory Accreditation Program
(NVLAP) and recommended by the National Institute of Standards Technology before it may
receive an EAC accreditation. NVLAP provides third party accreditation to testing and
calibration laboratories. NVLAP is in full conformance with the standards of the International
Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC),
including ISO/IEC Guide 17025 and 17011.
Voluntary Voting System Guidelines. Voluntary voting system standards developed, adopted,
and published by the EAC. The guidelines are identified by version number and date.
1.17. Acronyms and Abbreviations. For purposes of this Manual, the acronyms and
abbreviations listed below represent the following terms.
Certification Program. The EAC Voting System Testing and Certification Program
COTS. Commercial Off-the-Shelf
Decision Authority. Testing and Certification Decision Authority
EAC. United States Election Assistance Commission
HAVA. Help America Vote Act of 2002 (42 U.S.C. §15301 Et seq.)
Labs or Laboratories. Voting System Test Laboratories
LOC. Lines of Code
NASED. National Association of State Election Directors
NIST. National Institute of Standards and Technology
NVLAP. National Voluntary Laboratory Accreditation Program
Program Director. Director of the EAC’s Testing and Certification Program
VSS. Voting System Standards
OMB Control Number PENDING
10
EAC Voting System Testing and Certification Program Manual, Version 2.0
VSTL. Voting System Test Laboratory
VVSG. Voluntary Voting System Guidelines
OMB Control Number PENDING
11
EAC Voting System Testing and Certification Program Manual, Version 2.0
2. Manufacturer Registration
2.1. Overview. Manufacturer Registration is the process by which voting system Manufacturers
make initial contact with the EAC and provide information essential to participate in the EAC
Voting System Testing and Certification Program. Before a Manufacturer of a voting system
can submit an application to have a voting system certified by the EAC, the Manufacturer
must be registered. This process requires the Manufacturer to provide certain contact
information and agree to certain requirements of the Certification Program. After successfully
registering, the Manufacturer will receive an identification code.
2.2. Registration Required. To submit a voting system for certification or otherwise participate in
the EAC voluntary Voting System Testing and Certification Program, a Manufacturer must
register with the EAC. Registration does not constitute an EAC endorsement of the
Manufacturer or its products. Registration of a Manufacturer is not a certification of that
Manufacturer’s products.
2.3. Registration Requirements. The registration process will require the voting system
Manufacturer to provide certain information to the EAC. This information is necessary to
enable the EAC to administer the Certification Program and communicate effectively with the
Manufacturer. The registration process also requires the Manufacturer to agree to certain
Certification Program requirements. These requirements relate to the Manufacturer’s duties
and responsibilities under the program. For this program to succeed, it is vital that a
Manufacturer know and assent to these duties at the outset of the program.
2.3.1. Information. Manufacturers are required to provide the following information.
2.3.1.1. The Manufacturer’s organizational information:
2.3.1.1.1.
The official name of the Manufacturer.
2.3.1.1.2.
The address of the Manufacturer’s official place of business.
2.3.1.1.3.
A description of how the Manufacturer is organized (i.e., type of
corporation or partnership).
2.3.1.1.4.
Names of officers and/or members of the board of directors.
2.3.1.1.5.
Names of all partners and members (if organized as a partnership or
limited liability corporation).
2.3.1.1.6.
Identification of any individual, organization, or entity with a
controlling ownership interest in the Manufacturer.
OMB Control Number PENDING
12
EAC Voting System Testing and Certification Program Manual, Version 2.0
2.3.1.2. The identity of an individual authorized to represent and make binding
commitments and management determinations for the Manufacturer
(management representative). The following information is required for the
management representative:
2.3.1.2.1.
Name and title.
2.3.1.2.2.
Mailing and physical addresses.
2.3.1.2.3.
Telephone number, fax number, and e-mail address.
2.3.1.3. The identity of an individual authorized to provide technical information on
behalf of the Manufacturer (technical representative). The following information
is required for the technical representative:
2.3.1.3.1.
Name and title.
2.3.1.3.2.
Mailing and physical addresses.
2.3.1.3.3.
Telephone number, fax number, and e-mail address.
2.3.1.4. The Manufacturer’s written policies regarding its quality assurance system. This
policy must be consistent with guidance provided in the VVSG and this Manual.
2.3.1.5. The Manufacturer’s written policies regarding internal procedures for
controlling and managing changes to, and versions of, its voting systems. Such
polices shall be consistent with this Manual and guidance provided in the VVSG.
2.3.1.6. The Manufacturer’s written policies on document retention. Such policies must
be consistent with the requirements of this Manual.
A list of all manufacturing and/or assembly facilities used by the Manufacturer
and the name and contact information of a person at each facility. The term
“manufacturing and/or assembly facilities” applies to facilities that provide the
following manufacturing services:
2.3.1.6.1
Final system configuration and loading of programs for customer
delivery.
2.3.1.6.2.
Manufacturing of component units of the voting system.
2.3.1.6.3.
Manufacturing of major sub-assemblies of the voting system.
OMB Control Number PENDING
13
EAC Voting System Testing and Certification Program Manual, Version 2.0
Manufacturing facilities for COTS components and plastic modeling facilities are
not included in this definition and need not be reported to the EAC. The EAC
reserves the right to request additional information from manufacturers related
to the manufacturing process, including manufacturing facilities for the benefit of
the testing and certification program.
Manufacturers shall report all current facilities that meet the above criteria. If
manufacturing is not in progress at the time of a manufacturer’s submission of
their registration package to the EAC, the manufacturer shall report the last
manufacturing facility which meets the definitions in this section. Manufacturers
should also be aware that the reporting requirement is continuous and that when
new manufacturing facilities are engaged, the EAC registration package
submitted to the EAC must be updated to reflect the new facilities as required by
Section 2.5.2 of this Manual.
2.3.1.7. The following information is required for a person at each facility:
2.3.1.7.1.
Name and title.
2.3.1.7.2.
Mailing and physical addresses.
2.3.1.7.3.
Telephone number, fax number, and e-mail address.
2.3.2. Agreements. Manufacturers are required to take or abstain from certain actions to protect
the integrity of the Certification Program and promote quality assurance. Manufacturers are
required to agree to the following program requirements:
2.3.2.1. Represent a voting system as certified only when it is authorized by the EAC
and is consistent with the procedures and requirements of this Manual.
2.3.2.2. Produce and affix an EAC certification label to all production units of the
certified system. Such labels must meet the requirements set forth in Chapter 5 of
this Manual.
2.3.2.3. Notify the EAC of changes to any system previously certified by the EAC
pursuant to the requirements of this Manual (see Chapter 3). Such systems shall
be submitted for testing and additional certification when required.
2.3.2.4. Permit an EAC representative to verify the Manufacturer’s quality control by
cooperating with EAC efforts to test and review fielded voting systems
consistent with Section 8.6 of this Manual.
OMB Control Number PENDING
14
EAC Voting System Testing and Certification Program Manual, Version 2.0
2.3.2.5. Permit an EAC representative to verify the Manufacturer’s quality control by
conducting periodic inspections of manufacturing facilities consistent with
Chapter 8 of this Manual.
2.3.2.6. Cooperate with any EAC inquiries and investigations into a certified system’s
compliance with VVSG standards or the procedural requirements of this Manual
consistent with Chapter 7.
2.3.2.7. Report to the Program Director any known malfunction of a voting system
holding an EAC Certification. A malfunction is a failure of a voting system, not
caused solely by operator or administrative error, which causes the system to not
function as expected during a Federal election or otherwise results in data loss.
Initial Malfunction Reports should identify the location, nature, date, impact, and
status of resolution (if any) of the malfunction and be filed within 30 business
days of occurrence during or in preparation for a Federal election, as defined in
this Manual. Final malfunction Reports shall be submitted to the EAC after the
root cause of the malfunction has been determined and a permanent fix
developed.
2.3.2.8. Report to the Program Director the names of each State and/or local jurisdiction
using an EAC certified voting system within 5 business days of delivery of the
first production unit of the voting system to the jurisdiction.
2.3.2.9. Certify the entity is not barred or otherwise prohibited by statute, regulation, or
ruling from doing business in the United States.
2.3.2.10. Agree to participate in a Kick-off Meeting at the beginning of every
certification effort. The purposes of these meetings are to permit an in-depth
discussion of the candidate voting system and allow both the EAC and the VSTL
staff to have a live, hands-on demonstration of the voting system. The duration
of this meeting shall be mutually agreed upon by all parties, but shall not be less
than one business day. Topics of discussion during this meeting include, but are
not limited to:
•
•
•
•
2.3.2.11.
System architecture
System security design
System data flows
System limits
Adhere to all procedural requirements of this Manual.
OMB Control Number PENDING
15
EAC Voting System Testing and Certification Program Manual, Version 2.0
2.4. Registration Process. Generally, registration is accomplished through use of the EAC
registration form. After the EAC has received a registration form and other required
registration documents, the agency reviews the information for completeness before approval.
2.4.1. Application Process. To become a registered voting system Manufacturer, interested parties
must apply by submitting a Manufacturer Registration Application form (Appendix A).
This form will be used as the means for the Manufacturer to provide the information and
agree to the responsibilities required in Section 2.3, above.
2.4.1.1. Application Form. In order for the EAC to accept and process the registration
form, the applicant must adhere to the following requirements:
2.4.1.1.1.
All fields must be completed by the Manufacturer.
2.4.1.1.2.
All required attachments prescribed by the form and this Manual
must be identified, completed, and forwarded in a timely manner to
the EAC (e.g., Manufacturer’s quality control and system change
policies).
2.4.1.1.3.
The application form must be affixed with the hand written signature
(including a digital representation of the handwritten signature) of
the authorized representative of the vendor.
2.4.1.2. Availability and Use of the Form. The Manufacturer Registration Application Form
may be accessed through the EAC’s website at www.eac.gov. Instructions for
completing and submitting the form are included on the website along with
contact information regarding questions about the form or the application
process.
2.4.2. EAC Review Process. The EAC will review all registration applications.
2.4.2.1. After the application form and required attachments have been submitted, the
applicant will receive an acknowledgment that the EAC has received the
submission and that the application will be processed.
2.4.2.2. If an incomplete form is submitted, or an attachment is not provided, the EAC
will notify the Manufacturer and request the omitted information. Registration
applications will not be processed until they are deemed complete.
2.4.2.3. Upon receipt of the completed registration form and accompanying
documentation, the EAC will review the information for sufficiency. If the EAC
OMB Control Number PENDING
16
EAC Voting System Testing and Certification Program Manual, Version 2.0
requires clarification or additional information, the EAC will contact the
Manufacturer and request the needed information.
2.4.2.4. Upon the determination that an application has been satisfactorily completed,
the EAC will notify the Manufacturer that it has been registered.
2.5. Registered Manufacturers. After a Manufacturer has received notice that it is registered, it
will receive an identification code and will be eligible to participate in the voluntary voting
system Certification Program.
2.5.1. Manufacturer Code. Registered Manufacturers will be issued a unique, three-letter
identification code. This code will be used to identify the Manufacturer and its products.
2.5.2. Continuing Responsibility to Report. Registered Manufacturers are required to keep all
registration information up to date. Manufacturers must submit a revised application form
to the EAC within 30 days of any changes to the information required on the application
form. Manufacturers will remain registered participants in the program during this update
process.
2.5.3. Program Information Updates. Registered Manufacturers will automatically be provided
timely information relevant to the Certification Program.
2.5.4. Website Postings. The EAC will add the Manufacturer to the EAC’s listing of registered
voting system Manufacturers publicly available at www.eac.gov.
2.6. Suspension of Registration. Manufacturers are required to establish policies and operate
within the EAC Certification Program consistent with the procedural requirements presented
in this Manual. When Manufacturers violate the certification program’s requirements by
engaging in management activities inconsistent with this Manual or failing to cooperate with
the EAC, their registration may be suspended until such time as the issue is remedied.
2.6.1
Procedures. When a Manufacturer’s activities violate the procedural requirements of this
Manual, the Manufacturer will be notified of the violations, given an opportunity to
respond, and provided with the suggested steps to bring itself into compliance.
2.6.1.1. Notice. Manufacturers shall be provided written notice that they have taken
action inconsistent with or acted in violation of the requirements of this Manual.
The notice will state the violations and the specific steps required to cure them.
The notice will also provide Manufacturers with 30 days (or a greater period of
time as stated by the Program Director) to (1) respond to the notice and/or (2)
cure the defect.
OMB Control Number PENDING
17
EAC Voting System Testing and Certification Program Manual, Version 2.0
2.6.1.2. Manufacturer Action. The Manufacturer is required to either respond in a timely
manner to the notice (demonstrating it was not in violation of program
requirements) or cure the violations identified in a timely manner. The steps
required to cure a violation will include addressing the direct violation and the
underlying root cause. In any case, the Manufacturer’s action must be approved
by the Program Director to prevent suspension.
2.6.1.3. Non-Compliance. If the Manufacturer fails to respond in a timely manner, is
unable to provide a cure or response that is acceptable to the Program Director,
or otherwise refuses to cooperate, the Program Director may suspend the
Manufacturer’s registration. The Program Director shall issue a notice of his or
her intent to suspend and provide the Manufacturer five (5) business days to
object to the action and submit information in support of the objection.
2.6.1.4. Suspension. After notice and opportunity to be heard (consistent with the above),
the Program Director may suspend a Manufacturer’s registration. The
suspension shall be provided in writing and must inform the Manufacturer of
the steps available to remedy the violations and lift the suspension.
2.6.2. Effect of Suspension. A suspended Manufacturer may not submit a voting system for
certification under this program. This prohibition includes a ban on the submission of
modifications and changes to certified system. A suspension shall remain in effect until
lifted. Suspended Manufacturers will have their registration status reflected on the EAC
website. Manufacturers have the right to remedy a non-compliance issue at any time and
lift a suspension consistent with EAC guidance. Failure of a Manufacturer to follow the
requirements of this section may also result in Decertification of voting systems consistent
with Chapter 7 of this Manual.
OMB Control Number PENDING
18
EAC Voting System Testing and Certification Program Manual, Version 2.0
3. When Voting Systems Must Be Submitted for Testing and
Certification
3.1. Overview. An EAC certification signifies that a voting system has been successfully tested
against an identified voting system standard adopted by the EAC. Only the EAC can issue a
Federal certification. Ultimately, systems must be submitted for testing and certification under
this program to receive this certification. Systems will usually be submitted when (1) they are
new to the marketplace, (2) they have never before received an EAC certification, (3) they are
modified, or (4) the Manufacturer wishes to test a previously certified system to a different
(newer) standard. This chapter discusses the submission of de minimis change orders, which
may not require additional testing and certification. Additionally, this chapter outlines
provisional, pre-election emergency modifications, which provide for pre-election, emergency
waivers.
3.2. EAC Certification. Certification is the process by which the EAC, through testing and
evaluation conducted by an accredited Voting System Test Laboratory, validates that a voting
system meets the requirements set forth in existing voting system testing standards (VVSG),
and performs according to the Manufacturer’s specifications for the system. An EAC
certification may be issued only by the EAC in accordance with the procedures presented in
this Manual. Certifications issued by other bodies (e.g., the National Association of State
Election Directors and State certification programs) are not EAC certifications.
3.2.1. Types of Voting Systems Certified. The EAC Certification Program is designed to test and
certify electromechanical and electronic voting systems. Ultimately, the determination of
whether a voting system may be submitted for testing and certification under this program
is solely at the discretion of the EAC.
3.2.2. Voting System Standards. Voting systems certified under this program are tested to a set of
voluntary standards providing requirements that voting systems must meet to receive a
Federal certification. These standards are referred to as Voluntary Voting System
Guidelines (VVSG).
3.2.2.1. Versions—Availability and Identification. Voluntary Voting System Guidelines are
published by the EAC and are available on the EAC’s website (www.eac.gov).
The standards will be routinely updated and versions will be identified by
version number and/or release date.
3.2.2.2. Versions—Basis for Certification. The EAC will promulgate which version or
versions of the standards it will accept as the basis for testing and certification.
This effort may be accomplished through the setting of an implementation date
for a particular version’s applicability, the setting of a date by which testing to a
particular version is mandatory or the setting of date by which the EAC will no
longer test to a particular standard. The EAC will certify only those voting
OMB Control Number PENDING
19
EAC Voting System Testing and Certification Program Manual, Version 2.0
systems tested to standards that the EAC has identified as valid for
certification.
3.2.2.2.1.
End date. When a version’s status as the basis of an EAC certification
is set to expire on a certain date, the submission of the system’s test
report will be the controlling event (see Chapter 4). This requirement
means the system’s test report must be received by the EAC on or
before the expiration date to be certified to the terminating standard.
3.2.2.2.2.
Start date. When a version’s status as the basis of an EAC
certification is set to begin on a certain date, the submission of the
system’s application for certification will be the controlling event (see
Chapter 4). This requirement means the system’s application,
requesting certification to the new standard, will not be accepted by
the EAC until the start date.
3.2.2.3. Version—Manufacturer’s Option. When the EAC has authorized the option of
certification to more than one version of the standards, the Manufacturer must
choose which version it wishes to have its voting system tested against. The
voting system will then be certified to that version of the standards upon
successful completion of testing. Manufacturers must ensure all applications for
certification identify a particular version of the standards.
3.2.2.4. Emerging Technologies. If a voting system or component thereof is eligible for a
certification under this program (see Section 3.2.1.) and employs technology that
is not addressed by a currently accepted version of the VVSG the relevant
technology shall be subjected to full integration testing and shall be tested to
ensure that it operates to the Manufacturer’s specifications and that the proper
security risk assessments and quality assurance processes are in place. The
Technology Testing Agreement (TTA) process described below is intended to
provide additional clarification and guidance to enhance the testing and
certification process for voting systems incorporating new or emerging
technology. The remainder of the system will be tested to the applicable Federal
standards. Information on emerging technologies will be forwarded to the
EAC’s Technical Guidelines Development Committee (TGDC).
3.2.2.4.1.
TTA Meeting.
3.2.2.4.1.1. The manufacturer should contact the Certification Division as early
as possible in their design and development process to have a general
discussion regarding new or emerging technology in any voting
system product.
OMB Control Number PENDING
20
EAC Voting System Testing and Certification Program Manual, Version 2.0
3.2.2.4.1.2. A formal request for a TTA Meeting should be: a) clearly identified
as such and b) submitted via email or other secure means to the
Director of the EAC Testing and Certification Division. The EAC
expects that the submission will be as detailed as design and
development allow, but should include the following items:
• Description of the product, highlighting elements involving new
technologies testable requirements and other testing protocol issues.
This description should include, at a minimum:
o General product description
o Engineering drawing(s)
o Product composition/key components/materials
• Device specifications
• Analysis of potential failure modes and threat model/risk analysis
• Outline of the proposed conditions of use
• Summary of instructions for use of the product (voter and poll
worker/election official)
• Relevant performance information on the product, especially if
routinely used in other industries. This information may include:
o Published and/or unpublished data
o Summary of test data
o Summary of prior user experience.
3.2.2.4.1.3. Prior to the formal TTA Meeting, the manufacturer should arrange
for a preliminary meeting (videoconference or teleconference) to
review the submitted information and discuss any additional
questions that may arise prior to the actual formal TTA Meeting. The
manufacturer may then submit any additional information as
required, and finalize the date and time for formal TTA Meeting with
the EAC and VSTL. Because of logistics and budgetary considerations
for all parties, the location of the meeting (EAC, VSTL or manufacturer
location) will be mutually agreed upon. Meeting plans should
generally be finalized within 30 days of the preliminary meeting.
3.2.2.4.1.4. TTA Meetings should generally be face-to-face, or by
videoconference, and should be scheduled for approximately 2-4
hours or longer depending on the complexity of the issues to be
discussed. The EAC and VSTL staff may raise any questions for the
manufacturer about the product, but should be focused on the key
issues of the products test plan development and testing that will
ultimately lead to the Technology Testing Agreement. The Director of
the Certification Division will determine which EAC staff should
attend the meeting, but will generally include: the EAC Project
OMB Control Number PENDING
21
EAC Voting System Testing and Certification Program Manual, Version 2.0
Manager and any or all Technical Reviewers with interest or expertise
in specific areas under discussion.
3.2.2.4.2.
Post TTA Meeting Activities
3.2.2.4.2.1. At the end of the meeting, the EAC Project Manager for the voting
system will summarize the agreement(s) or explain any reasons for
tabling the agreement(s), including the date of any follow-on meeting,
if appropriate. A record of attendees and minutes of the meeting shall
be kept by both a designated EAC staff member and manufacturer
representative. EAC and the manufacturer should exchange their
respective meeting minutes for review following the meeting and
share the minutes with the VSTL. The minutes shall be in sufficient
detail to reflect the substance of the issues discussed at the meeting
and the final agreement.
3.2.2.4.2.2. The EAC Project Manager will prepare a memorandum outlining the
TTA. Within ten (10) business days of the meeting, a draft of the
memorandum should be circulated for comment among all TTA
Meeting participants. Comments shall be returned to EAC in 5
business days. The final memorandum shall be signed by the Director
and conveyed to the applicant and VSTL within 5 business days of the
receipt of final comments.
3.2.3. Significance of an EAC Certification. An EAC certification is an official recognition that a
voting system (in a specific configuration or configurations) has been tested by a VSTL to be
in conformance with an identified set of Federal voting standards. An EAC certification is
not:
3.2.3.1. An official endorsement of a Manufacturer, voting system, or any of the
system’s components.
3.2.3.2. A Federal warranty of the voting system or any of its components.
3.2.3.3. A determination that a voting system, when fielded, will be operated in a
manner that meets all HAVA requirements.
3.2.3.4. A substitute for State or local certification and testing.
3.2.3.5. A determination that any particular component of a certified system is itself
certified for use outside the certified configuration.
OMB Control Number PENDING
22
EAC Voting System Testing and Certification Program Manual, Version 2.0
3.3. When Certification Is Required Under the Program. To obtain or maintain an EAC
certification, Manufacturers must submit a voting system for testing and certification under
this program. Such action is usually required for (1) new systems not previously tested to any
standard; (2) existing systems not previously certified by the EAC; (3) previously certified
systems that have been modified; (4) systems or technology specifically identified for retesting
by the EAC; or (5) previously certified systems that the Manufacturer seeks to upgrade to a
higher standard (e.g., more recent version of the VVSG).
3.3.1. New System Certification. For purposes of this Manual, new systems are defined as voting
systems that have not been previously tested to applicable Federal standards. New voting
systems must be fully tested and submitted to the EAC according to the requirements of
Chapter 4 of this Manual.
3.3.2. System Not Previously EAC Certified. This term describes any voting system not
previously certified by the EAC, including systems previously tested and qualified by
NASED or systems previously tested and denied certification by the EAC. Such systems
must be fully tested and submitted to the EAC according to the requirements of Chapter 4
of this Manual.
3.3.3. Modification. A modification is any change to a previously EAC-certified voting system’s
hardware, software, or firmware that is not a de minimis change. Any modification to a
voting system will require testing and review by the EAC according to the requirements of
Chapter 4 of this Manual.
3.3.4. EAC Identified Systems. Manufacturers may be required to submit systems previously
certified by the EAC for retesting. This may occur when the EAC determines that the
original tests conducted on the voting system are now insufficient to demonstrate
compliance with Federal standards in light of newly discovered threats or information.
3.4. Changes to Voting Systems in the EAC Certification Program – Change Order. A change
order does not apply to a system under test; any changes made to a system under test are
considered part of the test campaign. A single change order can be applied to multiple
systems as long as a VSTL reviews and approves the change order for each EAC certified
system.
3.4.1. Change Order. A change order is a change to a previously EAC certified voting system’s
hardware, documentation, or data that is minor in nature and effect. Such changes,
however, require VSTL review and endorsement as well as EAC approval. Any proposed
change not accepted as a de minimis change is a modification and shall be submitted for
testing and review consistent with the requirements of this Manual.
3.4.2. De Minimis Change – Defined. A de minimis change is a change to a certified voting
system’s hardware, software, TDP, or data, the nature of which will not materially alter the
OMB Control Number PENDING
23
EAC Voting System Testing and Certification Program Manual, Version 2.0
system’s reliability, functionality, capability, or operation. Under no circumstance shall a
change be considered de minimis if it has reasonable and identifiable potential to impact the
system’s performance and compliance with the applicable voting Standard.
3.4.3. De Minimis Change – Procedure. Manufacturers who wish to implement a proposed de
minimis change must submit it for VSTL review and endorsement and EAC approval. A
proposed de minimis change may not be implemented as such until it has been approved in
writing by the EAC.
3.4.3.1. VSTL Review. Manufacturers must submit any proposed de minimis change to a
VSTL and the EAC for review and endorsement. The Manufacturer will provide
the VSTL: (1) a detailed description of the change; (2) a description of the facts
giving rise to or necessitating the change; (3) the basis for its determination that
the change will not alter the system’s reliability, functionality, or operation; (4)
upon request of the VSTL, a sample voting system at issue or any relevant
technical information needed to make the determination; (5) document any
potential impact to election officials currently using the system and any required
notifications to those officials; (6) a description of how this change will impact
any relevant system documentation;; and (7) any other information the EAC or
VSTL needs to make a determination. The VSTL will review the proposed de
minimis change and make an independent determination as to whether the
change meets the definition of de minimis change or requires the voting system
to undergo additional testing as a system modification. If the VSTL determines
that a de minimis change is appropriate, it shall endorse the proposed change as
a de minimis change. If the VSTL determines that modification testing and
certification should be performed, it shall reclassify the proposed change as a
modification. Endorsed de minimis changes shall be forwarded to the Voting
System’s Project Manager for final approval. Rejected changes shall be returned
to the Manufacturer for resubmission as system modifications.
3.4.3.2. VSTL Endorsed Changes. The VSTL shall forward to the EAC any change it has
endorsed as de minimis. The VSTL shall forward its endorsement in a package
that includes:
3.4.3.2.1.
The Manufacturer’s initial description of the de minimis change, a
narrative of facts giving rise to, or necessitating, the change, and the
determination that the change will not alter the system’s reliability,
functionality, or operation.
3.4.3.2.2.
The written determination of the VSTL’s endorsement of the de
minimis change. The endorsement document must explain why the
VSTL, in its engineering judgment, determined that the proposed de
OMB Control Number PENDING
24
EAC Voting System Testing and Certification Program Manual, Version 2.0
minimis change met the definition in this section and otherwise does
not require additional testing and certification.
3.4.3.3. EAC Action. The EAC will review all proposed de minimis changes endorsed by
a VSTL. The EAC has sole authority to determine whether any VSTL endorsed
change constitutes a de minimis change under this section. The EAC will inform
the Manufacturer and VSTL of its determination in writing.
3.4.3.3.1.
If the EAC approves the change as a de minimis change, it shall
provide written notice to the Manufacturer and VSTL. The EAC will
maintain copies of all approved de minimis changes and otherwise
track such changes.
3.4.3.3.2.
If the EAC determines that a proposed de minimis change cannot be
approved, it will inform the VSTL and Manufacturer of its decision.
The proposed change will be considered a modification and require
testing and certification consistent with this Manual. De minimis
changes cannot be made to voting systems currently undergoing
testing; these changes are merely changes to an uncertified system
and may require an Application update.
3.5. Changes to Voting Systems in the EAC Certification Program - Modification.
3.5.1. Modification – Defined. A modification is any change to a previously EAC‐certified voting
system’s hardware, software, or firmware that is not a de minimis change. Any modification
to a voting system will require testing and review by the EAC according to the
requirements of Chapter 4 of this Manual.
3.5.2. Modification – Procedure. Once a VSTL has submitted a modification application, a Test
Plan shall be created and submitted to the EAC for the Test Plan review process. Any
modification shall be subject to full testing of the modifications (delta ‐testing) and those
systems or subsystems altered or impacted by the modification (regression testing). The
system will also be subject to system integration testing to ensure overall functionality.
Once testing is completed, a Test Report will be generated by the VSTL and submitted to
the EAC for approval.
3.5.3. EAC Approval. If the EAC approves the change as a modification, it shall provide written
notice to the Manufacturer and VSTL and generate a Certificate of Conformance. The EAC
will maintain copies of all approved modifications and otherwise track such changes
OMB Control Number PENDING
25
EAC Voting System Testing and Certification Program Manual, Version 2.0
3.5.4. EAC Denial. If the EAC determines that a modification cannot be approved, it will inform
the VSTL and Manufacturer of its decision. The Denial of Certification appeals process
would govern this testing campaign.
3.5.5. Modification Change – Effect of EAC Approval. EAC approval of a modification permits
the Manufacturer to implement the proposed change (as identified, endorsed, and
approved). Fielding a change not approved by the EAC is a basis for system
Decertification.
3.6. Provisional, Pre-Election Emergency Modification. To address extraordinary pre-election
emergency situations, the EAC has developed a special provisional modification process. This
process is to be used only for the emergency situations indicated and only when there is a
clear and compelling need for temporary relief until the regular certification process can be
followed.
3.6.1. Purpose. The purpose of this section is to allow for a mechanism within the EAC
Certification Program for Manufacturers to modify EAC-certified voting systems in
emergency situations immediately before an election. This situation arises when a
modification to a voting system is required and an election deadline is imminent,
preventing the completion of the full certification process (and State and/or local testing
process) prior to Election Day. In such situations, the EAC may issue a waiver to the
Manufacturer authorizing it to make the modification without submission for modification
testing and certification.
3.6.2. General Requirements. A request for an emergency modification waiver may be made by a
Manufacturer only in conjunction with the State election official whose jurisdiction(s) would
be adversely affected if the requested modification were not implemented before Election
Day. Requests must be submitted at least 5 calendar days before an election. Only systems
previously certified are eligible for such a waiver. To receive a waiver, a Manufacturer must
demonstrate the following:
3.6.2.1. The modification is functionally or legally required; that is, the system cannot be
fielded in an election without the change.
3.6.2.2. The voting system requiring modification is needed by State or local election
officials to conduct a pending Federal election.
3.6.2.3. The voting system to be modified has previously been certified by the EAC.
3.6.2.4. The modification cannot be tested by a VSTL and submitted to the EAC for
certification, consistent with the procedural requirements of this Manual, at least
30 days before the pending Federal election.
OMB Control Number PENDING
26
EAC Voting System Testing and Certification Program Manual, Version 2.0
3.6.2.5. Relevant State law requires Federal certification of the requested modification.
3.6.2.6. The Manufacturer has taken steps to ensure the modification will properly
function as designed, is suitably integrated with the system, and otherwise will
not negatively affect system reliability, functionality, or accuracy.
3.6.2.7. The Manufacturer (through a VSTL) has completed as much of the evaluation
testing as possible for the modification and has provided the results of such
testing to the EAC.
3.6.2.8. The emergency modification is required and otherwise supported by the Chief
State Election Official seeking to field the voting system in an impending Federal
election.
3.6.3. Request for Waiver. A Manufacturer’s request for waiver shall be made in writing to the
Decision Authority and shall include the following elements:
3.6.3.1. A signed statement providing sufficient description, background, information,
documentation, and other evidence necessary to demonstrate that the request for
a waiver meets each of the eight requirements stated in Section 3.6.2 above.
3.6.3.2. A signed statement from the Chief State Election Official requiring the
emergency modification. This signed statement shall identify the pending
election creating the emergency situation and attest that (1) the modification is
required to field the system, (2) State law (citation) requires EAC action to field
the system in an election, and (3) normal timelines required under the EAC
Certification Program cannot be met.
3.6.3.3. A signed statement from a VSTL stating there is insufficient time to perform
necessary testing and complete the certification process. The statement shall also
state what testing the VSTL has performed on the modification to date, provide
the results of such tests, and state the schedule for the completion of testing.
3.6.3.4. A detailed description of the modification, the need for the modification, how it
was developed, how it addresses the need for which it was designed, its impact
on the voting system, and how the modification will be fielded or implemented
in a timely manner consistent with the Manufacturer’s quality control program.
3.6.3.5. All documentation of tests performed on the modification by the Manufacturer,
a laboratory, or other third party.
3.6.3.6. A stated agreement signed by the Manufacturer’s representative agreeing to
take the following action:
OMB Control Number PENDING
27
EAC Voting System Testing and Certification Program Manual, Version 2.0
3.6.3.6.1.
Submit for testing and certification, consistent with Chapter 4 of this
Manual, any voting system receiving a waiver under this section that
has not already been submitted. This action shall be taken
immediately.
3.6.3.6.2.
Abstain from representing the modified system as EAC certified. The
modified system has not been certified; rather, the originally certified
system has received a waiver providing the Manufacturer a
temporary exemption allowing its modification.
3.6.3.6.3.
Submit a report to the EAC regarding the performance of the
modified voting system within 60 days of the Federal election that
served as the basis for the waiver. This report shall (at a minimum)
identify and describe any (1) performance failures, (2) technical
failures, (3) security failures, and/or (4) accuracy problems.
3.6.4. EAC Review. The EAC will review all waiver requests submitted in a timely manner and
make determinations regarding the requests. Incomplete requests will be returned for
resubmission with a written notification regarding its deficiencies.
3.6.5. Letter of Approval. If the EAC approves the modification waiver, the Decision Authority
shall issue a letter granting the temporary waiver within five (5) business days of receiving
a complete request.
3.6.6. Effect of Grant of Waiver. An EAC grant of waiver for an emergency modification is not an
EAC certification of the modification. Waivers under this program grant Manufacturers
leave to only temporarily amend previously certified systems without testing and
certification for the specific election noted in the request. Without such a waiver, such
action would ordinarily result in Decertification of the modified system (See Chapter 7).
Systems receiving a waiver shall satisfy any State requirement that a system be nationally or
federally certified. In addition—
3.6.6.1. All waivers are temporary and expire 60 days after the Federal Election for
which the system was modified and the waiver granted.
3.6.6.2. Any system granted a waiver must be submitted for testing and certification.
This shall be accomplished as soon as possible.
3.6.6.3. The grant of a waiver does not predispose the modified system to being granted
a certification.
OMB Control Number PENDING
28
EAC Voting System Testing and Certification Program Manual, Version 2.0
3.6.7. Denial of Request for Waiver. A request for waiver may be denied by the EAC if the request
does not meet the requirements noted above, fails to follow the procedure established by
this section, or otherwise fails to sufficiently support a conclusion that the modification at
issue is needed, will function properly, and is in the public interest. A denial of a request
for an emergency modification by the EAC shall be final and not subject to appeal.
Manufacturers may submit for certification, consistent with Chapter 4 of this Manual,
modifications for which emergency waivers were denied.
3.6.8. Publication Notice of Waiver. The EAC will post relevant information relating to the
temporary grant of an emergency waiver on its website. This information will be posted
upon grant of the waiver and removed upon the waiver’s expiration. This posting will
include information concerning the limited nature and effect of the waiver.
OMB Control Number PENDING
29
EAC Voting System Testing and Certification Program Manual, Version 2.0
4. Certification Testing and Technical Review
4.1. Overview. This chapter discusses the procedural requirements for submitting a voting system
to the EAC for testing and review. The testing and review process requires an application,
employment of an EAC-accredited testing laboratory, and technical analysis of the laboratory
test report by the EAC. The result of this process is an Initial Decision on Certification by the
Decision Authority.
4.2. Policy. Generally, to receive an initial determination on an EAC certification for a voting
system, a registered Manufacturer must have (1) submitted an EAC-approved application for
certification, (2) had a VSTL submit an EAC-approved test plan, (3) had a VSTL test a voting
system to applicable voting system standards, (4) had a VSTL submit a test report to the EAC
for technical review and approval, and (5) received EAC approval of the report in an Initial
Decision on Certification.
4.3. Certification Application. The first step in submitting a voting system for certification is
submission of an application package. The nature of the submission will determine what
information is required from the manufacturer. Manufacturers must identify the nature of
their submission by selecting one of two submission types:
•
New system. A new system is a voting system not previously certified by the EAC. The
application package must contain an application form, Summative Usability Testing
Report and the Test Readiness Review.
•
Modification. A modification is a change to a system previously certified by the EAC. A
modification does not include de minimis changes to the system. The application package
must contain an application form, and updated Summative Usability Testing Report (if
modification impacts usability).
Manufacturers must use the appropriate application form for submitting a voting system for
testing. Any submission that is not on the EAC provided form will be rejected. In addition, a
manufacturer must submit a complete certification application package and receive
notification from the EAC that it is accepted prior to conducting any certification testing. Any
testing occurring after the execution of a contract or agreement for certification testing (not
including the Test Readiness Review) between a VSTL and a registered manufacturer is
presumed to be certification testing.
4.3.1. Information on Application Form- New System. The application provides the EAC with
essential information at the outset of the certification process. This information includes:
4.3.1.1. Manufacturer Information. Identification of the Manufacturer (name and threeletter identification code).
OMB Control Number PENDING
30
EAC Voting System Testing and Certification Program Manual, Version 2.0
4.3.1.2. Selection of Accredited Laboratory. Selection and identification of the VSTL that
will perform voting system testing and other prescribed laboratory action
consistent with the requirements of this Manual. Once selected, a Manufacturer
may NOT replace the selected VSTL without the express written consent of the
Program Director. Such permission is granted solely at the discretion of the
Program Director and only upon demonstration of good cause.
4.3.1.3. Voting System Standards Information. Identification of the VVSG, including the
document’s date and version number, to which the Manufacturer wishes to have
the identified voting system tested and certified.
4.3.1.4. Identification of the Voting System/system overview. Manufacturers must identify
the system submitted for testing by providing its name and version number. If
the system submitted was previously fielded, but the Manufacturer wishes to
change its name or version number after receipt of EAC certification, it must
provide identification information on both the past name or names and the new,
proposed name.
4.3.1.4.1.
Separate identification of each device that is part of the voting
system. This includes all COTS components. A keyboard, mouse,
accessibility peripheral or printer connected to a programmed voting
device, as well as any optical drive, hard drive or similar component
installed within it, are considered components of the voting device,
not separate devices.
4.3.1.5. Voting variations. Manufactures must identify the voting variations supported
by the voting system. These variations are described in Volume 1 Section 2.1.7.2
of the 2005 VVSG.
4.3.1.6. Languages support. The electronic display or printed document on which the
user views the ballot must be capable of rendering an image of the ballot in any
of the languages required by the Voting Rights Act of 1965, as amended.
4.3.1.7. List of accessibility capabilities. Manufacturers must provide a detailed explanation
of the accessibility capabilities present in their system
4.3.1.8. Device capacities and limits.
4.3.1.9. Coding Standard. Each voting system component must have a single coding
convention selected for every programming language used in the voting system.
This information must include:
• System Component
• Language Used
• Specified Coding Convention
OMB Control Number PENDING
31
EAC Voting System Testing and Certification Program Manual, Version 2.0
•
Source of Coding Convention
4.3.1.10. Functional Diagrams. Diagram(s) that display all components and how the
components relate and interact in each configuration.
4.3.1.11. Certification Number. Manufacturers must provide their desired EAC
certification number.
4.3.1.12. Test Readiness Review. Manufacturers must submit confirmation that a test
readiness review has been completed by a VSTL per Section 4.5 of this manual.
4.3.1.13. Date Submitted. Manufacturers must note the date the application was
submitted for EAC approval.
4.3.1.14. Signature. The Manufacturer must affix the signature of the authorized
management representative.
4.3.2. Information on Application Form- Modification. If submitting an application for
modification, the application must contain:
4.3.2.1. Manufacturer Information. Identification of the Manufacturer (name and threeletter identification code).
4.3.2.2. Selection of Accredited Laboratory. Selection and identification of the VSTL that
will perform voting system testing and other prescribed laboratory action
consistent with the requirements of this Manual. Once selected, a Manufacturer
may NOT replace the selected VSTL without the express written consent of the
Program Director. Such permission is granted solely at the discretion of the
Program Director and only upon demonstration of good cause.
4.3.2.3. Voting System Standards Information. Identification of the VVSG, including the
document’s date and version number, to which the Manufacturer wishes to have
the identified voting system tested and certified.
4.3.2.4. Manufacturers must provide a detailed overview of the modification containing:
4.3.2.4.1.
Modified system components
4.3.2.4.2.
Component version numbers
4.3.2.4.3.
Detailed description of the change(s)
4.3.2.4.4.
Listing of all TDP documents impacted by the change
OMB Control Number PENDING
32
EAC Voting System Testing and Certification Program Manual, Version 2.0
4.3.2.4.5.
Usability impact
4.3.2.4.6.
Functional diagram(s) that display all components and how the
components relate and interact in each configuration if impacted by
modification.
4.3.2.5. Certification Number. Manufacturers must provide their desired EAC certification
number.
4.3.2.6. Date Submitted. Manufacturers must note the date the application was submitted
for EAC approval.
4.3.2.7. Signature. The Manufacturer must affix the signature of the authorized
management representative.
4.3.3. Submission of the Application Package. Manufacturers must submit a copy of the
application form described above and the required additional information. Manufacturers
must submit the required information in a concise and efficient manner.
4.3.3.1. Submission. Applications and accompanying documentation must be submitted
in Adobe PDF, Microsoft Word, or other electronic format as prescribed by the
Program Director. Applications must be submitted via the VRT.
4.4. EAC Review. Upon receipt of a Manufacturer’s application package, the EAC will review the
submission for completeness and accuracy. The manufacturer will be notified of acceptance or
rejection of the application package within five business days of the EAC’s receipt of the
application. If the application package is incomplete or inaccurate, the EAC will return it to
the Manufacturer with instructions for resubmission. If the form submitted is acceptable, the
Manufacturer will be notified and provided a unique application number.
4.5. Test Readiness Review. The Test Readiness Review (TRR) is the mechanism used by the EAC
to ensure that test and evaluation resources are not committed to a voting system that is not
ready for testing by a VSTL. The TRR determines if the submitted voting system and
documentation are ready to enter certification testing. The TRR shall be completed by the
VSTL and the subsequent Test Readiness Acknowledgement must be received by the EAC
prior to the initiation of any certification testing. To assess the readiness of a voting system for
certification testing, the VSTL shall review:
• System Technical Data Package (TDP): The voting system technical data package shall be
reviewed to ensure all elements required by the VVSG are present.
• System Components: The VSTL shall review the submitted voting system to ensure all
components required to configure the voting system as defined in the system TDP are
delivered to the VSTL and appear to be operational and in good working order. System
OMB Control Number PENDING
33
EAC Voting System Testing and Certification Program Manual, Version 2.0
•
•
•
Component information should match the Manufacturer’s application submitted to the
EAC. All components submitted for testing must be equivalent to the final production
model of the voting system in fit, form and function. Any component not available at the
time of this review shall be delivered to the VSTL by the voting system manufacturer
within 30 days of the initial TRR, or testing of the system will be halted and the EAC
notified that the system is not ready for testing.
Preliminary Source Code Review: The VSTL shall conduct a preliminary review of no less
than 1% of the total lines of code (LOC) of every software package or product submitted for
testing in order to ensure that the code is mature and does not contain any systematic nonconformities.
Mark Reading: The system shall be able to read a fully filled mark if it is an optical scan
system.
Summary of COTS components. This summary should outline which components of the
voting system are COTS products and shall be updated with each test campaign.
4.5.1. Test Readiness Notification. Upon completion of the TRR, the VSTL shall submit a signed
statement to the EAC confirming that the voting system completed the TRR and the VSTL
determined that the system is ready for certification testing to applicable Voluntary Voting
System Guidelines.
4.5.2. Test Readiness Acknowledgement. Upon receipt of the Test Readiness Notification from the
VSTL, the EAC shall issue an acknowledgement in writing stating that the VSTL and
manufacturer may commence certification testing. This acknowledgement will be issued
within 3 business days of receipt of the Notification. Systems not passing the Test Readiness
Review will be remanded to the manufacturer for additional work as noted in the Test
Readiness Notification.
4.6. Test Plan. The Manufacturer shall authorize the VSTL identified in its application to submit a
test plan directly to the EAC. The test plan shall document the strategy and plan for testing
each section of the applicable version of the VVSG and is to be used as a key tool to manage
the test campaign and to verify that a voting system or component meets all VVSG and
program defined requirements. The test plan shall be written with completeness and clarity
that will allow all constituents to understand what testing will be conducted, to assess each
group of VVSG requirements, and to assure the test plan will remain a living document
throughout the life of the test campaign. The objective is to address each section of the VVSG
in detail, and to clearly and succinctly describe the strategy and/or approach for testing each
section.
4.6.1. Development. An accredited laboratory will develop test plans that use appropriate test
protocols, standards, or test suites developed by the laboratory. Laboratories must use all
applicable protocols, standards, or test suites issued by the EAC. Care should be taken to
clearly communicate the scope and requirements of testing, the test strategies, and the
OMB Control Number PENDING
34
EAC Voting System Testing and Certification Program Manual, Version 2.0
resource needs. This information identifies the purpose and boundaries of the test
campaign:
•
•
•
What will be tested,
How it will be tested,
What resources are needed for testing.
Because future events in any test campaign cannot be 100% predicted and controlled, the
initial submission of the test plan is viewed as a baseline that enables periodic updates as
events cause the plan to change. The VSTL is expected to update specific sections of the
plan and resubmit as necessary to enable all stakeholders (Manufacturer, EAC, the public
and states or jurisdictions) to understand and use the test plan. As the Target of
Evaluation changes via Change Orders, component changes, or COTS products change,
the test plan shall be updated since these changes may significantly impact the testing.
These test plan changes might also alter the original schedule and may require an
updated schedule be submitted with the revised test plan. The following are examples of
instances that would likely require updating the test plan:
•
•
•
Changes to the manufacturer’s application for testing.
Engineering Changes that alter the scope or function of the voting system.
Information discovered during testing that change the strategy on how best to test the
voting system.
For the test plan to be an effective, living document it needs to be clear and complete so
stakeholders (VSTL project manager, VSTL testers, third party lab personnel,
manufacturer, EAC, states and jurisdictions) can review the plan and understand what
needs to be done to complete the project. In order to accomplish these goals the following
general topics, which are further defined in the test plan outline later in the document,
shall be included in the test plan.
•
A detailed, comprehensive knowledge of the scope of evaluation
o
o
•
•
That each requirement or set of requirements is going to be evaluated for
compliance
That all features, interfaces and characteristics of the individual devices and the
system are evaluated to applicable standards
Titles of test lab personnel who will be responsible for each aspect of the test
campaign
Detailed project schedule including what the critical path is for timely project
completion
OMB Control Number PENDING
35
EAC Voting System Testing and Certification Program Manual, Version 2.0
•
What test methods will be used to evaluate each section of the standards (more than
one test method may be used for a section)
o
o
o
o
o
What is tested in document review
What is tested in source code review
What is operationally tested
What is tested at a component/subsystem level
What is tested at a system only level
4.6.2.Required Testing. Test plans shall be developed to ensure a voting system is functional and
meets all requirements of the applicable, approved voting system standards. The highest
level of care and vigilance is required to ensure comprehensive test plans are created. A test
plan should ensure the voting system meets all applicable standards and test results, and
other factual evidence of the testing, are clearly documented. System testing must meet all
of the requirements of the VVSG. Generally, full testing will be required of any voting
system applying for certification, regardless of previous certification history.
4.6.2.1. New System. A new system shall be subject to full testing of all hardware and
software according to applicable voting system standards.
4.6.2.2. System Not Previously EAC Certified. A system not previously certified by the
EAC shall be fully tested as a new system.
4.6.2.3. Modification. A modification to a previously EAC-certified voting system shall be
tested in a manner necessary to ensure all changes meet applicable voting system
standards and the modified system (as a whole) will function properly and
reliably. Any system submitted for modification shall be subject to full testing of
the modifications (delta-testing) and those systems or subsystems altered or
impacted by the modification (regression testing). The system will also be subject
to system integration testing to ensure overall functionality. The modification
will be tested to the version or versions of the VVSG currently accepted for
testing and certification by the EAC. This requirement, however, does not mean
that the full system must be tested to such standards. If the system has been
previously certified to a VVSG version deemed acceptable by the EAC (see
Section 3.2.2.2), it may retain that level of certification with only the modification
being tested to the current version(s).
4.6.2.3.1.
OMB Control Number PENDING
Modification Test Plans. Test Plans submitted for modifications to
previously EAC certified voting systems should be brief and
structured to minimize test plan development and review, while
enabling the EAC to maintain solid control of the certification
process. The test plan shall concisely document the strategy and plan
for testing those sections of the VVSG applicable to the modification
36
EAC Voting System Testing and Certification Program Manual, Version 2.0
or modifications submitted. The test plan shall be written with clarity
that will allow all constituents to understand what testing will be
conducted, to verify compliance to the VVSG standard, and to assure
that the test plan will remain a living document throughout the life of
the test campaign for the modification.
Care should be taken to clearly communicate the scope and
requirements of testing, the test strategies, and the resource needs. In
order to accomplish these goals the following general topics shall be
included in all modification test plans.
•
•
•
•
•
•
•
•
•
•
•
•
•
OMB Control Number PENDING
Complete definition of the baseline certified system.
Detailed description of all the engineering changes and/or
modifications to the certified system and why the modification
was implemented.
Cite the standard (VVSG) to which the original system was
certified.
Cite the standard (VVSG) to which the modified system is to be
tested.
Detailed description of which specific components, including
version, are tested to which standard.
An initial assessment of the impact the changes have on the
current system and any previous certification.
An initial assessment of the impact the changes have on TDP
documents.
A table or list indicating how each of the existing NOCs/RFIs will
be addressed and why this plan is valid for this test campaign.
Description of what will be tested (regression) to establish
assurance that the change(s) have no adverse impact on the
compliance, integrity, or the performance of the equipment.
Description of what will be tested (regression) to establish
assurance that the change(s) create no inconsistencies with the
TDP and further are correctly documented and reflected in the
TDP.
A summary of the test methods that will be used to validate
compliance. This summary may include existing, modified or
new test methods, test cases or test sequences.
Titles of test lab personnel who will be responsible for each
aspect of the test campaign.
Detailed project schedule including what the critical path is for
timely project completion.
37
EAC Voting System Testing and Certification Program Manual, Version 2.0
4.6.2.4. EAC Identified Systems. Previously certified systems identified for retesting by
the EAC (see Section 3.4.4) shall be tested as directed by the Program Director
(after consultation with technical experts as necessary).
4.6.2.5. Certification Upgrade. A previously certified system submitted for testing to a
new voting system standard (without modification) shall be tested in a manner
necessary to ensure that the system meets all requirements of the new standard.
The VSTL shall create a test plan that identifies the differences between the new
and old standards and, based upon the differences, fully retest all hardware and
software components affected.
4.6.3. Format. Test labs shall issue test plans consistent with the format outlined in Appendix D of
this document and any applicable EAC guidance.
4.6.4. EAC Approval. All test plans are subject to EAC approval. No test report will be accepted
for technical review unless the test plan on which it is based has been approved by EAC’s
Program Director.
4.6.4.1. Review. All test plans must be reviewed for adequacy by the Program Director.
For each submission, the Program Director will determine whether the test plan
is acceptable or unacceptable. Unacceptable plans will be returned to the VSTL
for further action. Acceptable plans will be approved and appropriate
notifications will be made. Although Manufacturers may direct test labs to begin
testing before approval of a test plan, the Manufacturer bears the full risk that the
test plan (and thus any tests preformed) may be deemed unacceptable.
4.6.4.2. Unaccepted Plans. If a test plan is not accepted, the Program Director will return
the submission to the Manufacturer’s identified VSTL for additional action.
Notice of unacceptability will be provided in writing to the laboratory and
include a description of the deficiencies identified and steps required to remedy
the test plan. A copy of this notice will also be sent to the Manufacturer.
Questions concerning the notice shall be forwarded to the Program Director in
writing. Plans that have not been accepted may be resubmitted for review after
remedial action is taken.
4.6.4.3. Effect of Approval. Approval of a test plan is required before a test report may be
filed. EAC approval of the Test Plan does not mean the EAC accepts
responsibility that efforts described in the Test Plan will produce full and
adequate testing to certify the system. The following discuss the meaning of
EAC’s approval of the Test Plan:
• Approval simply signifies that the tests proposed, if performed properly,
appear to be sufficient to fully test the system. A final determination of
the sufficiency of the testing is a global evaluation based on the test plan,
OMB Control Number PENDING
38
EAC Voting System Testing and Certification Program Manual, Version 2.0
•
•
•
test cases, and test report reviews, as well as the EAC’s Quality
Monitoring Process outlined in Ch. 8 of the Program Manual.
Approval allows the VSTL to begin test case development, testing, and
test report submittal.
A test plan is approved based on information submitted; therefore it is
unknown if relevant information was omitted that would affect the
testing campaign.
The test plan is a living document and is expected to change and be
updated during various phases of the testing life cycle. A final version
that reflects all of the testing completed (TDP, S/W, Hardware, Software
etc) should be submitted to the EAC at the completion of testing. If this
final “as run” test plan does not reflect all the testing required the EAC
reserves the right to request further updates to the test plan and possibly
additional testing.
4.7. Testing. During testing, Manufacturers are responsible for enabling VSTLs to report any
changes to a voting system, or an approved test plan, directly to the EAC. Manufacturers shall
also enable VSTLs to report all test failures or anomalies directly to the EAC.
4.7.1. Changes. Any changes to a voting system, initiated as a result of the testing process, will
require submission of an updated Implementation Statement, functional diagram, and
System Overview document and, potentially, an updated test plan. Test plans must be
updated whenever a change to a voting system requires deviation from the test plan
originally approved by the EAC. Changes requiring alteration or deviation from the
originally approved test plan must be submitted to the EAC (by the VSTL) for approval
before the completion of testing and shall include an updated Implementation Statement,
functional diagram, and System Overview, as needed. Changes not affecting the test plan
shall be reported in the test report and shall include an updated Implementation Statement,
functional diagram, and System Overview document, as needed.
4.7.2. Test Anomalies or Failures. Manufacturers shall enable VSTLs to notify the EAC directly
and independently of any test anomalies or failures during testing. The VSTLs shall ensure
all anomalies or failures are addressed and resolved before testing is completed. All test
failures, anomalies and actions taken to resolve such failures and anomalies shall be
documented by the VSTL in an appendix to the Test Report submitted to the EAC. These
matters shall be reported in a matrix, or similar format, that identifies the failure or
anomaly, the applicable voting system standards, and a description of how the failure or
anomaly was resolved. Associated or similar anomalies/failures may be summarized and
reported in a single entry on the report (matrix) as long as the nature and scope of the
anomaly/failure is clearly identified. The manufacturer shall conduct a root cause analysis
for each anomaly following the format provided by the EAC. This analysis must be
OMB Control Number PENDING
39
EAC Voting System Testing and Certification Program Manual, Version 2.0
provided to the VSTL and the EAC prior to the beginning the Test Report phase of the test
campaign.
4.7.3. Deficiency Criteria. The EAC has developed a number of metrics to determine if voting
systems under test by a VSTL shall be removed from the EAC’s Testing and Certification
Program and returned to a manufacturer for further readiness review and/or QA testing.
These metrics include:
•
The duration of time a system is in a VSTL for testing
•
Significant delays/inactivity during a test campaign
•
Type or significance of deficiencies found
•
Total number of deficiencies, excluding source code coding convention deficiencies
•
Defect Density Ratio
•
A maximum number of errors in each of four categories, as defined later on in this
section
Note: A deficiency, for the purposes of this document, is considered a non-conformity to the
voting standard to which the voting system is being certified.
Voting systems shall be returned to a manufacturer for further readiness review and/or QA
testing if any of the following conditions occur:
•
Testing continues for more than 18 months without a test report being issued;
•
Inactivity as a result of a manufacturer’s decision or lack of action, which hinders
the reasonable progression of the test campaign, that exceeds 90 calendar days;
•
A significant deficiency caused by one or more major architectural flaws, requiring
significant redesign to adequately eliminate the deficiency;
o
Two factors shall be considered in determining the significance of a
deficiency:
The consequences of the deficiency to proper voting system function.
The extent of redesign necessary to fully remedy the deficiency. A
full remedy goes beyond a superficial response to the symptoms,
which leaves an underlying architectural flaw unaddressed, creating
the potential for other manifestations of the deficiency to re-occur. A
full remedy addresses the root cause of the deficiency and removes
the cause of the problem that created the deficiency.
•
The occurrence of 250 or more unique deficiencies, excluding coding convention
deficiencies;
•
Software defect density ratio (errors per 1000 lines of code) of
o
> 2.00 for voting systems of less than 250,000 lines of code
OMB Control Number PENDING
40
EAC Voting System Testing and Certification Program Manual, Version 2.0
o
> 1.75 for voting systems between 250,000 and 500,000 lines of code
o
> 1.50 for voting systems of more than 500,000 lines of code;
Additionally, the following four categories of deficiencies shall also be used to determine
when to return a voting system to a manufacturer:
•
Category 1 (Fatal Deficiency): A logic defect responsible for the incorrect
recording, tabulation, or reporting of a vote.
o
Voting systems shall be returned to a manufacturer if one or more
unique fatal deficiencies are discovered during one test campaign. 1
•
Category 2 (Severe Deficiency): A deficiency that causes program execution to
abort or causes a program not to perform properly or to produce unreliable
results.
o
Voting systems shall be returned to a manufacturer if 10 or more unique
severe deficiencies are discovered during one test campaign.
•
Category 3 (Significant Deficiency): A deficiency that is not a Category 1 or
Category 2 deficiency.
o
Voting systems shall be returned to a manufacturer if 200 or more
unique significant deficiencies are discovered during one test campaign.
•
Category 4 (Insignificant Deficiency): A minor deficiency, (including a source
code coding convention deficiency, e.g. naming convention, control construct,
coding or comment convention deficiency), a documentation deficiency, or a
deficiency caused by a typographical error, and is not a Category 1, 2, or 3
deficiency.
o
Category 4 deficiencies are unlimited.
Two or more instances of a deficiency are considered to be the same unique deficiency if: 1)
the outputs of each instance are identical; and 2) the same, specific remedy cures all
instances of the deficiency. If a second deficiency is discovered that results in the same
output as the first deficiency, but requires a different remedy to cure it, it shall be
considered a second unique deficiency. Two similar deficiencies that require a
modification within different areas of the source code to remedy the deficiency are to be
considered separate and unique deficiencies.
Note that for some requirements, for example the accuracy requirements, some errors are allowed, so
long as they remain below a specified threshold. A voting system that had errors under the specified
threshold would not be judged deficient because the system meets the requirements of the voting system
standard.
1
OMB Control Number PENDING
41
EAC Voting System Testing and Certification Program Manual, Version 2.0
Note: The above categories of deficiencies describe conditions under which the voting
system is returned to a manufacturer for further readiness review and/or QA testing.
However, even if the thresholds for return to a manufacturer are not met, all deficiencies
shall be corrected for a certificate to be issued.
The VSTL shall make the initial assignment for each deficiency into one of the four
categories described above. The VSTL shall ensure that each deficiency is described and
documented accurately in order to ensure the correct categorization of each deficiency. The
EAC shall review the determinations of the VSTL and make the final determinations as to
the categorization of deficiencies.
When a voting system is returned to a manufacturer for reasons described in this section,
the manufacturer shall perform a thorough QA analysis to determine the reason for the
deficiencies (root cause analysis). In addition, the manufacturer shall review its quality
process and perform an analysis of how the identified deficiencies passed through its
quality system. The manufacturer shall perform an extensive quality review to determine
the extent of the QA issues and shall document the appropriate measures that are
implemented to ensure that similar deficiencies do not occur again. Specifically, the
manufacturer shall detail the specific changes made to its quality process and then the
voting system to remedy the failures in the design and the quality process. All such
documentation shall be submitted to the EAC for review. The manufacturer may re-apply
for certification only after the EAC makes the determination that the QA analysis/review
and the measures put in place, in both the quality system and the voting system design, are
deemed adequate.
4.8. Test Report. Manufacturers shall enable VSTLs to submit test reports directly to the EAC. The
VSTL shall submit test reports only if the voting system has been tested and all tests identified
in the test plan have been successfully performed.
4.8.1. Submission. The test reports shall be submitted to the Program Director. The Program
Director shall review the submission for completeness. Any reports showing incomplete or
unsuccessful testing will be returned to the test laboratory for action and resubmission.
Notice of this action will be provided to the Manufacturer. Test reports shall be submitted
in Adobe PDF, Microsoft Word, or other electronic formats as prescribed by the Program
Director. Information on how to submit reports will be posted on the EAC’s website:
www.eac.gov.
4.8.2. Format. Manufacturers shall ensure VSTLs submit reports consistent with the requirements
in the VVSG and in the format outlined in Appendix E of this Manual. All information
provided in the Test Report shall be provided in a clear, complete and unambiguous
manner, so that a wide range of readers and users of the document will be able to
understand the evaluation supporting a system’s certification. In addition, the Test Report
OMB Control Number PENDING
42
EAC Voting System Testing and Certification Program Manual, Version 2.0
must show that all mandatory (shall) requirements in the standard have been tested and
successfully completed by the voting system as a prerequisite to certification.
4.8.3. Technical Review. A technical review of the test report, technical documents, and test plan
will be conducted by EAC technical experts. The EAC may require the submission of
additional information from the VSTL or Manufacturer if deemed necessary to complete the
review. These experts will submit a report outlining their findings to the Program Director.
The report will provide an assessment of the completeness, appropriateness, and adequacy
of the VSTL’s testing as documented in the test report.
4.8.4. Program Director’s Recommendation. The Program Director shall review the report and
take one of the following actions:
4.8.4.1. Recommend certification of the voting system consistent with the reviewed test
report and forward it to the Decision Authority for action (Initial Decision); OR
4.8.4.2. Refer the matter back to the technical reviewers for additional, specified action
and resubmission.
4.9. Initial Decision on Certification. Upon receipt of the report and recommendation of the
Program Director, the Decision Authority shall issue an Initial Decision on Certification. The
decision shall be forwarded to the Manufacturer consistent with the requirements of this
Manual.
4.9.1. An Initial Decision granting certification shall be processed consistent with Chapter 5 of this
Manual.
4.9.2. An Initial Decision denying certification shall be processed consistent with Chapter 6 of this
Manual.
OMB Control Number PENDING
43
EAC Voting System Testing and Certification Program Manual, Version 2.0
5. Grant of Certification
5.1. Overview. The grant of certification is the formal process through which the EAC
acknowledges that a voting system has successfully completed conformance testing to an
appropriate set of standards or guidelines. The grant of certification begins with the Initial
Decision of the Decision Authority. This decision becomes final after the Manufacturer
confirms that the final version of the software that was certified, and which the Manufacturer
will deliver with the certified system, has been subject to a trusted build (see Section 5.6),
placed in an EAC-approved repository (see Section 5.7), and can be verified using the
Manufacturer’s system identification tools (see Section 5.8). After a certification is issued, the
Manufacturer is provided a Certificate of Conformance and relevant information about the
system is added to the EAC’s website. Manufacturers with certified voting systems are
responsible for ensuring that each system it produces is properly labeled as certified.
5.2. Applicability of This Chapter. This chapter applies when the Decision Authority makes an
Initial Decision to grant a certification to a voting system based on the materials and
recommendation provided by the Program Director.
5.3. Initial Decision. The Decision Authority shall make a written decision on all voting systems
submitted for certification and issue the decision to a Manufacturer. When such decisions
result in a grant of certification, the decision shall be considered preliminary and referred to as
an Initial Decision pending required action by the Manufacturer. The Initial Decision shall:
5.3.1. State the preliminary determination reached (granting certification).
5.3.2. Inform the Manufacturer of the steps that must be taken to make the determination final
and receive a certification. This action shall include providing the Manufacturer with
specific instructions, guidance, and procedures for confirming and documenting that the
final certified version of the software meets the requirements for:
5.3.2.1. Performing and documenting a trusted build pursuant to Section 5.6 of this
chapter.
5.3.2.2. Depositing software in an approved repository pursuant to Section 5.7 of this
chapter.
5.3.2.3. Creating and making available system verification tools pursuant to Section 5.8
of this chapter.
5.3.3. Certification is not final until the Manufacturer accepts the certification and all conditions
placed on the certification.
OMB Control Number PENDING
44
EAC Voting System Testing and Certification Program Manual, Version 2.0
5.4. Pre-Certification Requirements. Before an Initial Decision becomes final and a certification is
issued, Manufacturers must ensure certain steps are taken. They must confirm that the final
version of the software that was certified and which the Manufacturer will deliver with the
certified system has been subject to a trusted build (see Section 5.6), has been delivered for
deposit in an EAC-approved repository (see Section 5.7), and can be verified using
Manufacturer-developed identification tools (see Section 5.8). The Manufacturer must provide
the EAC documentation demonstrating compliance with these requirements.
5.5. Trusted Build. A software build (also referred to as a compilation) is the process whereby
source code is converted to machine-readable binary instructions (executable code) for the
computer. A “trusted build” (or trusted compilation) is a build performed with adequate
security measures implemented to give confidence that the executable code is a verifiable and
faithful representation of the source code. The primary function of a trusted build is to create a
chain of evidence which allows stakeholders to have an approved model to use for verification
of a voting system. Specifically, the build will:
5.5.1. Demonstrate that the software was built as described in the TDP.
5.5.2. Show that the tested and approved source code was actually used to build the executable
code used on the system.
5.5.3. Demonstrate that no elements other than those included in the TDP were introduced in the
software build. The vendor or source from which each COTS product was procured must
be included in the TDP.
5.5.4. Document for future reference the configuration of the system certified.
5.5.5. Demonstrate that all COTS products are unmodified by requiring the VSTL to
independently obtain all COTS products from an outside source.
5.6. Trusted Build Procedure. A trusted build is a three-step process: (1) the build environment is
constructed, (2) the executable code and installation disks are created, and (3) the VSTL
verifies that the trusted build was created and functions properly. The process may be
simplified for a modification to a previously certified system. In each step, a minimum of two
witnesses from different organizations are required to participate. These participants must
include a VSTL representative and a manufacturer representative. Before creating the trusted
build, the VSTL must complete the source code review of the software delivered from the
manufacturer for compliance with the VVSG and must produce and record file signatures of
all source code modules. Hashes shall use a current FIPS 140-2 level 1 or higher validated
cryptographic module. After the trusted build is completed, there shall be no other “final”
build. As the final step, the trusted build must be submitted to the EAC on two separate forms
of media.
OMB Control Number PENDING
45
EAC Voting System Testing and Certification Program Manual, Version 2.0
5.6.1. Constructing the Build Environment. The VSTL shall construct the build environment in
an isolated environment controlled by the VSTL, as follows:
5.6.1.1. The device that will hold the build environment shall be completely erased, in
accordance with Department of Defense or NIST approved methods. The VSTL
shall ensure a complete erasure of the device.
5.6.1.2. The VSTL, with manufacturer observation, shall construct the build
environment.
5.6.1.3. After construction of the build environment, the VSTL shall produce and record
a file signature of the build environment.
5.6.1.4. A clone of the build environment computer’s main storage media shall be
created. File signatures shall be taken by the VSTL for verification purposes.
5.6.2. Creating the Executable Code and Installation Disks. After successful source code
review the VSTL shall:
5.6.2.1. Check the file signatures of the source code modules and build environment to
ensure they are unchanged from their original form.
5.6.2.2. Load the source code onto the build environment and produce and record the
file signature of the resulting combination.
5.6.2.3. Produce the executable code, and produce and record file signatures of the
executable code. A clone of the computer’s main storage on which the executable
code was created shall be created, with the file signatures verified by the VSTL.
5.6.2.4. The VSTL shall create installation disk(s) from the executable code, and
produce and record file signatures of the installation disk(s).
5.6.3. Verification of the Created Media. Upon completion of all the tasks outlined above, the
VSTL shall perform the following tasks:
5.6.3.1. Install the executable code onto the system submitted for testing and
certification before the completion of system testing.
5.6.3.2. Produce and record file signatures of each voting system file resident on each
device.
OMB Control Number PENDING
46
EAC Voting System Testing and Certification Program Manual, Version 2.0
5.6.3.3. Verify that all media to be included in the Trusted Build and submitted to the
EAC functions properly.
5.6.4. Trusted Build for Modifications. The process of building new executable code when a
previously certified system has been modified can be somewhat simplified, if the build
environment of the modification’s original certification can be obtained.
5.6.4.1. The build environment used in the original certification is removed from
storage and its file signature verified.
5.6.4.2. After source code review, the modified files are placed onto the verified build
environment and new executable files are produced.
5.6.4.3. If the original build environment is unavailable or its file signatures cannot be
verified against those recorded from the original certification, then the full
process of creating the build environment must be performed. Further source
code review may be required to validate that files are unmodified from the
originally certified versions.
5.7. Depositing Software in the EAC Repository. After EAC certification has been granted, the
VSTL’s project manager, or an appropriate designee of the project manager, shall deliver for
deposit the following elements into the EAC repository:
5.7.1. Description of items located on the deposit media, including a description of items to be
deposited. This description should include:
5.7.1.1. Deposit size (physical and logical).
5.7.1.2. Utilities or third-party applications used to create the deposit such as OS utilities
or third party software.
5.7.1.3. Encryption information, required passwords and/or crypto-keys or software
programs required to access the deposited materials.
5.7.2. Source code used for the trusted build and its file signatures.
5.7.3. List identifying all known dependencies between components.
5.7.4. The final TDP of the voting system submitted for testing including all product bills of
material, assembly drawings and schematics for the version being certified.
5.7.5. Build environment setup and configuration, including configuration settings for all
compilers and third party components and whether the build process requires source
code to be loaded to a specific location.
5.7.6. Build control files and/or scripts that control the build process.
OMB Control Number PENDING
47
EAC Voting System Testing and Certification Program Manual, Version 2.0
5.7.7. Executable code produced by the trusted build and the file signatures of all files
produced.
5.7.8. A detailed description of the Build Environment.
5.7.9. Installation device(s) and the file signatures of the installation devices.
5.7.10. Build instructions describing how to compile the escrow deposit and build executable
code. (Include hardware descriptions and OS system requirements, particularly any
custom settings required.
5.7.11. Names of all required applications necessary to compile and build executable code,
objects, dynamic libraries, etc.
5.7.12.
A working copy of the certified version of the EMS for the voting system.
5.7.13.
The computer on which the trusted build was created shall have its hard disk drive,
or other applicable storage media that contained the trusted build, removed and
submitted to the EAC.
5.7.14.
The Manufacturer must provide hashes to the EAC.
5.8. System Identification Tools. The Manufacturer shall provide tools through which a fielded
voting system may be identified and demonstrated to be unmodified from the system that was
certified. The purpose of this requirement is to make such tools available to Federal, State, and
local officials to identify and verify that the equipment used in elections is unmodified from its
certified version. Manufacturers may develop and provide these tools as they deem
appropriate or as required by the EAC. The tools, however, must provide the means to
identify and verify hardware and software. The EAC may review the system identification
tools developed by the Manufacturer to ensure compliance. VSTLs shall test system
identification tools during the test campaign to make sure they function properly and as
intended. System identification tools include the following examples:
5.8.1. Hardware is commonly identified by a model number and revision number on the unit, its
printed wiring boards (PWBs), and major subunits. Typically, hardware is verified as
unmodified by providing detailed photographs of the PWBs and internal construction of
the unit. These images may be used to compare to the unit being verified.
5.8.2. Software operating on a host computer will typically be verified by providing a self-booting
compact disk (CD) or similar device that verifies the file signatures of the voting system
application files and the signatures of all nonvolatile files the application files access during
their operation. Note that the creation of such a CD requires having a file map of all
nonvolatile files used by the voting system. Such a tool must be provided for verification
using the file signatures of the original executable files provided for testing. If during the
certification process modifications are made and new executable files created, then the tool
must be updated to reflect the file signatures of the final files to be distributed for use. For
OMB Control Number PENDING
48
EAC Voting System Testing and Certification Program Manual, Version 2.0
software operating on devices in which a self-booting CD or similar device cannot be used,
a procedure must be provided to allow identification and verification of the software that is
being used on the device.
5.9. Documentation. Manufacturers shall provide documentation to the Program Director
verifying the trusted build has been performed, software has been deposited in an approved
repository, and system identification tools are available to election officials. The Manufacturer
shall submit a letter, signed by both its management representative and a VSTL official, stating
(under penalty of law) that it has (1) performed a trusted build consistent with the
requirements of Section 5.6 of this Manual, (2) deposited software consistent with Section 5.7
of this Manual, and (3) created and made available system identification tools consistent with
Section 5.8 of this Manual. This letter shall also include (as attachments) a copy and
description of the system identification tool developed under Section 5.8 above.
5.10. Agency Decision. Upon receipt of documentation demonstrating the successful completion
of the requirements above and recommendation of the Program Director, the Decision
Authority will issue an Agency Decision granting certification and providing the
Manufacturer with a certification number and Certificate of Conformance.
5.11. Certification Document. A Certificate of Conformance will be provided to Manufacturers
for voting systems that have successfully met the standard of the EAC Certification Program.
The document will serve as the Manufacturer’s evidence that a particular system is certified to
a particular set of voting system standards. The EAC certification and certificate apply only to
the specific voting system configuration(s) identified, submitted and evaluated under the
Certification Program. Any modification to the system not authorized by the EAC will void
the certificate. The certificate will include the product (voting system) name, the specific
model or version of the product tested, the name of the VSTL that conducted the testing,
identification of the standards to which the system was tested, the EAC certification number
for the product, and the signature of the EAC Executive Director. The certificate will also
identify each of the various configurations of the voting system’s components that may be
represented as certified.
5.12. Certification Number and Version Control. Each system certified by the EAC will receive
a certification number unique to the system which will remain with the system until such time
as the system is decertified, sufficiently modified, or tested and certified to newer standards.
Generally, when a previously certified system is issued a new certification number, the
Manufacturer will be required to change the system’s name or version number.
5.12.1. New Voting Systems and Those Not Previously Certified by the EAC. All systems
receiving their first certification from the EAC will receive a new certification number.
Manufacturers must provide the EAC with the voting system’s name and version number
during the application process (Chapter 4). Systems previously certified by another body
may retain the previous system name and version number unless the system was modified
OMB Control Number PENDING
49
EAC Voting System Testing and Certification Program Manual, Version 2.0
before its submission to the EAC. Such modified systems must be submitted with a new
naming convention (i.e., a new version number).
5.12.2. Modifications. Voting systems previously certified by the EAC and submitted for
certification of a modification will generally receive a new voting system certification
number. Such modified systems must be submitted with a new naming convention. In rare
instances, the EAC may authorize retention of the same certification and naming
convention when the modification is so minor that is does not represent a substantive
change to the voting system. A request for such authorization must be made and approved
by the EAC during the application phase of the program.
5.12.3. Certification Upgrade. Voting systems previously certified and submitted (without
modification) for testing to a new version of the VVSG will receive a new certification
number. In such cases, however, the Manufacturer will not be required to change the
system name or version number.
5.12.4. De Minimis Change. Voting systems previously certified and implementing an approved
De Minimis Change Order (per Chapter 3) will not be issued a new certification number
and are not required to implement a new naming convention.
5.13. Publication of EAC Certification. The EAC will publish and maintain on its website a list
of all certified voting systems, including copies of all Certificates of Conformance, supporting
test reports, and voting system and Manufacturer information. Such information will be
posted immediately following the Manufacturer’s receipt of the EAC Final Decision and
Certificate of Conformance.
5.14. Representation of EAC Certification. Manufacturers may not represent or imply a voting
system is EAC certified unless it has received a Certificate of Conformance for the system.
Statements regarding EAC certification in brochures, on websites, on displays, and in
advertising/sales literature must be made solely in reference to specific systems. Any action by
a Manufacturer to suggest EAC endorsement of its product or organization is strictly
prohibited and may result in a Manufacturer’s suspension or other action pursuant to Federal
civil and criminal law. Manufacturers must provide a copy of the Certificate and Scope of
Certification document (found at www.eac.gov) to any jurisdiction purchasing an EAC
certified system.
5.15. Mark of Certification Requirement. Manufacturers shall post a Mark of Certification
(Mark) on all EAC-certified voting systems produced. This mark or label must be securely
attached to the system before sale, lease, or release to third parties. A mark of certification
shall be made using an EAC-mandated template. These templates identify the version of the
VVSG or VSS to which the system is certified. Use of this template shall be mandatory and the
EAC will provide the Mark as a template in .jpg, .eps, .pdf, and .tif formats. Manufacturers
who need access to the Mark pursuant to labeling an EAC certified voting system should send
OMB Control Number PENDING
50
EAC Voting System Testing and Certification Program Manual, Version 2.0
a formal request, via email or letter, to the Director. The request must include the specific
voting system and version number(s), indication of where the Mark will be displayed on the
voting system, and specification of the format in which the Mark will be reproduced. The EAC
Mark must be displayed as follows:
5.15.1. The Manufacturer may use only the Mark of Certification that accurately reflects the
certification held by the voting system as a whole. The certification of individual
components or modifications shall not be independently represented by a Mark of
Certification. In the event a system has components or modifications tested to various
(later) versions of the VVSG, the system shall bear only the Mark of Certification of the
standard to which the system (as a whole) was tested and certified (i.e. the lesser standard).
Ultimately, a voting system shall only display the Mark of Certification of the oldest or least
rigorous standard to which any of its components are certified.
5.15.2. The Mark shall be placed on the outside of a unit of voting equipment in a place readily
visible to election officials. The Mark need not be affixed to each of the voting system’s
components. The Mark shall be affixed to either (1) each unit that is used to cast ballots or
(2) each unit that is used to tabulate ballots.
5.15.3. All labels bearing the EAC Mark of Certification shall be designed and applied to voting
equipment so that the labels will remain in place and be clear and legible during the
customary conditions of distribution, storage, voting and routine testing and maintenance.
The materials used for the label, printing and adhesives shall be reasonably expected to last
the normal and projected lifespan of the voting system. If using an adhesive type label for
the Mark of Certification, the label stock material shall be such that the label cannot be
removed intact and reapplied. The label shall also be designed to resist the effects of
cleaning agents specified by the manufacturer. The Mark of Certification shall remain clear
and legible after the use of any recommended cleaning agents as specified by the
manufacturer and adhesive labels, if used, shall not have become loose or curled at the
edges.
5.15.4. If the EAC determines a voting system is not in compliance with the VVSG, and the
system has already been sold or otherwise distributed bearing the Mark of Certification, the
EAC shall provide written notice to the Manufacturer. If the Manufacturer fails to take
corrective action within 15 days of receipt of such notice, the EAC shall have the right to
announce publicly that the voting system may no longer comply with its original
certification, and may choose to initiate decertification actions as outlined in Chapter 7 of
the Manual, and/or suspension of Manufacturer Registration as outlined in Section 2.6 of the
Manual. Corrective action may include modification of the voting system to bring it into
compliance with the VVSG, or removal of the Mark of Certification from the product.
5.16. Information to Election Officials Purchasing Voting Systems. The user’s manual or
instruction manual for a certified voting system shall warn purchasers that any changes or
OMB Control Number PENDING
51
EAC Voting System Testing and Certification Program Manual, Version 2.0
modifications to the system not tested and certified by the EAC will void the EAC certification
of the voting system. In cases in which the manual is only provided in a form other than
paper, such as on a CD or via the Internet, the information required in this section may be
included in this alternative format provided the election official can reasonably be expected to
have the capability to access information in that format.
OMB Control Number PENDING
52
EAC Voting System Testing and Certification Program Manual, Version 2.0
6. Denial of Certification
6.1. Overview. When the Decision Authority issues an Initial Decision denying certification, the
Manufacturer has certain rights and responsibilities. The Manufacturer may request an
opportunity to cure the defects identified by the Decision Authority. In addition, the
Manufacturer may request that the Decision Authority reconsider the Initial Decision after the
Manufacturer has had the opportunity to review the record and submit supporting written
materials, data, and the rationale for its position. Finally, in the event reconsideration is
denied, the Manufacturer may appeal the decision to the Appeal Authority.
6.2. Applicability of This Chapter. This chapter applies when the Decision Authority makes an
Initial Decision to deny an application for voting system certification based on the materials
and recommendation provided by the Program Director.
6.3. Form of Decisions. All agency determinations shall be made in writing. All materials and
recommendations reviewed, or used by agency decision makers in making an official
determination, shall be in written form.
6.4. Effect of Denial of Certification. Upon receipt of the agency’s decision denying certification—
or in the event of an appeal, subject to the decision on appeal—the Manufacturer’s application
for certification shall be denied. Such systems will not be reviewed again by the EAC for
certification unless the Manufacturer alters the system, retests it, and submits a new
application for system certification.
6.5. The Record. The Program Director shall maintain all documents related to a denial of
certification. Such documents shall constitute the procedural and substantive record of the
decision making process. Records may include the following:
6.5.1. The Program Director’s report and recommendation to the Decision Authority.
6.5.2. The Decision Authority’s Initial Decision and Final Decision.
6.5.3. Any materials gathered by the Decision Authority that serve as a basis for a certification
determination.
6.5.4. All relevant and allowable materials submitted by the Manufacturer upon request for
reconsideration or appeal.
6.5.5. All correspondence between the EAC and a Manufacturer after the issuance of an Initial
Decision denying certification.
6.6. Initial Decision. The Decision Authority shall make and issue a written decision for voting
systems submitted for certification. When such decisions result in a denial of certification, the
OMB Control Number PENDING
53
EAC Voting System Testing and Certification Program Manual, Version 2.0
decision shall be considered preliminary and referred to as an Initial Decision. Initial Decisions
shall be in writing and contain (1) the Decision Authority’s basis and explanation for the
decision and (2) notice of the Manufacturer’s rights in the denial of certification process.
6.6.1. Basis and Explanation. The Initial Decision of the Decision Authority shall accomplish:
6.6.1.1. Clearly state the agency’s decision on certification.
6.6.1.2. Explain the basis for the decision, including:
6.6.1.2.1.
The relevant facts.
6.6.1.2.2.
The applicable EAC voting system standard.
6.6.1.2.3.
The relevant analysis in the Program Director’s recommendation.
6.6.1.2.4.
The reasoning behind the decision.
6.6.1.3. State the actions the Manufacturer must take, if any, to cure all defects in the
voting system and obtain a certification.
6.6.2. Manufacturer’s Rights. The written Initial Decision must also inform the Manufacturer of its
procedural rights under the program, including the following:
6.6.2.1. Right to request reconsideration. The Manufacturer shall be informed of its right
to request a timely reconsideration (see Section 6.9). Such request must be made
within 10 calendar days of the Manufacturer’s receipt of the Initial Decision.
6.6.2.2. Right to request a copy or otherwise have access to the information that served
as the basis of the Initial Decision (the record).
6.6.2.3. Right to cure system defects prior to Final Decision (see Section 6.8). A
Manufacturer may request an opportunity to cure within 10 calendar days of its
receipt of the Initial Decision.
6.7. No Manufacturer Action on Initial Decision. If a Manufacturer takes no action (by either
failing to request an opportunity to cure or request reconsideration) within 10 calendar days of
its receipt of the Initial Decision, the Initial Decision shall become the agency’s Final Decision
on certification. In such cases, the Manufacturer is determined to have foregone its right to
reconsideration, cure, and appeal. The certification application shall be considered denied.
6.8. Opportunity to Cure. Within 10 calendar days of receiving the EAC’s Initial Decision on
certification, a Manufacturer may request an opportunity to cure the defects identified in the
OMB Control Number PENDING
54
EAC Voting System Testing and Certification Program Manual, Version 2.0
EAC’s Initial Decision. If the request is approved, a compliance plan must be created,
approved, and followed. If this cure process is successfully completed, a voting system
denied certification in an Initial Decision may receive a certification without resubmission.
6.8.1. Manufacturer’s Request to Cure. The Manufacturer must request to cure in writing to the
Program Director within 10 calendar days of receipt of an Initial Decision.
6.8.2. EAC Action on Request. The Decision Authority will review the request and notify the
manufacturer in writing if the request to cure is approved or denied. The Decision
Authority will deny a request to cure only if the proposed plan to cure is inadequate or
does not present a viable way to remedy the identified defects. If the Manufacturer’s
request to cure is denied, it shall have 10 calendar days from the date it received such notice
to request reconsideration of the Initial Decision pursuant to Section 6.6.2.
6.8.3. Manufacturer’s Compliance Plan. Upon approval of the Manufacturer’s request for an
opportunity to cure, the manufacturer shall submit a compliance plan to the Decision
Authority for approval. This compliance plan must set forth steps to be taken to cure all
identified defects. It shall include the proposed changes to the system, updated technical
information (as required by Section 4.3.2), and a new test plan created and submitted
directly to the EAC by the VSTL (testing the system consistent with Section 4.4.2.3). The
plan shall also provide for the testing of the amended system and submission of a test
report by the VSTL to the EAC for approval. It should provide an estimated date for receipt
of this test report and include a schedule of periodic VSTL progress reports to the Program
Director.
6.8.4. EAC Action on the Compliance Plan. The Decision Authority must review and approve the
compliance plan. The Decision Authority may require the Manufacturer to provide
additional information and modify the plan as required. If the Manufacturer is unable or
unwilling to provide a compliance plan acceptable to the Decision Authority, the Decision
Authority shall provide written notice terminating the cure process. The Manufacturer shall
have 10 calendar days from the date it receives such notice to request reconsideration of the
Initial Decision pursuant to Section 6.6.2.
6.8.5. Compliance Plan Test Report. The VSTL shall submit the test report created pursuant to its
EAC-approved compliance plan. The EAC shall review the test report, along with the
original test report and other materials originally provided. The report will be technically
reviewed by the EAC consistent with the procedures laid out in Chapter 4 of this Manual.
6.8.6. EAC Decision on the System. After receipt of the test plan, the Decision Authority shall
issue a decision on a voting system amended pursuant to an approved compliance plan.
This decision shall be issued in the same manner and with the same process and rights as an
Initial Decision on Certification.
OMB Control Number PENDING
55
EAC Voting System Testing and Certification Program Manual, Version 2.0
6.9. Requests for Reconsideration. Manufacturers may request reconsideration of an Initial
Decision.
6.9.1. Submission of Request. A request for reconsideration must be made within 10 calendar
days of the Manufacturer’s receipt of an Initial Decision. The request shall be made and sent
to the Decision Authority.
6.9.2. Acknowledgment of Request. The Decision Authority shall acknowledge receipt of the
Manufacturer’s request for reconsideration. This acknowledgment shall either enclose all
information that served as the basis for the Initial Decision (the record) or provide a date by
which the record will be forwarded to the Manufacturer.
6.9.3. Manufacturer’s Submission. Within 30 calendar days of receipt of the record, a
Manufacturer may submit written materials in support of its position, including the
following:
6.9.3.1. A written argument responding to the conclusions in the Initial Decision.
6.9.3.2. Documentary evidence relevant to the issues raised in the Initial Decision.
6.9.4. Decision Authority’s Review of Request. The Decision Authority shall review and consider
all relevant submissions of the Manufacturer. In making a decision on reconsideration, the
Decision Authority shall also consider all documents that make up the record and any other
documentary information he or she determines relevant.
6.10. Agency Final Decision. The Decision Authority shall issue a written Final Decision after
review of the Manufacturer’s request for reconsideration. This Decision shall be the decision of
the agency and shall include:
6.10.1. The agency’s determination on the application for certification.
6.10.2. The issues raised by the Manufacturer in its request for reconsideration.
6.10.3. All facts, evidence, and EAC voting system standards that serve as the basis for the
decision.
6.10.4. The reasoning behind the determination.
6.10.5. Any additional documentary information identified and provided as an attachment that
serves as a basis for the decision and was not part of the Manufacturer’s submission or the
prior record.
6.10.6. The Manufacturer notice of its right to appeal.
OMB Control Number PENDING
56
EAC Voting System Testing and Certification Program Manual, Version 2.0
6.11. Appeal of Agency Final Decision. A Manufacturer may, upon receipt of a Final Decision
denying certification, issue a request for appeal.
6.11.1. Requesting Appeal. A Manufacturer may appeal a final decision of the agency by issuing
a written request for appeal.
6.11.1.1. Submission. Requests must be submitted in writing to the Program Director,
addressed to Chair of the U.S. Election Assistance Commission.
6.11.1.2. Timing of Appeal. The Manufacturer may request an appeal within 20
calendar days of receipt of the Agency Final Decision. Late requests will not be
considered.
6.11.1.3.
Contents of Request.
6.11.1.3.1. The request must clearly state the specific conclusions of the Final
Decision it wishes to appeal.
6.11.1.3.2. The request may include additional written argument.
6.11.1.3.3. The request may not reference or include any factual material not in
the record.
6.11.2. Consideration of Appeal. All timely appeals will be considered by the Appeal Authority.
6.11.2.1. The Appeal Authority shall be two or more EAC Commissioners or other
individuals appointed by the Commissioners who have not previously served as
the initial or reconsideration authority on the matter.
6.11.2.2.
All decisions on appeal shall be based on the record.
6.11.2.3. The determination of the Decision Authority shall be given deference by the
Appeal Authority. Although it is unlikely that the scientific certification process
will produce factual disputes, in such cases, the burden of proof shall belong to
the Manufacturer, to demonstrate by clear and convincing evidence, that its
voting system met all substantive and procedural requirements for certification.
In other words, the determination of the Decision Authority will be overturned
only when the Appeal Authority finds the ultimate facts in controversy highly
probable.
6.12. Decision on Appeal. The Appeal Authority shall make a written, final Decision on Appeal
and shall provide it to the Manufacturer.
OMB Control Number PENDING
57
EAC Voting System Testing and Certification Program Manual, Version 2.0
6.12.1. Contents. The following are required to be contained in the Decision on Appeal:
6.12.1.1.
The final determination of the agency.
6.12.1.2.
The matters raised by the Manufacturer on appeal.
6.12.1.3.
The reasoning behind the decisions.
6.12.1.4.
Statement that the Decision on Appeal is final.
6.12.2. Determinations. The Appeal Authority may make one of two determinations.
6.12.2.1. Grant of Appeal. If the Appeal Authority determines that the conclusions of
the Decision Authority shall be overturned in full, the appeal shall be granted. In
such cases, certification will be approved subject to the requirements of Chapter
5.
6.12.2.2. Denial of Appeal. If the Appeal Authority determines that any part of the
Decision Authority’s determination shall be upheld, the appeal shall be denied.
In such cases, the application for appeal is denied.
6.12.3. Effect. All Decisions on Appeal shall be final and binding on the Manufacturer. No
additional appeal shall be granted.
OMB Control Number PENDING
58
EAC Voting System Testing and Certification Program Manual, Version 2.0
7. Decertification
7.1. Overview. Decertification is the process by which the EAC revokes a certification previously
granted to a voting system. It is an important part of the Certification Program because it
serves to ensure the standards of the program are followed and certified voting systems
fielded for use in Federal elections maintain the same level of quality as those presented for
testing. Decertification is a serious matter. Its use will significantly affect Manufacturers, State
and local governments, the public, and the administration of elections. As such, the process for
Decertification is complex. It is initiated when the EAC receives information that a voting
system may not be in compliance with the Voluntary Voting System Guidelines or the
procedural requirements of this Manual. Upon receipt of this information, the Program
Director may initiate an Informal Inquiry to determine the credibility of the information. If the
information is credible and suggests the system is non-compliant, a Formal Investigation will
be initiated. If the results of the Formal Investigation demonstrate non-compliance, the
Manufacturer will be provided a Notice of Non-Compliance. Before a final decision on
Decertification is made, the Manufacturer will have the opportunity to remedy any defects
identified in the voting system and present information for consideration by the
Decertification Authority. A Decertification of a voting system may be appealed in a timely
manner.
7.2. Decertification Policy. Voting systems certified by the EAC are subject to Decertification.
Systems shall be decertified if (1) they are shown not to meet applicable Voluntary Voting
System Guidelines standards, (2) they have been modified or changed without following the
requirements of this Manual, or (3) the Manufacturer has otherwise failed to follow the
procedures outlined in this Manual and the quality, configuration, or compliance of the
system is in question. Systems will be decertified only after completion of the process outlined
in this chapter.
7.3. Informal inquiry. An Informal Inquiry is the first step taken when information is presented to
the EAC that suggests a voting system may not be in compliance with the Voluntary Voting
System Guidelines standards or the procedural requirements of this Manual.
7.3.1. Informal Inquiry Authority. The authority to conduct an Informal Inquiry shall rest with the
Program Director.
7.3.2. Purpose. The sole purpose of the Informal Inquiry is to determine whether a Formal
Investigation is warranted. The outcome of an Informal Inquiry is limited to a decision on
referral for investigation.
7.3.3. Procedure. Informal Inquiries do not follow a formal process.
7.3.3.1. Initiation. Informal Inquiries are initiated at the discretion of the Program
Director. They may be initiated any time the Program Director receives
OMB Control Number PENDING
59
EAC Voting System Testing and Certification Program Manual, Version 2.0
attributable, relevant information that suggests a certified voting system may
require Decertification. The information shall come from a source that has
directly observed or witnessed the reported occurrence. Such information may
be a product of the Certification Quality Monitoring Program (see Chapter 8).
Information may also come from State and local election officials, voters or
others who have used or tested a given voting system. The Program Director
may notify a Manufacturer that an Informal Inquiry has been initiated, but such
notification is not required. Initiation of an inquiry shall be documented through
the creation of a Memorandum for the Record.
7.3.3.2. Inquiry. The Informal Inquiry process is limited to inquiries necessary to
determine whether a Formal Investigation is required. In other words, the
Program Director shall conduct such inquiry necessary to determine (1) the
accuracy of the information obtained; and (2) if the information, if true, would
serve as a basis for Decertification. The nature and extent of the inquiry process
will vary depending on the source of the information. For example, an Informal
Inquiry initiated as a result of action taken under the Certification Quality
Monitoring Program will often require the Program Director merely to read the
report issued as a result of the Quality Monitoring action. On the other hand,
information provided by election officials or by voters who have used a voting
system may require the Program Director (or assigned technical experts) to
perform an in-person inspection or make inquiries of the Manufacturer.
7.3.3.3. Conclusion. An Informal Inquiry shall be concluded after the Program Director
determines the accuracy of the information that initiated the inquiry and whether
that information, if true, would warrant Decertification. The Program Director
may make only two conclusions: (1) refer the matter for a Formal Investigation or
(2) close the matter without additional action or referral.
7.3.4. Closing the Matter without Referral. If the Program Director determines, after Informal
Inquiry, a matter does not require a Formal Investigation, the Program Director shall close
the inquiry by filing a Memorandum for the Record. This document shall state the focus of
the inquiry, the findings of the inquiry and the reasons a Formal Investigation was not
warranted.
7.3.5. Referral. If the Program Director determines, after Informal Inquiry, a matter requires a
Formal Investigation, the Program Director shall refer the matter in writing to the Decision
Authority. In preparing this referral, the Program Director:
7.3.5.1. State the facts that served as the basis for the referral.
7.3.5.2. State the findings of the Program Director.
OMB Control Number PENDING
60
EAC Voting System Testing and Certification Program Manual, Version 2.0
7.3.5.3. Attach all documentary evidence that served as the basis for the conclusion.
7.3.5.4. Recommend a Formal Investigation, specifically stating the system to be
investigated and the scope and focus of the proposed investigation.
7.4. Formal Investigation. A Formal Investigation is an official investigation to determine whether
a voting system warrants Decertification. The end result of a Formal Investigation is a Report
of Investigation.
7.4.1. Formal Investigation Authority. The Decision Authority shall have the authority to initiate
and conclude a Formal Investigation by the EAC.
7.4.2. Purpose. The purpose of a Formal Investigation is to gather and document relevant
information sufficient to make a determination on whether an EAC-certified voting system
warrants Decertification consistent with the policy put forth in Section 7.2.
7.4.3. Initiation of Investigation. The Decision Authority shall authorize the initiation of an EAC
Formal Investigation.
7.4.3.1. Scope. The Decision Authority shall clearly set the scope of the investigation by
identifying (in writing) the voting system (or systems) and specific procedural or
operational non-conformance to be investigated. The non-conformance to be
investigated shall be set forth in the form of numbered allegations.
7.4.3.2. Investigator. The Program Director shall be responsible for conducting the
investigation unless the Decision Authority appoints another individual to
conduct the investigation. The Program Director (or Decision Authority
appointee) may assign staff or technical experts, as required, to investigate the
matter.
7.4.4. Notice of Formal Investigation. Upon initiation of a Formal Investigation, notice shall be
given to the Manufacturer of the scope of the investigation, which shall include:
7.4.4.1. Identification of the voting system and specific procedural or operation nonconformance being investigated (scope of investigation).
7.4.4.2. An opportunity for the manufacturer to provide relevant information in writing.
7.4.4.3. An estimated timeline for the investigation.
7.4.5. Investigation. Investigations shall be conducted impartially, diligently, promptly, and
confidentially and shall utilize appropriate techniques to gather the necessary information.
OMB Control Number PENDING
61
EAC Voting System Testing and Certification Program Manual, Version 2.0
7.4.5.1. Fair and Impartial Investigation. All Formal Investigations shall be conducted in a
fair and impartial manner. All individuals assigned to an investigation must be
free from any financial conflicts of interest.
7.4.5.2. Diligent Collection of Information. All investigations shall be conducted in a
meticulous and thorough manner. Investigations shall gather all relevant
information and documentation that is reasonably available. The diligent
collection of information is vital for informed decision making.
7.4.5.3. Prompt Collection of Information. Determinations that may affect the
administration of Federal elections must be made in a reasonable, yet expedited
manner. The EAC’s determinations on Decertification will affect the actions of
State and local election officials conducting elections and as such, all
investigations regarding Decertification must proceed with an appropriate sense
of urgency.
7.4.5.4. Confidential Collection of Information. Consistent with Federal law, information
pertaining to a Formal Investigation should not be made public until the Report
of Investigation is complete. The release of incomplete and unsubstantiated
information or predecisional opinions that may be contrary or inconsistent with
the final determination of the EAC could cause public confusion or could
unnecessarily negatively affect public confidence in active voting systems. Such
actions could serve to impermissibly affect election administration and voter
turnout. All predecisional investigative materials must be appropriately
safeguarded.
7.4.5.5. Methodologies. Investigators shall gather information by means consistent with
the four principles noted above. Investigative tools include (but are not limited
to) the following:
7.4.5.5.1.
Interviews. Investigators may interview individuals (such as State
and local election officials, voters, or manufacturer representatives).
All interviews shall be reduced to written form; each interview
should be summarized in a statement that is reviewed, approved,
and signed by the interviewee.
7.4.5.5.2.
Field audits.
7.4.5.5.3.
Manufacturer site audits.
7.4.5.5.4.
Written interrogatories. Investigators may pose specific, written
questions to the Manufacturer for the purpose of gathering
information relevant to the investigation. The Manufacturer shall
OMB Control Number PENDING
62
EAC Voting System Testing and Certification Program Manual, Version 2.0
respond to the queries within a reasonable timeframe (as specified in
the request).
7.4.5.5.5.
System testing. Testing may be performed in an attempt to reproduce
a condition or failure that has been reported. This testing will be
conducted at a VSTL as designated by the EAC.
7.4.5.6. Report of Investigation. The end result of a Formal Investigation is a Report of
Investigation.
7.4.6. Report of Investigation. The Report of Investigation serves primarily to document: (1) all
relevant and reliable information gathered in the course of the investigation; and (2) the
conclusion reached by the Decision Authority.
7.4.6.1. When Complete. The report is complete and final when certified and signed by
the Decision Authority.
7.4.6.2. Contents of the Report of Investigation. The following shall be included in the
written report:
7.4.6.2.1.
The scope of the investigation, identification of the voting system and
specific matter investigated.
7.4.6.2.2.
Description of the investigative process employed.
7.4.6.2.3.
Summary of the relevant and reliable facts and information gathered
in the course of the investigation.
7.4.6.2.4.
All relevant and reliable evidence collected in the course of the
investigation that documents the facts shall be documented and
attached.
7.4.6.2.5.
Analysis of the information gathered.
7.4.6.2.6.
Statement of the findings of the investigation.
7.4.7. Findings, Report of Investigation. The Report of Investigation shall state one of two
conclusions. After gathering and reviewing all applicable facts, the report shall find each
allegation investigated to be either (1) substantiated or (2) unsubstantiated.
7.4.7.1. Substantiated Allegation. An allegation is substantiated if a preponderance of the
relevant and reliable information gathered requires the voting system in question
OMB Control Number PENDING
63
EAC Voting System Testing and Certification Program Manual, Version 2.0
to be decertified (consistent with the policy set out in Section 7.2). If any
allegation is substantiated a Notice of Non-Compliance shall be issued.
7.4.7.2. Unsubstantiated Allegation. An allegation is unsubstantiated if the preponderance
of the relevant and reliable information gathered does not warrant
Decertification (see Section 7.2). If all allegations are unsubstantiated, the matter
shall be closed and a copy of the report forwarded to the Manufacturer.
7.4.8. Publication of Report. The report shall not be made public nor released to the public until
final.
7.5. Effect of Informal Inquiry or Formal Investigation on Certification. A voting system’s EAC
certification is not affected by the initiation or conclusion of an Informal Inquiry or Formal
Investigation. Systems under investigation remain certified until a final Decision on
Decertification is issued by the EAC.
7.6. Notice of Non-Compliance. If an allegation in a Formal Investigation is substantiated, the
Decision Authority shall send the Manufacturer a Notice of Non-Compliance. The Notice of
Non-Compliance is not, itself, a Decertification of the voting system. The purpose of the notice
is to (1) notify the Manufacturer of the non-compliance and the EAC’s intent to Decertify the
system; and (2) inform the Manufacturer of its procedural rights so that it may be heard prior
to Decertification.
7.6.1. Noncompliance Information. The following shall be included in a Notice of NonCompliance:
7.6.1.1. A copy of the Report of Investigation to the Manufacturer.
7.6.1.2. The non-compliance, consistent with the Report of Investigation.
7.6.1.3. Notification to the Manufacturer that if the voting system is not made
compliant, the voting system will be decertified.
7.6.1.4. State the actions the Manufacturer must take, if any, to bring the voting system
into compliance and avoid Decertification.
7.6.2. Manufacturer’s Rights. The written Notice of Non-compliance shall also inform the
Manufacturer of its procedural rights under the program, which include the following:
7.6.2.1. Right to Present Information Prior to Decertification Decision. The Manufacturer
shall be informed of its right to present information to the Decision Authority
prior to a determination of Decertification.
OMB Control Number PENDING
64
EAC Voting System Testing and Certification Program Manual, Version 2.0
7.6.2.2. Right to Have Access to the Information That Will Serve as the Basis of the
Decertification Decision. The Manufacturer shall be provided the Report of
Investigation and any other materials that will serve as the basis of an agency
Decision on Decertification.
7.6.2.3. Right to Cure System Defects Prior to the Decertification Decision. A Manufacturer
may request an opportunity to cure within 20 calendar days of its receipt of the
Notice of Non-Compliance.
7.7. Procedure for Decision on Decertification. The Decision Authority shall make and issue a
written Decision on Decertification whenever a Notice of Non-Compliance is issued. The
Decision Authority will not take such action until the Manufacturer has had a reasonable
opportunity to cure the non-compliance and submit information for consideration.
7.7.1. Opportunity to Cure. The Manufacturer shall have an opportunity to cure a nonconformant voting system in a timely manner prior to Decertification. A cure shall be
considered timely when the process can be completed before the next Federal election,
meaning that any proposed cure must be in place before any individual jurisdiction fielding
the system holds a Federal election. The Manufacturer must request the opportunity to cure
and if the request is approved, a compliance plan must be created, approved by the EAC,
and adhered to. If the cure process is successfully completed, a Manufacturer may modify a
non-compliant voting system, remedy procedural discrepancies, or otherwise bring its
system into compliance without resubmission or Decertification.
7.7.1.1. Manufacturer’s Request to Cure. Within 10 calendar days of receiving the EAC’s
Notice of Non-Compliance, a Manufacturer may request an opportunity to cure
all defects identified in the Notice of Non-Compliance in a timely manner. The
request must be sent to the Decision Authority and outline how the
Manufacturer intends to modify the system, update the technical information (as
required by Section 4.3.2), have a VSTL create a test plan and test the system, and
obtain EAC approval before the next election for Federal office.
7.7.1.2. EAC Action on Request. The Decision Authority will review the request and
approve it if the defects identified in the Notice of Non-Compliance may
reasonably be cured before the next election for Federal office.
7.7.1.3. Manufacturer’s Compliance Plan. Upon approval of the Manufacturer’s request for
an opportunity to cure, the Manufacturer shall submit a compliance plan to the
Decision Authority for approval. This compliance plan must set forth the steps to
be taken (including time frames) to cure all identified defects in a timely manner.
The plan shall describe the proposed changes to the system, provide for
modification of the system, update the technical information required by Section
4.3.2, include a test plan delivered to the EAC by the VSTL (testing the system
OMB Control Number PENDING
65
EAC Voting System Testing and Certification Program Manual, Version 2.0
consistent with Section 4.4.2.3), and provide for the VSTL’s testing of the system
and submission of the test report to the EAC for approval. The plan shall also
include a schedule of periodic progress reports to the Program Director. 2
7.7.1.4. EAC Action on the Compliance Plan. The Decision Authority must review and
approve the compliance plan. The Decision Authority may require the
Manufacturer to provide additional information and modify the plan as
required. If the Manufacturer is unable or unwilling to provide a Compliance
Plan acceptable to the Decision Authority, the Decision Authority shall provide
written notice terminating the “opportunity to cure” process.
7.7.1.5. VSTL’s Submission of the Compliance Plan Test Report. The VSTL shall submit the
test report created pursuant to the Manufacturer’s EAC-approved Compliance
Plan. The EAC shall review the test report and any other necessary or relevant
materials. The report will be reviewed by the EAC in a manner similar to the
procedures described in Chapter 4 of this Manual.
7.7.1.6. EAC Decision on the System. After receipt of the VSTL’s test report, the Decision
Authority shall issue a decision within 20 working days.
7.7.2. Opportunity to Be Heard. The Manufacturer may submit written materials in response to
the Notice of Non-Compliance and Report of Investigation. These documents shall be
considered by the Decision Authority when making a determination on Decertification. The
Manufacturer shall ordinarily have 20 calendar days from the date it received the Notice of
Non-Compliance (or in the case of a failed effort to cure, the termination of that process) to
deliver its submissions to the Decision Authority. When warranted by public interest
(because a delay in making a determination on Decertification would affect the timely, fair,
and effective administration of a Federal election), the Decision Authority may request a
Manufacturer to submit information within a condensed timeframe. This alternative period
(and the basis for it) must be stated in the Notice of Non-Compliance and must allow the
Manufacturer a reasonable amount of time to gather its submissions. Submissions may
include the following materials:
7.7.2.1. A written argument responding to the conclusions in the Notice of NonCompliance or Report of Investigation.
7.7.2.2. Documentary evidence relevant to the allegations or conclusions in the Notice of
Non-Compliance.
2
Manufacturers should also be cognizant of State certification procedures and local pre-election logic and accuracy
testing. Systems that meet EAC guidelines will also be impacted by independent State and local requirements.
These requirements may also prevent a system from being fielded, irrespective of EAC Certification.
OMB Control Number PENDING
66
EAC Voting System Testing and Certification Program Manual, Version 2.0
7.7.3. Decision on Decertification. The Decision Authority shall make an agency determination on
Decertification.
7.7.3.1. Timing. The Decision Authority shall promptly make a decision on
Decertification. The Decision Authority may not issue such a decision, however,
until the Manufacturer has provided all of its written materials for consideration
or the time allotted for submission (usually 20 calendar days) has expired.
7.7.3.2. Considered Materials. The Decision Authority shall review and consider all
relevant submissions by the Manufacturer. To reach a decision on
Decertification, the Decision Authority shall consider all documents that make
up the record and any other documented information deemed relevant.
7.7.3.3. Agency Decision. The Decision Authority shall issue a written Decision on
Decertification after review of applicable materials. This decision shall be the
final decision of the agency. The following shall be included in the decision:
7.7.3.3.1.
The agency’s determination on the Decertification, specifically
addressing the areas of non-compliance investigated.
7.7.3.3.2.
The issues raised by the Manufacturer in the materials it submitted
for consideration.
7.7.3.3.3.
Facts, evidence, procedural requirements, and/or voting system
standards (VVSG or VSS) that served as the basis for the decision.
7.7.3.3.4.
The reasoning for the decision.
7.7.3.3.5.
Documented information, identified and provided as an attachment,
that served as a basis for the decision and that was not part of the
Manufacturer’s submission or the Report of Investigation.
7.7.3.3.6.
Notification to the Manufacturer of its right to appeal.
7.8. Effect of Decision Authority’s Decision on Decertification. The Decision Authority’s
Decision on Decertification is the determination of the agency. A Decertification is effective
upon the EAC’s Publication or Manufacturer’s receipt of the decision (whichever is earlier). A
Manufacturer that has had a voting system decertified may appeal that decision.
7.9. Appeal of Decertification. A Manufacturer may, upon receipt of a Decision on Decertification,
request an appeal in a timely manner.
7.9.1. Requesting Appeal.
OMB Control Number PENDING
67
EAC Voting System Testing and Certification Program Manual, Version 2.0
7.9.1.1. Submission. Requests must be submitted by the Manufacturer in writing to the
Chair of the U.S. Election Assistance Commission.
7.9.1.2. Timing of Appeal. The Manufacturer may request an appeal within 20 calendar
days of receipt of the Agency Final Decision on Decertification. Late requests will
not be considered.
7.9.1.3. Contents of Request. The following actions are necessary for the Manufacturer to
write and submit a request for appeal:
7.9.1.3.1.
Clearly state the specific conclusions of the Final Decision the
Manufacturer wishes to appeal.
7.9.1.3.2.
Include additional written argument, if any.
7.9.1.3.3.
Do not reference or include any factual material not previously
considered or submitted to the EAC.
7.9.1.4. Effect of Appeal on Decertification. The initiation of an appeal does not affect the
decertified status of a voting system. Systems are decertified upon notice of
Decertification in the agency’s Decision on Decertification (see Section 7.8).
7.9.2. Consideration of Appeal. All timely appeals will be considered by the Appeal Authority.
7.9.2.1. The Appeal Authority shall consist of two or more EAC Commissioners or other
individual(s) designated by the Commissioners who has not previously served
as an investigator, advisor, or decision maker in the Decertification process.
7.9.2.2. All decisions on appeal shall be based on the record.
7.9.2.3. The decision of the Decision Authority shall be given deference by the Appeal
Authority. Although it is unlikely that the scientific certification process will
produce factual disputes, in such cases, the burden of proof shall belong to the
Manufacturer to demonstrate by clear and convincing evidence that its voting
system met all substantive and procedural requirements for certification. In other
words, the determination of the Decision Authority will be overturned only
when the Appeal Authority finds the ultimate facts in controversy highly
probable.
7.9.3. Decision on Appeal. The Appeal Authority shall issue a written, final Decision on Appeal
that shall be provided to the Manufacturer. Each Decision on Appeal shall be final and
OMB Control Number PENDING
68
EAC Voting System Testing and Certification Program Manual, Version 2.0
binding and no additional appeal shall be granted. The following shall be included in a
Decision on Appeal:
7.9.3.1. The final determination of the agency.
7.9.3.2. The matters raised by the Manufacturer on appeal.
7.9.3.3. The reasoning behind the decision.
7.9.3.4. Statement that the decision on appeal is final.
7.9.4. Effect of Appeal.
7.9.4.1. Grant of Appeal. If a Manufacturer’s appeal is granted in whole, the decision of
the Decision Authority shall be reversed and the voting system shall have its
certification reinstated. For purposes of this program, the system shall be treated
as though it was never decertified.
7.9.4.2. Denial of Appeal. If a Manufacturer’s appeal is denied in whole or in part, the
decertification decision of the Decision Authority shall be upheld. Therefore, the
voting system shall remain decertified and no additional appeal shall be made
available.
7.10. Effect of Decertification. A decertified voting system no longer holds an EAC certification
under the EAC Certification Program. For purposes of this Manual and the program, a
decertified system will be treated as any other uncertified voting system. As such, the effects
of Decertification are as follows:
7.10.1. The Manufacturer may not represent the voting system as certified.
7.10.2. The voting system may not be labeled with a Mark of Certification.
7.10.3. The voting system will be removed from the EAC’s list of certified systems.
7.10.4. The EAC will notify State and local election officials of the Decertification.
7.11. Recertification. A decertified system may be resubmitted for certification. Such systems
shall be treated as any other system seeking certification. The Manufacturer shall present an
application for certification consistent with the instructions of this Manual.
OMB Control Number PENDING
69
EAC Voting System Testing and Certification Program Manual, Version 2.0
8. Quality Monitoring Program
8.1. Overview. The quality of any product, including a voting system, depends on two specific
elements: (1) the design of the product or system; and (2) the consistency of the manufacturing
process. The EAC’s testing and certification process focuses on voting system design by
ensuring that a representative sample of a system meets the technical specifications of the
applicable EAC voting system standards. This process, commonly called ‘type acceptance,’
determines whether the representative sample submitted for testing meets the standards.
What type acceptance does not do is explore whether variations in manufacturing may allow
production of non-compliant systems. Generally, the quality of the manufacturing is the
responsibility of the Manufacturer. After a system is certified, the vendor assumes primary
responsibility for compliance of the products produced. This level of compliance is
accomplished by the Manufacturer’s configuration management and quality control processes.
The EAC’s Quality Monitoring Program, as outlined in this chapter, however, provides an
additional layer of quality control by allowing the EAC to perform manufacturing site
reviews, carry out fielded system reviews, and gather information on voting system anomalies
from election officials. These additional tools help ensure that voting systems continue to meet
the EAC’s voting system standards as the systems are manufactured, delivered, and used in
Federal elections. These aspects of the program enable the EAC to independently monitor the
continued compliance of fielded voting systems.
8.2. Purpose. The purpose of the Quality Monitoring Program is to ensure systems used by
election jurisdictions are identical to those tested and certified by the EAC as well as to
monitor the completeness and adequacy of testing with the desired performance in fielded
voting systems. This level of quality control is accomplished primarily by identifying (1)
potential quality problems in manufacturing, (2) uncertified voting system configurations, and
(3) field performance issues with certified systems.
8.3. Manufacturer’s Quality Control. The EAC’s Quality Monitoring Program shall not be
considered a substitute for the Manufacturer’s own quality control program. As stated in
Chapter 2 of this Manual, all Manufacturers must have an acceptable quality control program
in place before they may be registered. The EAC’s program serves as an independent and
complementary process of quality control that works in tandem with the Manufacturer’s
efforts.
8.4. Quality Monitoring Methodology. Provides the EAC with four primary tools for assessing
the level of effectiveness of the certification process and the compliance of fielded voting
systems, which includes: (1) manufacturing site reviews; (2) fielded system reviews; (3) a
means for receiving anomaly reports from the field; and (4) technical bulletins or product
advisories created by the manufacturer.
8.5. Manufacturing Site Review. Facilities that produce certified voting systems will be reviewed
periodically, at the discretion of the EAC, to verify that the system being manufactured,
OMB Control Number PENDING
70
EAC Voting System Testing and Certification Program Manual, Version 2.0
shipped, and sold is the same as the certified system. All registered Manufacturers must
cooperate with such site reviews as a condition of program participation.
8.5.1. Notice. The site review may be conducted as either a pre-scheduled or as an impromptu
visit, at the discretion of the EAC; however a Manufacturer will be given at least 24 hours
notice. Scheduling and notice of site reviews will be coordinated with, and provided to, the
manufacturing facility’s representative and the Manufacturer’s representative.
8.5.2. Frequency. At a minimum, at least one manufacturing facility of a registered Manufacturer
shall be subject to a site review at least once every 4 years.
8.5.3. The Review. The production facility and production test records must be made available for
review. When requested, production schedules must be provided to the EAC. Production or
production testing may be witnessed by EAC representatives. If equipment is not being
produced during the inspection, the review may be limited to production records. During
the inspection, the Manufacturer must make available to the EAC’s representative the
Manufacturer’s quality manual and other documentation sufficient to enable the
representative to evaluate the following factors of the facility’s production:
8.5.3.1. Manufacturing quality controls.
8.5.3.2. Final inspection and testing.
8.5.3.3. History of deficiencies or anomalies and corrective actions taken.
8.5.3.4. Equipment calibration and maintenance.
8.5.3.5. Corrective action program.
8.5.3.6. Policies on product labeling and the application of the EAC mark of
certification.
8.5.4. Exit Briefing. EAC representatives will provide the manufacturing facility’s representative
with a verbal exit briefing regarding the preliminary observations of the review.
8.5.5. Written Report. A written report documenting the review will be drafted by the EAC and
provided to the Manufacturer. The report will detail the findings of the review and identify
actions that are required to correct any identified deficiencies.
8.6. Fielded System Review and Testing. Upon invitation, or with the permission of a State or
local election authority, the EAC may, at its discretion, conduct a review of fielded voting
systems. Such reviews will be conducted to ensure that a fielded system is comprised of the
same configuration as what was certified by the EAC and that the proper Mark of Certification
OMB Control Number PENDING
71
EAC Voting System Testing and Certification Program Manual, Version 2.0
has been applied. This review may include the testing of a fielded system, if deemed
necessary. Any anomalies found during this review will be provided to the appropriate
election jurisdiction(s) and the Manufacturer. In addition, this review will evaluate the
correspondence of the actual configuration and use of the voting system in the field with that
envisioned during testing. If anomalies occur, these reviews seek to determine the direct
cause, underlying root cause and appropriate remedial and/or preventative actions.
8.7. Field Anomaly Reporting. The EAC will collect information from election officials with
fielded EAC-certified voting systems. Information on the actual field performance of a voting
system shall be used as a means for assessing the effectiveness of the Certification Program
and the manufacturing quality and version control. The EAC will provide a mechanism for
election officials to provide input related to voting system anomalies.
8.7.1. Anomaly Report. Election officials may submit notices of voting system anomalies directly
to the EAC in either WORD or .pdf format consistent with the requirements in Section 8.7.3
below.
8.7.2. Who May Report? State or local election officials who have experienced voting system
anomalies in their jurisdiction may file anomaly reports. The individuals reporting must
identify themselves and have firsthand knowledge of or official responsibility over the
anomaly being reported. Anonymous or hearsay reporting will not be accepted.
8.7.3. What Is Reported? Election officials shall report voting system anomalies. An anomaly is
defined as an irregular or inconsistent action or response from the voting system, or system
component, which resulted in the system or component not functioning as intended or
expected. Anomalies resulting from administrator error or procedural deficiencies shall not
be considered anomalies for purposes under this chapter. The report shall include:
8.7.3.1. The official’s name, title, contact information, and jurisdiction.
8.7.3.2. A description of the voting system that experienced the anomaly.
8.7.3.3. The date and location of the reported occurrence.
8.7.3.4. The type of election.
8.7.3.5. A description of the anomaly witnessed with applicable supporting
documentation, if available.
8.7.4. Distribution of Reports. Reports which are deemed to contain credible information will be
distributed to State and local election jurisdictions with similar systems, to the
Manufacturer of the voting system, and to the VSTLs. Reports are deemed credible if:
OMB Control Number PENDING
72
EAC Voting System Testing and Certification Program Manual, Version 2.0
8.7.4.1. The definition of an anomaly under Section 8.7.3 was met;
8.7.4.2. A complete report, per the requirements of Sections 8.7.3.1 – 8.7.3.5 was
submitted;
8.7.4.3. Information contained within the report was confirmed by others present at the
time of the anomaly; and
8.7.4.4. Was verified by the relevant state’s chief election official.
8.8. Manufacturer Created Technical Bulletins or Product Advisories. Manufacturers are
required to provide any technical bulletins or product advisories issued on EAC certified
voting systems to the EAC at the time they are issued to jurisdictions impacted by the
advisory. EAC must receive these via email or postal mail within 24 hours of issuance.
8.9. Use of Quality Monitoring Information. Ultimately, the information the EAC gathers from
manufacturing site reviews, fielded system reviews, and field anomaly reports will be used to
improve the program and ensure the quality of voting systems. The Quality Monitoring
Program is not designed to be punitive but to be focused on improving the process.
Information gathered will be used to accomplish the following:
8.9.1. Identify areas for improvement in the EAC’s Testing and Certification Program.
8.9.2. Improve the manufacturing quality and change control processes.
8.9.3. Increase voter confidence in voting technology.
8.9.4. Inform Manufacturers, election officials, and the EAC of issues associated with voting
systems in a real-world environment.
8.9.5. Share information among jurisdictions that use similar voting systems.
8.9.6. Resolve problems associated with voting technology or manufacturing in a timely manner
by involving Manufacturers, election officials, and the EAC.
8.9.7. Strengthen the coordination between certification testing and the desired performance in
deployed voting systems.
8.9.8. Provide feedback to the EAC, National Institute of Standards and Technology (NIST), and
the Technical Guidelines Development Committee (TGDC) regarding issues that may need
to be addressed through a revision to the Voluntary Voting System Guidelines.
OMB Control Number PENDING
73
EAC Voting System Testing and Certification Program Manual, Version 2.0
8.9.9. Initiate an investigation when information suggests Decertification is warranted (see
Chapter 7).
OMB Control Number PENDING
74
EAC Voting System Testing and Certification Program Manual, Version 2.0
9. Requests for Interpretations
9.1. Overview. A Request for Interpretation is a means by which a registered Manufacturer or
VSTL may seek clarification on a specific EAC voting system standard. An Interpretation is a
clarification of the voting system standards and guidance on how to properly evaluate
conformance to it. Suggestions or requests for modifications to the standards are provided by
other processes. This chapter outlines the policy, requirements, and procedures for submitting
a Request for Interpretation.
9.2. Policy. Registered Manufacturers or VSTLs may request the EAC to provide definitive
Interpretation of EAC-accepted voting system standards when, in the course of developing or
testing a voting system, the meaning of a particular standard becomes ambiguous or unclear.
The EAC may self-initiate such a request when it agents identifies a need for interpretation
within the program. An Interpretation issued by the EAC will serve to clarify what a given
standard requires and how to properly evaluate compliance. An Interpretation does not
amend voting system standards, but serves only to clarify existing standards.
9.3. Requirements for Submitting a Request for Interpretation. An EAC Interpretation is limited
in scope. The purpose of the Interpretation process is to provide Manufacturers or VSTLs, in
the process of developing or testing a voting system a means for resolving the meaning of a
VVSG requirement. A Request for Interpretation must: (1) be submitted by a registered
manufacturer or VSTL; (2) request interpretation of an applicable VVSG requirement; (3)
present an actual controversy; and (4) seek clarification on a matter of unsettled ambiguity.
9.3.1. Proper Requestor. A Request for Interpretation may be submitted only by a registered
Manufacturer or a VSTL. Requests for Interpretation will not be accepted from any other
parties.
9.3.2. Applicable Standard. A Request for Interpretation is limited to queries regarding
requirements contained in EAC VVSG. A Manufacturer or VSTL may only submit a
Request for Interpretation on a version of EAC VVSG to which the EAC currently offers
certification.
9.3.3. Existing Factual Controversy. To submit a Request for Interpretation, a Manufacturer or
VSTL must present a question relative to a specific voting system or technology proposed
for use in a voting system. A Request for Interpretation on hypothetical issues will not be
addressed by the EAC. To submit a Request for Interpretation, the need for clarification
must have arisen during the development or testing of a voting system. A factual
controversy exists when an attempt to apply a specific section of the VVSG to a specific
system or piece of technology creates ambiguity.
9.3.4. Unsettled, Ambiguous Matter. Requests for Interpretation must involve actual
controversies that have not been previously clarified.
OMB Control Number PENDING
75
EAC Voting System Testing and Certification Program Manual, Version 2.0
9.3.4.1. Actual Ambiguity. A proper Request for Interpretation must contain an actual
ambiguity. The interpretation process is not a means for challenging a clear
VVSG requirement or to recommend changes to requirements. An ambiguity
arises (in applying a voting system standard to a specific technology) when one
of the following occurs:
9.3.4.1.1.
The language of the standard is unclear on its face.
9.3.4.1.2.
One section of the standard seems to contradict another, relevant
section.
9.3.4.1.3.
The language of the standard, though clear on its face, lacks sufficient
detail or breadth to determine its proper application to a particular
technology.
9.3.4.1.4.
The language of a particular standard, when applied to a specific
technology, conflicts with the established purpose or intent of the
standard.
9.3.4.1.5.
The language of the standard is clear, but the proper means to assess
compliance is unclear.
9.3.4.2. Not Previously Clarified. The EAC will not accept a Request for Interpretation
when the issue has previously been clarified.
9.4. Procedure for Submitting a Request for Interpretation. A Request for Interpretation shall be
made in writing to the Program Director. EAC interpretations are based upon, and limited to,
the facts presented; therefore all requests should be complete and as detailed as possible. .
Failure to provide complete information may result in an Interpretation that is non-applicable
and ultimately immaterial to the issue at hand. The following shall be included in a Request
for Interpretation:
9.4.1. Establish Standing To Make the Request. To make a request, one must meet the
requirements identified in Section 9.3. Thus, the written request must provide sufficient
information for the Program Director to conclude that the requestor is: (1) a proper
requester; (2) requesting an Interpretation of an applicable voting system standard; (3)
presenting an actual factual controversy; and (4) seeking clarification on a matter of
unsettled ambiguity.
9.4.2. Identify the EAC VVSG Requirement To Be Clarified. The request must identify the specific
VVSG requirement or requirement(s) to which the requestor seeks clarification. The request
OMB Control Number PENDING
76
EAC Voting System Testing and Certification Program Manual, Version 2.0
must state the version of the voting system standards at issue (if applicable) and quote and
correctly cite the applicable requirement(s).
9.4.3. State the Facts Resulting in Ambiguity. The request must provide the facts associated with
the voting system technology that resulted in the ambiguity. The requestor must provide all
necessary information in a clear, concise manner. Any Interpretation issued by the EAC will
be based on the facts provided.
9.4.4. Identify the Ambiguity. The request must identify the ambiguity it seeks to resolve and
shall:
9.4.4.1. Clearly state a concise question.
9.4.4.2. Be related to, and reference, the voting system standard and voting system
technology.
9.4.4.3. Be limited to a single issue. Each question or issue arising from an ambiguous
requirement must be stated separately. Compound questions are unacceptable. If
multiple issues exist, they should be presented as individual, numbered
questions.
9.4.4.4. Be stated in a way that can ultimately be answered yes or no.
9.4.5. Provide a Proposed Interpretation. A Request for Interpretation should propose an answer
to the question posed. The answer should interpret the voting system standard in the
context of the facts presented and it should provide the basis and reasoning behind the
proposed interpretation.
9.5. EAC Action on a Request for Interpretation. Upon receipt of a Request for Interpretation, the
EAC shall:
9.5.1. Review the Request. The Program Director shall review the request to ensure it is complete,
clear, and meets the requirements of Section 9.3. Upon review, the Program Director may:
9.5.1.1. Request Clarification. If the Request for Interpretation is incomplete, or additional
information is otherwise required, the Program Director may request the
Manufacturer or VSTL clarify its Request for Interpretation and identify any
additional information required.
9.5.1.2. Reject the Request for Interpretation. If the Request for Interpretation does not meet
the requirements of Section 9.3, the Program Director may reject it. Such rejection
must be provided in writing to the Manufacturer or VSTL and must state the
basis for the rejection.
OMB Control Number PENDING
77
EAC Voting System Testing and Certification Program Manual, Version 2.0
9.5.1.3. Notify Acceptance of the Request. If the Request for Interpretation is accepted, the
Program Director will notify the Manufacturer or VSTL in writing and provide it
with an estimated date of completion. A Request for Interpretation may be
accepted in whole or in part and the notice of acceptance shall state the issues
accepted for interpretation.
9.5.2. Consideration of the Request. After a Request for Interpretation has been accepted, the
matter shall be analyzed and researched. Such action may require the EAC to employ
technical experts and may also require the EAC to request additional information from the
Manufacturer or VSTL. The Manufacturer or VSTL shall respond promptly to such
requests.
9.5.3. Interpretation. The Program Director shall be responsible for making determinations on a
Request for Interpretation. After this determination has been made, a written Interpretation
shall be sent to the Manufacturer or VSTL. The following actions shall be included in the
Interpretation:
9.5.3.1. The question or questions investigated.
9.5.3.2. The relevant facts that served as the basis of the Interpretation.
9.5.3.3. The voting system standards interpreted.
9.5.3.4. The conclusion reached.
9.5.3.5. The effect of an Interpretation (see Section 9.6).
9.6. Effect of Interpretation. Interpretations are fact specific and case specific. They are not tools of
policy, but specific, fact-based guidance useful for resolving a particular problem. Ultimately,
an Interpretation is determinative and conclusive only with regard to the case presented.
Nevertheless, Interpretations do have some value as precedent. Interpretations published by
the EAC shall serve as reliable guidance and authority over identical or similar questions of
interpretation. These Interpretations will help users understand and apply the individual
requirements of EAC VVSG.
9.7. Library of Interpretations. To better serve Manufacturers, VSTLs and those interested in the
EAC’s voting system standards, the Program Director shall publish EAC Interpretations. All
proprietary information contained in an Interpretation will be redacted before publication
consistent with Chapter 10 of this Manual. The library of published Interpretations is posted
on the EAC’s website: www.eac.gov.
OMB Control Number PENDING
78
EAC Voting System Testing and Certification Program Manual, Version 2.0
10. Release of Certification Program Information
10.1. Overview. Manufacturers participating in the Certification Program will be required to
provide the EAC with a variety of documents. In general, these documents will be releasable
to the public and, in many cases, the information provided will be affirmatively published by
the EAC. In limited cases, however, documents may not be released if they include trade
secrets, confidential commercial information, or personal information. While the EAC is
ultimately responsible for determining which documents Federal law protects from release,
Manufacturers must identify the information they believe is protected and ultimately provide
substantiation and a legal basis for withholding. This chapter discusses the EAC’s general
policy on the release of information and provides Manufacturers with standards, procedures,
and requirements for identifying documents as trade secrets or confidential commercial
information.
10.2. EAC Policy on the Release of Certification Program Information. The EAC seeks to make
its Voting System Testing and Certification Program as transparent as possible. The agency
believes such action benefits the program by increasing public confidence in the process and
creating a more informed and involved public. As such, it is the policy of the EAC to make all
documents, or severable portions thereof, available to the public consistent with Federal law
(e.g. Freedom of Information Act (FOIA) and the Trade Secrets Act).
10.2.1. Requests for Information. As in any Federal program, members of the public may
request access to Certification Program documents under FOIA (5 U.S.C. §552). The EAC
will promptly process such requests per the requirements of the Act.
10.2.2. Publication of Documents. Beyond the requirements of FOIA, the EAC intends to
affirmatively publish program documents (or portions of documents) it believes will be of
interest to the public. This publication will be accomplished through the use of the EAC’s
website (www.eac.gov). The published documents will cover the full spectrum of the
program, including information pertaining to:
10.2.2.1.
Registered Manufacturers;
10.2.2.2.
VSTL Test Plans;
10.2.2.3.
VSTL Test Reports;
10.2.2.4.
Agency decisions;
10.2.2.5.
Denials of Certification;
10.2.2.6.
Issuance of Certifications;
OMB Control Number PENDING
79
EAC Voting System Testing and Certification Program Manual, Version 2.0
10.2.2.7. Information on a certified voting system’s operation, components, features
or capabilities;
10.2.2.8.
Appeals;
10.2.2.9.
Reports of investigation and Notice of Non-compliance;
10.2.2.10. Decertification actions;
10.2.2.11. Manufacturing facility review reports;
10.2.2.12. Official Interpretations (VVSG); and
10.2.2.13. Other topics as determined by the EAC.
10.2.3. Trade Secret and Confidential Commercial Information. Federal law places a number of
restrictions on a Federal agency’s authority to release information to the public. Two such
restrictions are particularly relevant to the Certification program: (1) trade secrets
information; and (2) privileged or confidential commercial information. Both types of
information are explicitly prohibited from release by the FOIA and the Trade Secrets Act (18
U.S.C. §1905).
10.3. Trade Secrets. A secret, commercially valuable plan, process, or device used for the making
or processing of a product and that is the end result of either innovation or substantial effort. It
relates to the productive process itself, describing how a product is made. It does not relate to
information describing end product capabilities, features, or performance.
10.3.1. The following examples illustrate productive processes that may be trade secrets:
10.3.1.1.
Plans, schematics, and other drawings useful in production.
10.3.1.2.
Specifications of materials used in production.
10.3.1.3. Voting system source code used to develop or manufacture software where
release would reveal actual programming.
10.3.1.4. Technical descriptions of manufacturing processes and other secret
information relating directly to the production process.
10.3.2. The following examples are likely not trade secrets:
10.3.2.1.
Information pertaining to a finished product’s capabilities or features.
OMB Control Number PENDING
80
EAC Voting System Testing and Certification Program Manual, Version 2.0
10.3.2.2.
Information pertaining to a finished product’s performance.
10.3.2.3. Information regarding product components that would not reveal any
commercially valuable information regarding production.
10.4. Privileged or Confidential Commercial Information. Privileged or confidential
commercial information is information submitted by a Manufacturer that is commercial or
financial in nature and privileged or confidential.
10.4.1. Commercial or Financial Information. The terms commercial and financial should be given
their ordinary meanings. They include records in which a submitting Manufacturer has any
commercial interest.
10.4.2. Privileged or Confidential Information. Commercial or financial information is privileged or
confidential if its disclosure would likely cause substantial harm to the competitive position
of the submitter. The concept of harm to one’s competitive position focuses on harm
flowing from a competitor’s affirmative use of the proprietary information. It does not
include incidental harm associated with upset customers or employees.
10.5. EAC’s Responsibilities. The EAC is ultimately responsible for determining whether or not
a document (in whole or in part) may be released pursuant to Federal law. In doing so,
however, the EAC will require information and input from the Manufacturer submitting the
documents. This requirement is essential for the EAC to identify, track, and make
determinations on the large volume of documentation it receives. The EAC has the following
responsibilities:
10.5.1. Managing Documentation and Information. The EAC will control the documentation it
receives by ensuring that documents are secure and released to third parties only after the
appropriate review and determination.
10.5.2. Contacting Manufacturer on Proposed Release of Potentially Protected Documents. In the
event a member of the public submits a FOIA request for documents provided by a
Manufacturer or the EAC otherwise proposes the release of such documents, the EAC will
take the following action:
10.5.2.1. Review the documents to determine if they are potentially protected from
release as trade secrets or confidential commercial information. The documents
at issue may have been previously identified as protected by the Manufacturer
when submitted (see Section 10.7.1 below) or identified by the EAC on review.
10.5.2.2. Grant the submitting Manufacturer an opportunity to provide input. In the
event the information has been identified as potentially protected from release as
a trade secret or confidential commercial information, the EAC will notify the
OMB Control Number PENDING
81
EAC Voting System Testing and Certification Program Manual, Version 2.0
submitter and allow them an opportunity to submit their position on the issue
prior to release of the information. The submitter shall respond consistent with
Section 10.7.1 below.
10.5.3. Final Determination on Release. After providing the submitter of the information an
opportunity to be heard, the EAC will make a final decision on release. The EAC will
inform the submitter of this decision.
10.6. Manufacturer’s Responsibilities. Although the EAC is ultimately responsible for
determining if a document, or any portion thereof, is protected from release as a trade secret
or confidential commercial information, the Manufacturer shall be responsible for identifying
documents, or portions of documents, it believes warrant such protection. Moreover, the
Manufacturer will be responsible for providing the legal basis and substantiation for their
determination regarding the withholding of a document. This responsibility arises in two
situations: (1) upon the initial submission of information; and (2) upon notification by the EAC
that it is considering the release of potentially protected information.
10.6.1. Initial Submission of Information. When a Manufacturer submits documents to the EAC
as required by the Certification Program, it is responsible for identifying any document or
portion of a document that it believes is protected from release by Federal law.
Manufacturers shall identify protected information by the following:
10.6.1.1. Submitting a Notice of Protected Information. This notice shall identify the
document, document page, or portion of a page that the Manufacturer believes
should be protected from release. This identification must be done with
specificity. For each piece of information identified, the Manufacturer must state
the legal basis for its protected status.
10.6.1.1.1. Cite the applicable law that exempts the information from release.
10.6.1.1.2. Clearly discuss why that legal authority applies and why the
document must be protected from release.
10.6.1.1.3. If necessary, provide additional documentation or information. For
example, if the Manufacturer claims a document contains
confidential commercial information, it would also have to provide
evidence and analysis of the competitive harm that would result
upon release.
10.6.1.2. Label Submissions. Label all submissions identified in the notice as
“Proprietary Commercial Information.” Label only those submissions identified
as protected. Attempts to indiscriminately label all materials as proprietary will
render the markings moot.
OMB Control Number PENDING
82
EAC Voting System Testing and Certification Program Manual, Version 2.0
10.6.2. Notification of Potential Release. In the event a Manufacturer is notified that the EAC is
considering the release of information that may be protected, the Manufacturer shall:
10.6.2.1. Respond to the notice within 15 calendar days. If additional time is needed,
the Manufacturer must promptly notify the Program Director. Requests for
additional time will be granted only for good cause and must be made before the
15-day deadline. Manufacturers that do not respond in a timely manner will be
viewed as not objecting to release.
10.6.2.2.
Clearly state one of the following in the response:
10.6.2.2.1. There is no objection to release; OR
10.6.2.2.2. The Manufacturer objects to release. In this case, the response must
clearly state which portions of the document the Manufacturer
believes should be protected from release. The Manufacturer shall
follow the procedures discussed in Section 10.7.1.
10.7. Personal Information. Certain personal information is protected from release under FOIA
and the Privacy Act (5 U.S.C. §552a). This information includes private information about a
person that, if released, would cause the individual embarrassment or constitute an
unwarranted invasion of personal privacy. Generally, the EAC will not require the submission
of private, individual information and the incidental submission of such information should
be avoided. If a Manufacturer believes it is required to submit such information, it should
contact the Program Director. If the information will be submitted, it must be properly
identified. Examples of such information include:
10.7.1. Social Security Number.
10.7.2. Bank account numbers.
10.7.3. Home address.
10.7.4. Home phone number.
OMB Control Number PENDING
83
EAC Voting System Testing and Certification Program Manual, Version 2.0
Appendix A
Manufacturer Registration Application Form
Available in electronic format at www.eac.gov
OMB Control Number PENDING
84
EAC Voting System Testing and Certification Program Manual, Version 2.0
Appendix B
Application for Voting System Testing Form
Available in electronic format at www.eac.gov
OMB Control Number PENDING
85
EAC Voting System Testing and Certification Program Manual, Version 2.0
Appendix C
Voting System Test Plan Outline
OMB Control Number PENDING
86
EAC Voting System Testing and Certification Program Manual, Version 2.0
This outline is provided solely as an aid to test plan development. Note that these items may
change significantly, depending on the specific project planned.
1 Introduction
1.1 References
1.2 Terms and Abbreviations
1.3 Testing Responsibilities
1.3.1 Project schedule with
1.3.1.1 Owner assignments
1.3.1.2 Test case development
1.3.1.3 Test procedure development and validation
1.3.1.4 3rd party tests
1.3.1.5 EAC and Manufacturer dependencies
1.4 Target of Evaluation Description
1.4.1 System Overview
1.4.2 Block diagram
1.4.3 System Limits
1.4.4 Supported Languages
1.4.5 Supported Functionality
1.4.5.1 Standard VVSG Functionality
1.4.5.2 Manufacturer Extensions
2. Pre‐Certification Testing and Issues
2.1 Evaluation of prior VSTL testing
2.1.1 Reason for testing and results, listing of modifications from previous to current
system
2.2 Evaluation of prior non‐VSTL testing
2.2.1 Reason for testing and results, states, other 3rd party entities
2.3 Known Field Issues
2.3.1 Listing of relevant issues uncovered during field operations
3 Materials Required for Testing
3.1 Software
3.2 Equipment
3.3 Test Materials
3.4 Deliverable Materials
4 Test Specifications
4.1 Requirements
4.1.1 Mapping of requirements to equipment type and features
4.1.2 Rationale for why some requirements are NA for this campaign
4.2 Hardware Configuration and Design
OMB Control Number PENDING
87
EAC Voting System Testing and Certification Program Manual, Version 2.0
4.3 Software System Functions
4.4 Test Case Design
4.4.1 Hardware Qualitative Examination Design
4.4.1.1 Mapping of requirements to specific interfaces
4.4.2 Hardware Environmental Test Case Design
4.4.3 Software Module Test Case Design and Data
4.4.4 Software Functional Test Case Design and Data
4.4.5 System‐level Test Case Design
4.5 Security functions
4.6 TDP evaluation
4.7 Source Code review
4.8 QA & CM system review
5 Test Data
5.1 Data Recording
5.2 Test Data Criteria
5.3 Test Data Reduction
6 Test Procedure and Conditions
6.1 Facility Requirements
6.2 Test Set‐up
6.3 Test Sequence
7 Test Operations Procedures
Proprietary Data
OMB Control Number PENDING
88
EAC Voting System Testing and Certification Program Manual, Version 2.0
Appendix D
Voting System Modification Test Plan Outline
OMB Control Number PENDING
89
EAC Voting System Testing and Certification Program Manual, Version 2.0
Test Plans submitted for modifications to previously EAC certified voting systems should be brief
and structured to minimize test plan development and review, while enabling the EAC to maintain
solid control of the certification process. The test plan shall concisely document the strategy and
plan for testing those sections of the VVSG applicable to the modification or modifications
submitted. The test plan shall be written with clarity that will allow all constituents to understand
what testing will be conducted, to verify compliance to VVSG requirements, and to assure that the
test plan will remain a living document throughout the life of the test campaign for the
modification.
This outline is provided solely as an aid to test plan development. Note that these items may
change significantly, depending on the specific project planned.
1. Introduction
1.1Description and Overview of EAC certified system being modified
1.1.1 Complete definition of the baseline certified system.
1.1.2 Detailed description of the engineering changes and/or modifications to the
certified system and why the modification was implemented.
1.1.3 An initial assessment of the impact that the modifications have on the system
and past certification.
1.1.4 Description of what will be regression tested to establish assurance that the
modifications have no adverse impact on the compliance, integrity or performance
of the system.
1.2 References
1.3 Terms and Abbreviations
1.4 Project Schedule
1.5 Scope of testing
1.5.1 Block diagram (if applicable)
1.5.2 System limits (if applicable)
1.5.3 Supported Languages
1.5.4 Supported Functionality
1.5.5 VVSG
1.5.6 RFIs
1.5.7 NOCs
2. Pre‐Certification Testing and Issues
2.1 Evaluation of prior VSTL testing
2.2 Evaluation of prior non‐VSTL testing (if applicable)
2.3 Known Field Issues (if applicable)
3. Materials Required for Testing
3.1 Software
3.2 Equipment
OMB Control Number PENDING
90
EAC Voting System Testing and Certification Program Manual, Version 2.0
3.3 Test Materials
3.4 Deliverable
3.5 Proprietary Data
4. Test Specifications
4.1 Requirements
4.1.1 Mapping of requirements to equipment type and features
4.1.2 Rationale for why some requirements are NA for this campaign
4.2 Hardware Configuration and Design (if applicable)
4.3 Software System Functions (if applicable)
4.4 Test Case Design
4.4.1 Hardware Qualitative Examination Design (if applicable)
4.4.2 Hardware Environmental Test Case Design (if applicable)
4.4.3 Software Module Test Case Design and Data (if applicable)
4.4.4 Software Functional Test Case Design and Data (if applicable)
4.4.5 System‐level Test Case Design
4.5 Security functions (if applicable)
4.6 TDP evaluation
4.7 Source Code review (if applicable)
4.8 QA & CM system review
5. Test Data
5.1 Test Data Recording
5.2 Test Data Criteria
6. Test Procedure and Conditions
6.1 Test Facilities
6.2 Test Set‐up
6.3 Test Sequence
6.4 Test Operations Procedure
OMB Control Number PENDING
91
EAC Voting System Testing and Certification Program Manual, Version 2.0
Appendix E
Voting System Test Report Outline
OMB Control Number PENDING
92
EAC Voting System Testing and Certification Program Manual, Version 2.0
Test Reports produced by VSTLs shall follow the format outlined below. Deviations from this
format may be used upon prior written approval of the Program Director.
1. System Identification and Overview
2. Certification Test Background
2.1 Revision History
2.2 Implementation Statement
3. Test Findings and Recommendation
3.1 Summary Finding and Recommendation
3.2 Reasons for Recommendation to Reject
3.3 Anomalies
3.4 Correction of Deficiencies
Appendix A. Additional Findings
Appendix B. Warrant of Accepting Change Control Responsibility
Appendix C. Trusted Build
Appendix D. Test Plan
Appendix E. State Test Reports
OMB Control Number PENDING
93
EAC Voting System Testing and Certification Program Manual, Version 2.0
Appendix F
Voting System Modification Test Report Outline
OMB Control Number PENDING
94
EAC Voting System Testing and Certification Program Manual, Version 2.0
Test Reports produced by VSTLs shall follow the format outlined below. Deviations from this
format may be used upon prior written approval of the Program Director.
1. Introduction
1.1Description of EAC certified system being modified
1.1 References
1.2 Terms and Abbreviations
2. Certification Test Background
2.1 Revision History
2.2 Scope of testing
2.2.1 Modification Overview
2.2.1.1 Detailed list of changes
2.2.2 Block diagram (if applicable)
2.2.3 Supported Languages
2.2.4 VVSG
2.2.5 RFIs
2.2.6 NOCs
3. Test Findings and Recommendation
3.1 Summary Finding and Recommendation
3.1.1 Hardware Testing
3.1.2 System Level Testing
3.1.3 Source code review
3.2 Anomalies and Resolutions
3.3 Deficiencies and Resolutions
4. Recommendation for Certification
Appendix A. Additional Findings
Appendix B. Deficiency report (if applicable)
Appendix C. Anomaly report (if applicable)
Appendix D. Test Plan
Appendix E. State Test Reports (if applicable)
OMB Control Number PENDING
95
| File Type | application/pdf |
| File Modified | 2015-04-20 |
| File Created | 2015-04-20 |