Download:
pdf |
pdfVoting System Test Laboratory
Program Manual
Version 2.0
Effective May 31, 2015
United States Election Assistance Commission
1335 East West Highway, Suite 4300, Silver Spring, MD 20910
www.eac.gov
OMB Control # Pending
Voting System Test Laboratory Program Manual, Version 2.0
Paperwork Reduction Act
The reporting requirements in this manual have been approved under the Paperwork
Reduction Act of 1995, Office of Management and Budget Control (OMB) Number 0000-0000,
expiring May 31, 2015. Persons are not required to respond to this collection of information
unless it displays a currently valid OMB number. Information gathered pursuant to this
document and its forms will be used solely to administer the EAC Testing & Certification and
Laboratory Accreditation Program. This program is voluntary. Individuals who wish to
participate in the program, however, must meet its requirements. The estimated total annual
hourly burden on the voting system manufacturing industry and election officials is 322 hours.
This estimate includes the time required for reviewing the instructions, gathering information,
and completing the prescribed forms. Send comments regarding this burden estimate or any
other aspect of this collection, including suggestions for reducing this burden to the U.S.
Election Assistance Commission, Voting System Testing and Certification Program, Office of
the Program Director, 1335 East West Highway, Suite 4300, Silver Spring, MD, 20910.
i
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Contents
INTRODUCTION .......................................................................................................................................................5
1.1.
1.2.
1.3.
1.4.
1.5.
1.6.
1.7.
1.8.
1.9.
1.10.
1.11.
1.12.
1.13.
1.14.
1.15.
BACKGROUND ............................................................................................................................................5
AUTHORITY. ...............................................................................................................................................5
ROLE OF THE NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY. ..............................................5
SCOPE. ........................................................................................................................................................5
MANUAL MAINTENANCE AND REVISION.................................................................................................6
CLARIFICATION OF PROGRAM REQUIREMENTS AND PROCEDURES. ........................................................6
PROGRAM PERSONNEL ..............................................................................................................................6
SUBMISSION OF DOCUMENTS ....................................................................................................................6
RECEIPT OF DOCUMENTS—VSTL. ............................................................................................................7
RECEIPT OF DOCUMENTS—EAC...............................................................................................................7
RECORD RETENTION—EAC......................................................................................................................7
PUBLICATION AND RELEASE OF DOCUMENTS ..........................................................................................7
REFERENCES ...............................................................................................................................................7
DEFINITIONS...............................................................................................................................................8
ACRONYMS AND ABBREVIATIONS.............................................................................................................9
PROGRAM REQUIREMENTS .............................................................................................................................. 11
2.1.
2.2.
2.3.
2.4.
2.5.
2.6.
2.7.
2.8.
2.9.
2.10.
2.11.
2.12.
2.13.
2.14.
2.15.
2.16.
2.17.
2.18.
2.19.
2.20.
2.21.
2.22.
2.23.
OVERVIEW ................................................................................................................................................ 11
PROGRAM REQUIREMENTS - GENERALLY ............................................................................................... 11
NIST RECOMMENDATION ....................................................................................................................... 11
NVLAP ACCREDITATION ....................................................................................................................... 11
CONFLICT OF INTEREST AND PROHIBITED PRACTICES PROGRAM.......................................................... 12
PERSONNEL POLICIES............................................................................................................................... 19
NOTIFICATION OF CHANGES ................................................................................................................... 19
SITE VISITS ................................................................................................................................................ 20
NOTICE OF LAWSUITS .............................................................................................................................. 20
TESTING, TECHNICAL PRACTICES AND REPORTING ............................................................................... 20
TEST READINESS REVIEW ......................................................................................................................... 20
TECHNOLOGY TESTING AGREEMENT. ..................................................................................................... 21
TEST PLAN PACKAGE............................................................................................................................... 21
TESTING .................................................................................................................................................... 22
TEST REPORT PACKAGE. .......................................................................................................................... 25
ACCEPTANCE OF PRIOR TESTING ............................................................................................................ 26
TERMINATION OF TESTING PRIOR TO COMPLETION ............................................................................... 27
VSTL VERIFICATION OF TRUSTED BUILD................................................................................................ 27
LABORATORY INDEPENDENCE ................................................................................................................ 27
AUTHORITY TO DO BUSINESS IN THE UNITED STATES ............................................................................ 31
COMMUNICATIONS .................................................................................................................................. 31
RESOURCES AND FINANCIAL STABILITY ................................................................................................. 31
RECORDKEEPING ...................................................................................................................................... 31
ACCREDITATION PROCESS ............................................................................................................................... 32
ii
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
3.1.
3.2.
3.3.
3.4.
3.5.
3.6.
3.7.
3.8.
3.9.
3.10.
3.11.
3.12.
3.13.
3.14.
3.15.
OVERVIEW. ............................................................................................................................................... 32
NIST RECOMMENDATION ....................................................................................................................... 32
EAC INVITATION ..................................................................................................................................... 33
APPLICATION ........................................................................................................................................... 33
EAC REVIEW OF APPLICATION PACKAGE .............................................................................................. 35
GRANT OF ACCREDITATION .................................................................................................................... 37
EFFECT OF ACCREDITATION .................................................................................................................... 38
EXPIRATION AND RENEWAL OF ACCREDITATION .................................................................................. 39
DENIAL OF ACCREDITATION ................................................................................................................... 39
REQUESTING APPEAL............................................................................................................................... 39
EAC ACTION ON A REQUEST FOR APPEAL ............................................................................................. 40
SUBMISSION OF APPEAL........................................................................................................................... 40
CONSIDERATION OF APPEAL ................................................................................................................... 40
COMMISSIONER’S DECISION ON APPEAL ................................................................................................ 40
EFFECT OF DENIAL OF ACCREDITATION ................................................................................................. 41
COMPLIANCE MANAGEMENT PROGRAM .................................................................................................... 42
4.1.
4.2.
4.3.
4.4.
4.5.
4.6.
4.7.
4.8.
4.9.
4.10.
PURPOSE ................................................................................................................................................... 42
COMPLIANCE MANAGEMENT PROGRAM, GENERALLY.......................................................................... 42
VSTL NOTIFICATION OF CHANGES ........................................................................................................ 42
REQUEST FOR DOCUMENTS AND INFORMATION .................................................................................... 42
PROFICIENCY TESTING ............................................................................................................................. 43
ON SITE LABORATORY REVIEW—GENERALLY ....................................................................................... 43
ON SITE LABORATORY REVIEW—FREQUENCY ....................................................................................... 44
ON SITE LABORATORY REVIEW—PROCEDURE ....................................................................................... 44
EAC COMPLIANCE MANAGEMENT REPORTS ......................................................................................... 46
CORRECTIVE ACTION............................................................................................................................... 47
REVOCATION OF ACCREDITATION ................................................................................................................ 49
5.1.
5.2.
5.3.
5.4.
5.5.
5.6.
5.7.
5.8.
5.9.
5.10.
5.11.
5.12.
OVERVIEW ................................................................................................................................................ 49
REVOCATION POLICY............................................................................................................................... 49
REVOCATION—GENERALLY.................................................................................................................... 49
NOTICE OF INTENT TO SUSPEND ............................................................................................................. 49
SUSPENSION OF ACCREDITATION ............................................................................................................ 51
COMMISSIONERS’ DECISION ON REVOCATION OF ACCREDITATION...................................................... 53
EFFECT OF REVOCATION OF ACCREDITATION ........................................................................................ 54
REQUESTING APPEAL............................................................................................................................... 55
EAC ACTION ON A REQUEST FOR APPEAL ............................................................................................. 55
SUBMISSION OF APPEAL........................................................................................................................... 55
CONSIDERATION OF APPEAL ................................................................................................................... 55
COMMISSIONER’S DECISION ON APPEAL ................................................................................................ 56
REQUESTS FOR INTERPRETATIONS ............................................................................................................... 58
6.1.
6.2.
6.3.
6.4.
6.5.
OVERVIEW ................................................................................................................................................ 58
POLICY...................................................................................................................................................... 58
REQUIREMENTS FOR SUBMITTING A REQUEST FOR INTERPRETATION .................................................... 58
PROCEDURE FOR SUBMITTING A REQUEST FOR INTERPRETATION ......................................................... 59
EAC ACTION ON A REQUEST FOR INTERPRETATION .............................................................................. 60
iii
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
6.6.
6.7.
EFFECT OF INTERPRETATION ................................................................................................................... 61
LIBRARY OF INTERPRETATIONS................................................................................................................ 62
RELEASE OF LABORATORY ACCREDITATION PROGRAM INFORMATION ....................................... 63
7.1.
7.2.
7.3.
7.4.
7.5.
7.6.
7.7.
OVERVIEW ................................................................................................................................................ 63
EAC POLICY ON THE RELEASE OF CERTIFICATION PROGRAM INFORMATION ...................................... 63
TRADE SECRETS ........................................................................................................................................ 64
PRIVILEGED OR CONFIDENTIAL COMMERCIAL INFORMATION .............................................................. 65
EAC’S RESPONSIBILITIES ......................................................................................................................... 65
VSTL’S RESPONSIBILITIES ........................................................................................................................ 66
PERSONAL INFORMATION ....................................................................................................................... 67
APPENDIX A............................................................................................................................................................. 69
APPENDIX B ............................................................................................................................................................. 72
APPENDIX C............................................................................................................................................................. 75
APPENDIX D............................................................................................................................................................. 77
APPENDIX E ............................................................................................................................................................. 79
APPENDIX F ............................................................................................................................................................. 82
iv
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Introduction
1.1. Background. In late 2002, Congress passed the Help America Vote Act of 2002 (HAVA).
HAVA created the U.S. Election Assistance Commission (EAC) and assigned to the EAC
the responsibility for both setting voting system standards and providing for the
voluntary testing and certification of voting systems. This mandate represented the first
time the Federal government provided for the voluntary testing, certification, and
decertification of voting systems nationwide. In response to this HAVA requirement, the
EAC has developed the voting system standards in the form of the Voluntary Voting
System Guidelines (VVSG), a voting system certification program in the form of the
Voting System Testing and Certification Program Manual and this document, the Voting
System Test Laboratory Manual.
1.2. Authority. HAVA Section 231(b) (42 U.S.C. §15371(b)) requires that the EAC provide for
the accreditation and revocation of accreditation of independent, non-federal laboratories
qualified to test voting systems to Federal standards. Generally, the EAC considers for
accreditation those laboratories evaluated and recommend by the National Institute of
Standards and Technology (NIST) pursuant to HAVA Section 231(b)(1). However,
consistent with HAVA Section 231(b)(2)(B), the Commission may also vote to accredit
laboratories outside of those recommended by NIST upon publication of an explanation of
the reason for any such accreditation.
1.3. Role of the National Institute of Standards and Technology. Section 231(b) (1) of HAVA
requires that the National Institute of Standards and Technology “conduct an evaluation
of independent, non-federal laboratories and shall submit to the Commission a list of
those laboratories…to be accredited….” Additionally, HAVA Section 231(c) requires
NIST to monitor and review the performance of EAC accredited laboratories. NIST has
chosen its National Voluntary Laboratory Accreditation Program (NVLAP) to carry out
these duties. NVLAP conducts a review of applicant laboratories in order to provide a
measure of confidence that such laboratories are capable of performing testing of voting
systems to Federal standards. Additionally, the NVLAP program monitors laboratories
by requiring regular assessments. Laboratories are reviewed one year after their initial
accreditation and biennially thereafter. The EAC has made NVLAP accreditation a
requirement of its Laboratory Accreditation Program. However, a NVLAP accreditation is
not an EAC accreditation. EAC is the sole Federal authority for the accreditation and
revocation of accreditation of Voting System Test Laboratories (VSTL).
1.4. Scope. This Manual provides the procedural requirements of the EAC voting system
Laboratory Accreditation Program. Although participation in the program is voluntary,
adherence to the program’s procedural requirements is mandatory for participants. The
procedural requirements of this Manual supersede any prior laboratory accreditation
requirements issued by the EAC. This manual shall be read in conjunction with the EAC
Voting System Testing and Certification Manual.
5
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
1.5. Manual Maintenance and Revision. The Manual will be reviewed periodically and
updated to meet the needs of the EAC, VSTLs, election officials, and public policy. The
EAC is responsible for revising this document. All revisions will be made consistent with
Federal law. Substantive input from stakeholders and the public will be sought whenever
possible. Changes in policy requiring immediate implementation will be noticed via
policy memoranda and will be issued to each VSTL and registered Manufacturers.
Changes, addendums, or updated versions will also be posted to the EAC Web site at
www.eac.gov.
1.6. Clarification of Program Requirements and Procedures. VSTLs and registered
Manufacturers may request clarification regarding the requirements and procedures set
forth in this manual. Requests for clarification must be based upon ambiguity arising
from the application of this manual. Hypothetical questions will not be considered.
Requests shall be submitted to the Program Director in writing. The request shall clearly
identify the section of the manual and issue to be clarified, a proposed interpretation and
all relevant facts. Clarifications issued by the EAC will be provided to all EAC VSTLs,
registered Manufacturers and placed on EAC’s Web site.
1.7. Program Personnel. All EAC personnel and contractors associated with this program will
be held to the highest ethical standards. All agents of the EAC involved in the
Accreditation Program will be subject to conflict-of-interest reporting and review,
consistent with Federal law and regulation.
1.8. Submission of Documents. Any documents submitted pursuant to the requirements of
this Manual shall be submitted:
1.8.1.
If sent electronically, via secure e-mail or physical delivery of a compact disk,
unless otherwise specified. The submitted electronic files shall be in Microsoft
Word or Adobe PDF format, formatted to protect the document from alteration.
1.8.2.
With a proper signature when required by this Manual. Documents that require
an authorized signature may be signed with an electronic representation or
image of the signature of an authorized management representative.
1.8.3.
If sent via physical delivery, by Certified Mail™ (or similar means that allows
tracking) to the following address:
Testing and Certification Program Director
U.S. Election Assistance Commission
1335 East West Highway
Suite 4300
Silver Spring, MD 20910
6
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
1.9. Receipt of Documents—VSTL. For purposes of this Manual, a document, notice, or other
communication is considered received by a VSTL upon one of the following:
1.9.1.
The actual, documented date the correspondence was received (either
electronically or physically) at the VSTL, or
1.9.2.
If no documentation of the actual delivery date exists, the date of constructive
receipt of the communication. For electronic correspondence, documents will be
constructively received the day after the date sent. For mail correspondence, the
document will be constructively received 3 days after the date sent.
1.9.3.
The term “receipt” shall mean the date a document or correspondence arrives
(either electronically or physically) at the VSTL’s place of business. Arrival does
not require that an agent of the VSTL open, read, or review the correspondence.
1.10. Receipt of Documents—EAC. For purposes of this Manual, a document, notice, or other
communication is considered received by the EAC upon its physical or electronic arrival
at the agency. All documents received by the agency will be physically or electronically
date stamped. This stamp shall serve as the date of receipt. Documents received after the
regular business day (5:00 PM Eastern Standard Time), will be treated as if received on the
next business day.
1.11. Record Retention—EAC. The EAC shall retain all records associated with accreditation of
Voting System Test Laboratories. The records shall otherwise be retained or disposed of
consistent with Federal statutes and regulations.
1.12. Publication and Release of Documents. The EAC will release documents consistent with
the requirements of Federal law. It is EAC policy to make the laboratory accreditation
process as open and public as possible. Any documents (or portions thereof) submitted
under this program will be made available to the public unless specifically protected from
release by law. The primary means for making this information available is through the
EAC Web site. See Chapter 7 of this Manual for additional information.
1.13. References. The following documents are referenced in this Manual. For dated
references, only the edition cited applies. For undated references, the latest edition of the
referenced document (including any amendments) applies.
- ISO/IEC 17011, Conformity assessment- General requirements for accreditation bodies
accrediting conformity assessment bodies.
- ISO/IEC 17025, General requirements for the competence of testing and calibration
laboratories.
7
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
- NIST Handbook 150, (NVLAP) Procedures and General Requirements.
- NIST Handbook 150-22, (NVLAP) Voting System Testing.
1.14. Definitions. For purposes of this Manual, the terms listed below have the following
definitions.
Applicant Laboratory. An independent, non-Federal laboratory which has applied for
EAC accreditation after receipt of an invitation.
Commission. The U.S. Election Assistance Commission, as an agency.
Commissioners. The serving commissioners of the U.S. Election Assistance
Commission.
Contracted Third Party Laboratory. A laboratory contracted or otherwise providing
testing services to a VSTL to meet program requirements.
Days. Calendar days, unless otherwise noted. When counting days, for the purpose of
submitting or receiving a document, the count shall begin on the first full calendar day
after the date the document was received.
Election Official. A State or local government employee who has as one of his or her
primary duties the management or administration of a Federal election.
Federal Election. Any primary, general, runoff, or special Election in which a candidate
for Federal office (President, Senator, or Representative) appears on the ballot.
Fielded Voting System. A voting system purchased or leased by a State or local
government that is being use in a Federal election.
Gift. A Gift includes any gratuity, favor, discount, entertainment, travel, service,
hospitality, loan, meal, forbearance, or other item having monetary value.
Integration Testing. The end-to-end testing of a full system configured for use in an
election to assure that all legitimate configurations meet applicable standards.
Key Laboratory Staff. Laboratory employees serving as approval authorities of test
reports (approved signatories per NIST Handbook 150) or otherwise responsible for the
supervision of individuals performing voting system testing.
8
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Lead Voting System Test Laboratory. The accredited Voting System Test Laboratory
identified on an EAC approved Application for Testing (EAC Voting System Testing and
Certification Program Manual, Sec. 4.3, Certification Application).
Manufacturer. The entity with ownership and control over a voting system submitted
for certification.
Memorandum for the Record. A written statement drafted to document an event or
finding, without a specific addressee other than the pertinent file.
Proprietary Information. Commercial information or trade secrets protected from
release under the Freedom of Information Act (FOIA) and the Trade Secrets Act.
Recommended Laboratory. A laboratory recommended for EAC accreditation by the
Director of NIST after evaluation by NVLAP.
Scope of Accreditation. The version or versions of the Federal voting system standards
(VVSG) to which a VSTL is authorized to test.
Technical Reviewers. Technical experts in the areas of voting system technology and
conformity assessment appointed by the EAC to provide expert guidance.
Testing and Certification Decision Authority. The EAC Executive Director or Acting
Executive Director.
Testing and Certification Program Director. The individual appointed by the EAC
Executive Director to administer and manage the Testing and Certification Program.
Voting System. The total combination of mechanical, electromechanical, and electronic
equipment (including the software, firmware, and documentation required to program,
control, and support the equipment) that is used to define ballots, cast and count votes,
report or display election results, interface the voting system to the voter registration
system, and maintain and produce any audit trail information.
Voting System Test Laboratories (VSTLs). Laboratories accredited by the EAC to test
voting systems to EAC approved voting system standards.
Voluntary Voting System Guidelines. Voluntary voting system standards developed,
adopted, and published by the EAC. The guidelines are identified by version number
and date.
1.15. Acronyms and Abbreviations. For purposes of this Manual, the acronyms and
abbreviations listed below represent the following terms.
9
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Accreditation Program. The EAC Voting System Test Laboratory Accreditation
Program
Certification Program. The EAC Voting System Testing and Certification Program
EAC. United States Election Assistance Commission
FEC. Federal Election Commission
HAVA. Help America Vote Act of 2002 (42 U.S.C. §15301 et seq.)
ISO/IEC. The International Organization for Standardization & The International
Electrotechnical Commission.
NASED. National Association of State Election Directors
NIST. National Institute of Standards and Technology
NVLAP. National Voluntary Laboratory Accreditation Program
Program Director. Director of the EAC Testing and Certification Program
VSTL. Voting System Test Laboratory
VVSG. Voluntary Voting System Guidelines
10
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Program Requirements
2.1. Overview. This chapter lists the requirements of the EAC’s Voting System Test
Laboratory Program. Adherence to these requirements is a condition of accreditation and
a continuing obligation. Failure to demonstrate compliance with the requirements of this
chapter may result in the denial of an application for accreditation, suspension of
accreditation, or revocation of accreditation.
2.2. Program Requirements - Generally. In order to be considered for, receive, and maintain
an EAC accreditation as a VSTL, laboratories must demonstrate compliance with the
requirements of EAC’s Voting System Test Laboratory Program. The program
requirements are set forth in this Chapter.
2.2.1.
Continuing Compliance Obligation. VSTLs have a continuing obligation to meet
the requirements set forth in this Chapter. VSTLs are required to maintain their
compliance with the program’s requirements as long as they hold an EAC
accreditation.
2.2.2.
Requests to Document Compliance. VSTLs may be required by the EAC to
document compliance at any time. Such requests will be in writing and VSTLs
shall respond timely, consistent with the request (see Chapter 4 of this Manual).
2.2.3.
Failure to Comply, Effect. Failure to meet each of the program’s requirements
may result in the denial of an application for accreditation, suspension of
accreditation, or revocation of accreditation, consistent with the procedures of
Chapter 5 of this Manual.
2.3. NIST Recommendation. As a condition of accreditation, all laboratories must be
recommended to the EAC by the National Institute of Standards and Technology (NIST),
unless the emergency provisions of Chapter 3 apply. NIST is responsible, pursuant to the
Help America Vote Act of 2002, Section 231(b), for performing a technical evaluation of
laboratories and identifying and recommending those competent to test voting systems.
This recommendation is provided directly to the EAC from NIST.
2.4. NVLAP Accreditation. As a condition of accreditation, all VSTLs must hold a valid
accreditation from NIST’s National Voluntary Laboratory Accreditation Program
(NVLAP), unless the emergency provisions of Chapter 3 apply. NVLAP accreditation is
the primary means by which the EAC may ensure that each VSTL meets and continues to
meet the technical requirements of the EAC program. It sets the standards for each
VSTL’s technical, physical and personnel resources, as well as its testing, management,
and quality assurance policies and protocols. The loss or suspension of a NVLAP
accreditation will result in the suspension and possible revocation of any EAC
accreditation consistent with the procedures of Chapter 5 of this Manual. VSTLs are
11
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
required to immediately report any change in their NVLAP accreditation status to the
EAC. Whenever possible, the EAC will conduct the required accreditation audit and any
follow up on site visits at the same time as NVLAP accreditation audit or follow up on site
visits.
2.5. Conflict of Interest and Prohibited Practices Program. As a condition of accreditation, all
laboratories must maintain and enforce policies which prohibit and prevent conflicts of
interest or the appearance of conflicts of interest. A laboratory shall ensure that neither
the Laboratory, its parent corporation, contracted third party laboratories, nor any
individual staff member involved in the testing of voting systems have any vested interest
in the outcome of the test process. Laboratories must have a written policy in place. This
policy must, at a minimum, (1) prohibit conflicts of interest and other prohibited practices
and (2) provide for enforcement, consistent with the subsections below.
2.5.1.
Prohibited Conflicts of Interest. The purpose of a conflict of interest policy is to
prevent situations where the exercise of an official duty directly impacts the
actor’s financial interests. For the purposes of this program, a prohibited
conflict of interest exists if the duties and responsibilities of a laboratory,
parent corporation, or a laboratory employee involved in the testing of voting
systems under EAC’s Certification Program will have a direct and predictable
effect on the financial interest of that laboratory, parent corporation, or a
laboratory employee. 1 For example, an employee who is responsible for testing a
voting system on behalf of a VSTL would be prohibited from holding a financial interest
in the entity whose product is being tested or a direct competitor of that entity. A
prohibited conflict of interest would also include a contractual or other fiduciary
relationship between a VSTL or VSTL employee and a Manufacturer (outside an
agreement for State or Federal certification testing) when that VSTL or VSTL employee is
concurrently responsible for conducting certification testing for that Manufacturer under
this program. Additionally, financial interests may be imputed or attributed to
a laboratory, Parent Corporation, or a laboratory employee through a
relationship with a third party. For example, a VSTL employee responsible for the
testing of a voting system would be conflicted from performing his or her duties if his or
her spouse owned a financial interest in the manufacture of the voting system.
2.5.1.1. Involved in Testing—Defined. For the purposes of a financial conflict of
interest, an organization is involved in the testing of a voting system
any time it contractually or otherwise takes on the responsibility for
testing a voting system to Federal standards under EAC’s Certification
Program. For the purposes of a financial conflict of interest, an
For the purpose of this Program, agreements with voting system manufactures to provide testing
pursuant to the requirements of EAC or a State’s certification program do not constitute a prohibited
conflict of interest. Certification testing is considered a duty and responsibility of a VSTL, not an outside
financial interest.
1
12
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
employee is involved in the testing of a voting system when the
individual’s duties as a VSTL employee require him or her to perform
testing on the system, manage the testing process or supervise those
who perform testing on the system.
2.5.1.2. Financial Interest--Defined. The term includes any current or contingent
ownership, equity, or security interest in real or personal property or a
business and may include indebtedness or compensated employment
relationship. It thus includes, for example, interests in the nature of
stocks, bonds, partnership interests, fee and leasehold interests, and
other property rights, deeds of trust, and liens, and extends to any right
to purchase or acquire any such interest, such as a stock option or
commodity future.
2.5.1.3. Direct Effect—Defined. A matter will have a direct effect on a financial
interest if there is a close causal link between any decision or action to
be taken in the matter and any expected effect of the matter on the
financial interest. An effect may be direct even though it does not occur
immediately. A matter will not have a direct effect on a financial
interest, however, if the chain of causation is attenuated or is contingent
upon the occurrence of events that are speculative or that are
independent of, and unrelated to, the matter. A matter that has an effect
on a financial interest only as a consequence of its effects on the general
economy does not have a direct effect within the meaning of this
section.
2.5.1.4. Predictable Effect—Defined. A matter will have a predictable effect if
there is a real, as opposed to a speculative possibility that the matter
will affect the financial interest. It is not necessary, however, that the
magnitude of the gain or loss be known, and the dollar amount of the
gain or loss is immaterial.
2.5.1.5. Imputed Interests—Defined. An imputed interest is a financial interest
held by a third party individual or organization that serves to disqualify
an employee or laboratory to the same extent as if they were the
employee’s or laboratory’s own interest. These interests include:
2.5.1.5.1.
The financial interests of a spouse or dependent child shall
be imputed to an employee.
2.5.1.5.2.
The financial interest of any organization in which a
laboratory, Parent Corporation, or a laboratory employee
serves as an employee, officer, board member, partner,
13
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
consultant, director, trustee or similar position shall be
imputed.
2.5.2.
2.5.1.5.3.
The interests of any contracted third party laboratory shall
be imputed to the utilizing VSTL.
2.5.1.5.4.
The financial interest of a person or organization with whom
an employee is negotiating or has an arrangement
concerning prospective employment shall be imputed.
Prohibited Practices. Furthermore, irrespective of the existence of a conflict of
interest, it is a prohibited practice for a laboratory, parent corporation, or
laboratory employee to be involved in the development of a voting system or
solicit or receive a gift from a voting system Manufacturer. No laboratory,
parent corporation, or laboratory employee may:
2.5.2.1. Voting System Development and Testing. Provide, or have provided,
consultation, developmental testing or other services to a voting system
developer such that the independence, or appearance of independence,
in the testing of a particular voting system or system component would
be compromised.
2.5.2.1.1.
A laboratory or individual may not be involved in both the
development of a voting system and the certification of a
system. Voting system development includes any testing,
consultation or design work performed in order to ready a
specific system for the marketplace or the certification
process. Generally, any testing performed on behalf of a
voting system manufacture that was not otherwise
performed pursuant to a State or Federal voting system
certification program will be considered developmental in
nature.
2.5.2.1.2.
The prohibition barring participation in both development
and testing is voting system specific. An employee or
laboratory that was previously involved 2 in product
development with a Manufacturer is not prohibited from
testing all systems produced by that Manufacturer, just
those systems in which the employee or laboratory
participated directly in development. As voting systems are
The prohibition relates to a VSTL’s prior involvement in system development. Concurrent development
work and testing may constitute a prohibited conflict of interested under Section 2.5.2 of this Manual.
2
14
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
subject to change over time, for the purposes of this
prohibition, a voting system shall be considered altered to
the degree that it is a different system when:
2.5.2.1.3.
2.5.2.1.2.1.
A period of at least three years has passed
since the VSTL or employee was involved in
the system’s development;
2.5.2.1.2.2.
The system has been subject to both software
and hardware modification since the VSTL or
employee was involved in the system’s
development. De minimis changes (as
defined in EAC Voting System Testing and
Certification Program Manual) are not
modifications; AND
2.5.2.1.2.3.
The system has received a certification after
being tested by a different independent
laboratory since the VSTL or employee was
involved in the system’s development.
The prohibition barring participation in both development
and testing does not prohibit a VSTL from allowing a
Manufacturer to perform onsite hardware mitigation on a
voting system in response to a minor system failure or
anomaly. In such cases the VSTL:
2.5.2.1.3.1.
Shall suspend all hardware testing;
2.5.2.1.3.2.
Shall not participate or assist the
Manufacturer in remediation;
2.5.2.1.3.3.
May provide testing equipment and qualified
operators to the Manufacturer for its use;
2.5.2.1.3.4.
Shall monitor and document the
Manufacturer’s access to the system
consistent with Section 2.11.1. of this manual;
and
2.5.2.1.3.5.
Shall document in the test report the failure or
anomaly and remedial action taken by the
Manufacturer consistent with Section
15
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
2.10.5.2.1 of this Manual and Chapter 4 of
EAC’s Certification Manual (anomaly matrix).
2.5.2.2. Gifts. Solicit or receive a gift, directly or indirectly, from any entity
which holds a financial interest in the development, production, or sale
of voting systems, or is otherwise impacted by the testing and
certification of voting systems. Gifts given or received under
circumstances which make it clear that the gift is motivated by a family
relationship or personal friendship rather than position are not
prohibited. Relevant factors in making such a determination include the
history of the relationship and whether the family member or friend
personally pays for the gift.
2.5.3.
Program Enforcement Elements. Prohibited conflicts and practices shall be
enforced through a written program which:
2.5.3.1. Regarding Employees Involved in the Testing of Voting Systems.
2.5.3.1.1.
Annually collects standard information from each employee,
including assets, debts, outside or prior
activities/employment, gifts, and any work on voting system
development sufficient to demonstrate compliance with
Section 2.5.1. and 2.5.2. of this Manual. The information
collection must also reflect the financial interests of those
individuals (like spouses and minor children) whose
interests are imputed to the employee;
2.5.3.1.2.
Requires and documents the review of information collected
for potential conflicts and prohibited practices; and
2.5.3.1.3.
Resolves all identified conflicts of interest or prohibited
practices prior to the employee or laboratory’s involvement
in the testing of any voting system. Such resolution shall be
documented. Resolutions may include the divestiture of
assets or gifts, employee resignation from outside
organizations, or the altering of an employee’s
responsibilities by prohibiting participation in Voting
System Testing or the testing of a specific system.
2.5.3.2. Regarding the VSTL or VSTL’s Parent Corporation.
2.5.3.2.1.
Annually collects information pertaining to the holdings and
activities of the VSTL and its parent corporation(s), sufficient
16
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
to demonstrate compliance with Section 2.5.1. and 2.5.2. of
this Manual;
2.5.3.2.2.
Requires and documents the review of collected information
for potential conflicts and prohibited practices; and
2.5.3.2.3.
Resolves all identified conflicts of interest or prohibited
practices prior to the laboratory’s testing of any voting
system. Such resolution shall be documented. Resolutions
may include the divestiture of assets or gifts, the termination
or rejection of conflicted or prohibited testing work.
2.5.3.3. Regarding Contracted Third Party Laboratories. The interest of a contracted
third party laboratory may be imputed to a VSTL. VSTLs may meet and
enforce the program requirements of this section with regard to this
relationship in one of two ways:
2.5.3.3.1.
Collection of third party laboratory information, review of
information and resolution of conflicts or prohibited
practices:
2.5.3.3.1.1.
Collect information pertaining to the holdings
and activities of the third party laboratory
and its employees, sufficient to demonstrate
compliance with Section 2.5.1. and 2.5.2. of
this Manual. This includes gathering
information concerning any involvement by
the third party laboratory or its employees in
the development of specific voting systems.
This collection of information shall be
performed prior to the execution of any
contract for the testing of voting systems
under this program and annually thereafter if
the contract exceeds one year in duration.
2.5.3.3.1.2.
Require and document the review of collected
information for potential conflicts, and
2.5.3.3.1.3.
Resolve all identified conflicts of interest prior
to the laboratory’s testing of any voting
system.
17
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
2.5.3.3.2.
2.5.4.
VSTL Supervision of third party laboratories performing
non-core testing. Where a third party laboratory is subject to
direct VSTL supervision and observation, the third party
laboratory’s conflicts of interest or prohibited practices will
not be imputed to the lead VSTL. Direct VSTL supervision
under this section requires that a VSTL employee is
physically present during the third party testing and directly
observes and supervises the testing. This VSTL employee
must: (1) have been properly vetted for conflict of interest
and prohibited practices pursuant to Section 2.5 of this
Manual, (2) be competent to supervise the testing being
performed and (3) have no financial interest in the third
party laboratory they are supervising.
Waivers. In rare circumstances, prohibited practices or conflicts of interest may
be waived by the EAC after the conflict or prohibited practice is properly
disclosed to the agency. Waivers may be granted at the sole discretion of the
Program Director.
2.5.4.1. Requesting a Waiver. A request for a waiver shall be made in writing to
the EAC Program Director. The request shall fully disclose the conflict
of interest or prohibited practice for which the waiver is sought. The
request shall also describe all steps taken to resolve the conflict or
prohibited practice and the reasons why such attempts were
unsuccessful or otherwise untenable. The request shall also state why
the waiver should be granted, consistent with the standard in Section
2.5.4.2.
2.5.4.2. Waiver Standard. A disqualifying conflict of interest or prohibited
practice is subject to waiver when the issuance of a waiver is in the best
interest of the EAC Certification Program and the identified conflict or
practice is unlikely to affect the integrity or impartiality of the VSTL or
VSTL employee’s services under the EAC Certification Program. The
Program Director may consider the following factors in making a
waiver determination:
2.5.4.2.1.
The value of any disqualifying financial interest;
2.5.4.2.2.
The nature and impact of any prohibited practice;
2.5.4.2.3.
The role and responsibility of the employee subject to the
conflict of interest or prohibited practice;
18
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
2.5.4.2.4.
The availability of other employees, VSTLs or laboratories to
conduct the testing without a conflict or prohibited practice.
2.5.4.2.5.
The level of discretion or sensitivity required to perform the
conflicted or prohibited duties under the certification
program;
2.5.4.2.6.
The ability of an EAC waiver to adjust a VSTL or VSTL
employee’s testing process and duties or otherwise mandate
additional safeguards which would limit or abrogate the
impact of the conflict of interest or prohibited practice.
2.5.4.3. Issuing a Waiver. Any waiver issued by the Program Director shall be
made in writing to the requestor. The waiver shall state with specificity
the conflict of interest or prohibited practice waived. The waiver shall
also clearly state any conditions for it issuance, such as mitigating
processes or procedures or safeguards. The VSTL is responsible for
meeting all waiver conditions prior to engaging in the waived activity.
Failure to meet such condition may result in the revocation of a VSTLs
accreditation. The Program Director shall publish all waivers on the
EAC Web site.
2.5.4.4. Denying a Request for a Waiver. Any decision denying a request for a
waiver shall be made by the Program Director in writing and provided
to the VSTL. The Program Director shall publish all waiver denials on
the EAC Web site.
2.6. Personnel Policies. As a condition of accreditation, all laboratories shall have in place
written policies to ensure that the Laboratory does not employ individuals, in any capacity
related to the testing of voting systems, who have been convicted of a felony offense or
any criminal offense involving fraud, misrepresentation, or deception under either Federal
or State law. The VSTL shall have a program in place to enforce this policy and document
such enforcement.
2.7. Notification of Changes. As a condition of accreditation, all laboratories shall agree to
notify the EAC in writing within fifteen (15) calendar days of any significant changes in
laboratory operations from what the Laboratory described in any assertion that served as
the basis for its EAC accreditation, including any assertions made to NIST’s NVLAP or to
the EAC pursuant to Chapter 3 of this Manual. Examples of events that require written
notification include, but are not limited to:
2.7.1.
A Laboratory’s decision to withdraw from the EAC’s program;
19
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
2.7.2.
Changes in ownership of the Laboratory (other than minor—less that 15%—
change in stock ownership),
2.7.3.
A change in location of the Laboratory facility, or
2.7.4.
Personnel changes in key staff positions.
2.8. Site Visits. As a condition of accreditation, all laboratories shall allow EAC
representatives to enter their voting system testing and management facilities pursuant to
the procedures and requirements of Chapter 4 of this Manual.
2.9. Notice of Lawsuits. As a condition of accreditation, all laboratories shall provide notice to
the EAC of any lawsuits or claims filed against it, its subcontractors, subsidiaries,
employees, officers, owners, operators, or insurers while the Laboratory holds an EAC
accreditation and which relate to the work performed in, or management of, the
Laboratory’s voting system testing program.
2.10. Testing, Technical Practices and Reporting. As a condition of accreditation, each VSTL
shall perform testing in conformance with the relevant standards of the applicable Federal
Standards (VVSG). Additionally, the VSTL shall create written reports of such testing
consistent with the requirements of the latest version of the VVSG, EAC’s Voting System
Testing and Certification Manual, any applicable test suites mandated by the EAC, and any
other written guidance published by the EAC.
2.11. Test Readiness Review. The Test Readiness Review (TRR) is the mechanism used by the
EAC to ensure that test and evaluation resources are not committed to a voting system
that is not ready for testing by a VSTL. The TRR determines if the submitted voting
system and documentation are ready to enter certification testing. The TRR shall be
completed by the VSTL and the subsequent Test Readiness Acknowledgement must be
received by the EAC prior to the initiation of any certification testing. The TRR does not
apply to modifications. To assess the readiness of a voting system for certification testing,
the VSTL shall review:
•
•
System Technical Data Package (TDP): The voting system technical data package shall
be reviewed to ensure all elements required by the VVSG are present.
System Components: The VSTL shall review the submitted voting system to ensure all
components required to configure the voting system as defined in the system TDP are
delivered to the VSTL and appear to be operational and in good working order. System
Component information should match the Manufacturer’s application submitted to the
EAC. All components submitted for testing must be equivalent to the final production
model of the voting system in fit, form and function. Any component not available at
the time of this review shall be delivered to the VSTL by the voting system manufacturer
20
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
•
•
•
within 30 days of the initial TRR, or testing of the system will be halted and the EAC
notified that the system is not ready for testing.
Preliminary Source Code Review: The VSTL shall conduct a preliminary review of no
less than 1% of the total lines of code (LOC) of every software package, module or
product submitted for testing in order to ensure that the code is mature and does not
contain any systematic non-conformities.
Mark Reading: The system shall be able to read a fully filled mark if it is an optical scan
system.
Summary of COTS components. This summary should outline which components of
the voting system are COTS products and shall be updated with each test campaign.
2.11.1. Test Readiness Notification. Upon completion of the TRR, the VSTL shall submit
a signed statement to the EAC confirming that the voting system completed the
TRR and the VSTL determined that the system is ready for certification testing to
applicable Voluntary Voting System Guidelines.
2.11.2. Test Readiness Acknowledgement. Upon receipt of the Test Readiness
Notification from the VSTL, the EAC shall issue an acknowledgement in writing
stating that the VSTL and manufacturer may commence certification testing.
This acknowledgement will be issued within 3 business days of receipt of the
Notification.
2.12. Technology Testing Agreement. The VSTL shall participate in all meetings related to
development of Technology Testing Agreements. VSTLs are expected to participate and
sign on to the agreement reach between the EAC, manufacturer and VSTL.
2.13. Test Plan Package. The VSTL shall submit a test plan package directly to the EAC
consistent with the requirements of the Voting System Testing and Certification Manual, the
latest version of the VVSG, this Manual and any other written guidance from the EAC. A
test plan package includes:
2.13.1. Virtual Review Tool (VRT). The VRT is a web based application developed by
the EAC which, in addition to other information, identifies each requirement
found in VVSG. VSTLs will be required to use the tool to identify the standards
that apply to the system being tested, identify the testing to be performed and
provide additional information as required. The EAC will provide log-in
information and grant specific access to VSTL staff upon accreditation of the
laboratory, and to manufacturer representative upon the acceptance of a
manufacturer’s registration with the EAC. The VRT will serve as both a tool to
identify and a means to document what should be tested and how.
2.13.2. Test Plan. The purpose of the Test Plan is to provide information regarding test
methods. The Test Plan contains more detail than the VRT.
21
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
2.13.2.1. Format. VSTLs shall format each test plan consistent with the
requirements of Appendix A of this Manual.
2.13.2.2. Content. Each test plan shall identify applicable voting system
standards and contain a description of the testing proposed to verify
conformance. Also, each test plan shall contain a statement indicating
the scope of the labs accreditation.
Required Content. For each test, the test plan shall
provide detailed information referencing testing to be
performed, including facility requirements, test set-up,
test sequence, data recording requirements and pass
criteria. 3
Exception. Where a VSTL utilizes EAC mandated or
approved test methods, the test plan may simply
reference these methods and identify, with specificity,
all deviations. Mandated test methods are those test
methods required for use by the EAC. Approved test
methods are standard, verified VSTL test methods
approved by the EAC. VSTLs may submit standard
test methods for approval by submitting them in
writing to the Program Director.
2.13.3. Test Case. After approval of the VSTLs Test Plan, the VSTL shall develop Test
Cases. A Test Case is a system specific, step-by-step test procedure or laboratory
testing process that provides detailed test operation procedures sufficient for
trained laboratory personnel to fully conduct a given test and produce repeatable
results. The VSTL shall inform the EAC, in writing, when all test cases for the
voting system under test have been completed. This notice shall include an
index identifying each test case created to test the system. The notification
should indicate if these are standard test cases, modified standard test cases, or a
new test case. These test cases shall be available to the EAC for review and
approval upon request.
2.14. Testing. The highest standards shall be applied to the testing of voting systems. VSTLs
shall perform testing in conformance with the relevant standards of the applicable Federal
Standards (VVSG) and consistent with any written EAC interpretations of these standards.
This requirement is consistent with International Standards Organization requirements, which serve as
a basis for NIST NVLAP’s accreditation and recommendation to the EAC. Where established and
approved test methods do not exist, ISO Standard 17025, Section 5.4.4., Non-Standard Method requires the
testing to be validated by the laboratory prior to use. The EAC will review and approve the validated test
methods.
3
22
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
VSTLs shall test system identification tools during the test campaign to make sure they
function properly and as intended. The Laboratory shall maintain its technical practices
consistent with the standards which served as the basis for its NVLAP accreditation.
These standards include International Standard ISO/IEC 17025, General Requirements for the
Competence of Testing and Calibration Laboratories; NIST Handbook 150, Procedures and
General Requirement; NIST Handbook 150-22, Voting System Testing; any documents
supplementing, updating or replacing these standards or handbooks; and any pertinent
EAC guidance. When conducting testing under EAC’s program, VSTLs shall only
perform testing of voting systems consistent with the scope of their accreditation.
2.14.1. Third Party Testing. Lead VSTL’s may contract or otherwise provide for the
testing of voting systems by third parties under this program. However, the lead
VSTL shall be responsible for the accuracy, quality assurance, and results of all
tests performed. Under this program, no VSTL may perform or contract for the
performance of testing outside the scope of its accreditation. Testing performed
directly by lead VSTL personnel using third party contractor equipment and
facilities is not considered third party testing.
2.14.1.1. Core Testing. Core voting system testing may only be performed by
VSTLs. Core testing includes: Technical Data Package review, physical
configuration audit, source code review, functional configuration audit,
system integration testing, volume testing, and security testing (not
including cryptographic testing).
2.14.1.2. Non-Core Testing. Non-core testing may be performed by non-VSTLs if
they hold an EAC recognized accreditation to perform the relevant
testing. The EAC recognizes two national accreditation bodies, NIST’s
NVLAP program and the American Association of Laboratory
Accreditation (A2LA). Generally, a VSTL may only contract or
otherwise provide for the non-core testing of voting systems if it uses a
NVLAP or A2LA laboratory accredited to the specific scope of testing
necessary. Non-core testing includes: electromagnetic compatibility
testing, telecommunications testing, environmental testing, electrical
testing, acoustical testing, accessibility testing, usability testing, and
cryptographic testing. 4 In limited circumstances, laboratories not
holding a recognized accreditation may be used by VSTLs for non-core
testing only after approval by EAC’s Program Director. Requests for
such approval must be made in writing and demonstrate: (1) That there
is no recognized laboratory available within a reasonable window of
availability and geographic proximity (generally within the continental
For the purposes of the EAC’s Voting System Test Laboratory Program, non-core cryptographic
testing includes all testing involving evaluation of cryptographic operation and key management.
4
23
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
United States) and (2) that the VSTL has conducted a thorough
assessment of the third party laboratory’s capabilities, quality system,
management system, and/or alternative accreditations and have
determined and documented that the laboratory is qualified to perform
testing. The EAC may visit, interview or audit any non-accredited
laboratory at any time before, during, or after the testing has occurred
to verify their qualifications.
2.14.1.3. VSTL Responsibilities. Lead VSTLs are responsible for all tests performed
on voting systems submitted to them by Manufacturers under EAC’s
Testing and Certification Program. This includes testing (both core and
non-core) performed by third party laboratories under their direction
(including third party VSTL laboratories). Any procedural or
substantive irregularities or errors which occur during the third party
testing process will be imputed to the responsible lead VSTL. Such
failures may serve as a basis for the revocation of accreditation. Lead
VSTLs using third party laboratories (consistent with Sections 2.10.4.1
through 2.10.4.2, above) shall take steps to ensure that the third party
laboratories they employ meet the standards of this Program. At a
minimum, the lead VSTLs shall ensure:
2.14.1.3.1. The third party laboratory provides the lead VSTL verifiable
documentation regarding its relevant accreditation;
2.14.1.3.2. Any hardware tested by the qualified third party laboratory
is first validated by the lead VSTL as the same hardware
presented to it for certification;
2.14.1.3.3. The third party laboratory provides the lead VSTL with
evidence that it will direct its activities in compliance with
any and all relevant VVSG requirements for testing and that
the testing was, in fact, performed consistent with such
specific requirements. Any special procedures, tools, or
testing software necessary to meet VVSG requirements must
be validated by the lead VSTL prior to use. For example, the
VVSG requires that systems be tested while operating and that
such operation be in manner and under conditions that simulate
election use. In such cases, the lead VSTL must ensure that the
third party laboratory will properly implement the VVSG
requirements, validate its election simulation tools, and properly
performed the testing;
24
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
2.14.1.3.4. The lead VSTL performs all system accuracy, reliability,
functionality and integration testing; and
2.14.1.3.5. The third party laboratory issues a report to the lead VSTL
that fully documents its testing such that the lead VSTL may
demonstrate compliance with this section and produce a
report consistent with Section 2.10.5 of this Manual.
2.15. Test Report Package. The Test Report Package represents the culmination of the testing
process. As such, it is vital that it accurately and completely document the testing
performed and the results of such testing. VSTLs shall submit Test Report Packages
directly to the EAC. The packages shall include:
2.15.1. Virtual Review Tool (VRT). VSTLs shall update the VRT information originally
submitted with its test plan (see Section 2.13 above). The final updates to the
VRT will serve as verification that the VSTL performed the testing required to
demonstrate compliance with voting system standards.
2.15.2. Test Report. VSTLs shall provide a test report.
2.15.3. Content. All test reports shall document the testing process, including the
documentation and justification of any divergence from the EAC approved test
plan, methods, or cases and the identification of all failures and/or anomalies
along with any remedial action taken 5 (see Chapter 4 of the EAC’s Voting System
Testing and Certification Manual regarding the anomaly matrix). VSTLs shall not
include any proprietary test cases in the Test Report. Test cases shall be uploaded
to the VRT as requested by the EAC. Test reports shall also document any
prescribed maintenance or modifications, performed by the Manufacturer, to a
voting system in testing. Such maintenance or modifications shall be monitored
by the VSTL consistent with Section 2.11.1 of this Manual.
2.15.4. Format. To the greatest extent possible, VSTLs shall write reports such that they
are understandable to non-technical persons. As the EAC will publish these
reports (barring portions prohibited by law), VSTLs shall refrain from including
in them trade secrets or other commercial information protected from release
unless substantively required. Where information protected from release may be
included, it shall be identified consistent with Chapter 7 of this Manual. VSTLs
shall format each test report consistent with the requirements of Appendix B of
this Manual.
VSTLs must report all errors and anomalies identified in the test campaign even when an error is
identified during the testing of unrelated functionality.
5
25
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
2.15.5. VSTL Attestation. The VSTL shall provide a letter, signed by a representative
authorized to take action on behalf of the VSTL (see Sections 2.13 and 3.4.1.6. of
this Manual), which attests that (1) all testing prescribed by the test plan or
amended test plan was performed as identified or the divergence from the test
plan was properly documented, (2) all identified voting system anomalies or
failures were reported and resolved, (3) that the test report is accurate and
complete, and (4) the VSTL recommends the system for certification.
2.16. Acceptance of Prior Testing. Testing previously performed on a voting system by a VSTL
or by a third party test laboratory operating at the direction of a VSTL, may be reused at
the discretion of the lead VSTL and the EAC. The EAC encourages VSTLs to use such
testing to fulfill certification requirements. The VSTL must attain written approval from
the EAC for all reuse requests. In order for the EAC to accept prior testing, lead VSTLs
must provide evidence that the requirements below are met. Prior testing is valid when:
2.16.1. The discrete software or hardware component of the voting system previously
tested is demonstrably identical to the voting system presently offered for
testing. Lead VSTLs must examine and/or compare the components and
documentation to ensure there is no change in the voting system. When valid
prior testing is used, the system presented must be subject to regression testing,
functional testing and system integration testing, and any other testing deemed
necessary to ensure compliance with the VVSG and this manual;
2.16.2. The requirements, standards and relevant EAC Request for Interpretations
applicable to the prior and current testing are identical;
2.16.3. The test methods used are equivalent or identical to current test methods
accepted by the EAC;
2.16.4. The prior testing was reviewed by the VSTL, with no apparent errors or
omissions and fully complies with the VVSG and this manual;
2.16.5. Testing from previous EAC test campaigns can only be submitted for reuse if the
EAC accepted a final test report for that campaign; and
2.16.6. The use of prior testing must be noted in the test plan, with test report titles,
numbers, and descriptions, along with EAC approval.
2.16.7. The use of prior testing must be noted in the body of the test report. Like all
testing, prior testing is subject to EAC review and approval.
26
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
2.17. Termination of Testing Prior to Completion. In the event testing is terminated prior to
completion, VSTLs are required to notify the EAC Program Director. This notification
shall be in writing and state the reasons for termination provide a list of all testing
completed, and produce a matrix of test anomalies or failures pursuant to Section 4.5.2 of
the EAC Testing and Certification Program Manual.
2.17.1. Termination Defined. Voting system testing shall be considered terminated
when the testing process is permanently ended or otherwise halted without a
specific plan to recommence within 120 calendar days of the last test performed.
2.17.2. Effect of Termination. Notification of termination will result in the suspension of
the Manufacturer’s Certification Application. Additionally, the termination and
VSTL’s written notice shall be posted on EAC’s Web site.
2.17.3. Resubmission after Termination. Manufacturers may resubmit a system
previously terminated by submitting an updated application consistent with
Chapter 4 of the Voting System Testing and Certification Program Manual. Pursuant
to Section 2.11 of this Manual and Section 4.3.1.2 of the Voting System Testing and
Certification Program Manual, a system resubmitted to the EAC after termination
must be tested by the VSTL identified on the original application.
2.18. VSTL Verification of Trusted Build. At the conclusion of each test campaign, VSTLs
shall verify the trusted build and associated materials required to be escrowed in the EAC
Repository (See Section 5.5 of the Testing and Certification Program Manual.) The verification
process shall include:
2.18.1. Catalog all files contained in the escrow package and confirm the ability to read
the media.
2.18.2. Test the functionality of the compile to be deposited.
2.19. Laboratory Independence. As a condition of accreditation, all laboratories shall maintain
their independence from voting system Manufacturers, consistent with their roles and
responsibilities as a key component of the EAC Certification program. VSTLs shall
maintain an arm’s length relationship with the manufactures and avoid even the
appearance of improper conduct. In order to maintain independence, VSTLs shall adhere
to the following independence principles and requirements:
2.19.1. Testing Independence. Consistent with the requirements of this Manual, only
the lead VSTL identified on a voting system’s application form may test or
oversee the testing of that system. Under no circumstances may a Manufacturer
perform or participate in any testing which will serve as the basis of an EAC
certification. Participation includes but is not limited to the observation of testing by
27
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
the Manufacturer. 6 Additionally, lead VSTL’s shall ensure that Manufactures’ do
not have access to a system under test unless accompanied and monitored by a
VSTL representative. The EAC recognizes that in some cases there is value in
allowing manufacturers to witness a particular test or a re-creation of a test in
order to allow them to comment on the proper system set up or operation.
However, any such participation must be (1) at the discretion of the VSTL, (2)
supervised by the VSTL and (3) clearly documented in order to maintain
laboratory independence. Therefore, the EAC finds the following three
situations to be allowable under this Section:
2.19.1.1. The VSTL may at any time, and at its own discretion, halt an active
certification test and bring the manufacturer into the testing room for a
re-creation of the test being performed. If the VSTL chooses to do this it
must:
2.19.1.1.1. Document the time and circumstance that cause a halt in
testing.
2.19.1.1.2. Document the reason why the manufacturer’s presence is
needed.
2.19.1.1.3. Document the result of the test prior to re-creating the test
for the manufacturer.
2.19.1.1.4. Document any re-running of the official EAC Certification
Test. This documentation must include any change that
occurred to the “as run” test case as a result of the recreation and the result of the official test.
2.19.1.1.5. Have the test supervisor in charge of the project present for
the re-creation of the test. If the test engineer conducting
the test is also the test supervisor in charge of the project,
one other VSTL employee must be present in the room
during the re-creation of the test. The test supervisor
present should be the most senior engineer or personnel
assigned to the testing engagement. Documentation of the
re-creation of the test should include lab personnel present
at the time of the re-creation.
Not all activities required for EAC Certification are “testing” activities. Examples of certification
requirements that do not fall into the category of “testing” include trusted and witness builds.
6
28
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
2.19.1.1.6. All documentation must be retained according to NVLAP
and EAC requirements.
2.19.1.2. The VSTL may, at its own discretion, create for the manufacturer either
a closed circuit video feed or web cam feed of the official EAC
Certification Testing being conducted and allow for real time
correspondence between test engineers and the manufacturers
provided that:
2.19.1.2.1. All correspondence (i.e., letters, emails, memos, recorded
video calls, etc.) between the test engineers and the
manufacturer is documented and retained.
2.19.1.2.2. Any changes to the testing that results from correspondence
between the manufacturers and the VSTL is signed off by
the VSTL project manager and provided to the EAC as part
of the test report package.
2.19.1.3. The VSTL may, at its discretion, provide supervised access to the
manufacturer prior to and during the official EAC Certification Testing
to perform unscheduled and non-routine maintenance provided that:
2.19.1.3.1. All documentation related to the maintenance activities is
recorded within the "as run" test case.
2.19.1.3.2. Any unscheduled maintenance that is performed is
documented in the discrepancy report included as part of
the test report materials.
2.19.2. Decision Making. Determinations regarding testing, test requirements, and test
results shall be made on the basis and for the purpose of ensuring that the
systems tested meet Federal voting system standards. A VSTL’s primary
purpose shall be to serve the public interest through adherence to the EAC
Testing and Certification Program.
2.19.3. Single Laboratory Requirement. EAC’s Testing and Certification Program
prohibits Manufacturers from changing laboratories during the testing process.
Once a lead VSTL is identified to the EAC by the Manufacturer to test a system, a
test report will not be accepted by the EAC from any other laboratory unless
authorized pursuant to Chapter 4 of the EAC’s Voting System Testing and
Certification Program Manual. This strict policy supports VSTLs in their
independent decision making role. VSTLs shall immediately report to the EAC
Certification Program Director any time a Manufacturer withdraws a product
29
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
from testing or the testing is otherwise terminated (see Section 2.10.7. of this
Manual).
2.19.4. Fee for Service. All fees paid by a Manufacturer to a VSTL shall be solely for
services rendered. No payment may be accepted by a VSTL that is not directly
linked to services necessary to complete system testing. No payment may be
accepted by a VSTL that is conditioned or dependent on testing outcome.
2.19.5. Communications. To ensure and document the independent relationship
between test laboratories and Manufacturers, all substantive discussions
regarding the outcome, cost, payment and testing of a voting system shall be
documented in writing by the VSTL. This includes, but is not limited to: letters,
emails, reports, meetings and telephone calls. These records shall be maintained
consistent with Section 2.23 of this Manual. Examples of substantive discussions
between the lead VSTL and a Manufacturer include but are not limited to:
2.19.5.1. All contracts and amendments thereto;
2.19.5.2. All discussions regarding the set up and operation of the voting system
during testing;
2.19.5.3. All discussions with the Manufacturer regarding the test plan, test cases,
testing, or the test report; and
2.19.5.4. All discussions regarding implementation or interpretation of the
standards.
2.19.6. Cooperation with EAC. Cooperate with any EAC inquiries and investigations
into a certified system’s compliance with VVSG standards and any VSTL testing
related to that system consistent with Chapter 7 of the Testing and Certification
Program Manual.
2.19.7. Testing Facilities. To avoid the appearance of impropriety and otherwise
maintain laboratory independence, VSTLs shall not conduct testing 7 at a
Manufacturer owned or controlled facility. If exceptional circumstances exist
requiring that the VSTL use Manufacturer facilities, the VSTL may request a
waiver from this prohibition. The request must be in writing to the Program
Director and clearly state why such testing is necessary. A waiver may be
granted at the sole discretion of the Program Director and may impose necessary
As noted in footnote 6, above, this requirement only applies to “testing” and does not include other
certification activities such as trusted and witness builds.
7
30
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
restrictions, limitations and requirements on testing. Waivers will be granted
only in exceptional circumstances.
2.19.8. Improper Influence. Any attempt by a Manufacturer to unduly influence the test
process shall be immediately reported to the EAC’s Certification and Testing
Program Director.
2.20. Authority to do Business in the United States. As a condition of accreditation, all
laboratories shall be lawfully entitled or otherwise not prohibited from doing business
with the United States or its citizens or operating in the United States.
2.21. Communications. As a condition of accreditation, all laboratories shall designate and
identify an individual or individuals who may speak for and take action on behalf of the
VSTL. VSTLs shall maintain an open line of communication with EAC’s Testing and
Certification Program Director, providing prompt response to requests for information
regarding the Program.
2.22. Resources and Financial Stability. As a condition of accreditation, all VSTLs shall
allocate sufficient resources to enable the laboratory to properly use and maintain its test
equipment, personnel, and facility and to satisfactorily perform all required laboratory
functions. The laboratory shall maintain insurance policies sufficient to indemnify itself
against financial liabilities or penalties that may result from its operations. VSTLs shall:
2.22.1. Maintain insurance policies (see Section 3.4.1.8.) that indemnify the laboratory
against the potential losses identified in its liability assessment (see Section
3.4.1.9.); and
2.22.2. Document solvency through demonstrating that the laboratory’s assets are
greater than its liabilities in its audited financial statement (see Section 3.4.1.16.).
2.23. Recordkeeping. As a condition of accreditation, all laboratories shall have a written
policy regarding the proper storage, management and retention of all records relating to
the testing of voting systems. At a minimum, this policy shall require all forms, reports,
test records, observations, calculations, and derived data for all tests performed on a given
voting system (or component of said system) be retained for a period of at least 5 years
after the last test performed on any version of that system (or component of any version
of said system). The policy shall require that all documents are maintained in a safe and
secure environment and stored in a manner that provides for organized and timely
identification and retrieval. Additionally, all records must be kept in a data format usable
and available to the EAC.
31
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Accreditation Process
3.1. Overview. This chapter sets forth the required steps Applicant Laboratories must
perform in order to receive an EAC Voting System Test Laboratory Accreditation. The
process generally includes an application for and receipt of a NIST recommendation;
receipt of an EAC invitation to apply; and the successful submission, acceptance and
review of an EAC application.
3.2. NIST Recommendation. The Election Assistance Commission (EAC) is mandated under
Section 231 of the Help America Vote Act of 2002 (HAVA) (42 U.S.C. §15371(b)) to “…
provide for the certification, de-certification and re-certification of voting system
hardware and software by accredited laboratories.” As part of this process, HAVA
requires the National Institute of Standards and Technology (NIST) to evaluate
independent non-Federal test laboratories. NIST selects those laboratories technically
qualified to test voting systems and recommends them to the EAC for accreditation.
Generally, a Laboratory must have a NIST recommendation before it may be considered
for EAC accreditation.
3.2.1.
NIST Recommendation Process. NIST utilizes its National Voluntary Laboratory
Accreditation Program (NVLAP) to perform this evaluation. NIST, through the
NVLAP process, assesses laboratory technical capabilities, procedures and
personnel before recommending a laboratory for EAC accreditation. The
requirements, procedures and application process for requesting consideration
by NIST (for recommendation to the EAC) may be found at
www.nist.gov/NVLAP or by contacting NIST at, National Voluntary Laboratory
Accreditation Program, Standards Services Division, NIST, 100 Bureau Drive,
Stop 2140, Gaithersburg, MD, 20899-2140.
3.2.2.
Emergency EAC Accreditation without NIST Recommendation. HAVA
authorizes the EAC to consider and accredit laboratories without a NIST
recommendation (42 U.S.C. §15371(b)(2)(B)). The EAC will accredit laboratories
without a NIST recommendation only as an emergency action.
3.2.2.1.
Emergency Action—Defined. The EAC will take emergency action only in
instances where (1) there is a significant national need for accredited
laboratory testing capacity that cannot be met by existing VSTL’s, (2) the
shortage of laboratory testing capacity may cause a disruption in the
orderly administration of Federal elections, and (3) NIST is not capable
of timely providing new laboratories to meet needs. Consistent with
HAVA, the EAC will publish its basis for emergency action following
the above standards.
32
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
3.2.2.2.
Emergency Action—Process. Laboratories shall be accredited by the EAC
in an emergency action only after they have been properly assessed
according to international standards and applicable NIST Guidance.
These standards include International Standard ISO/IEC 17025, General
Requirements for the Competence of Testing and Calibration Laboratories;
NIST Handbook 150, Procedures and General Requirement; NIST
Handbook 150-22, Voting System Testing; and/or any documents
supplementing, updating or replacing these standards or handbooks.
3.2.2.3.
Emergency Action—Provisional. Any accreditation provided by the EAC
through its emergency action authority will be provisional in nature and
limited in scope. All emergency accreditations must expire on a date
certain.
3.3. EAC Invitation. After receipt of a NIST list of recommended laboratories, the EAC will
send a letter to the laboratories inviting them to apply for EAC accreditation under the
VSTL program. No laboratory may apply for EAC accreditation without an invitation
from the Commission. The letter of invitation will identify the scope of accreditation for
which the laboratory may apply. The invited laboratories must follow the application
procedure noted in Section 3.4, below.
3.4. Application. EAC is the sole authority for Voting System Test Laboratory Accreditation.
While NIST’s recommendation serves as a reliable indication of potential technical
competency, the EAC must take additional steps to ensure that laboratory policies are in
place regarding issues like conflict of interest, record maintenance, and financial stability.
It must also ensure that the candidate laboratory is willing and capable to work with EAC
in its Certification Program. To that end, applicant laboratories are required to submit a
Letter of Application requesting accreditation. The letter shall be addressed to the Testing
and Certification Program Director and attach (in either hard copy or on CD/DVD) (1) all
required information and documentation; (2) a signed letter of agreement; and (3) a signed
certification of conditions and practices.
3.4.1.
Information and Documents. The applicant laboratory must submit the
information and documents identified below as a part of its application. These
documents will be reviewed by the EAC in order to determine whether the
applicant laboratory meets the program requirements identified in Chapter 2.
The grant of EAC accreditation is subject to receipt of the information and EAC’s
review and approval of the materials. The applicant laboratory shall properly
label any documents, or portions of documents, it believes are protected from
release under Federal law.
3.4.1.1. The legal name of the laboratory
33
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
3.4.1.2.
Mailing address of the laboratory
3.4.1.3. Physical location of the laboratory (if different than the mailing
address).
3.4.1.4. Name, phone number, fax number and e-mail address of the voting
system testing program manager or individual otherwise immediately
responsible for the voting system testing program.
3.4.1.5. Name, phone number, fax number, and e-mail address of the
individual, CEO, president or otherwise titled head of the laboratory.
3.4.1.6. Name, title, phone number, fax number, and e-mail address of the
individual or individuals designated to speak for and take action on
behalf of the laboratory pursuant to Section 2.21 of this Manual.
3.4.1.7. The business contact information (such as point of contact, address,
Web site, e-mail address) to be posted by the EAC on its Web site.
3.4.1.8. The identity of the laboratory’s insurer(s), name of insured, and
coverage limits for any comprehensive general liability policies, errors
and omissions policies, professional liability policies, and bailee policies.
3.4.1.9. A written assessment of the laboratory’s commercial general liability.
3.4.1.10. A signed statement certifying that it maintains workman’s
compensation policy coverage sufficient to meet the applicable State’s
minimum requirements.
3.4.1.11. A copy of the laboratory’s organizational chart which includes the
names of key staff responsible for the testing of voting systems.
3.4.1.12. A copy of the laboratory’s conflict of interest policy which implements
the standards of Section 2.5 of this Manual.
3.4.1.13. A copy of the laboratory’s personnel policy which implements the
standards of Section 2.6 of this Manual.
3.4.1.14. A copy of the laboratory’s recordkeeping policy which implements the
standards of Section 2.23 of this Manual.
3.4.1.15. A copy of the laboratory facilities brochure.
34
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
3.4.1.16. A copy of the most recent annual report, the names of the current board
of directors and the previous year’s board of directors, the names of any
majority shareholders, and audited financial statements of the
companies or entities that own and operate the laboratory. Laboratories
not incorporated should provide comparable information.
3.4.2.
Letter of Agreement. The applicant laboratory must submit a signed letter of
agreement as a part of its application. This letter shall be signed by an official
vested with the legal authority to speak for, contract on behalf of or otherwise
bind the applicant laboratory (see Section 2.21). The purpose of this letter is to
document that the applicant laboratory is aware of and agrees to abide by the
requirements of the EAC Voting System Testing Laboratory Accreditation
Program. No applicant laboratory will be considered for accreditation unless it
has properly submitted a letter of agreement. The letter shall unequivocally state
the following:
The undersigned representative of ___________ (hereinafter “Laboratory”),
being lawfully authorized to bind Laboratory and having read the EAC Voting
System Test Laboratory Program Manual, accepts and agrees on behalf of
Laboratory to follow the program requirements as laid out in Chapter 2 of the
Manual. Laboratory shall meet all program requirements as they relate to
NVLAP accreditation; conflict of interest and prohibited practices; personnel
policies; notification of changes; resources; site visits, notice of law suits; testing,
technical practices and reporting; laboratory independence; authority to do
business in the United States; VSTL communications; financial stability; and
recordkeeping. Laboratory further recognizes that meeting these program
requirements is a continuing responsibility. Failure to meet each of the
requirements may result in the denial of an application for accreditation, a
suspension of accreditation or a revocation of accreditation.
3.4.3.
Certification of Laboratory Conditions and Practices. The applicant laboratory
must submit a signed Certification of Laboratory Conditions and Practices as a part of
its application. No applicant laboratory will be considered for accreditation
unless it has properly affirmed its conditions and practices through the
certification document. A Certification of Laboratory Conditions and Practices form
may be found at Attachment C and is available electronically at www.eac.gov.
By signing the certification, a laboratory affirms that it, in fact, has in place the
policies, procedures, practices, resources and personnel stated in the document.
Any false representations made in the certification process may result in the
revocation of accreditation and/or criminal prosecution.
3.5. EAC Review of Application Package. The EAC will perform a review of each Applicant
Laboratory’s application package to ensure that it is complete and the laboratory meets
35
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
the program requirements. Each package will be reviewed to identify any apparent
nonconformities or deficiencies. If necessary, the Program Director will notify Applicant
Laboratories of any such nonconformities or deficiencies and provide them an
opportunity to cure problems prior to forwarding the package to the Commissioners. The
Program Director will issue a recommendation to the Commissioners when forwarding
any application package. Consistent with HAVA, a laboratory will receive an
accreditation only upon a vote of the Commissioners.
3.5.1.
Program Director Review. Application packages shall be sent to the Program
Director. The Program Director will perform a review of the packages before
forwarding them to the Commissioners with a recommendation. Upon receipt of
an application package the Testing and Certification Program Director shall
review the package to ensure:
3.5.1.1. The package is complete. No application may be forwarded to the
Commission for a vote on accreditation unless is contains all required
documentation (Section 3.4.1), a proper letter of agreement (Section
3.4.2), and a signed Certification of Laboratory Conditions and Practices
(Section 3.4.3).
3.5.1.2. Evidence of compliance with program requirements. The Program Director
shall also review the submissions to ensure that the information
provided properly reflects and documents compliance with program
requirements.
3.5.2.
Notice of Nonconformity. In the event the Program Director identifies (1)
missing documentation or information and/or (2) issues of non-compliance, the
Program Director shall notify the Applicant Laboratory of the deficiencies prior
to forwarding a recommendation to the Commissioners. The written notice of
nonconformity shall:
3.5.2.1. Identify any missing documentation or information;
3.5.2.2. Identify any issues of potential non-compliance; and
3.5.2.3. Provide Applicant Laboratory a reasonable time period to submit
additional information or amend their application package in response
to identified non-conformities.
3.5.3.
Applicant Laboratory Action on Notice of Nonconformity. Applicant
Laboratories shall respond to a notice of nonconformity within the timeframe
identified by the Program Director. Responses shall include any missing
36
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
documents identified in the notice, as well as any additional or clarifying
information or documentation responsive to an issue of non-compliance.
3.5.3.1. Request for Additional Time. Applicant Laboratories may request
additional time in writing. Such request must state the basis for the
request and identify a reasonable time period for response. The grant of
additional time is at the sole discretion of the Program Director.
3.5.3.2. Failure to Respond—Missing Documentation or Information. If an
Applicant Laboratory fails to provide required information or
documentation within the timeframe provided in the notice of
noncompliance, the Program Director shall reject the application as
incomplete, returning the package to the applicant for resubmission
consistent with the requirements of this Chapter.
3.5.3.3. Failure to Respond—Issue of Noncompliance. If, within the timeframe
provided in the notice of noncompliance, an Applicant Laboratory (who
has provide all required documentation) fails to provide additional,
clarifying information or documentation in response to an identified
issue of program noncompliance, the Program Director shall forward
the original application to the Chair of the Commission for action.
3.5.4.
Recommendation to Commissioners. After review, and if necessary an
opportunity for the applicant to amend their application, the Program Director
shall forward each application to the Chair of the Commission with a
recommendation as to disposition. This application package shall include all
documents and correspondence between the applicant laboratory and the EAC
Program Director.
3.5.5.
Vote by Commissioners. Upon receipt of an application package and
recommendation from the Testing and Certification Program Director, the Chair
of the Commission shall forward the information to each EAC Commissioner.
After a reasonable time to review the forwarded materials, the Chair of the
Commission shall bring the matter to a vote, consistent with the rules of the
Commission. The measure presented for a vote shall take the form of a written
Commissioners’ Decision which (1) makes a clear determination as to
accreditation and (2) states the basis for the determination.
3.6. Grant of Accreditation. Upon a vote of the EAC Commissioners to accredit a laboratory,
the Testing and Certification Program Director shall inform the laboratory of the decision,
issue a Certificate of Accreditation and post information regarding the laboratory on the
EAC Web site.
37
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
3.6.1.
Certificate of Accreditation. A Certificate of Accreditation shall be issued to each
laboratory accredited by vote of the Commissioners. The certificate shall be
signed by the Chair of the Commission and state:
3.6.1.1. The name of the VSTL;
3.6.1.2. The scope of accreditation, by stating the Federal standard or standards
to which the VSTL is competent to test;
3.6.1.3. The effective date of the certification, which shall not exceed a period of
two (2) years; and
3.6.1.4. The technical standards to which the laboratory was accredited.
3.6.2.
Post Information on Web Site. The Program Director shall make information
pertaining to each accredited laboratory available to the public on EAC’s Web
site. This information shall include (but is not limited to):
3.6.2.1. NIST’s Recommendation Letter;
3.6.2.2. The VSTL’s Letter of Agreement;
3.6.2.3. The VSTL’s Certification of Conditions and Practices;
3.6.2.4. The Commissioner’s Decision on Accreditation; and
3.6.2.5. The Certificate of Accreditation.
3.7. Effect of Accreditation. Receipt of an EAC Accreditation indicates that a laboratory has
met the applicable technical, procedural, management and staffing requirements and may
serve as a Voting System Test Laboratory (VSTL) under EAC’s Testing and Certification
Program.
3.7.1.
Scope of Accreditation. A laboratory shall operate within the limits of the scope
of accreditation as stated on its Certificate of Accreditation.
3.7.2.
Representation. No VSTL may make representations regarding its accreditation
beyond its scope of accreditation.
3.7.3.
No Endorsement. A Certificate of Accreditation is not an endorsement of the
recipient laboratory. A VSTL may not state or imply EAC endorsement.
38
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
3.7.4.
Accreditation Logo. A VSTL may display the EAC laboratory accreditation logo.
Only the EAC authorized logo may be used. The display must be used in a
manner consistent Sections 3.7.1. - 3.7.3., above. Specifications for the
reproduction and use of the EAC logo are found in Appendix D.
3.8. Expiration and Renewal of Accreditation. A grant of accreditation is valid for a period
not to exceed two years. A VSTL’s accreditation expires on the date annotated on the
Certificate of Accreditation. VSTLs in good standing shall renew their accreditation by
submitting an application package to the Program Director, consistent with the
procedures of Section 3.4 of this Chapter, no earlier than 60 days before the accreditation
expiration date and no later than 30 days before that date. Laboratories that timely file the
renewal application package shall retain their accreditation while the review and
processing of their application is pending. VSTLs in good standing shall also retain their
accreditation should circumstances leave the EAC without a quorum to conduct the vote
required under Section 3.5.5.
3.9. Denial of Accreditation. Upon a vote of the EAC Commissioners not to accredit a
laboratory, the Testing and Certification Program Director shall inform the laboratory of
the decision and post relevant information on the EAC Web site.
3.9.1.
Notice of Denial. The Program Director shall inform the applicant laboratory (in
writing) of the Commissioners’ Decision. This notice must include:
3.9.1.1. A statement of the decision and brief summary explanation of the basis
for the decision;
3.9.1.2. Notice of the Applicant Laboratory’s right to appeal; and
3.9.1.3. A copy of the Commissioners’ Decision.
3.9.2.
Post Information on Web Site. The Program Director shall publish on EAC Web
site:
3.9.2.1. A copy of the Commissioners’ Decision, and
3.9.2.2. The Notice of Denial.
3.10. Requesting Appeal. An applicant laboratory that has been denied accreditation by a vote
of the Commissioners shall have the right to appeal. An Applicant Laboratory may
appeal a Denial of Accreditation by first issuing a written request for appeal.
3.10.1. Submission. Requests must be submitted in writing to the Program Director,
addressed to the Chair of the U.S. Election Assistance Commission.
39
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
3.10.2. Timing of Appeal. The Applicant Laboratory may request an appeal within 7
calendar days of receipt of the Notice of Denial. Late requests will not be
considered.
3.10.3. Contents of Request. The request must petition for reconsideration of the
Commissioners’ Decision and clearly state the specific conclusions of the
Decision the Applicant Laboratory wishes to appeal.
3.11. EAC Action on a Request for Appeal. The Program Director shall accept any request for
appeal timely submitted. Untimely requests shall be rejected. Upon receipt of a request
for appeal, the Program Director shall notify the requestor applicant laboratory, in
writing, as to whether their appeal has been accepted as timely. The notice for accepted
requests shall inform the applicant laboratory of the requirements for submitting their
appeal per Section 3.12 of this Manual.
3.12. Submission of Appeal. After submission of a timely request for appeal, the Applicant
Laboratory shall submit its appeal. This appeal shall (1) clearly identify the specific
conclusions of the Commissioners’ Decision the Laboratory wishes to challenge, (2)
provide the basis for its position on appeal and (3) submit a written argument in support
of its appeal. In addition, the applicant laboratory may submit documentary or other
relevant, physical evidence in support of the appeal. The Appeal and all supporting
materials must be received by the EAC within 20 days of the applicant laboratory’s receipt
of the Program Director’s notice of acceptance of the request to appeal.
3.13. Consideration of Appeal. All timely appeals will be considered by the Commissioners.
Upon receipt of an appeal, the Chair of the Commission shall forward to each EAC
Commissioner the Applicant Laboratory’s appellate submission, along with the original
application package, Commissioners’ Decision, and Program Director’s recommendation.
After a reasonable time to review and consider the forwarded materials, the Chair of the
Commission shall bring the matter to a vote, consistent with the rules of the Commission.
The measure presented for a vote shall take the form of a written Commissioners’
Decision on Appeal.
3.14. Commissioner’s Decision on Appeal. The Commissioners shall make a written, final
Decision on Appeal and shall provide it to the Applicant Laboratory.
3.14.1. Contents. The Decision on Appeal shall:
3.14.1.1. State the final determination of the Commission.
3.14.1.2. Address the matters raised by the Applicant Laboratory on appeal.
40
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
3.14.1.3. Provide the reasoning behind the decision.
3.14.1.4. State that the Decision on Appeal is final.
3.14.2. Determinations. The Commissioners shall make one of two determinations on
appeal.
3.14.2.1. Grant of Appeal. If the Commissioners determine that the previous
Decision of the Commission shall be overturned in full, the appeal shall
be granted. In such cases, the Applicant Laboratory shall be granted
accreditation.
3.14.2.2. Denial of Appeal. If the Commissioners determine that any part of the
previous Decision of the Commission shall be upheld such that the
procedural requirements of Chapter 3 or the Program requirements of
Chapter 2 of this manual will not be met in full, the appeal shall be
denied. In such cases, the application for appeal is finally denied.
3.14.3. Effect. All Decisions on Appeal shall be final and binding on the Applicant
Laboratory. No additional request for appeal shall be granted.
3.15. Effect of Denial of Accreditation. An EAC denial of accreditation indicates only that an
applicant laboratory has failed to document or otherwise demonstrate that it has the
procedures, policies, management or personnel in place to meet the requirements of the
Accreditation Program. A denial of accreditation is based upon current policy and
procedure and is not an indicator of past performance. Laboratories denied accreditations
have the right to cure any identified defect and reapply by resubmitting their application
package consistent with Section 3.4 of this Chapter.
41
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Compliance Management Program
4.1. Purpose. The purpose of the Compliance Management Program is to improve EAC’s
Laboratory Accreditation Program and testing; increase coordination, communication and
understanding between the EAC and its VSTLs; and increase public confidence in
elections by facilitating VSTL accountability. The program accomplishes this by
increasing personal interaction between EAC staff and VSTL personnel, collecting
information and performing reviews to ensure continued compliance with program
requirements, and requiring that VSTLs promptly remedy any identified areas of
noncompliance.
4.2. Compliance Management Program, Generally. The Compliance Management Program
meets its purposes by gathering information on the procedures and practices of its VSTLs.
There are three main sources of information: (1) VSTL Notifications of Changes, (2) EAC
Requests for Documents or Information and (3) EAC On Site Reviews. The information
collected is reviewed by the EAC to ensure that VSTLs are meeting all program
requirements. Any areas of noncompliance or recommendations for improvement are
presented to VSTLs in a Compliance Management Report. VSTLs are required to
promptly remedy any noncompliance or face revocation of accreditation.
4.3. VSTL Notification of Changes. VSTLs are obligated to report any significant changes
regarding the information, agreements or certifications made to the EAC as a condition of
accreditation (see Section 2.7). This requirement serves as the primary means by which
the EAC maintains VSTL compliance. Failure to report changes in conditions or practices
may result in suspension or revocation of accreditation consistent with the requirements
and procedures of Chapter 5.
4.4. Request for Documents and Information. The Program Director may request a VSTL to
provide the EAC information and/or documents to demonstrate the laboratory’s
continuing compliance with the Accreditation Program requirements noted in Chapter 2
(See Section 2.2).
4.4.1.
EAC Request. A request for documents or information shall be made in writing
by the Program Director and provide a reasonable timeframe for VSTL response.
The request may be for documents, information or both:
4.4.1.1. Request for Documents. A request for documents must identify the
specific documents sought. A request for documents is not a demand
for the VSTL to create a document, but to provide the EAC a copy of
any existing documentation responsive to the request.
42
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
4.4.1.2. Request for Information. Requests for information shall take the form of
interrogatories. Each inquiry shall take the form of a discrete question.
VSTLs are expected to provide complete answers to each question.
4.4.2.
VSTL Response. VSTLs shall respond within the timeframe provided by the
Program Director. If additional time is needed, VSTLs may request an extension.
Such requests must be made within the timeframe of the original request. The
grant of additional time is at the sole discretion of the Program Director.
4.4.2.1. Request for Documents. VSTLs shall respond to requests for documents
by having knowledgeable staff conduct a thorough search of VSTL
records. VSTLs shall provide copies of all documents responsive to the
request. If any document responsive to a request is considered
privileged or otherwise protected from release under Federal law, it
should be properly labeled. If no documents responsive to the request
are found, the VSTL shall state that no records were found.
4.4.2.2. Request for Information. VSTLs shall respond to requests for information
by having knowledgeable staff answer each question posed. VSTLs
shall ensure that each question is answered completely and accurately.
The VSTL may submit documents in support of its responses.
4.4.3.
Failure to Respond. Failure to timely respond to a request for documents or
information may result in a suspension or revocation of accreditation consistent
with the requirements and procedures of Chapter 5.
4.5. Proficiency Testing. VSTLs will comply with any guidelines and tests developed and
administered by the EAC. This will include, but is not limited to, a written test focusing
on scenario based and knowledge based questions.
4.6. On Site Laboratory Review—Generally. The Program Director shall provide for regular
on site reviews of VSTLs. There are two types of onsite review:
4.6.1.
On Site Review—Policy, Procedures and Practices Review. The most common
type of review is the Policy, Procedure and Practices Review. This type of review
requires EAC personnel to enter a VSTL facility, examine a variety of
documentation and meet with VSTL personnel to confirm that the VSTL’s
policies, procedures and practices meet the requirements of the Laboratory
Accreditation Program (Chapter 2).
4.6.2.
On Site Review—Testing Observation and Technical Assessment. A Testing
Observation and Technical Assessment Review requires an expert EAC
laboratory assessor to enter a VSTL facility and assess the laboratory’s technical
43
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
procedures, policies, management and personnel to verify compliance with
applicable laboratory standards. Additionally, the EAC assessor may observe
VSTL employees during the testing of voting systems to ensure that VSTL
practices match technical policies. 8
4.7. On Site Laboratory Review—Frequency. The Program Director shall ensure that each
VSTL receives an On Site Policy, Procedures and Practices Review at least once every two
years. Whenever possible, the EAC will conduct the required audits or follow up on site
visits at the same time as NVLAP audits or follow up visits.
4.8. On Site Laboratory Review—Procedure. The Program Director shall determine when
and what type of onsite review will be conducted for each VSTL. Before any on site
review, the Program Director shall provide the VSTL with reasonable notice. Reviews
shall be conducted with as little impact as possible on the activities of the VSTL. The
VSTL and its employees are required to participate in the review and cooperate with on
site EAC personnel. Finally, the reviewer shall provide the VSTL a short exit briefing
prior to the termination of the onsite review.
4.8.1.
Notice. The Program Director shall coordinate on site reviews with VSTL
management. As reviews require the availability of laboratory documents and
key personnel, a notice of onsite review shall be in writing and be provided to
the VSTL at least 15 calendar days before the onsite review date. The notice shall
provide the VSTL with the following information:
4.8.1.1. Duration of Review. The notice shall provide an estimated timeframe
during which EAC reviewers will be on site.
4.8.1.2. Type of Review. The notice shall identify the type of review to be
performed (see Section 4.6).
4.8.1.3. Scope of Review. The notice shall provide information regarding the
scope of review. This information shall be sufficient to allow the VSTL
to identify the documents, personnel and testing it must make available
to EAC reviewers. The notice shall specifically identify:
4.8.1.3.1.
The type of documents and/or program areas to be
reviewed.
EAC’s authority to observe testing and conduct technical assessments serves only as an additional tool
to ensure technical compliance. The primary means by which EAC ensures technical compliance is
through NIST’s NVLAP program. The NVLAP program monitors laboratories by requiring regular
assessments. Laboratories are reviewed one year after their initial accreditation and biennially thereafter.
8
44
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
4.8.1.3.2.
4.8.1.4.
The testing that is to be observed.
VSTL’s Responsibilities. The notice shall briefly inform the VSTL of its
responsibility to coordinate and cooperate with the EAC throughout the
onsite review process.
4.8.2.
VSTL Response to Notice. Upon receipt of a notice of onsite review, the VSTL
shall coordinate the logistics of the review with the Program Director. In the
event the noticed date or timeframe makes access to the required personnel,
documents or testing untenable, the VSTL shall contact the Program Director in
writing and identify, (1) The conflict or other problem which makes the proposed
date and timeframe untenable, and (2) a proposed alternative date for the onsite
review. The acceptance of an alternative on site review date is at the sole
discretion of the Program Director.
4.8.3.
Review. An onsite review begins upon the arrival of EAC personnel at the
VSTL’s facility. EAC reviewers will ordinarily conduct reviews during the
VSTL’s normal working hours. The reviewers will make every effort to work as
efficiently as possible and avoid impacting the laboratory’s routine operations.
The VSTL and its employees are required to cooperate with EAC reviewers. This
cooperation includes providing a private, physical location for EAC personnel to
review documents and speak with VSTL employees. Generally, the VSTL shall
be responsible for ensuring:
4.8.3.1. Document Access and Availability. That the reviewers have access to all
requested VSTL documents. All documents specifically identified in the
notice of onsite review shall be presented to reviewers upon arrival.
4.8.3.2. Personnel Access and Availability. That the reviewers have reasonable
access to requested personnel. The VSTL shall ensure that key
personnel for each substantive area identified in the notice of onsite
review be available to EAC reviewers during the noticed review period.
4.8.3.3. Facilities and Testing Access and Availability. That the reviewers have
access to VSTL facilities involved in the testing of voting systems,
including the facilities of third party contractor laboratories.
Additionally, VSTLs must coordinate access to view testing consistent
with the notice of onsite review.
4.8.4.
Exit Briefing. EAC reviewers shall provide the VSTL personnel an exit briefing.
Exit briefings shall be informal. The briefing shall identify any documents,
information or personnel which the VSTL remains responsible for making
available to the reviewers; inform the VSTL of the next steps in the review
45
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
process; and provide the VSTL an opportunity to ask questions about the
process.
4.9. EAC Compliance Management Reports. The EAC shall issue a written Compliance
Management Report after performing any on site review. A Compliance Management
Report shall also be issued after a Request for Documents/Information or VSTL
Notification of Change when either indicates a noncompliance with program
requirements. All reports shall be posted on the EAC Web site and (1) provide a brief
summary of the review process, request for information or VSTL Notification of Change
(2) state any findings resulting from the review, and (3) identify any corrective action
required.
4.9.1.
Purpose. The purpose of the report is to provide the VSTL with EAC’s findings
regarding its program so that:
4.9.1.1. Items of noncompliance may be identified and rectified,
4.9.1.2. Exceptional practices may be identified and encouraged, and
4.9.1.3. EAC recommendations (beyond the program requirements) may be put
forth in an effort to improve the VSTL’s program.
4.9.2.
Summary of Process. The report shall provide a brief summary of the review
process, request for information or VSTL Notification of Change. The purpose of
this summary is to provide background information regarding how the
information supporting EAC findings was collected. This includes identifying
sources of information, methodology and standards. For the purposes of onsite
reviews, the summary shall state:
4.9.2.1. The dates of the review,
4.9.2.2. The type of review performed,
4.9.2.3. The program areas reviewed, including any specific documents and
personnel discussions which were integral to the report findings, and
4.9.2.4. The processes used by the reviewers to determine compliance.
4.9.3.
Findings. The report shall outline any findings of the review, request for
information or VSTL Notification of Change. A finding is any factual
determination that the VSTL is not in compliance with the program requirements
identified in Chapter 2 of this Manual or an EAC recommendation for program
improvement which does not rise to the level of noncompliance. While reports
46
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
may also contain recognition of exceptional practices, such statements are not
considered findings. Reports shall identify three types of findings:
4.9.3.1. Critical. A critical finding is a determination that the VSTL has not met
a requirement of the program that is fundamentally critical to the
VSTL’s technical capability to test voting systems. A critical
noncompliance is a violation of program requirements that by its very
nature comprises the integrity of the EAC Testing and Certification
Program.
4.9.3.2. Required. A required finding is a determination that the VSTL has failed
to meet a requirement of the program that is not considered technically
critical pursuant to Section 4.8.3.1., above.
4.9.3.3. Recommended. A recommended finding is a determination that VSTL
practices could be improved, but that the identified improvement is not
required by the program. In some cases, recommended practices may
be practices the EAC plans to make program requirements.
4.9.4.
Corrective Action. The report shall specify the action to be taken by the EAC
and/or VSTL based upon the review findings.
4.10. Corrective Action. Based upon the Compliance Management Report, corrective action
may be required. EAC action and VSTL responsibilities will vary depending upon the
nature of the report’s findings.
4.10.1. Critical. Critical Findings require the EAC to initiate the immediate suspension
of the VSTL consistent with the requirements and procedures of Chapter 5,
Revocation of Accreditation. The VSTL’s rights to remedy its noncompliance or be
heard are laid out in Chapter 5.
4.10.2. Required. Required Findings obligate the VSTL to resolve the identified noncompliance within 20 days. Failure to do so within the 20 day timeframe will
result in suspension or revocation of accreditation consistent with the procedures
laid out in Chapter 5, Revocation of Accreditation. The VSTL may resolve a
Required Finding by:
4.10.2.1. Challenging the Finding. The VSTL may challenge a finding if it believes
its procedures and practices were in compliance with program
requirements at the time of the review. A VSTL shall challenge a
Required Finding by providing factual information which documents its
claim of compliance. Challenges must be filed within 5 days of receipt
of the EAC Report. The challenge must be in writing, state the basis for
47
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
the challenge, address the facts and conclusions in the EAC report, and
provide information which unambiguously documents that the VSTL
was in compliance at the time of the review, request for information or
VSTL Notification of Change. The EAC Program Director will accept or
reject a VSTL’s challenge in writing. If a challenge is accepted, no
corrective action will be required. If the challenge is rejected, the VSTL
will have 20 days from receipt of the notice of rejection to perform
remedial action.
4.10.2.2. Conducting Remedial Action. VSTLs may take corrective action by
submitting a remedial plan within 20 days of receipt of the report. The
remedial plan shall (for each finding of noncompliance) identify the
noncompliance, outline the steps to be taken to achieve compliance,
state the timeframe for each step and identify the means and final date
by which the VSTL will document compliance. A remedial plan is
subject to approval from the Program Director. A VSTL’s failure to
obtain approval of a remedial plan or unauthorized deviation from an
approved plan’s requirements or deadlines will result in suspension or
revocation of accreditation consistent with the procedures laid out in
Chapter 5, Revocation of Accreditation.
4.10.3. Recommended. Recommended findings do not require VSTL action. The
proposed remedial actions for recommended findings are not program
requirements, but EAC suggested practices.
48
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Revocation of Accreditation
5.1. Overview. This chapter puts forth the process for revoking the accreditation of an EAC
VSTL. The process for revocation begins with factual findings made pursuant to the
Compliance Management Program (Chapter 4). Prior to any revocation of accreditation,
VSTLs which fail to comply with program requirements are provided notice of (1) EAC’s
intent to suspend, (2) suspension and (3) an opportunity to be heard or cure
noncompliance. A laboratory that has its accreditation revoked has the right to appeal.
5.2. Revocation Policy. EAC Accreditation is subject to revocation. The EAC shall revoke an
accreditation upon a factual finding that a VSTL has failed to meet a requirement of the
Accreditation Program and is unable or unwilling to timely and properly remedy the noncompliance.
5.3. Revocation—Generally. The EAC monitors its VSTLs through its Compliance
Management Program (Chapter 4). This program monitors compliance through (1) the
VSTL’s continuing obligation to provide EAC Notifications of Changes, (2) EAC’s
authority to issue Requests for Documents or Information and (3) the performance of On
Site Reviews. Determinations that a VSTL is not complying with program requirements
shall be made in Compliance Management Reports (findings of non-compliance). The
process outlined in this chapter to suspend and revoke a VSTL’s accreditation shall be
initiated (1) immediately for Critical Findings of noncompliance and (2) after an
opportunity to remedy the noncompliance for Required Findings (consistent with the
process mandated by Section 4.9). Revocation of Accreditation is a three step process:
5.3.1.
Notice of Intent to Suspend;
5.3.2.
Suspension of Accreditation; and
5.3.3.
Commissioners’ Decision on Revocation of Accreditation.
5.4. Notice of Intent to Suspend. The revocation process shall be initiated by issuing a Notice
of Intent to Suspend to a non-compliant VSTL. Such notices shall be issued by the
Program Director. VSTLs shall have three days to submit a response to the notice. The
EAC will issue a decision on suspension after consideration of the VSTL’s submission.
5.4.1.
Written Notice. The Notice of Intent to Suspend shall be in writing and:
5.4.1.1. Inform the VSTL of the EAC’s intent to suspend the laboratory;
5.4.1.2. Identify the program requirement or requirements with which the VSTL
has failed to comply;
49
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
5.4.1.3. State the factual finding or findings that serve as the basis of the action;
5.4.1.4. Provide a copy of the relevant Compliance Management Report; and
5.4.1.5. Inform the VSTL of its right to file a response to the notice.
5.4.2.
VSTL Response. The VSTL may respond to the notice of intent to suspend.
Responses must be received by the EAC Program Director within three days of
the VSTLs receipt of the Notice of Intent to Suspend to be eligible for
consideration. The VSTL response:
5.4.2.1. Must be in writing;
5.4.2.2. Must be timely submitted to be considered;
5.4.2.3. Must challenge the factual finding or findings that serve as the basis of
the suspension;
5.4.2.4. May include relevant documentation in support of its challenge.
5.4.3.
EAC Consideration of Response. The EAC shall consider the timely submission
of a VSTL before issuing a Decision of Suspension. The EAC may consult
experts, perform research and request additional information from the VSTL
during the consideration process.
5.4.4.
EAC Decision on Suspension. The EAC shall issue a Decision on Suspension.
The decision shall be made in writing by the Program Director. A decision shall
state (1) the decision of the Program Director, (2) the basis for and reasoning
behind the decision and (3) the VSTL’s obligations and rights during suspension
(if applicable). A Decision on Suspension shall be provided to the VSTL, issued
to all registered Manufacturers and posted on EAC’s Web site. The Program
Director may make one of two determinations in a Decision on Suspension:
5.4.4.1. Program Compliance. Based upon the EAC’s consideration of a VSTL’s
response to the notice of intent to suspend, the Program Director may
overturn the factual findings that served as the basis of the notice. In
such cases, the Program Director shall determine that the VSTL is in
compliance with all program requirements. A decision that the VSTL is
in compliance shall end the revocation process.
5.4.4.2. Suspension. The Program Director shall suspend the VSTL consistent
with the notice of intent to suspend when the preponderance of the
50
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
evidence indicates noncompliance with program requirements.
Suspension is effective as of the VSTL’s receipt of the decision.
5.5. Suspension of Accreditation. Suspension is the second step in the revocation process.
The purpose of Suspension is (1) to provide the suspended VSTL an opportunity to timely
cure the noncompliance which served as the basis of Suspension or (2) grant the
suspended VSTL an opportunity to be heard prior to revocation of accreditation. A
suspended VSTL shall have 20 days to either cure its noncompliance or request an
opportunity to be heard. If no action is taken by the suspended VSTL within the 20 days,
the EAC Commissioners shall make a decision on revocation.
5.5.1.
Effect of Suspension. A suspended VSTL shall immediately cease all testing of
voting systems under the EAC’s Certification Program. Any testing performed
by a suspended VSTL during its suspension will not be accepted by the EAC
under its Voting System Certification Program. Any period of suspension must
be clearly documented in a VSTL’s test report (see Chapter 4 of the EAC Voting
System Testing and Certification Manual). Testing under the EAC Certification
Program shall not resume unless the suspension is lifted or the VSTL is otherwise
authorized by the EAC (in writing) to recommence testing.
5.5.2.
Opportunity to Cure. A suspended VSTL may request the opportunity to cure
its noncompliance within 20 days of its receipt of the Program Director’s
Decision on Suspension. The request must include a detailed remedial plan. If
this plan is accepted, properly executed and verified, the VSTL’s suspension will
be lifted and it may resume testing.
5.5.2.1. Remedial Plan. A request to cure noncompliance must include a plan by
which the VSTL outlines how it will timely bring its laboratory into full
compliance with the program. The remedial plan shall:
5.5.2.1.1.
Identify each noncompliance which served as the basis of its
suspension;
5.5.2.1.2.
For each identified noncompliance, outline the steps to be
taken to achieve compliance. This includes identifying the
resources and personnel needed for each step;
5.5.2.1.3.
Provide a timeframe for the completion of each identified
step and state the final date by which the VSTL will
complete the compliance plan;
5.5.2.1.4.
Provide a schedule of periodic progress reports to the
Program Director; and
51
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
5.5.2.1.5.
Require the VSTL to provide the EAC a written certification
attesting to its completion of the remedial plan and full
compliance with program requirements at close of the
process.
5.5.2.2. EAC Action on Plan. A remedial plan is subject to approval by the
Program Director. The Program Director will work with the suspended
VSTL to develop and approve a Remedial Plan that appropriately
brings the laboratory into compliance within an acceptable timeframe.
Remedial Plans shall be approved in writing. Ultimately, a VSTL’s
failure to cooperate or otherwise obtain approval of a remedial plan will
result in the termination of the cure process. A determination to
terminate the cure process will be made in writing by the Program
Director. Upon receipt of a notice that the cure process has been
terminated, a suspended VSTL shall have 10 days to request an
opportunity to be heard on revocation of accreditation (see Section
5.5.3., below).
5.5.2.3. VSTL Implementation of Plan. After the remedial plan has been approved
by the Program Director, the VSTL shall begin implementation. The
VSTL shall not deviate from an approved plan’s procedures,
requirements or deadlines without the written consent of the Program
Director. Failure to follow the remedial plan will result in the
termination of the cure process. A determination to terminate the cure
process will be made in writing by the Program Director. Upon receipt
of a notice that the cure process has been terminated, a suspended VSTL
shall have 10 days to request an opportunity to be heard on revocation
of accreditation (see Section 5.5.3., below).
5.5.2.4. EAC Verification of Remedy. Upon a VSTL’s timely completion of the
remedial plan and receipt of the VSTL’s Certification (see Section
5.5.2.1.5.), the Program Director shall verify compliance. At the
discretion of the Program Director, he or she may verify compliance
through the acceptance of the VSTL’s Certification or through the
various components of the Compliance Management Program (Chapter
4). If the Program Director determines that the remedial plan was not
completed, he or she may terminate the cure process. A determination
to terminate the cure process will be made in writing. Upon receipt of a
notice that the cure process has been terminated, a suspended VSTL
shall have 10 days to request an opportunity to be heard on revocation
of accreditation (see Section 5.5.3., below).
52
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
5.5.2.5. Notice of Compliance. The Program Director shall document his or her
verification that the remedial plan was complete by providing a written
notice of compliance to the VSTL. This notice shall state that the VSTL
is in compliance with program requirements and that the suspension is
lifted. The notice shall be posted on the EAC’s Web site and provided to
all registered Manufacturers.
5.5.3.
Opportunity to be Heard on Revocation of Accreditation. A VSTL has the right
to timely challenge the revocation of its accreditation prior to an EAC Decision
on Revocation. Unless otherwise noted above, a VSTL has 20 days from the date
it received its Decision on Suspension to submit a challenge. Late submissions
will not be considered. All challenges of revocation will be heard by the EAC
Commissioners. A challenge of revocation shall be submitted to the Program
Director, and addressed to the Chair of the U.S. Election Assistance Commission.
Each challenge of revocation shall be in writing and:
5.5.3.1. Shall identify each noncompliance which served as the basis of its
suspension;
5.5.3.2. Shall identify, document and provide verification of any remedial action
completed;
5.5.3.3. Shall provide, for each identified noncompliance, a written argument
challenging the finding of noncompliance; and
5.5.3.4. May provide any documentation and information in support of the
written statement.
5.6. Commissioners’ Decision on Revocation of Accreditation. Pursuant to HAVA, a VSTL
may have its accreditation revoked only by a vote of the EAC Commissioners. Upon a
timely receipt of a challenge of revocation, the program Director shall provide each
Commissioner all relevant documentation including: (1) the VSTL’s submission
challenging revocation, (2) copies of any terminated cure plans (3) the Notice of Intent to
Suspend, (4) the Compliance Management Report; (5) any documents pertaining to
challenges or remedial plans provided by the VSTL in response to a relevant Compliance
Management report; and (5) a Program Director recommendation as to disposition.
5.6.1.
Consideration. Each Commissioner shall review and consider all relevant
materials he or she has been provided. A Commissioner may request the
Program Director to provide additional relevant materials or information held by
the EAC or VSTL. Such requests and any responsive materials shall be provided
to each Commissioner. The Chair of the Commission shall ensure that each
53
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Commissioner has sufficient time to consider the relevant material before a vote
is called.
5.6.2.
Process. After a reasonable time to review the forwarded materials, the Chair of
the Commission shall bring the Decision of Revocation of Accreditation to a vote,
consistent with the rules of the Commission. The measure presented for a vote
shall take the form of a written Commissioners’ Decision on Revocation, which:
5.6.2.1. Makes a clear determination as to revocation on accreditation. The
Commissioners shall ultimately make one of two decisions:
5.6.2.1.1.
Program Compliance. If the VSTL demonstrates that it
meets all program requirements, successfully challenging
all previous findings of noncompliance, the
Commissioners shall find the VSTL compliant, reject the
revocation of accreditation and lift the VSTL’s suspension.
5.6.2.1.2.
Revocation of Accreditation. If the VSTL does not
demonstrate that it meets all program requirements and at
least one previous finding of noncompliance stands, the
Commissioners shall find the VSTL noncompliant and
revoke its accreditation.
5.6.2.2. Provides a finding with regard to each identified noncompliance which served
as the basis of suspension; and
5.6.2.3. Identifies the documents and information that served as the basis for the
Decision.
5.6.3.
Decision—Notice. After a vote of the Commissioners adopting a Decision on
Revocation, the Program Director shall forward the decision to the VSTL. At that
time the Program Director shall provide the VSTL notice of decision which
includes a summary of the laboratory’s appeal rights consistent with Section 5.8.,
below.
5.6.4.
Decision—Publication. After a vote of the Commissioners adopting a Decision
on Revocation, the Program Director shall cause the decision to be posted on the
EAC’s Web site, issue a copy to each registered voting system Manufacturer and
provide the decision to the Director of NIST.
5.7. Effect of Revocation of Accreditation. A revocation of accreditation is effective upon the
vote of the Commissioners. Laboratories that have had their accreditation revoked may
no longer test voting systems or submit test reports under the EAC certification program.
54
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
The laboratories may not represent themselves as accredited by EAC. A laboratory which
has had its accreditation revoked may reapply for an EAC accreditation consistent with
the requirements of Chapter 2, only after the EAC receives a new recommendation for
their participation from NIST. Where a revocation of accreditation results in the
termination of testing prior to completion, the laboratory which has had its accreditation
revoked must provide information to the EAC consistent with 2.10.7. of this manual.
Manufacturers may request the EAC grant permission to replace their lead VSTL pursuant
to section 4.3.1.2. of the Voting System Testing and Certification Program Manual.
5.8. Requesting Appeal. A laboratory that has had its accreditation revoked by a vote of the
Commissioners shall have the right to appeal. A Laboratory may appeal a Decision to
Revoke an Accreditation by first issuing a written request for appeal.
5.8.1.
Submission. Requests must be submitted in writing to the Program Director,
addressed to the Chair of the U.S. Election Assistance Commission.
5.8.2.
Timing of Appeal. The laboratory may request an appeal within 7 calendar days
of receipt of the Notice of Decision. Late requests will not be considered.
5.8.3.
Contents of Request. The request must petition for reconsideration of the
Commissioners’ Decision on Revocation and clearly state the specific conclusions
of the Decision the laboratory wishes to appeal.
5.9. EAC Action on a Request for Appeal. The Program Director shall accept any request for
appeal timely submitted. Untimely requests shall be rejected. Upon receipt of a request
for appeal, the Program Director shall notify the requestor laboratory, in writing, as to
whether their appeal has been accepted as timely. The notice for accepted requests shall
inform the applicant laboratory of the requirements for submitting their appeal per
Section 5.10. of this Manual.
5.10. Submission of Appeal. After submission of a timely request for appeal, the Laboratory
shall submit its appeal. This appeal shall (1) clearly identify the specific conclusions of the
Commissioners’ Decision the laboratory wishes to challenge, (2) provide the basis for its
position on appeal and (3) submit a written argument in support of its appeal. In
addition, the applicant laboratory may submit documentary or other relevant, physical
evidence in support of the appeal. The Appeal and all supporting materials must be
received by the EAC within 20 days of the applicant laboratory’s receipt of the Program
Director’s notice of acceptance of the request to appeal.
5.11. Consideration of Appeal. All timely appeals will be considered by the Commissioners.
Upon receipt of an appeal, the Chair of the Commission shall forward to each EAC
Commissioner the laboratory’s appellate submission, along with the original information
considered during the Commissioners Decision on Revocation (see Section 5.6.). After a
55
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
reasonable time to review and consider the forwarded materials, the Chair of the
Commission shall bring the matter to a vote, consistent with the rules of the Commission.
The measure presented for a vote shall take the form of a written Commissioners’
Decision on Appeal.
5.12. Commissioner’s Decision on Appeal. The Commissioners shall make a written, final
Decision on Appeal and shall provide it to the laboratory.
5.12.1. Contents. The Decision on Appeal shall:
5.12.1.1. State the final determination of the Commission.
5.12.1.2. Address the matters raised by the laboratory on appeal.
5.12.1.3. Provide the reasoning behind the decision.
5.12.1.4. State that the Decision on Appeal is final.
5.12.2. Determinations. The Commissioners shall make one of two determinations on
appeal.
5.12.2.1. Grant of Appeal. If the Commissioners determine that the previous
Decision of the Commission shall be overturned in full, and the
laboratory meets all program requirements, the appeal shall be
granted. In such cases, the laboratory shall have its accreditation
immediately reinstated.
5.12.2.2. Denial of Appeal. If the Commissioners determine that any part of the
previous Decision of the Commission shall be upheld such that the
procedural requirements of Chapter 3 or the Program requirements of
Chapter 2 of this manual will not be met in full, the appeal shall be
denied. In such cases, the application for appeal is finally denied.
5.12.3. Effect. All Decisions on Appeal shall be final and binding on the Applicant
Laboratory. No additional request for appeal shall be granted.
5.12.4. Notice. After a vote of the Commissioners adopting a Decision on Appeal, the
Program Director shall forward the decision to the VSTL.
5.12.5. Publication. After a vote of the Commissioners adopting a Decision on Appeal,
the Program Director shall cause the decision to be posted on the EAC Web site,
issue a copy to each registered voting system Manufacturer and provide the
decision to the Director of NIST.
56
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
57
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Requests for Interpretations
6.1. Overview. A Request for Interpretation is a means by which a registered Manufacturer or
VSTL may seek clarification on a specific EAC voting system standard (VVSG). An
Interpretation is a clarification of the voting system standards and guidance on how to
properly evaluate conformance to it. Suggestions or requests for modifications to the
standards are provided by other processes. This chapter outlines the policy, requirements,
and procedures for submitting a Request for Interpretation.
6.2. Policy. Registered Manufacturers or VSTLs may request that the EAC provide a definitive
Interpretation of EAC-accepted voting system standards (VVSG) when, in the course of
developing or testing a voting system, facts arise that make the meaning of a particular
standard ambiguous or unclear. The EAC may self-initiate such a request when its agents
identify a need for interpretation within the program. An Interpretation issued by the
EAC will serve to clarify what a given standard requires and how to properly evaluate
compliance. An Interpretation does not amend voting system standards, but serves only
to clarify existing standards.
6.3. Requirements for Submitting a Request for Interpretation. An EAC Interpretation is
limited in scope. The purpose of the Interpretation process is to provide Manufacturers or
VSTLs who are in the process of developing or testing a voting system a means for
resolving the meaning of a voting system standard in light of specific voting system
technology without having to present a finished product to EAC for certification. To
submit a Request for Interpretation, one must (1) be a proper requester, (2) request
interpretation of an applicable voting system standard, (3) present an actual controversy,
and (4) seek clarification on a matter of unsettled ambiguity.
6.3.1.
Proper Requestor. A Request for Interpretation may be submitted only by a
registered Manufacturer or a VSTL. Requests for Interpretation will not be
accepted from any other parties.
6.3.2.
Applicable Standard. A Request for Interpretation is limited to queries on EAC
voting system standards (i.e., VVSG). Moreover, a Manufacturer or VSTL may
submit a Request for Interpretation only on a version of EAC voting system
standards to which the EAC currently offers certification.
6.3.3.
Existing Factual Controversy. To submit a Request for Interpretation, a
Manufacturer or VSTL must present a question relative to a specific voting
system or technology proposed for use in a voting system. A Request for
Interpretation on hypothetical issues will not be addressed by the EAC. To
submit a Request for Interpretation, the need for clarification must have arisen
from the development or testing of a voting system. A factual controversy exists
58
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
when an attempt to apply a specific section of the VVSG to a specific system or
piece of technology creates ambiguity.
6.3.4.
Unsettled, Ambiguous Matter. Requests for Interpretation must involve actual
controversies that have not been previously settled. This requirement mandates
that interpretations contain actual ambiguities not previously clarified.
6.3.4.1. Actual Ambiguity. A proper Request for Interpretation must contain an
actual ambiguity. The interpretation process is not a means for
challenging a clear EAC voting system standard. Recommended
changes to voting system standards are welcome and may be forwarded
to the EAC, but they are not part of this program. An ambiguity arises
(in applying a voting system standard to a specific technology) when
one of the following occurs:
6.3.4.1.1.
The language of the standard is unclear on its face;
6.3.4.1.2.
One section of the standard seems to contradict another,
relevant section;
6.3.4.1.3.
The language of the standard, though clear on its face, lacks
sufficient detail or breadth to determine its proper
application to a particular technology;
6.3.4.1.4.
The language of a particular standard, when applied to a
specific technology, clearly conflicts with the established
purpose or intent of the standard; or
6.3.4.1.5.
The language of the standard is clear, but the proper means
to assess compliance is unclear.
6.3.4.2. Not Previously Clarified. The EAC will not accept a Request for
Interpretation when the issue has previously been clarified.
6.4. Procedure for Submitting a Request for Interpretation. A Request for Interpretation
shall be made in writing to the Program Director. All requests should be complete and as
detailed as possible because Interpretations issued by the EAC are based on, and limited
to, the facts presented. Failure to provide complete information may result in an
Interpretation that is off point and immaterial to the issue at hand. The following steps
must be taken when writing a Request for Interpretation:
6.4.1.
Establish Standing To Make the Request. To make a request, one must meet the
requirements identified in Section 6.3. above. Thus, the written request must
59
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
provide sufficient information for the Program Director to conclude that the
requestor is (1) a proper requester, (2) requesting an Interpretation of an
applicable voting system standard, (3) presenting an actual factual controversy,
and (4) seeking clarification on a matter of unsettled ambiguity.
6.4.2.
Identify the EAC Voting System Standard To Be Clarified. The request must
identify the specific standard or standards for which the requestor seeks
clarification. The request must state the version of the voting system standards
at issue (if applicable) and quote and correctly cite the applicable standards.
6.4.3.
State the Facts Giving Rise to the Ambiguity. The request must provide the facts
associated with the voting system technology that gave rise to the ambiguity in
the identified standard. The requestor must be careful to provide all necessary
information in a clear, concise manner. Any Interpretation issued by the EAC
will be based on the facts provided.
6.4.4.
Identify the Ambiguity. The request must identify the ambiguity it seeks to
resolve. The ambiguity shall be identified by stating a concise question that
meets the following requirements:
6.4.4.1. Shall be clearly stated;
6.4.4.2. Shall be related to and reference the voting system standard and voting
system technology information provided; and
6.4.4.3. Shall be limited to a single issue. Each question or issue arising from an
ambiguous standard must be stated separately. Compound questions
are unacceptable. If multiple issues exist, they should be presented as
individual, numbered questions.
6.4.4.4. Shall be stated in a way that can ultimately be answered yes or no.
6.4.5.
Provide a Proposed Interpretation. A Request for Interpretation should propose
an answer to the question posed. The answer should interpret the voting system
standard in the context of the facts presented. It should also provide the basis
and reasoning behind the proposal.
6.5. EAC Action on a Request for Interpretation. Upon receipt of a Request for
Interpretation, the EAC shall take the following action:
6.5.1.
Review the Request. The Program Director shall review the request to ensure it
is complete, is clear, and meets the requirements of Section 6.3. Upon review, the
Program Director may take the following action:
60
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
6.5.1.1. Request Clarification. If the Request for Interpretation is incomplete or
additional information is otherwise required, the Program Director may
request that the Manufacturer or VSTL clarify its Request for
Interpretation and identify any additional information required.
6.5.1.2. Reject the Request for Interpretation. If the Request for Interpretation does
not meet the requirements of Section 6.3., the Program Director may
reject it. Such rejection must be provided in writing to the Manufacturer
or VSTL and must state the basis for the rejection.
6.5.1.3. Notify Acceptance of the Request. If the Request for Interpretation is
acceptable, the Program Director will notify the Manufacturer or VSTL
in writing and provide it with an estimated date of completion. A
Request for Interpretation may be accepted in whole or in part. A notice
of acceptance shall state the issues accepted for interpretation.
6.5.2.
Consideration of the Request. After a Request for Interpretation has been
accepted, the matter shall be investigated and researched. Such action may
require the EAC to employ technical experts. It may also require the EAC to
request additional information from the Manufacturer or VSTL. The
Manufacturer or VSTL shall respond promptly to such requests.
6.5.3.
Interpretation. The Decision Authority shall be responsible for making
determinations on a Request for Interpretation. After this determination has
been made, a written Interpretation shall be sent to the Manufacturer or VSTL.
The following actions are necessary to prepare this written Interpretation:
6.5.3.1. State the question or questions investigated;
6.5.3.2. Outline the relevant facts that served as the basis of the Interpretation;
6.5.3.3. Identify the voting system standards interpreted;
6.5.3.4. State the conclusion reached; and
6.5.3.5. Inform the Manufacturer or VSTL of the effect of an Interpretation (see
Section 6.6.).
6.6. Effect of Interpretation. Interpretations are fact specific and case specific. They are not
tools of policy, but specific, fact-based guidance useful for resolving a particular problem.
An Interpretation is determinative and conclusive only with regard to the case presented.
Nevertheless, Interpretations do have some value as precedent. Interpretations published
61
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
by the EAC shall serve as reliable guidance and authority over identical or similar
questions of interpretation. These Interpretations will help users understand and apply
the provisions of EAC voting system standards.
6.7. Library of Interpretations. To better serve Manufacturers, VSTLs, and those interested in
the EAC voting system standards, the Program Director shall publish EAC
Interpretations. All proprietary information contained in an Interpretation will be
redacted before publication consistent with Chapter 7 of this Manual. The library of
published opinions is posted on the EAC Web site: www.eac.gov.
62
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Release of Laboratory Accreditation Program Information
7.1. Overview. VSTLs participating in the Certification Program will be required to provide
the EAC a variety of documents. In general, these documents will be releasable to the
public. Moreover, in many cases, the information provided will be affirmatively
published by the EAC. In limited cases, however, documents may not be released if they
include trade secrets, confidential commercial information, or personal information.
While the EAC is ultimately responsible for determining which documents Federal law
protects from release, VSTLs must identify the information they believe is protected and
provide substantiation and a legal basis for withholding. This chapter discusses EAC’s
general policy on the release of information and provides VSTL’s with standards,
procedures, and requirements for identifying documents as trade secrets or confidential
commercial information.
7.2. EAC Policy on the Release of Certification Program Information. The EAC seeks to
make its Voting System Test Laboratory Program as transparent as possible. The agency
believes that such action benefits the program by increasing public confidence in the
process and creating a more informed and involved public. As such, it is the policy of the
EAC to make all documents, or severable portions thereof, available to the public
consistent with Federal law (e.g. Freedom of Information Act (FOIA) and the Trade Secrets
Act).
7.2.1.
Requests for information. As in any Federal program, members of the public
may request access to Certification Program documents under FOIA (5 U.S.C.
§552). The EAC will promptly process such requests per the requirements of that
Act.
7.2.2.
Publication of documents. Beyond the requirements of FOIA, the EAC intends to
affirmatively publish program documents (or portions of documents) it believes
will be of interest to the public. This publication will be accomplished through
the use of the EAC Web site (www.eac.gov). The published documents will
cover the full spectrum of the program, including information pertaining to:
7.2.2.1. Accredited Laboratories;
7.2.2.2. VSTL test plans;
7.2.2.3. VSTL test reports;
7.2.2.4. Agency decisions;
7.2.2.5. Denials of Certification;
63
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
7.2.2.6. Issuance of Certifications;
7.2.2.7. Compliance Management Reports;
7.2.2.8. Suspensions or Revocation of Accreditations;
7.2.2.9. Appeals;
7.2.2.10. Official Interpretations (VVSG); and
7.2.2.11. Other topics as determined by the EAC.
7.2.3.
Trade Secret and Confidential Commercial Information. Federal law places a
number of restrictions on a Federal agency’s authority to release information to
the public. Two such restrictions are particularly relevant to the Accreditation
Program: (1) trade secrets information and (2) privileged or confidential
commercial information. Both types of information are explicitly prohibited
from release by the FOIA and the Trade Secrets Act (18 U.S.C. §1905).
7.3. Trade Secrets. A trade secret is a secret, commercially valuable plan, process, or device
that is used for the making or processing of a product and that is the end result of either
innovation or substantial effort. It relates to the productive process itself, describing how
a product is made. It does not relate to information describing end product capabilities,
features, or performance.
7.3.1.
The following examples illustrate productive processes that may be trade secrets:
7.3.1.1. Plans, schematics, and other drawings useful in production.
7.3.1.2. Specifications of materials used in production.
7.3.1.3. Voting system source code used to develop or manufacture software
where release would reveal actual programming.
7.3.1.4. Technical descriptions of manufacturing processes and other secret
information relating directly to the production process.
7.3.2.
The following examples are likely not trade secrets:
7.3.2.1. Information pertaining to a finished product’s capabilities or features.
7.3.2.2. Information pertaining to a finished product’s performance.
64
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
7.3.2.3. Information regarding product components that would not reveal any
commercially valuable information regarding production.
7.4. Privileged or Confidential Commercial Information. Privileged or confidential
commercial information is that information submitted by a VSTL that is commercial or
financial in nature and privileged or confidential.
7.4.1.
Commercial or Financial Information. The terms commercial and financial should
be given their ordinary meanings. They include records in which a submitting
VSTL has any commercial interest.
7.4.2.
Privileged or Confidential Information. Commercial or financial information is
privileged or confidential if its disclosure would likely cause substantial harm to
the competitive position of the submitter. The concept of harm to one’s
competitive position focuses on harm flowing from a competitor’s affirmative
use of the proprietary information. It does not include incidental harm
associated with upset customers or employees.
7.5. EAC’s Responsibilities. The EAC is ultimately responsible for determining whether or
not a document (in whole or in part) may be released pursuant to Federal law. In doing
so, however, the EAC will require information and input from the VSTL submitting the
documents. This requirement is essential for the EAC to identify, track, and make
determinations on the large volume of documentation it receives. The EAC has the
following responsibilities:
7.5.1.
Managing Documentation and Information. The EAC will control the
documentation it receives by ensuring that documents are secure and released to
third parties only after the appropriate review and determination.
7.5.2.
Contacting a VSTL on Proposed Release of Potentially Protected Documents. In
the event a member of the public submits a FOIA request for documents
provided by a VSTL or the EAC otherwise proposes the release of such
documents, the EAC will take the following actions:
7.5.2.1. Review the documents to determine if they are potentially protected
from release as trade secrets or confidential commercial information.
The documents at issue may have been previously identified as
protected by the VSTL when submitted (see Section 7.6.1. below) or
identified by the EAC on review.
7.5.2.2. Grant the submitting VSTL an opportunity to provide input. In the
event the information has been identified as potentially protected from
release as a trade secret or confidential commercial information, the
65
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
EAC will notify the submitter and allow it an opportunity to submit its
position on the issue prior to release of the information. The submitter
shall respond consistent with Section 7.6.1. below.
7.5.3.
Final Determination on Release. After providing the submitter of the
information an opportunity to be heard, the EAC will make a final decision on
release. The EAC will inform the submitter of this decision.
7.6. VSTL’s Responsibilities. Although the EAC is ultimately responsible for determining if a
document, or any portion thereof, is protected from release as a trade secret or
confidential commercial information, the VSTL shall be responsible for identifying
documents, or portions of documents, it believes warrant such protection. Moreover, the
VSTL will be responsible for providing the legal basis and substantiation for its
determination regarding the withholding of a document. This responsibility arises in two
situations: (1) upon the initial submission of information and (2) upon notification by the
EAC that it is considering the release of potentially protected information.
7.6.1.
Initial Submission of Information. When a VSTL is submitting documents to the
EAC as required by the Accreditation or Certification Programs, it is responsible
for identifying any document or portion of a document that it believes is
protected from release by Federal law. VSTLs shall identify protected
information 9 by taking the following action:
7.6.1.1. Submitting a Notice of Protected Information. This notice shall identify the
document, document page, or portion of a page that the VSTL believes
should be protected from release. This identification must be done with
specificity. For each piece of information identified, the VSTL must
state the legal basis for its protected status.
7.6.1.1.1.
Cite the applicable law that exempts the information from
release.
7.6.1.1.2.
Clearly discuss why that legal authority applies and why the
document must be protected from release.
7.6.1.1.3.
If necessary, provide additional documentation or
information. For example, if the VSTL claims a document
contains confidential commercial information, it would also
Documents submitted by the VSTL may include information that is a trade secret or confidential
commercial information of a Manufacturer. The VSTL shall take steps to identify any information it
believes may be protected. The VSTL may seek the input of the Manufacturer when identifying
potentially protected information pursuant to the requirements of this chapter. All communications on
this matter shall be in writing.
9
66
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
have to provide evidence and analysis of the competitive
harm that would result upon release.
7.6.1.2. Label Submissions. Label all submissions identified in the notice as
“Proprietary Commercial Information.” Label only those submissions
identified as protected. Attempts to indiscriminately label all materials
as proprietary will render the markings moot.
7.6.2.
Notification of Potential Release. In the event a VSTL is notified that the EAC is
considering the release of information that may be protected, the VSTL shall take
the following action:
7.6.2.1. Respond to the notice in writing within 15 calendar days. If additional
time is needed, the VSTL must promptly notify the Program Director.
Requests for additional time will be granted only for good cause and
must be made before the 15-day deadline. VSTLs that do not respond in
a timely manner will be viewed as not objecting to release.
7.6.2.2. Clearly state one of the following in the response:
7.6.2.2.1.
There is no objection to release, or
7.6.2.2.2.
The VSTL objects to release. In this case, the response must
clearly state which portions of the document the VSTL
believes should be protected from release. The VSTL shall
follow the procedures discussed in Section 7.6.1 above.
7.7. Personal Information. Certain personal information is protected from release under
FOIA and the Privacy Act (5 U.S.C. §552a). This information includes private information
about a person that, if released, would cause the individual embarrassment or constitute
an unwarranted invasion of personal privacy. Generally, the EAC will not require the
submission of private information about individuals. The incidental submission of such
information should be avoided. If a VSTL believes it is required to submit such
information, it should contact the Program Director. If the information will be submitted,
it must be properly identified. Examples of such information include the following:
7.7.1.
Social Security Number.
7.7.2.
Bank account numbers.
7.7.3.
Home address.
7.7.4.
Home phone number.
67
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
68
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Appendix A.
Voting System Test Plan Outline
69
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
This outline is provided solely as an aid to test plan development. Note that these items may
change significantly, depending on the specific project planned.
1 Introduction
1.1 References
1.2 Terms and Abbreviations
1.3 Testing Responsibilities
1.3.1 Project schedule with
1.3.1.1 Owner assignments
1.3.1.2 Test case development
1.3.1.3 Test procedure development and validation
1.3.1.4 3rd party tests
1.3.1.5 EAC and Manufacturer dependencies
1.4 Target of Evaluation Description
1.4.1 System Overview
1.4.2 Block diagram
1.4.3 System Limits
1.4.4 Supported Languages
1.4.5 Supported Functionality
1.4.5.1 Standard VVSG Functionality
1.4.5.2 Manufacturer Extensions
2. Pre‐Certification Testing and Issues
2.1 Evaluation of prior VSTL testing
2.1.1 Reason for testing and results, listing of modifications from previous to current
system
2.2 Evaluation of prior non‐VSTL testing
2.2.1 Reason for testing and results, states, other 3rd party entities
2.3 Known Field Issues
2.3.1 Listing of relevant issues uncovered during field operations
3 Materials Required for Testing
3.1 Software
3.2 Equipment
3.3 Test Materials
3.4 Deliverable Materials
4 Test Specifications
4.1 Requirements
4.1.1 Mapping of requirements to equipment type and features
4.1.2 Rationale for why some requirements are NA for this campaign
4.2 Hardware Configuration and Design
4.3 Software System Functions
70
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
4.4 Test Case Design
4.4.1 Hardware Qualitative Examination Design
4.4.1.1 Mapping of requirements to specific interfaces
4.4.2 Hardware Environmental Test Case Design
4.4.3 Software Module Test Case Design and Data
4.4.4 Software Functional Test Case Design and Data
4.4.5 System‐level Test Case Design
4.5 Security functions
4.6 TDP evaluation
4.7 Source Code review
4.8 QA & CM system review
5 Test Data
5.1 Data Recording
5.2 Test Data Criteria
5.3 Test Data Reduction
6 Test Procedure and Conditions
6.1 Facility Requirements
6.2 Test Set‐up
6.3 Test Sequence
7 Test Operations Procedures
Proprietary Data
71
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Appendix B
Voting System Modification Test Plan Outline
72
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Test Plans submitted for modifications to previously EAC certified voting systems should be
brief and structured to minimize test plan development and review, while enabling the EAC to
maintain solid control of the certification process. The test plan shall concisely document the
strategy and plan for testing those sections of the VVSG applicable to the modification or
modifications submitted. The test plan shall be written with clarity that will allow all
constituents to understand what testing will be conducted, to verify compliance to VVSG
requirements, and to assure that the test plan will remain a living document throughout the life
of the test campaign for the modification.
This outline is provided solely as an aid to test plan development. Note that these items may
change significantly, depending on the specific project planned.
1. Introduction
1.1Description and Overview of EAC certified system being modified
1.1.1 Complete definition of the baseline certified system.
1.1.2 Detailed description of the engineering changes and/or modifications to the
certified system and why the modification was implemented.
1.1.3 An initial assessment of the impact that the modifications have on the
system and past certification.
1.1.4 Description of what will be regression tested to establish assurance that the
modifications have no adverse impact on the compliance, integrity or
performance of the system.
1.2 References
1.3 Terms and Abbreviations
1.4 Project Schedule
1.5 Scope of testing
1.5.1 Block diagram (if applicable)
1.5.2 System limits (if applicable)
1.5.3 Supported Languages
1.5.4 Supported Functionality
1.5.5 VVSG
1.5.6 RFIs
1.5.7 NOCs
2. Pre‐Certification Testing and Issues
2.1 Evaluation of prior VSTL testing
2.2 Evaluation of prior non‐VSTL testing (if applicable)
2.3 Known Field Issues (if applicable)
3. Materials Required for Testing
3.1 Software
3.2 Equipment
73
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
3.3 Test Materials
3.4 Deliverable
3.5 Proprietary Data
4. Test Specifications
4.1 Requirements
4.1.1 Mapping of requirements to equipment type and features
4.1.2 Rationale for why some requirements are NA for this campaign
4.2 Hardware Configuration and Design (if applicable)
4.3 Software System Functions (if applicable)
4.4 Test Case Design
4.4.1 Hardware Qualitative Examination Design (if applicable)
4.4.2 Hardware Environmental Test Case Design (if applicable)
4.4.3 Software Module Test Case Design and Data (if applicable)
4.4.4 Software Functional Test Case Design and Data (if applicable)
4.4.5 System‐level Test Case Design
4.5 Security functions (if applicable)
4.6 TDP evaluation
4.7 Source Code review (if applicable)
4.8 QA & CM system review
5. Test Data
5.1 Test Data Recording
5.2 Test Data Criteria
6. Test Procedure and Conditions
6.1 Test Facilities
6.2 Test Set‐up
6.3 Test Sequence
6.4 Test Operations Procedure
74
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Appendix C
Voting System Test Report Outline
75
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Test Reports produced by VSTLs shall follow the format outlined below. Deviations from this
format may be used upon prior written approval of the Program Director.
1. System Identification and Overview
2. Certification Test Background
2.1 Revision History
2.2 Implementation Statement
3. Test Findings and Recommendation
3.1 Summary Finding and Recommendation
3.2 Reasons for Recommendation to Reject
3.3 Anomalies
3.4 Correction of Deficiencies
Appendix A. Additional Findings
Appendix B. Warrant of Accepting Change Control Responsibility
Appendix C. Trusted Build
Appendix D. Test Plan
Appendix E. State Test Reports
76
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Appendix D
Voting System Modification Test Report Outline
77
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Test Reports produced by VSTLs shall follow the format outlined below. Deviations from this
format may be used upon prior written approval of the Program Director.
1. Introduction
1.1Description of EAC certified system being modified
1.1 References
1.2 Terms and Abbreviations
2. Certification Test Background
2.1 Revision History
2.2 Scope of testing
2.2.1 Modification Overview
2.2.1.1 Detailed list of changes
2.2.2 Block diagram (if applicable)
2.2.3 Supported Languages
2.2.4 VVSG
2.2.5 RFIs
2.2.6 NOCs
3. Test Findings and Recommendation
3.1 Summary Finding and Recommendation
3.1.1 Hardware Testing
3.1.2 System Level Testing
3.1.3 Source code review
3.2 Anomalies and Resolutions
3.3 Deficiencies and Resolutions
4. Recommendation for Certification
Appendix A. Additional Findings
Appendix B. Deficiency report (if applicable)
Appendix C. Anomaly report (if applicable)
Appendix D. Test Plan
Appendix E. State Test Reports (if applicable)
78
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Appendix E
Certification of Laboratory Conditions and Practices
Form
Available in electronic format at www.eac.gov
79
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Appendix E.
CERTIFICATION OF LABORATORY CONDITIONS AND
PRACTICES
I, the undersigned, having investigated or caused to be investigated each matter, below;
certify, affirm and acknowledge that each of the following numbered statements are true
and otherwise accurately reflect the status, condition and operations of
___________________________ (hereinafter “Laboratory”). I understand that by certifying
the information below, I am making a statement or representation to the U.S. Election
Assistance Commission required for receiving a Laboratory Accreditation under 42 U.S.C.
§15371(b). I further understand, that to the extent any of the below representations or
certifications are found to be materially false, the U.S. Election Assistance Commission may
revoke any Accreditations granted to the above named laboratory and that I may be subject
to criminal prosecution under 18 U.S.C. §1001.
1. Signing Official. I hereby certify that I am an officer, partner or other official vested
with the legal authority to speak for, contract on behalf of, or otherwise bind the above
noted company, corporation, partnership or organization (Laboratory).
2. Personnel. I certify, consistent with Section 2.6. of the EAC Voting System Test Laboratory
Accreditation Program Manual (hereinafter Laboratory Manual), that the laboratory has
written policies in place to ensure that it does not currently, and will not in the future,
employ any individuals in any capacity related to the testing of voting systems who
have been convicted of a felony offense or any criminal offense involving fraud,
misrepresentation, or deception under either Federal or State law.
3. Conflicts of Interest and Prohibited Practices. I certify, consistent with Section 2.5. of
the Laboratory Manual, that the Laboratory maintains and enforces written policies
which:
a.
Prohibit conflicts of interest or the appearance of conflicts of interest pursuant to
Section 2.5.1. of the Laboratory Manual.
b. Prohibit practices such as participation in both the development and testing of a
voting system or the solicitation or acceptance of gifts from a voting system
manufacture pursuant to Section 2.5.2. of the Laboratory Manual.
80
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
c. Provide clear mechanisms for enforcement of the prohibitions noted above
pursuant to Section 2.5.3. of the Laboratory Manual.
4. Financial Stability. I certify, consistent with Section 2.14. of the Laboratory Manual,
that the laboratory possess sufficient resources to enable it to properly use and maintain
its test equipment and facility, to satisfactorily perform all required functions, and to
adequately indemnify itself against financial liabilities or penalties that may result from
its operations.
5. Authority to do Business in the United States. I certify, consistent with Section 2.12. of
the Laboratory Manual, that the Laboratory is lawfully entitled or otherwise not
prohibited from doing business with the United States or its citizens or operating in the
United States.
6. Recordkeeping. I certify, consistent with Section 2.15. of the Laboratory Manual, that
the laboratory operates and manages a records system in which it maintains all forms,
reports, test records, observations, calculations and derived data for all tests performed
for a period of at least 5 years.
I, by signing my name below, certify, affirm and acknowledge, under penalty of Federal law,
that each of the above numbered paragraphs accurately represent the operations, conditions
and practices of _____________________________ (Laboratory).
Signed this day,_____________:
________________________________________(Signature)
_________________________________________(Name of Signing Official)
__________________________________________(Title of Signing Official)
81
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Appendix F
Specification for Reproduction and use of the EAC
Laboratory Accreditation Logo
Accreditation Logo Available in electronic format at www.eac.gov
82
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
Appendix F.
Specification for Reproduction and use of the EAC Laboratory Accreditation Logo
To maintain a high level of quality and consistency in a variety of applications, the following
guidelines have been developed for VSTL use of the EAC laboratory accreditation logo.
Use and Display
The EAC VSTL logo contains the following elements:
The “U.S. Election Assistance Commission” and “VSTL” logotype separated by a divider rule.
The EAC will provide all accredited VSTLs with high resolution digital files for use on
approved written or electronic documents.
The logo may only be used by EAC accredited VSTLs and shall not misrepresent the specific
standards or guidelines to which the VSTL has been accredited. The EAC VSTL logo may be
displayed on all reports and work documents that contain exclusive results from testing
activities that have been carried out within the labs’ EAC scope of accreditation. Accredited
laboratories may also incorporate the logo in publicity and/or advertising materials, including
brochures and organization publications, technical literature, business reports, Web sites and
quotations or proposals for work.
Only the approved version of the VSTL logo may be used. When using the logo:
•
•
•
•
•
•
•
•
Do not print the logo in black over a dark background.
Do not change any colors of the logo.
Do not configure the elements of the logo in a different format.
Do not crop or remove any part of the logo.
Do not distort the logo.
Do not tilt the logo in any direction.
Do not add shadows, effects or other elements to the logo.
Do not change the typeface/font used in the logo.
Minimum Size
The full VSTL logo must remain readable in all uses and should not be reduced to a size smaller
than 2.5 inch x 1 inch.
Minimum Clear Space
83
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
The clear space surrounding the VSTL logo is an integral part of the logo design. An area of
clear space must be maintained around the logo to prevent it from being in conflict with other
design elements on the page. The clear space should measure at least X on all sides, where X
equals ½ the height of the upper case letters “VSTL” in the logo. Do not place any other logo,
logotype, trademark, text, or other graphic element in the minimum clear space area.
One Color Printing
A black version of the logo may be printed on white or light color background paper. In these
instances, the logo should appear in 100% black.
Color Printing
Whenever possible, the full color version of the logo should be used. The appropriate colors are
provided below for 4 color process printing or RGB for electronic use.
Blue
CMYK = 98/78/0/29
RGB = 0/51/153
HSL = 156/255/77
Red
CMYK = 5/96/98/5
RGB = 204/51/0
HSL = 10/255/102
Embossing on “VSTL” = CMYK 97/92/0/65
84
OMB Control Number: PENDING
Voting System Test Laboratory Program Manual, Version 2.0
U.S. Election Assistance Commission
VSTL
85
OMB Control Number: PENDING
File Type | application/pdf |
File Modified | 2015-04-20 |
File Created | 2015-04-20 |