Elections: Status and Use of Federal Voting Equipment Standards
(15-OCT-01, GAO-02-52).
Events surrounding the last presidential election raised concerns
about the conduct of elections that extend to the people,
processes, and technology used to administer elections. GAO has
reported on the scope of congressional authority in election
administration and voting assistance to military and overseas
citizens. This report focuses on the status and use of federal
voting equipment standards, which define minimum functional and
performance requirements for voting equipment. The standards
define minimum life-cycle management processes for voting
equipment developers to follow, such as quality assurance. No
federal agency has been assigned explicit statutory
responsibility for developing voting equipment standards,
however, the Federal Election Commission (FEC) assumed this role
by developing voluntary standards in 1990 for computer-based
systems, and Congress has supported this role with
appropriations. No federal agency has been assigned
responsibility for or assumed the role of testing voting
equipment against the federal standards. Instead, the National
Association of State Election Directors, through its Voting
Systems Committee, has assumed responsibility for implementing
the federal voting equipment standards by accrediting independent
test authorities who test equipment against the standards.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-02-52
ACCNO: A02236
TITLE: Elections: Status and Use of Federal Voting Equipment
Standards
DATE: 10/15/2001
SUBJECT: Congressional oversight
Elections
Standards and standardization
Quality assurance
Quality control
Voting records
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-02-52
Report to Congressional Requesters
United States General Accounting Office
GAO
October 2001 ELECTIONS Status and Use of Federal Voting Equipment Standards
GAO- 02- 52
Page i GAO- 02- 52 Voting Equipment Standards Letter 1
Results in Brief 2 Background 4 FEC Has Developed Voting Equipment Standards
but Has Not
Maintained Them 6 NASED?s Process for Testing and Qualifying Voting
Equipment 12 Conclusions 15 Matters for Congressional Consideration 16
Recommendations for Executive Action 16 Agency Comments and Our Evaluation
16
Appendix I Objectives, Scope, and Methodology 19
Appendix II Comments From the Federal Election Commission 21
Appendix III System Requirements and Supporting Life- Cycle Processes and
Our Analysis 27
Tables
Table 1: Types of Requirements and Processes Satisfied in FEC?s 1990 Voting
Equipment Standards 10 Table 2: Types of Requirements and Processes Not
Satisfied in
FEC?s 1990 Standards but Satisfied in the Draft Standards 11 Table 3: NASED
Qualified Equipment by Voting Equipment
Category 15
Figures
Figure 1: States That Require the Use of FEC?s Voting Equipment Standards 7
Figure 2: FEC?s Revised Standards Development Time Frame 8 Figure 3:
Simplified Diagram of the Voting Equipment Standards
Development and Implementation Process 13 Contents
Page 1 GAO- 02- 52 Voting Equipment Standards
October 15, 2001 The Honorable Thomas Daschle Majority Leader The Honorable
Trent Lott Minority Leader United States Senate
The Honorable Christopher J. Dodd Chairman The Honorable Mitch McConnell
Ranking Minority Member Committee on Rules and Administration United States
Senate
Events surrounding the last presidential election have brought to light a
number of issues about the conduct of elections that extend to the people,
processes, and technology used to administer these elections. We were asked
by several congressional committees and members to review aspects of
elections throughout the United States. In response, we are issuing a series
of reports. To date, we have issued reports on the scope of congressional
authority in election administration and voting assistance to military and
overseas citizens. 1 Other forthcoming reports will examine voting
accessibility for people with disabilities; the factors that affected the
uncounted votes in the November 2000 presidential election; and the people,
processes, and technology challenges affecting the conduct and
administration of elections. We are also issuing a capping report that draws
upon our extensive body of work to identify the main issues and challenges
confronting our nation?s election system and to delineate an analytical
framework that Congress could use as it weighs the merits of various reform
proposals.
This report focuses on the status and use of federal voting equipment
standards. These standards define minimum functional and performance
requirements for voting equipment, such as accurately recording votes cast.
The standards also define minimum life- cycle management processes
1 Elections: The Scope of Congressional Authority in Election Administration
(GAO- 01- 470, March 13, 2001) and Elections: Voting Assistance to Military
and Overseas Citizens Should Be Improved (GAO- 01- 1026, September 28,
2001).
United States General Accounting Office Washington, DC 20548
Page 2 GAO- 02- 52 Voting Equipment Standards
for voting equipment developers to follow, such as quality assurance. As
agreed with your offices, our objectives were to (1) identify the Federal
Election Commission?s (FEC) role regarding voting equipment and assess how
well FEC is fulfilling its role and (2) identify the National Association of
State Election Directors? (NASED) process for testing and qualifying voting
equipment against FEC?s standards. Details of our objectives, scope, and
methodology are contained in appendix I.
No federal agency has been assigned explicit statutory responsibility for
developing voting equipment standards; however, the Federal Election
Commission assumed this role by developing voluntary standards in 1990 for
computer- based systems, 2 and the Congress has supported this role with
appropriations. These standards describe specific performance benchmarks,
and address many- but not all- types of system requirements. In 1997, the
Commission initiated efforts to evaluate the 1990 standards to identify
areas to be updated and in 1999, it initiated efforts to update the
standards. We reviewed available drafts of these updated standards and found
they describe most- but again not all- types of system requirements. The
Commission plans to issue revised standards in 2002. This update is
necessary because the Commission has not proactively maintained them, thus
allowing them to become out of date. According to Federal Election
Commission officials, the Commission has not proactively maintained the
standards because it has not been assigned explicit responsibility to do so.
Without current, relevant, and complete voting equipment standards, states
may choose not to follow them, resulting in the adoption of disparate
standards that could drive up the cost of voting equipment and produce
unevenness among states in the capabilities of their respective equipment.
No federal agency has been assigned responsibility for or assumed the role
of testing voting equipment against the federal standards. Instead, the
National Association of State Election Directors, through its Voting Systems
Committee, has assumed responsibility for implementing the federal voting
equipment standards by accrediting independent test authorities, 3 which in
turn, test equipment against the standards. To this
2 Performance and Test Standards for Punchcard, Marksense, and Direct
Recording Electronic Voting Systems (January 1990). 3 Independent test
authorities are contractors independent from the voting equipment vendors
and are responsible for testing voting equipment to ensure that they meet
the Commission?s standards. Results in Brief
Page 3 GAO- 02- 52 Voting Equipment Standards
end, the committee has developed procedures to accredit the independent test
authorities. 4 According to the test authorities, testing is generally
iterative, in which the voting equipment vendors are provided an opportunity
to correct deficiencies identified during testing and resubmit the modified
equipment for retesting. When testing is successfully completed, the
independent test authorities notify the National Association of State
Election Directors that the equipment has satisfied testing requirements. 5
As of July 3, 2001, the Association had qualified 21 different voting
equipment, representing 10 vendors.
Because development, maintenance, and implementation of voting equipment
standards are very important responsibilities, we are raising matters for
congressional consideration regarding the explicit assignment of
responsibility in these areas. Additionally, we are making recommendations
to the Federal Election Commission?s commissioners aimed at improving its
efforts to update its 1990 standards.
In written comments on a draft of this report, the Commission stated that it
generally agreed with most of our observations and recommendations, and that
it agreed with our matter for congressional consideration. However, the
Commission commented that it disagreed with our conclusions that it has not
proactively maintained its voting equipment standards and that its draft
updates to those standards do not satisfy security certification and quality
assurance requirements. While the Commission deserves credit for recognizing
the need over a decade ago for voting equipment standards and developing and
updating them, despite a lack of explicit statutory responsibility to do so,
we disagree that its efforts to update its 1990 standards have been
proactive. Nine years elapsed from the time the Commission issued the
standards and the time the Commission began updating them; this is too long
a period between updates given advances in the field and is the major reason
that the current standards are out of date. Further, while we agree that the
draft standards address quality assurance, we are concerned that they omit
4 According to the National Association of State Election Directors,
accreditation signifies formal recognition that the independent test
authority possesses or will acquire the competence to design and perform
specific test methods applicable to voting equipment hardware and software
and that the test authority has adequately demonstrated its competence for
voting equipment testing.
5 Independent test authorities notify the Election Center, which serves as
the National Association of State Election Directors? Secretariat and
maintains the list of the association?s approved voting equipment.
Page 4 GAO- 02- 52 Voting Equipment Standards
essential aspects of quality assurance, such as quality assurance plans and
process reviews. Regarding the draft standards satisfaction of security
requirements, subsequent to commenting on a draft of this report, the
Commission provided us additional draft standards, which address the
requirements for security certification. Therefore, we have modified this
report to recognize this new information. Last, we acknowledge the
Commission?s position, as stated in its comments, for not including certain
quality assurance standards areas in the revised draft standards was the
result of deliberative and collaborative interaction among NASED and
Commission staff, and we have modified this report to reflect this change.
In 1975 Congress created the FEC to administer and enforce the Federal
Election Campaign Act. To carry out this role, FEC discloses campaign
finance information, enforces provisions of the law such as limits and
prohibitions on contributions, and oversees the public funding of
presidential elections.
Within FEC, the Office of Election Administration (OEA) serves as a national
clearinghouse for information regarding the administration of federal
elections. As such, OEA assists state and local election officials by
developing voluntary voting equipment standards, responding to inquiries,
publishing research on election issues, and conducting workshops on all
matters related to election administration. In addition, it answers
questions from the public and briefs foreign delegations on the U. S.
election process, including voter registration and voting statistics.
FEC consists of six voting members, appointed by the President and confirmed
by the Senate. To encourage nonpartisan decisions, no more than three
commissioners can be members of the same political party, and at least four
votes are required for most official Commission actions.
FEC?s budget for fiscal year 2001 is $40.4 million, and of that amount,
$804,000 is allocated to support OEA functions. FEC has 357 full- time
staff, of which 5 are allocated to OEA functions.
The voting methods used in the United States can be placed into five
categories: paper ballots, mechanical lever machines, punch cards, optical
scan, and direct recording electronic. The last three methods use Background
Voting Equipment Used in the United States
Page 5 GAO- 02- 52 Voting Equipment Standards
computer- based equipment. Three of the five- paper ballots, punch cards,
and optical scan- use some kind of paper ballot to record voters? choices. 6
Paper Ballot. Voters use a paper ballot listing the names of the
candidates and issues and record their choice by placing a mark in a box
next to the candidate?s name or issue. After making their choices, the
ballots are dropped into a sealed ballot box to be manually tabulated.
Mechanical Lever. Voters pull a lever next to the candidate?s name or issue
and the machine records and tallies the votes using a counting mechanism.
Write- in votes must be recorded on a separate document. Election officials
tally the votes by reading the counting mechanism totals on each lever
voting machine. Punch Card. Voters can use one of two basic types of punch
cards-
Votomatic or Datavote. In both instances, voters use a computer- readable
card to cast their vote. The Votomatic uses a computer- readable card with
numbered boxes that correspond to a particular ballot choice. The choices
corresponding to those numbered boxes are indicated to the voter in a
booklet attached to a vote recording machine, with the appropriate places to
punch indicated for each candidate and ballot choice. The voter uses a
simple stylus to punch out the box corresponding to each candidate and
ballot choice. In the Datavote, the names of the candidates and issues are
printed on the card itself- there is no ballot booklet. The voter uses a
stapler- like punching device to punch out the box corresponding to each
candidate and ballot choice. To tally the votes in both instances, the
ballots are fed into a computerized tabulation machine that records the vote
by reading the holes in the ballots. Optical Scan. Voters use a computer-
readable paper ballot listing the
names of the candidates and issues. The voters record their choices by using
an appropriate writing instrument to fill in a box or oval, or complete an
arrow next to the candidate?s name or issue. The ballot is then fed into a
computerized tabulation machine, which senses or reads the marks on the
ballot, and records the vote. Direct Recording Electronic. Voters use a
ballot that is printed and posted
on the voting machine or displayed on a computer screen listing the names of
the candidates and issues. Voters record their choices by pushing a button
or touching the screen next to the candidate?s name or issue. When a voter
is finished, the vote is submitted by pressing a vote button, which
6 A more detailed description of each type of voting method and the
associated equipment can be found in Elections: Perspectives on Activities
and Challenges Across the Nation
(GAO- 02- 03, October, 2001).
Page 6 GAO- 02- 52 Voting Equipment Standards
stores the vote in a computer memory chip. Election officials tally the
votes by reading the votes totaled on each machine?s computer chip.
While neither FEC nor any other federal agency has explicit statutory
responsibility to develop voting equipment standards, the Congress has
appropriated funds for FEC to develop and update the standards. FEC first
issued voting equipment standards in 1990. These standards identify minimum
functional and performance requirements for punch card, optical scan, and
direct recording electronic voting equipment, and specify test procedures to
ensure that the equipment meet these requirements. 7 The functional and
performance requirements address what voting equipment should do and
delineate minimum performance thresholds, documentation provisions, and
security and quality assurance requirements. The test procedures describe
three stages of testing: qualification, certification, and acceptance.
According to FEC?s standards document:
Qualification testing is the process by which a voting equipment is shown
to comply with the requirements of its own design specification and with the
requirements of FEC standards. Certification testing, generally conducted
by individual states, determines
how well voting equipment conform to individual state laws and requirements.
Acceptance testing is generally performed by the local jurisdictions
procuring voting equipment and demonstrates that the equipment, as delivered
and installed, satisfies all the jurisdiction?s functional and performance
requirements.
The standards are voluntary; states are free to adopt them in whole, in
part, or reject them entirely. To date, 38 states require that voting
equipment used in the state meet FEC standards either in total or in part. 8
Figure 1 shows these states.
7 The FEC standards address only computer- based systems; therefore, the
standards do not address the paper and mechanical lever machine voting
methods. 8 This includes the District of Columbia. Four of the 38 states
reported that they followed the FEC standards in part. FEC Has Developed
Voting Equipment Standards but Has Not Maintained Them
Page 7 GAO- 02- 52 Voting Equipment Standards
Figure 1: States That Require the Use of FEC?s Voting Equipment Standards
Source: GAO survey results.
In September 1997, FEC initiated efforts to evaluate its voting equipment
standards and identify areas to be updated, and in July 1999, FEC initiated
efforts to revise the standards. 9 As part of this revision, FEC has been
working closely with state and local election officials and vendors to
incorporate industry comments on the draft standards. FEC plans to issue the
revised standards in multiple volumes: volume I is to include the
9 FEC has contracted with American Management Systems to support FEC in
revising the voting equipment standards.
Page 8 GAO- 02- 52 Voting Equipment Standards
functional and performance requirements for voting equipment; volume II is
to provide the detailed test procedures, including information to be
submitted by the vendor, tests to be conducted to ensure compliance with the
standards, and the criteria to be applied to pass the individual tests.
Figure 2 depicts FEC?s time frames for revising the standards.
Figure 2: FEC?s Revised Standards Development Time Frame
Source: FEC data.
Organizations such as the Department of Defense and the Institute of
Electrical and Electronics Engineers have developed guidelines for various
types of systems requirements and for the processes that are important to
managing the development of any system throughout its life cycle. These
types of systems requirements and processes include, for example:
Security and Privacy Protection. Requirements defining the security/
privacy environment, types of security needed (e. g., data confidentiality
and fraud prevention), risks the system must withstand, safeguards required
to withstand those risks, security/ privacy policies that must be met,
accountability (i. e., audit trails), and criteria for security
certification. 10
10 Security certification is the technical evaluation that establishes the
extent to which a computer system, application, or network design and
implementation meets specified security requirements. FEC?s Voting Equipment
Standards Address Most Key Types of Systems Requirements
Page 9 GAO- 02- 52 Voting Equipment Standards
Human Factors. Requirements defining the usability of the system,
including considerations for human capabilities and limitations, and the use
and accessibility of the system by persons with disabilities.
Documentation. Processes for recording information produced during the
system development life cycle, which includes identifying documents to be
produced; identifying the format, content, and presentation items for each
document; and developing a process for reviewing and approving each
document. Configuration Management. Processes to establish and maintain
the
integrity of work products through the system development life cycle,
including developing a configuration management plan, identifying work
products to be maintained and controlled, establishing a repository to
maintain and control them, evaluating and approving changes to the work
products, accounting for changes to the products, and managing the release
and delivery of products. Quality Assurance. Processes to provide
independent verification of the
requirements and processes used to develop and produce the system, which
include developing a quality assurance plan, determining what system
development product and process standards are supposed to be followed, and
conducting reviews to ensure that the product and process standards are
followed.
While FEC?s 1990 standards satisfy most of these areas, they do not satisfy
all. For example, in the area of security, the standards do not address the
security/ privacy environment in which the voting equipment must operate,
the types of security to be provided, the risks the equipment must
withstand, the security/ privacy policies that must be met, or the criteria
for security certification. Further, the standards do not specify
requirements for voting equipment usability, taking into account human
capabilities and limitations, or the use and accessibility of the voting
equipment by persons with disabilities. Table 1 summarizes the types of
requirements and processes satisfied in FEC?s 1990 voting equipment
standards.
Page 10 GAO- 02- 52 Voting Equipment Standards
Table 1: Types of Requirements and Processes Satisfied in FEC?s 1990 Voting
Equipment Standards
Types of requirements and processes Satisfied in the 1990 standards?
Functional and performance Yes System quality Yes Security and privacy
protection No Human factors No System external interface Yes Installation-
dependent data Yes System environment Yes Design constraints Yes Physical
Yes Safety Yes Personnel Yes Training Yes Logistics Yes Documentation No
Configuration management No Quality assurance No
Source: GAO analysis.
As part of FEC?s current effort to revise the 1990 standards, it has made
improvements in all five of the areas in which we identified missing types
of requirements and processes. For example, in the area of human factors,
the draft standards now include requirements for the use and accessibility
of voting equipment by persons with disabilities. 11 Further, for
documentation, the draft standards include requirements for identifying
documents to be produced; defining the format, content, and presentation
items for each document; and developing a process for reviewing and
approving each document. In addition, in the area of security, the standards
now address security types, risks, safeguards, policies, accountability, and
certification.
While FEC has made improvements, the draft standards do not satisfy two
areas- human factors and quality assurance. For example, in the area of
human factors, the draft standards do not address requirements for equipment
usability, including considerations for human capabilities and
11 Our review of these standards did not include validating that the
requirements are correct and complete, beyond determining whether the
standards addressed all the requirements and process elements.
Page 11 GAO- 02- 52 Voting Equipment Standards
limitations. Finally, the draft standards do not yet specify the development
of a quality assurance plan or the performance of quality assurance reviews
to ensure that the equipment development process requirements are being met.
Table 2 summarizes the types of requirements and processes not satisfied in
FEC?s 1990 voting equipment standards but satisfied in the draft standards.
Appendix III provides a detailed description of the requirements and process
types and our complete analysis of FEC?s 1990 voting standards and draft
standards.
Table 2: Types of Requirements and Processes Not Satisfied in FEC?s 1990
Standards but Satisfied in the Draft Standards
Type of requirements and processes Satisfied in the draft standards?
Security and privacy protection Yes Human factors No Documentation Yes
Configuration management Yes Quality assurance No
Source: GAO analysis.
In the area of quality assurance, FEC stated in its written comments on a
draft of this report that its decision to not include quality assurance
process reviews in the revised standards was the result of deliberative and
collaborative interaction among NASED?s Voting System Committee and FEC
staff. In addition, FEC did not include equipment usability because it was
determined not to be an area of immediate concern by the election community
during FEC?s evaluation of the standards to identify areas to be updated.
FEC agrees that equipment usability should be addressed in the standards and
has stated that it will fully do so once resources are available. Beyond
this stated commitment, FEC has not established any specific plans or
allocated specific resources for doing so. Until FEC addresses these missing
requirements, the voting equipment standards? value and utility will be
diminished.
Given the pace of today?s technological advances, standards must be
proactively maintained to ensure that they remain current, relevant, and
complete. Standards- setting bodies, such as the American National Standards
Institute and the National Institute of Standards and Technology, require
that standards be revised or reaffirmed at least once every 5 years. This is
particularly important with voting equipment standards, which must respond
to technological developments if they are FEC Has Not Proactively
Maintained the Voting Equipment Standards
Page 12 GAO- 02- 52 Voting Equipment Standards
to be current, complete, and relevant, and are to be useful to state and
local election officials in assuring the public that their voting equipment
are reliable.
FEC has not proactively maintained its voting equipment standards. As
previously stated, FEC is only now updating the 1990 standards. Because FEC
has not proactively maintained the standards, they have become out of date.
Vendors are using new technology and expanding voting equipment functions
that are not sufficiently covered by the 1990 standards. For example, the
1990 standards do not address election management systems, which are used to
prepare ballots and programs for use in casting and tallying votes, and to
consolidate, report, and display election results. According to a NASED
committee representative and the Independent Test Authority (ITA)
responsible for testing election management systems, the lack of adequate
standards to address election management software has forced them to
interpret the current voting equipment standards to accommodate the
development of this new software. Further, according to these
representatives, these interpretations have not been documented and formally
shared with FEC. As mentioned earlier, FEC is updating its standards, and
the draft standards now address election management systems.
FEC officials acknowledge the need to actively maintain the standards, but
state that they have not done so because they have not been assigned
explicit responsibility. By not ensuring that voting equipment standards are
current, complete, and relevant, states may choose not to follow them,
resulting in states adopting disparate standards. In turn, this could drive
up the cost of voting equipment being designed to multiple standards and
produce unevenness among states in the capabilities of voting equipment.
No federal agency, including FEC, has been assigned explicit responsibility
for testing voting equipment against FEC standards, and no federal agency
has assumed this role. Rather, NASED has assumed responsibility for
implementing the standards. 12 To do so, NASED established a voting systems
committee, which comprises selected state and local election officials and
technical advisers. This committee
12 NASED, which comprises chief election officials from each state and
territory of the United States, provides a forum for state election
officials to share information about their duties, responsibilities, methods
of operation, and suggestions for improving election laws. NASED?s Process
for
Testing and Qualifying Voting Equipment
Page 13 GAO- 02- 52 Voting Equipment Standards
accredits ITAs to test and qualify voting equipment against FEC standards.
Figure 3 illustrates the voting equipment standards program, from the
development of voting equipment standards through the testing and
qualification of voting equipment.
Figure 3: Simplified Diagram of the Voting Equipment Standards Development
and Implementation Process
Source: GAO, based on FEC and NASED information.
To accredit the ITAs, the NASED committee has developed requirements and
procedures, which include provisions for NASED to periodically reaccredit
the ITAs and conduct on- site inspection visits, both of which
Page 14 GAO- 02- 52 Voting Equipment Standards
are important to ensuring that the accredited laboratories continue to
comply with all requirements. 13 To date, the committee has not reaccredited
or inspected ITAs because, according to NASED committee representatives,
they rely on the committee?s technical advisers? ongoing conversations with
ITA officials and the officials? participation in committee meetings to
ensure that the ITAs are fulfilling their responsibilities effectively.
Currently, three ITAs are approved to test voting equipment against the FEC
standards. In 1994, the NASED committee accredited Wyle Labs to test the
hardware and machine- resident software components of proprietary vote cast
and tally equipment. In February 2001, Metamor (previously PSINet) applied
for accreditation to conduct qualification testing of vote tabulation and
election management software. 14 Also in 2001, SysTest applied for
accreditation to conduct qualification testing of voting tabulation and
election management software. While both Metamor and SysTest have been
granted an interim approval to test voting equipment, NASED has not yet
accredited either.
To test voting equipment, voting equipment vendors submit requests for
testing to the ITAs, who then prepare a test procedure. The test procedure
details the software and hardware testing requirements that the voting
equipment will be tested against and is based on both the FEC voting
equipment standards and the vendors? design specifications.
According to ITA officials responsible for testing voting equipment, the
testing process is generally an iterative one. Vendors are provided an
opportunity to correct deficiencies identified during testing and resubmit
the modified voting equipment for retesting. At the end of testing, the ITA
completes a test report and notifies the Election Center that the voting
equipment has successfully satisfied testing requirements. 15 The Election
13 NASED, Accreditation of Independent Testing Authorities For Voting System
Qualification Testing Handbook (May 1, 1992). 14 Software qualification
testing does not include testing the software which is in a permanent
machine resident status (e. g., programmed on a read only memory chip) nor
the hardware and operating systems on which the software operates. Also, in
1997, the committee accredited Nichols Research to test software and the
integration of the software with vote cast and tally equipment. However, as
of December 2000, this responsibility was moved to PSINet.
15 The Election Center is a nonprofit organization dedicated to training
local election officials in election administration and serves as the NASED
voting systems committee secretariat, assisting NASED in implementing
national voting equipment standards.
Page 15 GAO- 02- 52 Voting Equipment Standards
Center then assigns a NASED number to the specific equipment model and
firmware release that was tested and maintains the list of qualified voting
equipment. 16 Each time a vendor issues a new model or software release, the
vendor is to submit a request for testing to the ITAs in order to qualify
the new model or release. As of July 3, 2001, NASED had qualified 21 models
of voting equipment and 7 election management systems, representing 10
vendors. See table 3 for a breakout of the types of equipment qualified.
Table 3: NASED Qualified Equipment by Voting Equipment Category Equipment
category Number qualified
Punch card 1 Optical scan 6 Direct recording electronic 14 Election
management systems a 7
Total 28
a Not specifically a separate voting equipment category. Rather these
systems support other voting equipment categories by preparing ballots and
programs for use in casting and tallying votes, and consolidating,
reporting, and displaying election results.
Source: NASED data, as of July 3, 2001.
The ITAs stated that the testing process generally takes about 2 to 3
months. This is contingent, however, upon the vendors having the proper
documentation in order. If documentation is missing or incomplete, the
process may take longer. According to the ITAs, the cost of qualification
testing ranges from $40, 000 for vote cast and tally equipment to $75,000
for vote tabulation and election management software.
While not explicitly provided for in legislation, FEC and NASED have assumed
and are performing important roles by developing voting equipment standards
and testing and qualifying equipment against these standards, respectively.
Given the current pace of technological change for voting equipment, the
degree to which these standards are actively maintained and the extent to
which they are appropriately applied, can have a direct bearing on the
capabilities of voting equipment. This, in turn, can affect the successful
conduct of national, state, and local elections. Therefore, it is important
that responsibility for these roles be clearly
16 Firmware is software that is embedded in a hardware device that allows
reading and executing the software, but it does not allow modification.
Conclusions
Page 16 GAO- 02- 52 Voting Equipment Standards
assigned. By doing so, the appropriate federal role in these important areas
can be deliberated, decided, and explicitly defined, thereby avoiding
another situation where the standards are allowed to become out of date. It
is also important that these roles be executed effectively. In the case of
FEC?s ongoing update of the standards, this means that requirements for
equipment usability, and quality assurance should be developed.
As part of the ongoing debate and deliberation over election reform in
general, and the federal role in voting equipment standards in particular,
the Congress may wish to consider assigning explicit federal authority,
responsibility, and accountability for voting equipment standards, including
proactive and continuous update and maintenance of the standards. Given that
no federal or state entity has been assigned explicit authority or
responsibility for testing voting equipment against the FEC standards, the
Congress may wish to consider what, if any, federal role is appropriate,
regarding implementation of the standards, including the accreditation of
ITAs and the qualification of voting equipment.
To improve the quality of FEC?s voting equipment standards, we recommend
that the FEC Commissioners direct the OEA Director to accelerate the
development of requirements for equipment usability, including
considerations for human capabilities and limitations. To improve the
quality of FEC?s current efforts to update the voting equipment standards,
we also recommend that the FEC Commissioners direct the OEA Director to
develop requirements for quality assurance, including developing a quality
assurance plan and conducting quality assurance process reviews.
In its written comments on a draft of this report (reprinted in appendix
II), the FEC Chairman and Vice Chairman stated that FEC generally agrees
with most of our observations and recommendations, including that human
factors are not being addressed in the revised voting equipment standards
and that FEC needs to accelerate their development in future iterations of
the standards. Additionally, FEC agreed with our matter for congressional
consideration.
Nevertheless, FEC commented that it was concerned with the report?s
portrayal of the Commission as being insufficiently proactive in revising
voting equipment standards, stating that its efforts have been as timely as
Matters for
Congressional Consideration
Recommendations for Executive Action
Agency Comments and Our Evaluation
Page 17 GAO- 02- 52 Voting Equipment Standards
possible given certain practical constraints, which it described in a
chronology of events and circumstances. FEC also commented that it disagrees
with the draft report?s characterization of the Commission?s ongoing efforts
to update security and quality assurance standards as incomplete, describing
how both areas are being addressed. Subsequent to providing us with its
written comments on a draft of this report, FEC also provided us with
additional draft standards that address security requirements. Accordingly,
we have modified this report, including our recommendations, to reflect this
new information.
We do not agree with either of FEC?s other two points of concern. Regarding
FEC?s concern with the report?s portrayal of the Commission as being
insufficiently proactive in revising voting equipment standards, FEC states
in its comments that 7 years elapsed from the time that the standards were
first issued in 1990 to the time that FEC first began evaluating them to
identify areas that needed to be updated. Further, it states that another 2
years elapsed between the time FEC began evaluating the standards and the
time it began updating them. We recognize that FEC is performing, through
its own initiative, an important role in developing and updating the
standards, and deserves credit for doing so. However, in our view, allowing
9 years to pass before beginning to update the standards, regardless of the
practical circumstances that FEC cites, is too long and does not constitute
a proactive maintenance process and is the primary reason that the current
standards are out of date. Regarding FEC?s disagreement with the report?s
characterization of the Commission?s ongoing efforts to update quality
assurance standards as incomplete, we do not question, and in fact state in
this report, that the draft standards address requirements for quality
assurance. However, our main concern is that important and relevant aspects
of quality assurance requirements, such as quality assurance plans and
process reviews, respectively, are not addressed.
Concerning FEC?s decision to omit quality assurance standards areas from the
revised draft standards, we modified this report to reflect FEC?s position
that its decision resulted from deliberative and collaborative interaction
among NASED and FEC staff and was not, as we were told during the course of
our review by the OEA Director, areas that were overlooked.
We are sending copies of this report to the Chairmen and Ranking Minority
Members of the Senate Appropriations Subcommittee on Treasury and General
Government and the House Appropriations
Page 18 GAO- 02- 52 Voting Equipment Standards
Subcommittee on Treasury, Postal Service, and General Government; the
Director of the Office of Management and Budget; and the Chairman and Vice
Chairman of FEC. Copies will also be available at our Web site at www. gao.
gov. If you have any questions, please contact me at (202) 512- 6240 or by
email at hiter@ gao. gov. Key contributors to this assignment were Deborah
A. Davis, Richard Hung, and Eric Winter.
Randolph C. Hite Director, Information Technology
Systems Issues
Appendix I: Objectives, Scope, and Methodology
Page 19 GAO- 02- 52 Voting Equipment Standards
The objectives of our review were to (1) identify Federal Election
Commission?s (FEC) role regarding voting equipment and assess how well FEC
is fulfilling its role and (2) identify the National Association of State
Election Director?s (NASED) process for testing and qualifying voting
equipment against FEC?s voluntary voting equipment standards.
To identify FEC?s role regarding voting equipment, we researched FEC?s
statutory and legislative role in developing and maintaining voting
equipment standards. To further identify FEC?s role, we reviewed relevant
documents, including the Plan to Update the Voting Systems Standards, 1 the
standards update project contract, project work plans, and legislative
proposals, and interviewed key FEC officials, including the Director, OEA.
To assess FEC?s voting equipment standards, we examined relevant guidelines
and procedures for the development of system requirements. Specifically, we
examined the Department of Defense?s Data Item Description for System/
Subsystem Specifications; the Institute of Electrical and Electronics
Engineers? Standard 12207 on Software Life Cycle Processes, and the Software
Engineering Institute?s Software Development Capability Maturity Model? 2
and identified 13 types of systems requirements and 3 supporting life- cycle
processes that are important in the development of any system. We then
compared these types of requirements and processes against FEC?s 1990 voting
equipment standards to determine if all key elements were addressed. In
those areas where variances were noted, we compared the types of
requirements and processes against relevant sections of volumes I and II of
the draft standards to determine whether FEC had addressed any of these
missing requirements. 3 We only reviewed those portions of the draft
standards for which we identified missing types of requirements and
processes in the 1990 standards. In addition, our review of the standards
did not include validating that the requirements are correct and complete
beyond determining whether the standards addressed all of the requirements
and process key elements.
1 ManTech Advanced Systems International, Inc., February 9, 1999. 2 The
Software Engineering Institute is operated by Carnegie Mellon University as
a federally funded research and development center sponsored by the
Department of Defense. Capability Maturity Model is a registered mark of
Carnegie Mellon University.
3 We examined all but two sections (hardware standards and software/
firmware standards) in volume I and those sections pertaining to security
and certification in volume II. Appendix I: Objectives, Scope, and
Methodology
Appendix I: Objectives, Scope, and Methodology
Page 20 GAO- 02- 52 Voting Equipment Standards
To identify NASED?s process for testing and qualifying voting equipment
against FEC?s voting equipment standards, we interviewed officials from
NASED, the Election Center, and the two independent test authorities (ITA).
We also reviewed documentation describing NASED?s process, NASED?s
Accreditation of Independent Testing Authorities For Voting System
Qualification Testing Handbook, 4 ITAs? generic test plans, and NASED?s
policies, procedures, and by- laws. We also provided a copy of relevant
parts of this report to the Chairman of the NASED Voting System Committee
for comment. The Chairman stated that the report accurately reflected the
NASED process.
We also contacted officials in the State Election Director's offices in each
of the 50 states and the District of Columbia to determine which states
required that their voting equipment be in compliance with FEC's standards.
We did not verify the officials' responses.
We performed our work at FEC headquarters in Washington, D. C., NASED, the
Election Center, and the independent test authorities from March 2001
through September 2001, in accordance with generally accepted government
auditing standards.
4 NASED, May 1, 1992.
Appendix II: Comments From the Federal Election Commission
Page 21 GAO- 02- 52 Voting Equipment Standards
Appendix II: Comments From the Federal Election Commission
Note: GAO comments supplementing those in the report text appear at the end
of this appendix.
See comment 2. See comment 1.
Appendix II: Comments From the Federal Election Commission
Page 22 GAO- 02- 52 Voting Equipment Standards
Appendix II: Comments From the Federal Election Commission
Page 23 GAO- 02- 52 Voting Equipment Standards
See comment 6. See comment 5.
See comment 4. See comment 3.
Appendix II: Comments From the Federal Election Commission
Page 24 GAO- 02- 52 Voting Equipment Standards
Appendix II: Comments From the Federal Election Commission
Page 25 GAO- 02- 52 Voting Equipment Standards
The following are GAO?s comments on the Federal Election Commission letter
dated July 18, 2001.
1. See comments 2, 5, and 6. 2. We do not dispute either the chronology of
events provided in FEC?s
comments or its statement that it does not have explicit statutory authority
to develop and revise the standards. We provide the relevant elements of
this chronology in this report. Additionally, we state in this report that
FEC has assumed and is performing an important role by developing and
revising the standards, despite its lack of explicit statutory
responsibility.
We do not agree with FEC?s comment that it has been proactive in updating
the voting equipment standards. As FEC acknowledges in its comments, 7 years
elapsed from the time the standards were first issued in 1990 to the time
that FEC initiated efforts to assess the standards to identify areas that
needed to be updated. During that time, considerable experience with the
standards was accumulating, as vendors were developing voting equipment to
meet the FEC standards and ITAs were testing against them. Since then,
additional experience has been gained with the standards as vendors have
continued to develop voting equipment to meet the standards, and ITAs have
continued to test vendors? equipment against the standards. For example, we
state in this report that ITAs have had to interpret the 1990 standards in
the testing process to accommodate vendors? use of new technologies and
expanded equipment functions that are not addressed in the 1990 standards.
However, FEC does not formally receive these interpretations, any one of
which could be the basis for prompting an update to the standards. In our
view, waiting 9 years to begin updating the standards is too long, does not
constitute proactive maintenance, and is the primary reason that the current
standards are out of date.
3. FEC is correct in stating that we did not assess all of the revised draft
standards areas. However, we disagree that this assessment approach ignores
the collaborative and dynamic process of NASED?s Voting Systems Committee
and FEC?s staff in overseeing the development of the standards for two
reasons. First, this report recognizes that FEC worked closely with state
and local election officials in revising the standards. Second, this joint
FEC and NASED process has no relevance to our findings that certain
standards areas do not address GAO Comments
Appendix II: Comments From the Federal Election Commission
Page 26 GAO- 02- 52 Voting Equipment Standards
the full range of items associated with well- defined system requirements in
these areas. As we state in the objectives, scope, and methodology section
of this report, our approach was to assess all of the 1990 standards because
they are the standards against which voting equipment are currently being
developed and independently tested. In assessing drafts of the updated
standards, we assumed that those areas in the 1990 standards that we found
to be satisfactory would continue to be satisfactory in the updated
standards. As long as our findings are limited to the standards area that we
assessed, the issue of whether we assessed all or some of the draft
standards is not relevant.
4. We acknowledge that FEC?s position, as stated in its comments, concerning
standards areas omitted from the revised draft standards is that these were
based on decisions resulting from deliberative and collaborative interaction
among NASED and FEC staff, and were not, as we were told during the course
of our review by the OEA Director, areas that were overlooked. Accordingly,
we have modified our report to reflect this position.
5. Subsequent to providing us written comments on a draft of this report,
FEC provided us with a copy of volume II of the standards, which includes
the tests to be conducted to ensure compliance with the voting equipment
standards. Based on our review of the relevant security sections, the
standards satisfy the requirement for security certification. We have
modified our report, including the recommendations, to reflect this new
information.
6. We do not disagree that the draft standards discuss quality assurance and
have been strengthened from the 1990 standards. We acknowledge these
improvements to the standards in our report. However, as we state, quality
assurance includes a number of activities. While FEC?s draft standards
include some of these elements, they do not include all of them.
Specifically, the draft standards do not address requirements for developing
a quality assurance plan and conducting process reviews to ensure that the
product and process standards are followed.
Appendix III: System Requirements and Supporting Life- Cycle Processes and
Our Analysis
Page 27 GAO- 02- 52 Voting Equipment Standards
We identified 13 types of system requirements and 3 supporting life- cycle
processes that are often associated with complete system requirements. FEC?s
1990 voting equipment standards satisfied 11 of the 13 system requirements
areas and none of the life- cycle processes. We reviewed FEC?s draft
standards for those areas for which we identified variances in the 1990
standards and found that the draft standards had made improvements in all
five areas. However, the draft standards still do not satisfy human factors
and quality assurance. A detailed description of the system requirements
areas and our complete analysis follow.
1990 Draft Systems requirements Definition/ analysis Standards satisfied?
Functional/ Performance Required system capabilities based on the purpose of
the system; also includes parameters for response times, accuracy,
capacities, unexpected/ unallowed conditions, error- handling, and
continuity of operations.
1990 analysis: Identified areas include ballot definition, candidate/
measure selection, vote casting, ballot interpretation, voting reports,
accuracy and integrity, processing speed, response times, and error and
status messages.
Yes System quality Quantitative measures of quality including reliability
(perform correctly and
consistently), maintainability (easily serviced/ repaired/ corrected), and
availability (accessibility to be operated when needed).
1990 analysis: All identified areas included. Yes
Security/ privacy protection
Requirements for maintaining a secure system and protecting data privacy,
including (1) security/ privacy environment in which system must operate;
(2) types of security to be provided (e. g., data confidentiality and fraud
prevention); (3) risks the system must withstand; (4) safeguards required;
(5) security/ privacy policies that must be met; (6) accountability the
system must provide (i. e., audit trails); and (7) criteria for security
certification.
1990 analysis: Access control identified as a security safeguard, and
requirements defined for audit records produced by the system to provide
accountability. The other areas, however, are not addressed.
Draft analysis: In addition to access controls and audit records, the
security/ privacy environment, the types of security to be provided, the
risks the system must withstand, safeguards necessary, security policies,
and criteria for security certification are identified.
No Yes Human factors Requirements defining system usability of the system
that take into
account human capabilities and limitations, along with use and accessibility
by persons with disabilities.
1990 analysis: System usability and accessibility by persons with
disabilities are not identified.
Draft analysis: Requirements for the use and accessibility by persons with
disabilities are identified. System usability requirements are not.
No No System external interface
Characteristics of the interface between the voting system and other
systems, including data types, data formats, and timing.
1990 analysis: Removable storage media, communications devices, and printers
identified as external interfaces.
Yes
Appendix III: System Requirements and Supporting Life- Cycle Processes and
Our Analysis
Appendix III: System Requirements and Supporting Life- Cycle Processes and
Our Analysis
Page 28 GAO- 02- 52 Voting Equipment Standards
1990 Draft Systems requirements Definition/ analysis Standards satisfied?
Installation- dependent data Requirements for system configuration to meet
local operational requirements.
1990 analysis: Requirements defined for voting systems programming in
accordance with ballot requirements of the election and the jurisdiction in
which the equipment will be used.
Yes System environmental The natural environment that the system must
withstand during
transportation, storage, and operation, including (1) temperature, (2)
humidity, (3) rain, and (4) motion/ shock.
1990 analysis: Requirements identified for temperature, humidity, rain,
transit drop, and vibration.
Yes Design constraints Any commercial standards that must be used in the
system?s
development.
1990 analysis: Vendors are instructed to design equipment in accordance with
best commercial and industrial practice; software is to be designed in a
modular fashion, preferably using a high- level programming language.
Yes Physical The system?s physical characteristics, including size, weight,
color,
nameplates, markings of parts and serial/ lot numbers, transportability, and
parts interchangeability.
1990 analysis: All requirements identified. Yes
Safety Requirements for preventing or minimizing unintended hazards to
personnel, property, and the physical environment.
1990 analysis: All systems shall be designed to meet the requirements of the
Occupational Safety and Health Administration Yes Personnel Requirements for
who will use or support the system, such as number of
workstations and built- in help/ training features.
1990 analysis: Vendors instructed to include information on number of
personnel and skill level required to maintain the voting system.
Yes Training Requirements for training devices and materials to be included
with the
system.
1990 analysis: Vendors instructed to document information required for
system use and operator training, and orientation and training of poll
workers, user maintenance technicians, and vendor personnel.
Yes Logistics Requirements for system maintenance, software support, and
system
transportation.
1990 analysis: Vendors instructed to document information required in these
three areas.
Yes
Appendix III: System Requirements and Supporting Life- Cycle Processes and
Our Analysis
Page 29 GAO- 02- 52 Voting Equipment Standards
1990 Draft Life- cycle process Definition/ analysis Standards satisfied?
Documentation The process of recording information produced during the life-
cycle process. Describes and records information about a product, the
processes used to develop the product, and provides a history of what
happened during the development and maintenance of the product. Includes (1)
identification of documents to be produced and delivered to customer or
tester, (2) identification of format, content, and presentation items for
each document, and (3) review and approval process for each document.
1990 analysis: Requirements identify products to be produced, including the
content and format of the documents. Review and approval process not
specified.
Draft analysis: Products to be produced, including the content and format of
the documents, as well as the review and approval process is identified.
No Yes Configuration management The process to establish and maintain the
integrity of work products
throughout the life- cycle process; it involves establishing product
baselines and systematically controlling changes to them. The process should
include (1) developing a configuration management plan, (2) identifying work
products to be maintained and controlled, (3) establishing a repository to
maintain and control the work products, (4) evaluating and approving changes
to the products, (5) accounting for changes to the work products, and (6)
managing the release and delivery of them.
1990 analysis: Includes requirements for (1) identifying work products to be
maintained and controlled, (2) evaluating and approving changes to the
products, and (3) managing the release and delivery of work products. The
standards do not include requirements for developing a configuration
management plan, establishing a repository to maintain and control the work
products, and accounting for changes to the work products.
Draft analysis: All areas identified. No Yes
Quality assurance The process that provides adequate assurance of the system
development process. It typically involves independent review of work
products and activities to ensure compliance with applicable development
standards and procedures. The process should include (1) developing a
quality assurance plan, (2) determining system development product and
process standards to be followed, and (3) conducting reviews to ensure that
the product and process standards are followed.
1990 analysis: None of these areas specified.
Draft analysis: The need to document the hardware and software development
process is specified, but a quality assurance plan and quality assurance
reviews are not.
No No Source: GAO analysis.
(310210)
The General Accounting Office, the investigative arm of Congress, exists to
support Congress in meeting its constitutional responsibilities and to help
improve the performance and accountability of the federal government for the
American people. GAO examines the use of public funds; evaluates federal
programs and policies; and provides analyses, recommendations, and other
assistance to help Congress make informed oversight, policy, and funding
decisions. GAO?s commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents is through the
Internet. GAO?s Web site (www. gao. gov) contains abstracts and full- text
files of current reports and testimony and an expanding archive of older
products. The Web site features a search engine to help you locate documents
using key words and phrases. You can print these documents in their
entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ?Today?s Reports,? on its Web
site daily. The list contains links to the full- text document files. To
have GAO E- mail this list to you every afternoon, go to our home page and
complete the easy- to- use electronic order form found under ?To Order GAO
Products.?
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U. S. General Accounting Office P. O. Box 37050 Washington, D. C. 20013
To order by Phone: Voice: (202) 512- 6000 TDD: (301) 413- 0006 Fax: (202)
258- 4066
GAO Building Room 1100, 700 4th Street, NW (corner of 4th and G Streets, NW)
Washington, D. C. 20013
Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm, E- mail:
fraudnet@ gao. gov, or 1- 800- 424- 5454 (automated answering system).
Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202) 512- 4800 U. S.
General Accounting Office, 441 G. Street NW, Room 7149, Washington, D. C.
20548 GAO?s Mission
Obtaining Copies of GAO Reports and Testimony
Order by Mail or Phone Visit GAO?s Document Distribution Center
To Report Fraud, Waste, and Abuse in Federal Programs
Public Affairs
*** End of document. ***