Elections: Federal Program for Certifying Voting Systems Needs to
Be Further Defined, Fully Implemented, and Expanded (16-SEP-08,  
GAO-08-814).							 
                                                                 
The 2002 Help America Vote Act (HAVA) created the Election	 
Assistance Commission (EAC) and, among other things, assigned the
commission responsibility for testing and certifying voting	 
systems. In view of concerns about voting systems and the	 
important role EAC plays in certifying them, GAO was asked to	 
determine whether EAC has (1) defined an effective approach to	 
testing and certifying voting systems, (2) followed its defined  
approach, and (3) developed an effective mechanism to track	 
problems with certified systems and use the results to improve	 
its approach. To accomplish this, GAO compared EAC guidelines and
procedures with applicable statutes, guidance, and best 	 
practices, and examined the extent to which they have been	 
implemented.							 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-08-814 					        
    ACCNO:   A84202						        
  TITLE:     Elections: Federal Program for Certifying Voting Systems 
Needs to Be Further Defined, Fully Implemented, and Expanded	 
     DATE:   09/16/2008 
  SUBJECT:   Best practices					 
	     Certification and accreditation			 
	     Developmental testing				 
	     Elections						 
	     Federal regulations				 
	     Federal/state relations				 
	     Information management				 
	     Operational testing				 
	     Product evaluation 				 
	     Program evaluation 				 
	     Program management 				 
	     Reports management 				 
	     Requirements definition				 
	     Software verification and validation		 
	     Standards						 
	     Strategic planning 				 
	     System software					 
	     Systems analysis					 
	     Systems evaluation 				 
	     Systems testing					 
	     Testing						 
	     Voting						 
	     Voting systems					 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-08-814


This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to [email protected]. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Chairman, Committee on House Administration, House of 
Representatives: 

United States Government Accountability Office: 

GAO: 

September 2008: 

Elections: 

Federal Program for Certifying Voting Systems Needs to Be Further 
Defined, Fully Implemented, and Expanded: 

Federal Certification of Voting Systems: 

GAO-08-814: 

GAO Highlights: 

Highlights of GAO-08-814, a report to the Chairman, Committee on House 
Administration, House of Representatives. 

Why GAO Did This Study: 

The 2002 Help America Vote Act (HAVA) created the Election Assistance 
Commission (EAC) and, among other things, assigned the commission 
responsibility for testing and certifying voting systems. In view of 
concerns about voting systems and the important role EAC plays in 
certifying them, GAO was asked to determine whether EAC has (1) defined 
an effective approach to testing and certifying voting systems, (2) 
followed its defined approach, and (3) developed an effective mechanism 
to track problems with certified systems and use the results to improve 
its approach. To accomplish this, GAO compared EAC guidelines and 
procedures with applicable statutes, guidance, and best practices, and 
examined the extent to which they have been implemented. 

What GAO Found: 

EAC has defined an approach to testing and certifying voting systems 
that follows a range of relevant practices and statutory requirements 
associated with a product certification program, including those 
published by U.S. and international standards organizations, and those 
reflected in HAVA. EAC, however, has yet to define its approach in 
sufficient detail to ensure that certification activities are performed 
thoroughly and consistently. This lack of definition also has caused 
voting system manufacturers and test laboratories to interpret program 
requirements differently, and the resultant need to reconcile these 
differences has contributed to delays in certifying systems that 
several states were intending to use in the 2008 elections. According 
to EAC officials, these definitional gaps can be attributed to the 
programï¿½s youth and the commissionï¿½s limited resources being devoted to 
other priorities. Nevertheless, they said that they intend to address 
these gaps, but added that they do not yet have written plans for doing 
so. 

EAC has largely followed its defined approach for each of the dozen 
systems it is in the process of certifying, with one major exception. 
Specifically, it has not established an effective and efficient 
repository for certified versions of voting system software, or related 
procedures and tools, for states and local jurisdictions to use in 
verifying that their acquired voting systems are identical to what EAC 
has certified. Further, EAC officials told GAO that they do not have a 
documented plan or requirements for a permanent solution. As an interim 
solution, they stated that they will maintain copies of certified 
versions in file cabinets and mail copies of these versions upon their 
request by states and local jurisdictions. In GAOï¿½s view, this process 
puts states and local jurisdictions at increased risk of using a 
version of a system during an election that differs from the certified 
version. 

Under its voting system testing and certification program, EAC has 
broadly described an approach for tracking problems with certified 
voting systems and using this information to improve its certification 
program. While this approach is consistent with some aspects of 
relevant guidance, key elements are either missing or inadequately 
defined. According to EAC officials, while they intend to address some 
of these gaps, they do not have documented plans for doing so. In 
addition, even if EAC defines and implements an effective approach, it 
would not affect the vast majority of voting systems that are to be 
used in the 2008 elections. This is because the commissionï¿½s approach 
only applies to those voting systems that it has certified, and it is 
unlikely that any voting systems will be certified in time to be used 
in the upcoming elections. Moreover, because most states do not 
currently require EAC certification for their voting systems, it is 
uncertain if this situation will change relative to future elections. 
As a result, states and other election jurisdictions are on their own 
to discover, disclose, and address any shared problems with these 
noncertified systems. 

What GAO Recommends: 

GAO is making recommendations to EAC relative to establishing and 
implementing plans to better define and implement its certification 
program. GAO is also proposing that Congress consider expanding EACï¿½s 
role under HAVA to include facilitating understanding and resolution of 
shared problems with noncertified voting systems and providing it with 
the resources to accomplish this. EAC stated that it generally agrees 
with GAOï¿½s conclusion that its certification program needs to improve 
and that it accepts GAOï¿½s recommendations. It also provided other 
comments, some of which GAO used to clarify its findings, one 
recommendation, and its proposal to amend HAVA. 

To view the full product, including the scope and methodology, click on 
[http://www.gao.gov/cgi-bin/getrpt?GAO-08-814]. For more information, 
contact Randolph C. Hite at (202) 512-3439 or [email protected]. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

EAC's Approach Largely Follows Many Accepted Practices, but Needs to Be 
Further Defined: 

EAC Has Largely Followed Its Defined Certification Approach, but a Key 
System Verification Capability to Support States and Local 
Jurisdictions Is Not Yet in Place: 

EAC's Mechanism for Tracking and Resolving Problems with Certified 
Voting Systems Lacks Sufficient Definition, and Its Scope Is Limited: 

Conclusions: 

Recommendations for Executive Action: 

Matter for Congressional Consideration: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Comments from the Election Assistance Commission: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Table: 

Table 1: EAC Certification Program Roles and Responsibilities: 

Figures: 

Figure 1: A Voting System Life Cycle Model: 

Figure 2: EAC Voting System Testing and Certification Process: 

Abbreviations: 

EAC: Election Assistance Commission: 

FEC: Federal Election Commission: 

HAVA: Help America Vote Act: 

IEC: International Electrotechnical Commission: 

ISO: International Organization for Standardization: 

NASED: National Association of State Election Directors: 

NIST: National Institute of Standards and Technology: 

NSRL: National Software Reference Library: 

NVLAP: National Voluntary Laboratory Accreditation Program: 

VSS: Voting Systems Standards: 

VSTL: voting system test laboratory: 

VVSG: Voluntary Voting System Guidelines: 

United States Government Accountability Office: 

Washington, DC 20548: 

September 16, 2008: 

The Honorable Robert A. Brady: 
Chairman: 
Committee on House Administration: 
House of Representatives: 

Dear Mr. Chairman: 

Following the 2000 and 2004 general elections, we issued a series of 
reports and testified on virtually every aspect of our nation's 
election system, including the many challenges and opportunities 
associated with various types of voting systems.[Footnote 1] In this 
regard, we emphasized that the voting systems alone were neither the 
sole contributor nor solution to the problems that were experienced 
during the 2000 and 2004 elections, and that the overall election 
system depended on the effective interplay of people, process, and 
technology and involved all levels of government. Among other things, 
we specifically reported in 2001 that no federal entity was responsible 
for developing voting system standards and for testing and certifying 
these systems against such standards, and we raised the establishment 
of such an entity as a matter for congressional consideration.[Footnote 
2] 

Subsequently, Congress passed the Help America Vote Act (HAVA), which 
created the Election Assistance Commission (EAC) and assigned it 
responsibilities for, among other things, the testing, certification, 
decertification, and recertification of voting system hardware and 
software.[Footnote 3] In 2004, we testified on the challenges facing 
EAC in meeting its responsibilities, adding that the commission's 
ability to meet these challenges depended in part on having adequate 
resources. In 2007, EAC established and began implementing its voting 
system certification program. In view of the continuing concerns about 
voting systems and the important role the commission plays in 
certifying them, you asked us to determine whether EAC has (1) defined 
an effective approach to testing and certifying voting systems, (2) 
followed its defined approach, and (3) developed an effective mechanism 
to track problems with certified systems and use the results to improve 
its certification program. 

To accomplish this, we reviewed EAC policies, procedures, and standards 
for testing, certifying, decertifying, and recertifying voting systems 
and compared them, as appropriate, with applicable statutory 
requirements and leading practices, such as HAVA and guidance published 
by the National Institute of Standards and Technology (NIST), 
International Organization for Standardization (ISO), and International 
Electrotechnical Commission (IEC). We also compared EAC actions and 
artifacts for executing the voting system certification program with 
its policies, guidelines, and procedures. In addition, we interviewed 
officials from EAC, NIST, voting system test laboratories (VSTL), 
voting system manufacturers, and the National Association of State 
Election Directors (NASED).[Footnote 4] 

We conducted this performance audit at EAC offices in Washington, D.C., 
from September 2007 to September 2008 in accordance with generally 
accepted government auditing standards. Those standards require that we 
plan and perform the audit to obtain sufficient, appropriate evidence 
to provide a reasonable basis for our findings and conclusions based on 
our audit objectives. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objectives. Further details of our objectives, scope, and methodology 
are included in appendix I. 

Results in Brief: 

EAC has defined an approach to testing and certifying voting systems 
that follows statutory requirements reflected in HAVA and a range of 
relevant practices associated with a product certification program, 
including those published by U.S. and international standards 
organizations. In particular, its voting system certification program 
defines key roles and responsibilities; describes the steps that voting 
system manufacturers are to follow; provides for testing to be 
performed by accredited VSTLs; provides for interpretation of voting 
system standards; includes mechanisms for VSTLs and EAC reviewers to 
declare their impartiality and independence; and addresses how 
manufacturer complaints, appeals, and disputes will be handled. 
Nevertheless, the commission has not developed its approach in 
sufficient detail to ensure that its certification activities are 
performed thoroughly and consistently. For example, it has not defined 
procedures or specific criteria for many of its review activities, 
including the specific test procedures to be used and how decisions are 
to be documented. According to EAC officials, these gaps exist because 
the program is still new and evolving and resources are focused on 
other commission priorities. Officials further stated that they intend 
to carry out a number of activities to address these gaps; however, 
they said that written plans for doing so are not yet developed. Until 
these definitional gaps are fully addressed, EAC cannot adequately 
ensure that its certification approach is repeatable and verifiable 
across all manufacturers and systems. Moreover, these gaps have already 
led to different interpretations of program requirements among EAC's 
stakeholders, and the resultant need to reconcile differences has 
contributed to delays in certifying systems that several states had 
intended to use in the 2008 elections. 

EAC has largely followed its voting system testing and certification 
approach as defined. For example, it has received and reviewed 
manufacturer registration applications, system certification 
applications, and system test plans, as well as issued notices of 
noncompliance. However, contrary to its own certification program 
requirements, the commission has yet to establish a sufficient 
mechanism for states and local jurisdictions to use in verifying that 
the voting systems that they receive from manufacturers are identical 
to the ones that were tested and certified. Specifically, EAC has yet 
to designate a repository of trusted software versions for certified 
systems, and it has not established plans or time frames for doing so. 
Instead, EAC officials stated that they plan to store these software 
builds on compact disks and place the disks in fireproof filing 
cabinets in their offices, and mail copies of the disks to those states 
and local jurisdictions that request them, until they can develop a 
permanent solution. However, the commission has no activities planned 
or under way to establish one. In addition, EAC has yet to specify 
exactly what needs to be done to ensure that manufacturers provide 
effective and efficient tools and processes to support states and local 
jurisdictions in using the repository to verify that their respective 
systems match the certified version, and has not developed plans for 
ensuring that this occurs. Until such tools are in place, state and 
local election officials are likely to face difficulty in determining 
whether the systems that they receive from a manufacturer are identical 
to the system that was tested and certified. 

EAC has broadly described an approach to allow it to track and resolve 
problems with certified voting systems and use the information about 
these problems to improve its voting system testing and certification 
program. Under this approach, EAC is to investigate reports alleging 
defects with certified voting systems, and take action by, for example, 
issuing a notice of noncompliance, providing the manufacturer with an 
opportunity to correct the noncompliance, and decertifying a system if 
the manufacturer does not take sufficient action to bring the system 
into compliance. In addition, EAC is to periodically inspect the 
production facilities of voting system manufacturers. However, it has 
yet to specify how this broadly defined approach will be executed, 
including what steps will be followed, what criteria will be used to 
reach decisions, how it will know if system problems have been 
corrected, and how it will use the information to improve its testing 
and certification program. In addition, while EAC officials stated that 
they intend to address some of these gaps, they do not have defined 
plans or time frames for doing so. According to the EAC officials, this 
is because the certification program is still new and other priorities 
have consumed the commission's limited resources. Until these 
definitional limitations are addressed, it is unlikely that EAC will be 
able to effectively track and resolve problems that arise with the 
systems that it has certified. 

Even if EAC defines and implements an effective problem tracking and 
resolution process for certified systems, these actions will not affect 
the vast majority of voting systems that are to be used in the 2008 
elections, and it is uncertain when this situation will change relative 
to future elections. This is because of two factors. First, most of the 
systems to be used will not be systems that were certified by EAC, but 
instead will be systems that were qualified by the now-discontinued 
NASED program, or those that were not endorsed by any national entity. 
Second, HAVA does not explicitly assign EAC any responsibility for non- 
EAC-certified voting systems, and, according to commission officials, 
they do not have the authority or resources to identify and address 
problems with these systems in the same manner that they do for EAC- 
certified systems. Although EAC has established a mechanism, under its 
HAVA-required information clearinghouse responsibility, to post state 
and local officials' reports of problems and experiences with non-EAC- 
certified systems on its Web site, this mechanism does not involve any 
EAC actions to facilitate problem understanding or resolution. 

Notwithstanding the EAC's efforts to establish an effective voting 
systems testing and certification program, more can be done to build on 
its existing program to be more effective. Accordingly, we are making 
recommendations to the Chair of the EAC to develop and execute plans to 
(1) establish detailed procedures, review criteria, and documentation 
requirements to ensure that voting system testing and certification 
reviews are conducted thoroughly, consistently, and verifiably; (2) 
establish an accessible and available software repository for its 
certified systems, and procedures and review criteria for manufacturer- 
provided tools for using the repository; and (3) fully define and 
implement certified voting system problem tracking and resolution 
activities, and apply lessons learned to improve the program. We are 
also providing a matter for congressional consideration aimed at 
expanding EAC's role under HAVA such that, consistent with the 
nonregulatory and voluntary nature of its certification program, the 
commission is assigned responsibility for providing resources and 
services to facilitate understanding and resolution of problems with 
non-EAC-certified voting systems that will be used in future elections, 
and provided with the resources needed to accomplish this. 

EAC provided written comments on a draft of this report, signed by the 
EAC Executive Director and reprinted in their entirety in appendix II. 
In its comments, the commission described our review and report as 
helpful as it works to fully implement and improve its voting system 
certification program. It also stated that it agrees with the report's 
conclusions that more can be done to build on the program and ensure 
that certifications are based on consistently performed reviews. 
Further, it stated that it generally accepts our three recommendations, 
adding that it will work hard to implement them. EAC provided 
additional comments on the findings that underlie each of the 
recommendations, which it described as needed to clarify and avoid 
confusion about some aspects of its certification program. In response 
to these comments, we have modified the report, as appropriate, to 
clarify our findings and one of the recommendations. 

The commission also provided comments on our matter for congressional 
consideration, including characterizing it as intending "to affect a 
sea change in the way that EAC operates its testing and certification" 
program. We agree that EAC does not have the authority to compel the 
manufacturers or states and local jurisdictions to submit to its 
testing and certification program and that the wording of the matter in 
our draft inadvertently led EAC to believe that our proposal would 
require the commission to assume a more regulatory role. In response to 
EAC's comments, and in order to avoid any misunderstanding as to the 
intent of this matter for congressional consideration, we have modified 
its wording. 

Background: 

All levels of government share responsibility in the overall U.S. 
election system. At the federal level, Congress has authority under the 
Constitution to regulate presidential and congressional elections and 
to enforce prohibitions against specific discriminatory practices in 
all federal, state, and local elections. Congress has passed 
legislation that addresses voter registration, absentee voting, 
accessibility provisions for the elderly and handicapped, and 
prohibitions against discriminatory practices.[Footnote 5] At the state 
level, individual states are responsible for the administration of both 
federal elections and their own elections. States regulate the election 
process, including, for example, the adoption of voluntary voting 
system guidelines, the state certification and acceptance testing of 
voting systems, ballot access, registration procedures, absentee voting 
requirements, the establishment of voting places, the provision of 
election day workers, and the counting and certification of the vote. 

In total, the overall U.S. election system can be seen as an assemblage 
of 55 distinct election systems--those of the 50 states, the District 
of Columbia, and the 4 U.S. territories. Further, although election 
policy and procedures are legislated primarily at the state level, 
states typically decentralize election administration, so that it is 
carried out at the city or county levels, and voting is done at the 
local level. As we reported in 2001, local election jurisdictions 
number more than 10,000, and their sizes vary enormously--from a rural 
county with about 200 voters to a large urban county, such as Los 
Angeles County, where the total number of registered voters for the 
2000 elections exceeded the registered voter totals in 41 
states.[Footnote 6] Further, these thousands of jurisdictions rely on 
many different types of voting methods that employ a wide range of 
voting system makes, models, and versions. 

The U.S. Election System Depends on Effective Interactions among 
People, Processes, and Technology: 

Voting systems are but one facet of a multifaceted, continuous election 
system that involves the interplay of people, processes, and 
technology. All levels of government, as well as commercial voting 
system manufacturers and VSTLs, play key roles in ensuring that voting 
systems perform as intended. 

Electronic voting systems are typically developed by manufacturers, 
purchased as commercial off-the-shelf products, and operated by state 
and local election administrators. These activities can be viewed as 
three phases in a system's life cycle: product development, 
acquisition, and operations (see fig. 1). Spanning these life cycle 
phases are key processes, including managing the interplay of people, 
processes, and technologies, and testing the systems and components. In 
addition, voting system standards are important through all of these 
phases because they provide the criteria for developing, testing, and 
acquiring the systems, and they specify the necessary documentation for 
operating the systems. We discuss each of these phases after figure 1. 

Figure 1: A Voting System Life Cycle Model: 

This figure is a flowchart showing a voting system life cycle mode. 

[See PDF for image] 

Source: GAO analysis of EAC, NIST, and Institute of Electrical and 
Electronics Engineers (IEEE) publications. 

[End of figure] 

* The product development phase includes such activities as 
establishing requirements for the system, designing a system 
architecture, and developing software and integrating components. 
Activities in this phase are performed by the system manufacturer. 

* The acquisition phase includes such procurement-related activities as 
publishing a solicitation, evaluating offers, choosing a voting 
technology, choosing a vendor, and awarding and administering 
contracts. Activities in this phase are primarily the responsibility of 
state and local governments, but include responsibilities that are 
shared with the vendor, such as establishing contracts. 

* The operations phase consists of such activities as ballot design and 
programming, setup of systems before voting, pre-election testing, vote 
capture and counting during elections, recounts and system audits after 
elections, and storage of systems between elections. Responsibility for 
activities in this phase typically resides with local jurisdictions, 
whose officials may, in turn, rely on or obtain assistance from system 
vendors for aspects of these activities. 

* Standards for voting systems were developed at the national level by 
the Federal Election Commission (FEC) in 1990 and 2002 and were updated 
by EAC in 2005. Voting system standards serve as guidance for product 
developers in building systems, a framework for state and local 
governments to evaluate systems, and as the basis for documentation 
needed to operate the systems. 

* Testing processes are conducted throughout the life cycle of a voting 
system. For example, manufacturers conduct product testing during 
development of the system. Also, national certification testing of 
products submitted by system manufacturers is conducted by nationally 
accredited VSTLs. States and local jurisdictions also perform a range 
of system tests. 

* Management processes help to ensure that each life cycle phase 
produces desirable outcomes. Typical management activities include 
planning, configuration management, system performance review and 
evaluation, problem tracking and correction, human capital management, 
and user training. 

Testing and Certification Are Important to Ensuring Voting System 
Security and Reliability: 

Testing electronic voting systems for conformance with requirements and 
standards is critical to ensuring their security and reliability, and 
an essential means to ensuring that systems perform as intended. In 
addition, such testing can help find and correct errors in systems 
before they are used in elections. If done properly, testing provides 
voters with assurance and confidence that their voting systems will 
perform as intended. 

Testing is particularly important for electronic voting systems because 
these systems have become our Nation's predominant method of voting, 
and concerns have been raised about their security and reliability. As 
we reported in 2005,[Footnote 7] these concerns include weak security 
controls, system design flaws, inadequate system version control, 
inadequate security testing, incorrect system configuration, poor 
security management, and vague or incomplete voting system standards. 
Further, security experts and some election officials have expressed 
concerns that tests performed under the NASED program by independent 
testing authorities and state and local election officials did not 
adequately assess voting systems' security and reliability. Consistent 
with these concerns, most of the security weaknesses that we identified 
in our prior report[Footnote 8] related to systems that NASED had 
previously qualified. Our report also recognized that security experts 
and others pointed to these weaknesses as an indication that both the 
standards and the NASED testing program were not rigorous enough with 
respect to security, and that these concerns were amplified by what 
some described as a lack of transparency in the testing process. 

EAC's Responsibilities under HAVA include Certifying Voting Systems: 

Enacted in October 2002, HAVA affects nearly every aspect of the 
election system, from voting technology to provisional ballots and from 
voter registration to poll worker training.[Footnote 9] Among other 
things, the act authorized $3.86 billion in funding over several fiscal 
years for states to replace punch card and mechanical lever voting 
equipment, improve election administration and accessibility, and 
perform research and pilot studies. In addition, the act established 
EAC and assigned it responsibility for, among other things, (1) 
updating voting system standards, (2) serving as a clearinghouse for 
election-related information, (3) accrediting independent test 
laboratories, and (4) certifying voting systems. EAC began operations 
in January 2004. In 2004, we testified on the challenges facing EAC in 
meeting its responsibilities.[Footnote 10] For example, we reported 
that EAC needed to move swiftly to strengthen voting system standards 
and the testing associated with these standards. We also reported that 
the commission's ability to meet its responsibilities depended, in 
part, on the adequacy of the resources at its disposal. 

Updating standards: HAVA requires EAC to adopt a set of federal voting 
system standards, referred to as the Voluntary Voting System Guidelines 
(VVSG). In December 2005, the commission adopted the VVSG, which 
defines a set of specifications and requirements against which voting 
systems are to be designed, developed, and tested to ensure that they 
provide the functionality, accessibility, and security capabilities 
required to ensure the integrity of voting systems. As such, the VVSG 
specifies the functional requirements, performance characteristics, 
documentation requirements, and test evaluation criteria for the 
national certification of voting systems. In 2007, the Technical 
Guidelines Development Committee[Footnote 11] submitted its 
recommendations for the next iteration of the VVSG to EAC. The 
commission has yet to establish a date when the update will be approved 
and issued. 

Serving as an information clearinghouse: HAVA requires EAC to maintain 
a clearinghouse of information on the experiences of state and local 
governments relative to, among other things, implementing the VVSG and 
operating voting systems. As part of this responsibility, EAC posts 
voting system reports and studies that have been conducted or 
commissioned by a state or local government on its Web site. These 
reports must be submitted by a state or local government that certifies 
that the report reflects its experience in operating a voting system or 
implementing the VVSG. EAC does not review the information for quality 
and does not endorse the reports and studies. 

Accrediting independent test laboratories: HAVA assigned 
responsibilities for laboratory accreditation to both EAC and NIST. In 
general, NIST focuses on assessing laboratory technical qualifications 
and recommends laboratories to EAC for accreditation. EAC uses NIST's 
assessment results and recommendations, and augments them with its own 
review of related laboratory capabilities to reach an accreditation 
decision. As we have previously reported,[Footnote 12] EAC and NIST 
have defined their respective approaches to accrediting laboratories 
that address relevant HAVA requirements. However, neither approach 
adequately defines all aspects of an effective program to the degree 
needed to ensure that laboratories are accredited in a consistent and 
verifiable manner. Accordingly, we recently made recommendations to 
NIST and EAC aimed at addressing these limitations. 

Certifying voting systems: HAVA requires EAC to provide for the 
testing, certification, decertification, and recertification of voting 
system hardware and software. EAC's voting system testing and 
certification program is described in detail in the following section. 

Prior to HAVA, no federal agency was assigned or assumed responsibility 
for testing and certifying voting systems against the federal 
standards. Instead, NASED, through its voting systems committee, 
assumed this responsibility by accrediting independent test 
authorities, which in turn tested equipment against the standards. When 
testing was successfully completed, the independent test authorities 
notified NASED that the equipment satisfied testing requirements. NASED 
would then qualify the system for use in elections. According to a 
NASED official, the committee has neither qualified any new or modified 
systems, nor taken any actions to disqualify noncompliant systems, 
since the inception of EAC's testing and certification program in 
January 2007. 

Overview of EAC's Voting System Testing and Certification Program: 

EAC implemented its voting system testing and certification program in 
January 2007. According to the commission's Testing and Certification 
Program Manual, EAC certification means that a voting system has been 
successfully tested by an accredited VSTL, meets requirements set forth 
in a specific set of federal voting system standards, and performs 
according to the manufacturer's specifications. 

The process of EAC's voting system testing and certification program 
consists of seven major phases. Key stakeholders that are involved in 
this process include voting system manufacturers, accredited VSTLs, and 
state and local election officials. These seven phases are described in 
the following text and depicted in figure 2. 

1. Manufacturer Registration: 

All manufacturers must be registered to submit a voting system for 
certification. To register, a manufacturer must provide such 
information as organizational structure and contact(s); quality 
assurance, configuration management, and document retention procedures; 
and identification of all manufacturing and assembly facilities. In 
registering, the manufacturer agrees to certain duties and requirements 
at the outset of its participation in the program. These requirements 
include properly using and representing EAC's certification label, 
notifying the commission of any changes to a certified system, 
permitting EAC to verify the manufacturer's quality control procedures 
by inspecting fielded systems and manufacturing facilities, cooperating 
with any inquiries and investigations about certified systems, 
reporting any known malfunction of a system, and otherwise adhering to 
all procedural requirements of the program manual. 

Once a manufacturer submits a completed application form and all 
required attachments, EAC reviews the submission for sufficiency using 
a checklist that maps to the application requirements listed in the 
program manual. If the application passes the review, EAC provides the 
manufacturer with a unique identification code and posts the applicant 
as a registered manufacturer on the commission's Web site, along with 
relevant documentation. 

2. Voting System Certification Application: 

For each voting system that a manufacturer wishes to have certified, it 
submits an application package. The package includes an application 
form requiring the following: manufacturer information, accredited VSTL 
selection, applicable voting system standard(s), nature of submission, 
system name and version number, all system components and corresponding 
version numbers, and system configuration information. The package also 
includes the following documentation: system implementation 
statement,[Footnote 13] functional diagram, and system overview. EAC 
reviews the submission for completeness and accuracy and, if it is 
acceptable, notifies the manufacturer and assigns a unique application 
number to the system. 

3. Test Plan Development and Approval: 

Once the certification application is accepted, the accredited VSTL 
prepares and submits to EAC a test plan defining how it will ensure 
that the system meets applicable standards and functions as intended. 
When a laboratory submits its test plan, EAC's technical reviewers 
assess it for adequacy. If the plan is deemed not acceptable, the 
commission provides written notice to the laboratory that includes a 
description of the problems identified and the steps required to remedy 
the test plan. The laboratory may take remedial action and resubmit the 
test plan until it is accepted by EAC reviewers. 

4. Test Plan Execution: 

The VSTL executes the approved test plan and notifies EAC directly of 
any test anomalies or failures, along with any changes or modifications 
to the test plan as a result of testing. The laboratory then prepares a 
test results report. 

5. Test Report Review: 

The VSTL submits the test results report to EAC's Program Director who 
reviews it for completeness. If it is complete, the technical reviewers 
analyze the report in conjunction with related technical documents and 
the test plan for completeness, appropriateness, and adequacy. The 
reviewers submit their findings to the Program Director, who either 
recommends certification of the system to the Decision Authority, EAC's 
Executive Director, or refers the matter back to the reviewers for 
additional specified action and resubmission. 

6. Initial Certification Decision and Appeal: 

EAC's Decision Authority reviews the recommendation of the Program 
Director and supporting materials and issues a written decision to the 
manufacturer. If certification is denied, the manufacturer may request 
an opportunity to correct the basis for the denial or may request 
reconsideration of the decision after submitting supporting written 
materials, data, and a rationale for its position. The Decision 
Authority considers the request and issues a written decision. If the 
decision is to deny certification, the manufacturer may request an 
appeal in writing to the Program Director. The Appeal Authority, which 
consists of two or more EAC Commissioners or other individuals 
appointed by the Commissioners who have not previously served as the 
initial or reconsideration authority, consider the appeal. The Appeal 
Authority may overturn the decision if it finds that the manufacturer 
has demonstrated by clear and convincing evidence that its system met 
all substantive and procedural requirements for certification. 

7. Final Certification: 

The initial decision becomes final and EAC issues a Certificate of 
Conformance to the manufacturer, and posts the system on the list of 
certified voting systems on its Web site, when the manufacturer and 
VSTL successfully demonstrate that the voting system under test has 
been: 

* Subject to a trusted build: The voting system's source code is 
converted to executable code in the presence of at least one VSTL 
representative and one manufacturer representative, using security 
measures to ensure that the executable code is a verifiable and 
faithful representation of the source code. This demonstrates that (1) 
the software was built as described in the technical documentation, (2) 
the tested and approved source code was actually used to build the 
executable code on the system, and (3) no other elements were 
introduced in the software build. It also serves to document the 
configuration of the certified system for future reference. 

* Placed in a software repository: The VSTL delivers the following to 
one or more trusted repositories designated by EAC: (1) source code 
used for the trusted build and its file signatures;[Footnote 14] (2) 
disk image of the prebuild, build environment, and any file signatures 
to validate that it is unmodified; (3) disk image of the postbuild, 
build environment, and any file signatures to validate that it is 
unmodified; (4) executable code produced by the trusted build and its 
file signatures of all files produced; and (5) installation device(s) 
and its file signatures. 

* Verified using system identification tools: The manufacturer creates 
and makes available system identification tools that federal, state, 
and local officials can use to verify that their voting systems are 
unmodified from the system that was certified. These tools are to 
provide the means to identify and verify hardware and 
software.[Footnote 15] 

Figure 2: EAC Voting System Testing and Certification Process: 

This figure is a flowchart of the EAC voting system testing and 
certification process. 

[See PDF for image] 

Source: GAO analysis of EAC data. 

Note: At the review and approval of manufacturer registration 
applications, voting system certification applications, and test plans, 
EAC may request additional clarification or information before 
approval. 

[End of figure] 

EAC's Approach Largely Follows Many Accepted Practices, but Needs to Be 
Further Defined: 

To its credit, EAC has taken steps to develop an approach to testing 
and certifying voting systems that follows statutory requirements and 
many recognized and accepted practices. However, the commission has not 
developed its approach in sufficient detail to ensure that its 
certification activities are performed thoroughly and consistently. It 
has not, for example, defined procedures or specific criteria for many 
of its review activities, and for ensuring that the decisions made, and 
their basis, are properly documented. According to EAC officials, these 
gaps exist because the program is still new and evolving and resources 
are limited. Officials further stated that they do not yet have written 
plans for addressing these gaps. Until these gaps are addressed, EAC 
cannot adequately ensure that its approach is repeatable and verifiable 
across all manufacturers and systems. Moreover, this lack of definition 
has caused EAC stakeholders to interpret certification requirements 
differently, and the resultant need to reconcile these differences has 
contributed to delays in certifying systems that several states were 
planning on using in the 2008 elections. 

EAC's Approach Reflects Key Practices and Statutory Requirements: 

Product certification or conformance testing is a means by which a 
third party provides assurance that a product conforms to specific 
standards. In the voting environment, EAC is the third party that 
provides assurance to the buyer (e.g., state or local jurisdictions) 
that the manufacturer's voting system conforms to the federal voting 
standards set forth in FEC's 2002 Voting System Standards (VSS) or 
EAC's 2005 VVSG. 

Several organizations, such as NIST, ISO, and IEC, have individually or 
jointly developed guidance for product certification and conformance 
testing programs.[Footnote 16] This guidance includes, among other 
things, (1) defining roles and responsibilities for all parties 
involved in the certification process, (2) defining a clear and 
transparent process for applicants to follow, (3) ensuring that persons 
involved in the process are impartial and independent, (4) establishing 
a process for handling complaints and appeals, and (5) having testing 
conducted by competent laboratories. Further, HAVA established 
statutory requirements for a federal testing and certification program. 
These requirements include ensuring that the program covers testing, 
certification, decertification, and recertification of voting system 
hardware and software. 

EAC's defined voting system certification approach reflects these key 
practices. Specifically: 

* EAC has defined the roles and responsibilities for itself, the VSTLs, 
and manufacturers in its Testing and Certification Program Manual. 
These roles and responsibilities are described in table 1. 

Table 1: EAC Certification Program Roles and Responsibilities: 

Key players: EAC; 
Roles and responsibilities: * Establish and maintain the testing and 
certification program; 
* Be the authority for granting and withdrawing certification; 
* Provide the procedural requirement of the program; 
* Review manufacturer registration applications for sufficiency; 
* Review certification applications for completeness and accuracy; 
* Provide technical guidance; 
* Perform technical reviews of test plans, test reports, and other 
technical documents; 
* Resolve disputes about voting system standards and program 
administration. 

Key players: VSTLs; 
Roles and responsibilities: * Prepare test plans and test 
methodologies; 
* Conduct testing to approved test plans; 
* During testing, report any changes to approved test plans and all 
test failures or anomalies; 
* Record results of testing in a test report; 
* Meet all requirements of EAC's laboratory accreditation program. 

Key players: Manufacturers; 
Roles and responsibilities: * Provide required information to EAC; 
* Adhere to program requirements; 
* Submit voting system to EAC for certification; 
* Report malfunctions with certified voting systems and submit 
corrective action plans to address any issues that violate EAC's 
program. 

Source: GAO analysis of EAC documentation. 

[End of table] 

* EAC's testing and certification process is documented in its program 
manual. Among other things, the manual clearly defines the program's 
administrative requirements that manufacturers and VSTLs are to follow. 
EAC has made the program manual, along with supporting policies and 
clarifications, publicly available on its Web site, and has made 
program-related news and correspondence publicly accessible as they 
have come available. 

* EAC's certification program addresses impartiality and independence. 
For example, EAC policy states that all personnel and contractors 
involved in the certification program are subject to conflict-of- 
interest reporting and review. In addition, the policy mandates 
conflict-of-interest and conduct statements for the technical reviewers 
that support the program, requires conflict-of-interest reporting and 
reviews to ensure the independence of EAC personnel assigned to the 
program, and requires that all VSTLs maintain and enforce policies that 
prevent conflict-of-interest or the appearance of a conflict-of- 
interest, or other prohibited practices. 

* EAC's program manual outlines its process for the resolution of 
complaints, appeals, and disputes received from manufacturers and 
laboratories. These can be about matters relating to the certification 
process, such as test methods, procedures, test results, or program 
administration. Specifically, the program manual contains policies and 
procedures for submitting a Request for Interpretation, which is a 
means by which a registered manufacturer or accredited laboratory seeks 
clarification on a specific voting system standard, including any 
misunderstandings or disputes about a standard's interpretation or 
implementation. The manual also contains policies, requirements, and 
procedures for a manufacturer to file an appeal on a decision denying 
certification, request an opportunity to correct a problem, and request 
reconsideration of a decision. In addition, EAC provides for Notices of 
Clarification, which offer guidance and explanation on the requirement 
and procedure of the program. Notices may be issued pursuant to a 
clarification request from a laboratory or manufacturer. EAC may also 
issue a notice or interpretation if it determines that any general 
clarifications are necessary. 

* EAC has a VSTL accreditation program. This program is supported by 
NIST's National Voluntary Laboratory Accreditation Program (NVLAP), 
which is a long established and recognized laboratory accreditation 
program. According to EAC's program manual, all certification testing 
is to be performed by a laboratory accredited by NIST and EAC. Further, 
all subcontracted testing is to be performed by a laboratory accredited 
by either NIST or the American Association of Laboratory Accreditation 
for the specific scope of needed testing. Finally, any prior testing 
will only be accepted if it was conducted or overseen by an accredited 
laboratory and was reviewed and approved by EAC. 

HAVA also established certain requirements for EAC's voting system 
testing and certification program.[Footnote 17] Under HAVA, EAC is to 
provide for the testing, certification, decertification, and 
recertification of voting system hardware and software by accredited 
laboratories. EAC's defined approach addresses each of these areas. 

According to program officials, EAC's certification program reflects 
many leading practices because the commission consciously sought out 
these best practices during program development. Officials stated that 
their intention is to develop a program that stringently tests voting 
systems to the applicable standards; therefore, they consulted with 
experts to assist with drafting the program manual. For example, 
according to EAC officials, they met with officials from other federal 
agencies that conduct certification testing in order to benefit from 
their lessons learned. By reflecting relevant practices, standards, and 
legislative requirements in its defined approach, EAC has provided an 
important foundation for having an effective voting system testing and 
certification program. 

EAC's Approach Does Not Sufficiently Define the Details for 
Certifications to Be Performed Thoroughly and Consistently: 

EAC has yet to define its approach for testing and certifying 
electronic voting systems in sufficient detail to ensure that its 
certification activities are performed thoroughly and consistently. It 
has not, for example, defined procedures or specific criteria for many 
of its review activities and for ensuring that the decisions made are 
properly documented. EAC officials attributed this lack of definition 
to the fact that the program is still new and evolving, and they stated 
that available resources are constrained by competing priorities. Until 
these details are defined, EAC will be challenged to ensure that 
testing and review activities are repeatable across different systems 
and manufacturers, and that the activities it performs are verifiable. 
Moreover, this lack of definition is likely to result in different 
interpretations of program requirements by stakeholders, which has 
already resulted in the need to reconcile different interpretations and 
thereby caused delays in certifying systems that several states 
intended to use in the 2008 elections. In such cases, the delays are 
forcing states to either not require EAC certification or rely on an 
alternative system. 

According to federal and international guidance,[Footnote 18] having 
well-defined and sufficiently detailed program management controls help 
to ensure that programs are executed effectively and efficiently. 
Relative to a testing and certification program, such management 
controls include, among other things, having (1) defined procedures and 
established criteria for performing evaluation activities so that they 
will be performed in a comparable, unambiguous, and repeatable manner 
for each system and (2) required documentation to demonstrate that 
procedural evaluation steps and related decisions have been effectively 
performed, including provisions for review and approval of such 
documentation by authorized personnel. 

EAC's defined approach for voting system testing and certification 
lacks such detail and definition. With respect to the first management 
control, the commission has not defined procedures or specific criteria 
for many of its review activities, instead it relies on the personal 
judgment of the reviewers. Specifically, the program manual states that 
EAC, with the assistance of its technical experts, as necessary, will 
review manufacturer registration applications, system certification 
applications, test plans, and test reports, but it does not define 
procedures or criteria for conducting these reviews. For example: 

* The program manual states that upon receipt of a completed 
manufacturer registration application, EAC will review the information 
for sufficiency. However, it does not define what constitutes 
sufficiency and what this sufficiency review should entail. Rather, EAC 
officials said that this is left up to the individual reviewer's 
judgment. 

* The program manual lists the information that manufacturers are 
required to submit as part of their certification applications and 
states that EAC will review the submission for completeness and 
accuracy. While the commission has developed a checklist for 
determining whether the required information was included in the 
application, neither the program manual nor the checklist describe how 
reviewers should perform the review or assess the adequacy of the 
information provided. For example, EAC requires certification 
applications to include a functional diagram depicting how the 
components for the voting system function and interact, as well as a 
system overview that includes a description of the functional and 
physical interfaces between components. Although the checklist provides 
for determining whether these items are part of the application 
package, it does not, for example, provide for checking them for 
completeness and consistency. Moreover, we identified issues with 
completeness and consistency of these documents for approved 
certification application packages. Again, EAC officials said that 
these determinations are to be based on each reviewer's judgment. 

* The program manual states that test plans are to be reviewed for 
adequacy. However, it does not define adequacy or how such reviews are 
to be performed. This lack of detail is particularly problematic 
because EAC officials told us that the VSS and VVSG contain many vague 
and undefined requirements. According to these officials, reviewers 
have been directed to ensure that VSTLs stress voting systems during 
testing, based on what they believe are the most difficult and 
stringent conditions likely to be encountered in an election 
environment that are permissible under the standards. 

* The program manual states that EAC technical experts will assess test 
results reports for completeness, appropriateness, and adequacy. 
However, it does not define appropriateness and adequacy or the 
procedural steps for conducting the review. 

* The program manual requires VSTLs to use all applicable test suites 
issued by EAC when developing test plans. However, program officials 
stated that they currently do not have defined test suites and NIST 
officials said that they are focused on preparing test suites for the 
forthcoming version of the VVSG and not the 2005 version. As a result, 
each laboratory develops its own unique testing approach, which 
requires each to interpret what is needed to test for compliance with 
the standards and increases the risk of considerable variability in how 
testing is performed. To address this void, EAC has tasked NIST with 
developing test suites for both the 2005 VVSG and the yet-to-be- 
released update to these guidelines.[Footnote 19] Until then, EAC 
officials acknowledge that they will be challenged to ensure that 
testing conducted on different systems or at different VSTLs is 
consistent and comparable. 

With respect to the second management control, the commission has not 
defined the documentation requirements that would demonstrate that 
procedural steps, evaluations, and related decision making have been 
performed in a thorough, consistent, and verifiable manner. 
Specifically, while the program manual requires the Program Director to 
maintain documentation to demonstrate that procedures and evaluations 
were effectively performed, EAC has yet to specify the nature or 
content of this documentation. For example: 

* The program manual requires technical reviewers to assess test plans 
and test reports prepared by laboratories and then to submit reports of 
their findings to the Program Director. However, EAC does not require 
documentation of how these reviews were performed beyond completion of 
a recently developed checklist, and this checklist does not provide for 
capturing how decisions were reached, including steps performed and 
criteria applied. For example, the VVSG requires that systems permit 
authorized access and prevent unauthorized access, and lists examples 
of measures to accomplish this, such as computer-generated password 
keys and controlled access security. While the checklist cites this 
requirement and provides for the reviewer to indicate whether the test 
plan satisfies it, it does not provide specific guidance on how to 
determine whether the access control measures are adequate, and it does 
not provide for documenting how the reviewer made such a decision. 

* The program manual does not require supervisory review of work 
conducted by EAC personnel. Moreover, it does not require that the 
reviewers be identified. 

According to EAC officials, its approach does not yet include such 
details because it is still new and evolving and because the 
commission's limited resources have been devoted to other 
priorities.[Footnote 20] To address these gaps, EAC officials stated 
that they intend to undertake a number of activities. For example, the 
Program Director stated that in the near-term, the technical reviewers 
will collaborate and share views on each test plan and test report 
under review as a way to provide consistency, and they will use a 
recently finalized checklist for test plan and test report reviews. In 
the longer term, EAC intends to define more detailed procedures for 
each step in its process. However, it has yet to establish documented 
plans, including the level of resources needed to accomplish this. 
Until these plans are developed and executed, EAC will be challenged to 
ensure that its testing and certification activities are performed 
thoroughly and consistently across different systems, manufacturers, 
and VSTLs. 

Moreover, this lack of program definition surrounding certification 
testing and review requirements has already caused, and could continue 
to cause, differences in how EAC reviewers, VSTLs, and manufacturers 
interpret the requirements. For example, the program requires 
sufficient and adequate testing, but it does not define what 
constitutes sufficient and adequate. As a result, laboratory officials 
and manufacturer representatives told us that EAC reviewers have 
interpreted these requirements more stringently than they have, and 
that reconciling these different interpretations has already caused 
delays in the approval of test plans,[Footnote 21] and they will likely 
prevent EAC from certifying any systems in time for use in the upcoming 
2008 elections. This is especially problematic for those states that 
have statutory or other requirements to use federally certified systems 
and that have a need to acquire or upgrade existing systems for these 
elections. In this regard, 18 states reported to us that they were 
relying on EAC to certify systems for use in the 2008 elections, but 
now will have to adopt different strategies for meeting their states' 
respective EAC certification requirements. For example, officials for 
several states said that they would use the same system as in 2006, 
while officials for other states described plans to either undertake 
upgrades to existing systems without federal certification, or change 
state requirements to no longer require EAC certification. 

EAC Has Largely Followed Its Defined Certification Approach, but a Key 
System Verification Capability to Support States and Local 
Jurisdictions Is Not Yet in Place: 

EAC has largely followed its defined certification approach for each of 
the dozen voting systems that it is in the process of certifying, with 
one major exception. Specifically, it has not established sufficient 
means for states and local jurisdictions to verify that the voting 
systems that each receives from its manufacturer have system 
configurations that are identical to those of the system that the EAC 
certified, and it has not established plans or time frames for doing 
so. This means that states and local jurisdictions are at increased 
risk of using a version of a system during an election that differs 
from the certified version. This lack of an effective and efficient 
verification capability could diminish the value of an EAC system 
certification. 

EAC Has Largely Followed Its Defined Certification Approach: 

EAC has largely executed its voting system testing and certification 
program as defined. While no system has yet to complete all major steps 
of the certification process discussed in the Background section of 
this report, and thus receive certification, 12 different voting 
systems have completed at least the first step of the process, and some 
have completed several steps. Specifically, as of May 2008, EAC had 
received, reviewed, and approved 12 manufacturer registration 
applications, and these approved manufacturers have collectively 
submitted certification applications for 12 different systems, of which 
EAC has accepted 9. For these 9 systems, manufacturers have submitted 7 
test plans, of which EAC has reviewed and approved 2 plans. In 1 of 
these 2 cases, EAC has received a test results report, which it is 
currently in the process of reviewing. At the same time, EAC has 
responded to 8 requests for interpretations of standards from 
manufacturers and laboratories. EAC has also issued 6 notices of 
clarification, which provide further guidance and explanation on the 
program requirements and procedures. 

Our analysis of available certification-related documentation showed 
that in each of the 12 systems submitted for certification, all 
elements of each executed step in the certification process were 
followed. With respect to the manufacturer registration step, EAC 
reviewed and approved all 12 applications, as specified in its program 
manual. For the certification application step, EAC reviewed and 
approved 9 applications. In doing so, EAC issued 3 notices of 
noncompliance to manufacturers for failure to comply with program 
requirements. In each notice, it identified the area of noncompliance, 
and described what requested relevant information or corrective 
action(s) was needed in order to participate in the program. In 1 of 
the cases, EAC terminated the certification application due to the 
manufacturer's failure to respond within the established time frame, 
which is consistent with the program manual. These actions were 
generally consistent with its program manual. 

EAC Has Not Established a Sufficient Means for States and Local 
Jurisdictions to Verify That Their Systems Match the Certified Version: 

Notwithstanding EAC's efforts to follow its defined approach, it has 
not yet established a sufficient mechanism for states and local 
jurisdictions to use in verifying that the voting systems that they 
receive from manufacturers for use in elections are identical to the 
systems that were actually tested and certified. According to EAC's 
certification program manual, final certification is conditional upon 
(1) testing laboratories depositing certified voting system software 
into an EAC-designated repository and (2) manufacturers creating and 
making available system identification tools for states and local 
jurisdictions to use in verifying that their respective systems' 
software configurations match that of the software for the system that 
was certified and deposited into the repository. However, EAC has yet 
to establish a designated repository or procedures and review criteria 
for evaluating the manufacturer-provided tools, and has not established 
plans or time frames for doing so. While none of the ongoing system 
certifications have progressed to the point where these aspects of 
EAC's defined approach is applicable, they will be needed when the 
first system's test results are approved. Until both aspects are in 
place, state and local officials will likely face difficulties in 
determining whether the systems that they receive from manufacturers 
are the same as the systems that EAC certified. 

EAC's program requires the use of a designated software repository for 
certified voting systems. Specifically, the certification program 
manual states that final certification will be conditional upon, among 
other things, the manufacturer and VSTL creating and documenting a 
trusted software build,[Footnote 22] and the laboratory depositing the 
build in a designated repository. In its 2005 VVSG, EAC designated 
NIST's National Software Reference Library (NSRL) as its repository and 
required its use. However, program officials stated that the commission 
does not intend to use the NSRL as its designated repository because 
the library cannot perform all required functions. While these 
officials added that they may use the NSRL for portions of the 
certification program, they said it will not serve as EAC's main 
repository. 

Nevertheless, the commission has not established plans to identify and 
designate another repository, and has yet to define minimum 
requirements (functional, performance, or interface) for what it 
requires in a repository to efficiently support states and local 
jurisdictions. As an interim measure, an EAC official stated that they 
will store the trusted builds on compact disks and keep the disks in 
fireproof filing cabinets in their offices. According to the official, 
this approach is consistent with established program requirements 
because the program manual merely refers to the use of a trusted 
archive or repository designated by EAC. Under this measure, state and 
local election officials will have to request physical copies of 
material from EAC and wait for the materials to be physically packaged 
and delivered. This interim approach is problematic for several 
reasons, including the demand for EAC resources to keep up with the 
requests during a time when the Executive Director told us that a more 
permanent repository solution has not been a commission focus because 
its limited resources have been focused on other priorities. 

EAC has also not defined how it will ensure that manufacturers develop 
and provide to states and local jurisdictions tools to verify their 
respective systems' software against the related trusted software 
builds.[Footnote 23] According to the program manual, final 
certification of a voting system is also conditional upon manufacturers 
creating and making available to states and local jurisdictions tools 
to compare their systems' software with the trusted build in the 
repository. In doing so, the program manual states that manufacturers 
shall develop and make available tools of their choice, and that the 
manufacturer must submit a letter certifying the creation of the tools 
and include a copy and description of the tools to EAC. The commission 
may choose to review the tools. Further, the 2005 VVSG provides some 
requirements for software verification tools, for example, that the 
tools provide a method to comprehensively list all software files that 
are installed on the voting system. However, EAC has yet to specify 
exactly what needs to be done to ensure that manufacturers provide 
effective and efficient system identification tools and processes, and 
it has not developed plans for ensuring that this occurs. Instead, EAC 
has stated that until it defines procedures to supplement the program 
manual, it will review each tool submitted. However, the commission has 
yet to establish specific criteria for the assessment. 

Without an established means for effectively and efficiently verifying 
that acquired systems have the same system configurations as the 
version that EAC certified, states and local jurisdictions will not 
know whether they are using federally certified voting systems. The 
absence of such tools unnecessarily increases the risk of a system 
bearing EAC's mark of certification differing from the certified 
version. 

EAC's Mechanism for Tracking and Resolving Problems with Certified 
Voting Systems Lacks Sufficient Definition, and Its Scope Is Limited: 

As part of its voting system testing and certification program, EAC has 
broadly described an approach for tracking and resolving problems with 
certified voting systems, and using the information about these 
problems to improve its program. This approach reflects some key 
aspects of relevant guidance. However, other aspects are either missing 
or not adequately defined, and although EAC officials stated that they 
intend to address some of these gaps, the commission does not have 
defined plans or time frames for doing so. Commission officials cited 
limited resources and competing priorities as reasons for these gaps. 

In addition, EAC's problem tracking and resolution approach does not 
extend to any of the voting systems that are likely to be used in the 
2008 elections, and it is uncertain when, and to what extent, this 
situation will change. This is because its defined scope only includes 
EAC-certified systems; it does not include NASED-qualified systems or 
any other systems to be used in elections. EAC officials stated that 
the reason for this is because HAVA does not explicitly assign the 
commission responsibility for systems other than those it certifies. 
This means that no federal entity is currently responsible for tracking 
and facilitating the resolution of problems found with the vast 
majority of voting systems that are used across the country today and 
that could be used in the future, and thus states and local 
jurisdictions must deal with problems with their systems on their own. 

EAC's Approach for Tracking and Resolving Problems with Certified 
Systems Partially Reflects Relevant Guidance: 

According to published guidance,[Footnote 24] tracking and resolving 
problems with certified products is important. This includes, among 
other things, (1) withdrawing certification if a product becomes 
noncompliant; (2) regularly monitoring the continued compliance of 
products being produced and distributed; (3) investigating the validity 
and scope of reports of noncompliance; (4) requiring the manufacturer 
to take corrective actions when defects are discovered, and ensuring 
that such actions are taken; and (5) using information gathered from 
these activities to improve the certification program. 

EAC's approach for tracking and resolving problems with certified 
systems reflects some, but not all aspects of these five practices. 
First, its certification program includes provisions for withdrawing 
certification for noncompliant voting systems. For example, the program 
manual describes procedures for decertifying a noncompliant voting 
system if the manufacturer does not take timely or sufficient action to 
correct instances of noncompliance. According to these procedures, the 
decertification decision cannot be made until the manufacturer is 
formally alerted to the noncompliance issue and provided with an 
opportunity to correct the issue (problem) or to submit additional 
information for consideration. Also, a manufacturer can dispute the 
decision by requesting an appeal. The procedures also state that upon 
decertification, the manufacturer cannot represent the system as 
certified and the system may not be labeled with a mark of 
certification. In addition, EAC is to remove the system from its list 
of certified systems, alert state and local election officials to the 
system's decertification via monthly newsletters and e-mail updates, 
and post all correspondence regarding the decertification on its Web 
site. 

Second, EAC's certification program includes provisions for 
postcertification oversight of voting systems and manufacturers. 
Specifically, the program manual provides for reviewing and retesting 
certified voting systems to ensure that they have not been modified, 
and that they continue to comply with applicable standards and program 
requirements. In addition, the program manual calls for periodic 
inspections of manufacturers' facilities to evaluate their production 
quality, internal test procedures, and overall compliance with program 
requirements. However, the program manual states that reviewing and 
retesting of certified systems is an optional step, and does not 
specify the conditions under which this option is to be exercised. 
Further, while the program manual provides for conducting periodic 
inspections of manufacturers' facilities, it does not define, for 
example, who is to conduct the inspections, what procedures and 
evaluation criteria are to be used in conducting them, and how they are 
to be documented. 

Third, EAC's certification program includes provisions for 
investigating reports of system defects. According to the program 
manual, investigations of reports alleging defects with certified 
systems begin as informal inquiries that can potentially become formal 
investigations. Specifically, the Program Director is to conduct an 
informal inquiry in which a determination is made regarding whether the 
reported defect information is both credible and deserving of system 
decertification if found to be credible. If both conditions are met, 
then a formal investigation is to be conducted. Depending on the 
outcome, decertification of the system could result. However, the 
program manual does not call for assessing the scope or impact of any 
defects identified, such as whether a defect is confined to an 
individual unit or whether it applies to all such units. Further, the 
program manual does not include procedures or criteria for determining 
the credibility of reported defects, or any other aspect of the inquiry 
or investigation, such as how EAC will gain access to systems once they 
are purchased and fielded by states and local jurisdictions. This is 
particularly important because EAC does not have regulatory authority 
over state election authorities, and thus, it cannot compel their 
cooperation during an inquiry or investigation. 

Fourth, EAC's certification program does not address how it will verify 
that manufacturers take required corrective actions to fix problems 
identified with certified systems. Specifically, the program manual 
states that the manufacturer is to provide EAC with a compliance plan 
describing how it will address identified defects. However, the program 
manual does not define an approach for evaluating the compliance plan 
and confirming that a manufacturer actually implements the plan. 
According to EAC officials, they see their role as certifying a system 
and informing states of modifications. As a result, they do not intend 
to monitor how or whether manufacturers implement changes to fielded 
systems. In their view, the state ultimately decides if a system will 
be fielded. 

Fifth, EAC's certification program provides for using information 
generated by its problem tracking and resolution activities to improve 
the program. According to the program manual, information gathered 
during quality monitoring activities will be used to, among other 
things, identify improvements to the certification process and to 
inform the related standards-setting process. Further, the program 
manual states that information gathered from these activities will be 
used to inform relevant stakeholders of issues associated with 
operating a voting system in a real-world environment and to share 
information with jurisdictions that use similar systems. However, the 
program manual does not describe how EAC will compile and analyze the 
information gathered to improve the program, or how it will coordinate 
these functions with information gathered in performing its HAVA- 
assigned clearinghouse function.[Footnote 25] 

EAC officials attributed the state of their problem tracking and 
resolution approach to the newness of the certification program and the 
fact that the commission's limited resources have been devoted to other 
priorities. In addition, while these officials said that they intend to 
address some of these gaps, they do not have defined plans or time 
frames for doing so. For example, while EAC officials stated that they 
plan to develop procedures for investigating voting system problems and 
for inspecting manufacturing facilities, they said that it is the 
states' responsibility to ensure that corrective actions are 
implemented on fielded systems. To illustrate their resource 
challenges, these officials told us that three staff are assigned to 
the testing and certification program and each is also supporting other 
programs.[Footnote 26] In addition, they said that the commission's 
technical reviewers are experts who, under Office of Personnel 
Management regulation, work no more than one-half of the time of the 
year.[Footnote 27] 

Given that EAC has not yet certified a system, the impact of these 
definitional limitations has yet to be realized. Nevertheless, with 12 
systems currently undergoing certification, it is important for the 
commission to address them quickly. If it does not, EAC will be 
challenged in its ability to effectively track and resolve problems 
with the systems that it certifies. 

EAC's Problem Tracking and Resolution Efforts Do Not Include Most 
Voting Systems to Be Used in Elections: 

The scope of EAC's efforts to track and resolve problems with certified 
voting systems does not extend to those systems that were either 
qualified by NASED or were not endorsed by any national authority. 
According to program officials, the commission does not have the 
authority or the resources needed to undertake such a responsibility. 
Instead of tracking and resolving problems with these systems, EAC 
anticipates that they will eventually be replaced or upgraded with 
certified systems. Our review of HAVA confirmed that the act does not 
explicitly assign EAC any responsibilities for noncertified systems, 
although it also does not preclude EAC from tracking and facilitating 
the resolution of problems with these systems. 

As a result, the commission's efforts to track and resolve problems 
with voting systems do not include most of the voting systems that will 
be used in the 2008 elections. More specifically, while EAC has efforts 
under way relative to the certification of 12 voting systems, as we 
have previously described in this report, commission officials stated 
that it will be difficult to field any system that EAC anticipates 
certifying before 2008 in time for the 2008 elections. Thus, voting 
systems used in national elections will likely be either those 
qualified under the now-discontinued NASED program, or those not 
endorsed by any national entity. Moreover, this will continue to be the 
case until states voluntarily begin to adopt EAC-certified systems, 
which is currently unclear and uncertain because only 18 states 
reported having requirements to use EAC-certified voting systems. 
Restated, most states' voting systems will not be covered by EAC's 
problem tracking and resolution efforts, and when and if they will is 
not known. Moreover, manufacturers may or may not upgrade existing, 
noncertified systems, and they may or may not seek EAC certification of 
those systems. Thus, it is likely that many states, and their millions 
of voters, will not use EAC-certified voting systems for the 
foreseeable future. 

Nevertheless, EAC has initiated efforts under the auspices of its HAVA- 
assigned clearinghouse responsibility[Footnote 28] to receive 
information that is volunteered by states and local jurisdictions on 
problems and experiences with systems that it has not certified, and to 
post this information on the commission's Web site to inform other 
states and jurisdictions about the problems. In doing so, EAC's Web 
site states that the commission does not review the information for 
quality and does not endorse the reports and studies. Notwithstanding 
this clearinghouse activity, this means that no national entity is 
currently responsible for tracking and facilitating the resolution of 
problems found with the vast majority of voting systems that are in use 
across the country. This in turn leaves state and local jurisdictions 
on their own to discover, disclose, and address any shared problems 
with systems. While this increases the chances of states and local 
jurisdictions duplicating efforts to get problems fixed, it also 
increases the chances that problems addressed by one state or 
jurisdiction may not even be known to another. A key to overcoming this 
situation will be strong central leadership. 

Conclusions: 

The effectiveness of our nation's overall election system depends on 
many interrelated and interdependent variables. Among these are the 
security and reliability of the voting systems that are used to cast 
and count votes, which in turn depend largely on the effectiveness with 
which these systems are tested and certified. EAC plays a pivotal role 
in testing and certifying voting systems. To its credit, EAC has 
recently established and begun implementing a voting system testing and 
certification program that is to both improve the quality of voting 
systems in use across the country, and help foster public confidence in 
the electoral process. While EAC has made important progress in 
defining and executing its program, more can be done. Specifically, key 
elements of its defined approach, such as the extent to which 
certification activities are to be documented, are vague, while other 
elements are wholly undefined--such as threshold criteria for making 
certification-related decisions. Moreover, a key element that is 
defined--namely, giving states and local jurisdictions an effective and 
efficient means to access the certified version of a given voting 
system software--has yet to be implemented. While EAC acknowledges the 
need to address these gaps, it has yet to develop specific plans or 
time frames for completing them that, among other things, ensure that 
adequate resources for accomplishing them are sought. 

Addressing these gaps is very important because their existence not 
only increases the chances of testing and certification activities 
being performed in a manner that is neither repeatable nor verifiable, 
they also can create misunderstanding among manufacturers and VSTLs 
that can lead to delays in the time needed to certify systems. Such 
delays have already been experienced, to the point that needed upgrades 
to current systems will likely not be fielded in time for use in the 
2008 elections. Such situations ultimately detract from, and do not 
enhance, election integrity and voter confidence. Moreover, by not 
having established an effective means for states and local 
jurisdictions to verify that the systems each acquires are the same as 
the EAC-certified version, EAC is increasing the risk of noncertified 
versions ultimately getting used in an election. 

Beyond the state of EAC's efforts to define and follow an approach to 
testing and certifying voting systems, including efforts to track and 
resolve problems with certified systems and use this information to 
improve the commission's testing and certification program, a void 
exists relative to having a national focus on tracking and resolving 
problems with voting systems that EAC has not certified, and thus has 
not been assigned explicit responsibility or has the resources to 
address. Unless this void is filled, state and local governments will 
likely continue to be on their own for resolving performance and 
maintenance issues for the vast majority of voting systems in use today 
and the near future. 

Recommendations for Executive Action: 

To assist EAC in building upon and evolving its voting systems testing 
and certification program, we recommend that the Chair of the EAC 
direct the commission's Executive Director to ensure that plans are 
prepared, approved, and implemented for developing and implementing: 

* detailed procedures, review criteria, and documentation requirements 
to ensure that voting system testing and certification review 
activities are conducted thoroughly, consistently, and verifiably; 

* an accessible and available software repository for testing 
laboratories to deposit certified versions of voting system software, 
as well as procedures and review criteria for evaluating related 
manufacturer-provided tools to support stakeholders in comparing their 
systems with this repository; and: 

* detailed procedures, review criteria, and documentation requirements 
to ensure that problems with certified voting systems are effectively 
tracked and resolved, and that the lessons learned are effectively used 
to improve the certification program. 

Matter for Congressional Consideration: 

To address the potentially longstanding void in centrally facilitated 
problem identification and resolution for non-EAC-certified voting 
systems, we are raising for congressional consideration expanding EAC's 
role under HAVA such that, consistent with both the commission's 
nonregulatory mission and the voluntary nature of its voting system 
standards and certification program, EAC is assigned responsibility for 
providing resources and services to facilitate understanding and 
resolution of common voting system problems that are not otherwise 
covered under EAC's certification program, and providing EAC with the 
resources needed to accomplish this. 

Agency Comments and Our Evaluation: 

In written comments on a draft of this report, signed by the EAC 
Executive Director, and reprinted in appendix II, the commission stated 
that it agrees with the report's conclusion that more can be done to 
build on the existing voting system certification program and ensure 
that certifications are based on consistently performed reviews. In 
addition, EAC stated that it has found our review and report helpful in 
its efforts to fully implement and improve this program. It also stated 
that it generally accepts our three recommendations with little 
comment, adding that it will work hard to implement them. In this 
regard, it cited efforts that are planned or underway to address the 
recommendations. 

EAC provided additional comments on the findings that underlie each of 
the recommendations, which it described as needed to clarify and avoid 
confusion about some aspects of its certification program. According to 
EAC, these comments are intended to allay some of the concerns raised 
in our findings. We summarize and evaluate these comments below, and to 
avoid any misunderstanding of our findings and the recommendation 
associated with one of them, we have modified the report, as 
appropriate, in response to EAC's comments. 

EAC also provided comments on our matter for congressional 
consideration, including characterizing it as intending "to affect a 
sea change in the way that EAC operates its testing and certification" 
program. We agree that EAC does not have the authority to compel 
manufacturers or states and local jurisdictions to submit to its 
testing and certification program and that the wording of the matter in 
our draft inadvertently led EAC to believe that our proposal would 
require the commission to assume a more regulatory role. In response, 
we have modified the wording that we used to clarify any 
misunderstanding as to our intent. 

With respect to our first recommendation for developing and 
implementing plans for ensuring that voting system testing and 
certification review activities are governed by detailed procedures, 
criteria, and documentation requirements, EAC stated that it is 
committed to having a program that is rigorous and thorough, and that 
its program manual creates such a program. It also stated that it 
agrees with our recommendation and that it will work to further the 
process in its manual by implementing detailed procedures. In this 
regard, however, it took issue with five aspects of our finding. 

* With respect to our point that criteria and procedures have not been 
adequately defined for reviewing the information in the manufacturer 
registration application package, EAC stated that such criteria are not 
necessary because the package does not require a determination of 
sufficiency. We do not agree. According to section 2.4.2 of the program 
manual, EAC is to review completed registration applications for 
sufficiency. However, as our report states, the manual does not define 
criteria as to what constitutes sufficiency and what this sufficiency 
review should entail, and EAC officials stated this determination is 
left up to the individual reviewer's judgment. 

* Concerning our point that criteria and procedures have not been 
adequately defined for reviewing the information in the system 
certification application package, EAC stated that a technical review 
of the package is not required. Rather, it said that the package simply 
requires a determination that all necessary information is present, 
adding that it has a checklist to assist a reviewer in determining 
this. We do not agree with this comment for two reasons. First, our 
report does not state that the package review is technical in nature, 
but rather is a review to determine a package's completeness and 
accuracy. Second, the checklist does not include any criteria upon 
which to base a completeness and accuracy determination. As EAC's 
comments confirm, this is important because the information in the 
package is to be used by technical reviewers as they review test plans 
and test reports to ensure that the testing covers all aspects of the 
voting system. For example, EAC requires certification applications to 
include a functional diagram depicting how the components for the 
voting system function and interact, as well as a system overview that 
includes a description of the functional and physical interfaces 
between components. Although the checklist provides for determining 
whether these items are part of the application package, it does not 
provide for checking them for completeness and consistency. We have 
clarified this finding in our report by including this example. 

* As to our point that EAC has not defined how technical reviewers are 
to determine the adequacy of system test plans and reports, the 
commission stated that our report does not take into account what it 
described as a certification requirements traceability matrix that its 
technical reviewers use to assess the completeness and adequacy of the 
plans and reports. However, EAC also acknowledged in its comments that 
procedures have yet to be established relative to the use of the 
matrix. Further, we reviewed this matrix, which we refer to in our 
report as a checklist, and as we state in our report, this checklist 
does not provide for capturing how decisions were reached, including 
steps performed and criteria applied. For example, the VVSG requires 
that systems permit authorized access and prevent unauthorized access, 
and lists examples of measures to accomplish this, such as computer- 
generated password keys and controlled access security. While the 
checklist cites this requirement and provides for the reviewer to 
indicate whether the test plan satisfies it, it does not provide 
specific guidance on how to determine whether the access control 
measures are adequate, and it does not provide for documenting how the 
reviewer made such a decision. In response to EAC's comments, we have 
added this access control example to clarify our finding. 

* With regard to our point that the program manual does not include 
defined test suites, EAC commented on the purpose of these test suites 
and stated that it would not be appropriate to include them in the 
manual because the manual is not intended to define technical 
requirements for testing. We agree that the program manual should not 
include actual test suites, and it was not our intent to suggest that 
it should. Rather our point is that test suites do not yet exist. 
Accordingly, we have modified our report to more clearly reflect this. 
In addition, we acknowledge EAC's comment that NIST is currently in the 
process of developing test suites, and that it recently sent several 
test suites to the VSTLs and other stakeholders for review. However, as 
we state in our report, NIST officials said that they are focused on 
preparing test suites for the yet-to-be-released update to the VVSG and 
not the 2005 version. Further, EAC has not yet established plans or 
time frames for finalizing test suites for either versions of these 
guidelines, and the program manual does not make reference to the 
development of these test suites. 

* In commenting on our point that differences in interpretation of 
program requirements have resulted in test plan and report approval 
delays, EAC stated that its interpretation process provides a means for 
VSTLs and manufacturers to request clarification of the voting system 
standards that are ambiguous. We agree with this statement. However, we 
also believe that having the kind of defined procedures and established 
criteria that are embodied in our recommendation will provide a common 
understanding among EAC stakeholders around testing and certification 
expectations, which should minimize the need to reconcile differences 
in interpretations later in the process. 

With respect to our second recommendation for developing and 
implementing plans for an accessible and available software repository 
for certified versions of voting system software, as well as the 
related manufacturer-provided procedures and tools to support 
stakeholders in using this repository, EAC stated that it agrees that 
implementation of a repository is needed. However, it stated that there 
is some misunderstanding regarding the purpose of the repository and 
the creation of software identification tools. Specifically, the 
commission stated that the repository is intended for the commission's 
own use when conducting investigations of fielded systems, while the 
manufacturer-provided system identification tools are for use by state 
and local election officials to confirm that their systems are the same 
as the one certified by EAC. In addition, it described steps taken or 
under way to ensure that a repository and identification tools are in 
place when needed. This includes "placing the onus" on system 
manufacturers to create verification tools, investigating software 
storage options, and discussing with another government agency and 
outside vendors the possibility of providing secure storage for 
certified software. 

We agree with EAC that its repository serves as a tool for its internal 
use. However, the repository is also to serve state and local election 
officials in verifying that their respective systems are identical to 
the EAC-certified versions of the systems. According to the 2005 VVSG 
software distribution and setup validation requirements, the process 
for voting system purchasers in verifying that the version of the 
software that they receive from a manufacturer is the same as the 
version certified by EAC is to be performed by comparing it with the 
reference information generated by the designated repository. Further, 
commission officials told us that EAC's repository needs to be 
accessible and easy to use by state and local election officials. While 
we understand that the manufacturer-provided system identification 
tools serve a separate function from the repository, both tools 
together are required for state and local election officials to verify 
their systems. We also acknowledge that the commission has initiated 
steps relative to establishing a repository and identification tools. 
However, our point is that EAC does not have any plans or time frames 
for accomplishing this. Further, while we agree that the manufacturers 
are responsible for creating the identification tools, as we stated in 
our report, EAC has not defined how it will evaluate the manufacturer- 
provided tools. To avoid any misunderstanding as to these points, we 
have slightly modified our finding and related recommendation. 

Concerning our third recommendation for developing and implementing 
detailed procedures, review criteria, and documentation requirements 
for tracking and resolving problems with certified voting systems and 
applying lessons learned to improve the certification program, EAC 
stated that the report does not correctly represent its role in 
confirming that manufacturers actually correct anomalies in all fielded 
systems, and it added that the commission does not have the authority 
or the human capital to do so. Accordingly, EAC stated that it informs 
affected jurisdictions of system changes, but that it is at the 
discretion of the states and local jurisdictions, and beyond the scope 
of the commission, to determine whether fixes are made to individual 
systems in the field. 

We agree that the states and local jurisdictions have the 
responsibility and authority to determine whether they will implement 
EAC-approved fixes in the systems that they own. However, as we state 
in our report, published ISO guidance on tracking and resolving 
problems with certified products recognizes the importance of the 
certification body's decision to require manufacturers to take 
corrective actions when defects are discovered, and to ensure that such 
actions are taken. Although this guidance acknowledges the difficulty 
in ensuring corrective actions are implemented on all affected units, 
it states that products should be corrected "to the maximum degree 
feasible." Given EAC's authority over registered manufacturers, it can 
play a larger role in ensuring that problems with fielded system are in 
fact resolved, while maintaining the voluntary nature of its program, 
by monitoring the manufacturers' efforts to fix systems for those 
jurisdictions that choose to implement such corrections, and holding 
manufacturers accountable for doing so. To avoid any confusion about 
this point, we have slightly modified our finding. 

As to our matter for congressional consideration to amend HAVA to give 
EAC certain additional responsibilities relative to problem resolution 
on voting systems not certified by EAC, the commission voiced several 
concerns. Among other things, it stated that our proposal would "affect 
a sea change in the way that EAC operates its testing and 
certification" program, changing it from voluntary to mandatory. 
Further, it stated that it would, in effect, place EAC in a position to 
act in a regulatory capacity without having the specific authority to 
do so, as it would necessitate making both the voluntary voting system 
guidelines and the testing and certification program mandatory for all 
states. It also stated that it would require EAC to have specific 
authority to compel manufacturers of these noncertified voting systems 
to submit their systems for testing, and to compel states and local 
jurisdictions to report and resolve any identified system problems. 

We recognize that both the voting system guidelines and the testing and 
certification program are voluntary, and that EAC does not have the 
authority to compel manufacturers or states and local jurisdictions to 
submit to its testing and certification program, or to force them to 
correct any known problems or report future problems. We further 
acknowledge that the wording of the matter for congressional 
consideration in our draft report resulted in EAC interpreting 
accomplishment of it as requiring such unintended measures. Therefore, 
we have modified it to clarify our intent and to avoid any possible 
misunderstanding. In doing so, we have emphasized our intent for EAC to 
continue to serve its existing role as a facilitator and provider of 
resources and services to assist states and local jurisdictions in 
understanding shared problems, as well as the voluntary nature of both 
the system guidelines and the testing and certification program. 
Further, we seek to capitalize on EAC's unique role as a national 
coordination entity to address a potentially longstanding, situational 
awareness void as it pertains to voting systems in use in our nation's 
elections. As we state in our report, this void increases the chances 
of states and local jurisdictions duplicating efforts to fix common 
system problems, and of problems addressed by one state or local 
jurisdiction being unknown to others. We believe that a key to 
overcoming this will be strong central leadership, and that with the 
appropriate resources, EAC is in the best position to serve this role. 

We are sending a copy of this report to the Ranking Member of the House 
Committee on House Administration, the Chairman and Ranking Member of 
the Senate Committee on Rules and Administration, the Chairmen and 
Ranking Members of the Subcommittees on Financial Services and General 
Government, Senate and House Committees on Appropriations, and the 
Chairman and Ranking Member of the House Committee on Oversight and 
Government Reform. We are also sending copies to the Chair and 
Executive Director of the EAC, the Secretary of Commerce, the Acting 
Director of the NIST, and other interested parties. We will also make 
copies available to others on request. In addition, this report will be 
available at no charge on the GAO Web site at [hyperlink, www.gao.gov]. 

Should you or your staff have any questions on matters discussed in 
this report, please contact me at (202) 512-3439 or at [email protected]. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. Key contributors 
to this report are listed in appendix III. 

Sincerely yours, 

Signed by: 

Randolph C. Hite: 
Director, Information Technology Architecture and System Issues: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

Our objectives were to determine whether the Election Assistance 
Commission (EAC) has (1) defined an effective approach to testing and 
certifying voting systems, (2) followed its defined approach, and (3) 
developed an effective mechanism to track problems with certified 
systems and use the results to improve its certification program. 

To address the first and third objectives, we researched leading 
practices relevant to certification testing/conformity assessment and 
tracking and resolving problems with certified products, including 
published guidance from the National Institute of Standards and 
Technology (NIST), International Organization for Standardization, and 
International Electrotechnical Commission, and legal requirements in 
the Help America Vote Act. We obtained and reviewed relevant EAC 
policies and procedures for testing, certifying, decertifying, and 
recertifying voting systems. Specifically, we reviewed the EAC 
Certification Program Manual and other EAC-provided documents. We 
interviewed EAC officials and their technical reviewers, NIST 
officials, representatives from the industry trade association for 
voting system manufacturers, representatives from the voting system 
test laboratories, and the National Association of State Election 
Directors' point-of-contact for qualified systems. We then compared 
this body of evidence with the leading practices and related guidance 
we had researched, as well as applicable legal requirements, to 
determine whether EAC's program had been effectively defined. In 
addition, for the third objective, we reviewed the contents and policy 
of EAC's clearinghouse. 

To address our second objective, we obtained and reviewed actions and 
artifacts from EAC's execution of its certification program to date. We 
assessed this information against the policies, procedures, and 
standards outlined in the EAC Certification Program Manual and the 2005 
Voluntary Voting System Guidelines, and after discussing and confirming 
our findings with EAC officials, we determined whether EAC had followed 
its defined approach. 

In addition, to determine the impact of federal certification time 
frames, we included a question about EAC certification on a survey of 
officials from all 50 states, the District of Columbia, and 4 
territories. We also contacted officials from states that indicated 
their intent to use EAC certification for the 2008 elections to better 
understand how they plan to address voting system certification in 
their state relative to EAC's program. To develop our survey, we 
reviewed related previous and ongoing GAO work, and developed a 
questionnaire in collaboration with GAO's survey and subject matter 
experts. We conducted pretests in person and by telephone with election 
officials from 5 states to refine and clarify our questions. Our Web- 
based survey was conducted from December 2007 through April 2008. We 
received responses from 47 states, the District of Columbia, and all 4 
territories (a 95 percent response rate). Differences in the 
interpretation of our questions among election officials, the sources 
of information available to respondents, and the types of people who do 
not respond may have introduced unwanted variability in the responses. 
We examined the survey results and performed analyses to identify 
inconsistencies and other indications of error, which were reviewed by 
an independent analyst. We also contacted officials from those states 
whose survey response indicated their intent to use EAC certification 
for the 2008 elections to identify their plans and approaches for state 
certification in the event that federal certification could not be 
completed to meet their election preparation schedules. 

We conducted this performance audit at EAC offices in Washington, D.C., 
from September 2007 to September 2008 in accordance with generally 
accepted government auditing standards. Those standards require that we 
plan and perform the audit to obtain sufficient, appropriate evidence 
to provide a reasonable basis for our findings and conclusions based on 
our audit objectives. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objectives. 

[End of section] 

Appendix II: Comments from the Election Assistance Commission: 

U.S. Election Assistance Commission: 
1225 New York Ave. NW ï¿½ Suite 1100: 
Washington, DC 20005: 

July 16, 2008: 

Mr. Randolph C. Hite, Director: 
Information Technology Architecture and Systems: 
United States Government Accountability Office: 

Delivered via e-mail: 

Re: Draft Report GAO 08-814, Elections: Federal Program for Certifying 
Voting Systems Needs to be Further Defined, Fully Implemented, and 
Expended 

Thank you for the opportunity to provide comment on GAO report 08-814, 
regarding the Election Assistance Commission (EAC) voting system 
testing and certification program. The EAC appreciates the time and 
effort put forth by GAO during the preparation of this document and the 
willingness of GAO staff to discuss pertinent issues at length with the 
EAC. The EAC has found both the review process and report helpful as it 
works to fully implement and improve its HAVA required mandate to 
provide for testing, certification, decertification, and 
recertification of voting system hardware and software. 

GAO recognized that: "EAC has defined an approach to testing and 
certifying voting systems that follows a range of relevant practices 
and statutory requirements associated with a product certification 
program, including those published by U.S. and international standards 
organizations, and those reflected in HAVA." (pp. 5-6). The EAC 
generally agrees with the report's conclusion that more can be done in 
order to build on the EAC's existing certification program in order to 
make sure that certifications are based upon consistent reviews. 
However, EAC is concerned that GAO confuses some aspects of EAC's 
Testing and Certification Program and therefore doesn't recognize 
practices and procedures already in place as part of the EAC's program 
that would quell some of GAO's concerns. 

The report provides three recommendations for the EAC to better conform 
to certification program management guidance published by National 
Institute of Standards and Technology (NIST) and International 
Standards Organization (ISO)/International Electrotechnical Commission 
(IEC) . Generally, the EAC accepts the recommendations provided with 
little comment. However, the EAC feels the need to clarify several 
points related to the recommendations and the Matter for Congressional 
Consideration. The following are EAC's comments in response to each 
recommendation. 

1. Detailed procedures, review criteria, and documentation requirements 
to ensure that voting system testing and certification review 
activities are conducted thoroughly, consistently, and verifiably; 

GAO recognizes that EAC's Testing and Certification program has defined 
a testing and certification process that follows many recognized and 
accepted practices for conformance assessment and product certification 
(pp. 20-21). Specifically, GAO found that EAC's Testing and 
Certification Program covers the standard procedures established by 
NIST, ISO and IEC for testing and certification programs, including: 
(1) Defining roles and responsibilities for all parties involved in the 
certification process; (2) defining a clear and transparent process for 
applicants to follow; (3) ensuring that persons involved in the process 
are impartial and independent; (4) establishing a process for handling 
complaints and appeals; and (5) having testing conducted by competent 
laboratories. These procedures are outlined in the EAC's Voting System 
Testing and Certification Program Manual. As the GAO report notes, in 
addition to the five items listed above, the EAC's program manual also 
"clearly defines the program's administrative requirements that 
manufacturers and Voting System Test Laboratories (VSTLs) are to 
follow," addresses impartiality and independence of the testing 
process, and outlines processes for the resolution of complaints, 
appeals, and disputes received from manufacturers and laboratories (p. 
22). The GAO report also states, ". the EAC has provided an important 
foundation for having an effective voting system certification 
program."(p.23). 

A thorough testing and certification program is essential to ensuring 
public confidence in our electoral system. As GAO found in its report, 
prior efforts at testing and qualifying voting systems have left 
election administrators to deal with complaints and questions relating 
to the sufficiency and security of their voting systems. "As we 
reported in 2005, these concerns include weak security controls, system 
design flaws, inadequate system version control, inadequate security 
testing, incorrect system configuration, poor security management, and 
vague or incomplete voting system standards. Further, security experts 
and some election officials have expressed concerns that tests 
performed under the NASED program by independent testing authorities 
and state and local election officials did not adequately assess voting 
systems' security and reliability. Consistent with these concerns, most 
of the security weaknesses that we identified in our prior report 
related to systems that had previously been qualified by NASED. Our 
report also recognized that security experts and others pointed to 
these weaknesses as an indication that both the standards and the NASED 
testing program were not rigorous enough with respect to security, and 
that these concerns were amplified by what some described as a lack of 
transparency in the testing process". (p. 12, footnote omitted) 

EAC is committed to conducting a program that is rigorous and 
thoroughly tests systems to high standards for operation and security. 
EAC's Voting System Testing and Certification Manual creates such a 
program and EAC will work to further this process by implementing 
internal procedures consistent with the program manual. 

Although GAO did not find any instances in which EAC's Voting System 
Testing and Certification Program has been conducted in an inequitable 
or discriminatory manner, GAO offered several areas where additional 
internal procedures would further ensure consistency in the process. 
GAO recommends procedures for reviewing manufacturer registration and 
system application packages, procedures for assessing test plan and 
test reports, including test suites in the program manual, and 
providing a means to resolve differing interpretations of voting system 
standards. While we agree with GAO's overall recommendations, we are 
concerned that some of the examples cited in the report may cause 
confusion as to the EAC's current policies and procedures. 

GAO asserts that while the EAC requires manufacturers to register prior 
to submitting a system for certification, it does not adequately define 
the criteria for approval of the registration package. As stated in the 
EAC's program manual, the EAC will review manufacturer registration 
applications for completeness before approval (Chapter 2 of the 
Certification manual). The registration package simply contains contact 
information, a listing of manufacturing facilities, and a series of 
agreements by the manufacturer to comply with program requirements. 
Reviewing this information does not require the EAC to make a 
determination of sufficiency, but instead simply requires confirmation 
that all required information is present. To accomplish this, the EAC 
has created a checklist for the review of each manufacturer 
registration package. 

Similarly, GAO notes that in order for a system to begin the EAC's 
certification process, a manufacturer must submit a voting system 
application package. The voting system application package includes 
information related to the make up of the system. It is used by EAC 
technical reviewers as they review testing plans and reports to assure 
that those plans and reports cover all aspects of the voting system. 
Thus, the voting system application package does not require a 
technical review of the information provided but instead simply 
requires a determination that all necessary information is present. As 
such, the EAC has created a checklist for voting system application 
packages that allows a reviewer to document whether or not all required 
information is provided and if it is not provided to request the 
information from the manufacturer. 

The GAO report correctly finds that all test plans submitted to the EAC 
for approval are reviewed for sufficiency. In conducting this review, 
the EAC is looking to ensure that all requirements of the VSS or VVSG 
that are applicable to the system will be tested. The GAO report states 
that the EAC does not define how such reviews are to be performed. 
However, this assessment does not take into account the certification 
requirements traceability matrix used by EAC technical reviewers to 
assess the quality and completeness of the test plan and the test 
report. The requirements matrix lists the requirements to be tested and 
the requirements which must be met by a system before it can receive 
certification. Using the requirements matrix enables EAC to 
consistently assess each test plan and test report for completeness and 
adequacy. 

The GAO report describes the EAC program manual as excluding defined 
test suites. The EAC believes there is some confusion regarding exactly 
what test suites are and the role they will play in the certification 
process. As noted in the GAO report, NIST is currently in the process 
of developing test suites for use by the VSTLs. Recently, NIST sent 
several test suites to the laboratories and other stakeholders for 
review. These test suites are designed to be high level test methods 
that can be taken by a VSTL and adapted for the creation of system 
specific test cases. Each test suite will encompass a set of voting 
system standards to be tested on a given system. As such, it would not 
be appropriate to include the actual test suites in the EAC's program 
manuals because the manuals are designed to document the EAC's 
procedural program requirements not to represent the technical 
requirements for testing. However, the EAC does require use of the test 
suites in its program. In the EAC's Voting System Testing Laboratory 
Program Manual, which is scheduled to be voted on by the Commission at 
the July 2008 public meeting, the EAC requires the use of test suites 
in the creation of test plans and the testing of the system. Likewise, 
in Chapter 4 of the Testing and Certification Manual, the EAC 
establishes a requirement to submit a test plan and test report for EAC 
approval. Test suites will be included and reviewed in the submitted 
test plans and test reports. 

GAO asserts that differences in how the EAC, VSTLs, and manufacturers 
interpret voting system requirements have caused delays in the test 
plan and test report approval process. As the GAO report notes, the EAC 
provides a means by which the VSTLs and Manufacturers may request 
clarification of the standard to be tested to (p. 22)(Chapter 9 Testing 
and Certification Manual). As the EAC encounters ambiguities in the 
standard the EAC issues interpretations of the standard in order to aid 
VSTLs in their testing of the system. However, as GAO noted in its 
report, the establishment of EAC's Voting System Testing and 
Certification Program represents a paradigm shift from previous testing 
efforts. In order for this to have its greatest effect, all players in 
the process must participate and use the tools available to them. To 
date the EAC has issued eight interpretations based upon requests by 
system users all of which are available on the EAC's website at 
[hyperlink, http://www.eac.gov]. 

The EAC is working to develop the internal procedures recommended by 
GAO. To ensure consistent and verifiable review, EAC is creating 
standard report formats, review tools and checklists. These documents 
will include guidance regarding: 

* Review of Technical Data Package elements. 

* Review and use of the Requirements Traceability Matrix. 

* Review of Test Plans.

* Review of Test Cases.

* Review of Test Reports.

* Identifying and reporting of common anomalies found during technical 
reviews.

* Timelines and processes for extending timelines when necessary. 

* Documenting review findings in a standard organized report. 

* Protocols for communicating with VSTLs and manufacturers. 

* Reporting process through which technical problems identified with 
the Voluntary Voting System Guidelines (VVSG) can be identified and 
reported to the EAC. 

2. Ensure plans are prepared, approved and implemented for an 
accessible and available software repository for testing laboratories 
to deposit certified versions of voting system software as well as 
related manufacturer-provided procedures and tools to support 
stakeholders in using this repository. 

As stated in the GAO report the EAC has, "...largely executed its 
voting certification program as defined." (p. 28) GAO goes on to add 
that for each of the 12 systems submitted for certification, "all 
elements of each executed step in the certification process were 
followed." Included in these steps were the approval of registration 
applications, applications for system testing, approval of test plans 
and the issuance of three notices of non-compliance to manufacturers 
not conforming with the EAC's requirements (p. 29). However, GAO 
pointed out the need to implement the program's requirement for a 
voting system software repository. 

While the EAC agrees that the implementation of a software repository 
is needed, there is some misunderstanding regarding the purpose of the 
repository and the creation of software identification tools. The EAC 
requires two post certification steps as a part of its testing and 
certification program: (1) submission of software in an approved 
repository; and (2) creation of system identification tools by the 
manufacturer. The purpose of the software repository is to create a 
frozen image or picture of the system as certified through the EAC 
program. This will allow the EAC to use the information stored in the 
repository when conducting investigations of fielded, EAC-certified 
systems and ensure that the fielded system is an exact match to the 
system that was certified. The creation of system identification tools 
by the manufacturer allows the voting system users (states) to review 
and confirm that the systems and software that they purchase are the 
same as those certified by EAC. 

Thus, the software repository is a tool for EAC use, while the system 
identification tools are for use by state and local governments. GAO 
suggests that the repository is intended to serve the function of the 
system identification tools and that this process should be 
implemented. However, EAC must point out that it has made provision for 
this function by requiring the manufacturer of the system to create 
this type of verification tool. Because no system has successfully 
completed the EAC certification process, EAC has yet to approve any 
manufacturer's system identification tool. However, the EAC is 
currently working with one manufacturer in the development of the 
required system identification tools. 

As noted in the GAO report, EAC had intended to use NIST's National 
Software Reference Library (NSRL) as its software repository. However, 
EAC quickly realized that NSRL could not serve the function of both a 
repository and a system identification tool. This is because of the 
limitation on comparing installed software (including the nonstatic 
portions of that code) to the hashed code maintained by NSRL. EAC's 
program needed more. In discussion with NIST, they agreed that NSRL's 
functions could not meet the needs of EAC's program. As such, EAC 
placed the onus on the manufacturer to develop the system 
identification tools. EAC has also investigated other, simpler 
solutions to its need for software storage. EAC has considered the 
possibility of contracting with an outside vendor for the secure 
storage of the certified software. Likewise, EAC has entered into 
discussion with another government agency that can provide the same 
service. Prior to the final certification of its first system, EAC will 
have a mechanism in place to securely store the certified software. 

3. Detailed procedures, review criteria, and documentation requirements 
to ensure that problems with certified voting systems are effectively 
tracked and resolved, and that the lessons learned are effectively used 
to improve the certification program. 

As GAO notes, EAC has already "broadly described an approach to allow 
it to track and resolve problems with certified voting systems." The 
GAO report cites five activities that a certifying body should provide 
for in its monitoring of fielded certified equipment: (1) Withdrawing 
certification if a product becomes non-compliant; (2) regularly 
monitoring the continued compliance of products being produced and 
distributed; (3) investigating the validity and scope of reports of non-
compliance; (4) requiring the manufacturer to take corrective actions 
when defects are discovered and ensuring such actions are taken; and 
(5) using information gathered from these activities to improve the 
certification program (p. 33). The GAO report correctly identifies that 
Chapter 8 of the EAC's program manual provides for aspects of all five 
of these requirements (pp. 33-34). GAO cites the EAC's outlined 
procedures for decertification, post-certification oversight including 
periodic inspections of manufacturer facilities, investigations of 
system defects, compliance management, and the use of information 
regarding fielded systems to improve the program. 

Additionally, GAO recognizes that the EAC compliance management process 
requires voting system manufacturers to report anomalies with fielded 
EAC-certified voting systems and to create a compliance plan to fix the 
anomaly found. However, the GAO report does not correctly represent the 
EAC's role in confirming that the manufacturer actually implements the 
solution in all fielded systems. As the certifying body EAC does not 
have the authority or the manpower to ensure that a manufacturer has 
instituted its solution in all fielded versions of the certified 
system. EAC has the responsibility to ensure that a noncompliant system 
has come into compliance. The EAC does this through its compliance 
management program and the compliance plan noted above. As noted in the 
EAC's program manual, after a compliance plan and compliance test 
report have been approved by the EAC, the EAC will make a decision on 
the amended voting system. All compliance plans including test reports 
and EAC decisions on amended systems will be made public. After a 
decision has been made on an amended system, the EAC will inform all 
jurisdictions impacted of the decision made. It is then up to the 
individual election jurisdictions to ensure that the change noted in 
the amended system decision is implemented. HAVA is explicit in stating 
that the EAC's program is voluntary and that it is the states' 
responsibility to determine if and how to use the EAC's program. 
Therefore, fixes made to individual systems in the field are at the 
discretion of the state and out of the scope of the EAC's program. 

In addition to the compliance management program requirements already 
noted by GAO, the EAC has already begun to develop specific procedures 
for use in investigating anomalies with certified voting systems. The 
procedures will include details on when and how the EAC will work with 
State and local election officials to investigate these anomalies and 
include general timeframes for all aspects of the investigations. 
Included in these procedures will be details on conducting 
manufacturing site audits. The information collected via the EAC's 
Compliance Management Program will be used to: 

* Identify areas for improvement in the EAC Testing and Certification 
Program. 

* Improve manufacturing quality and change control processes. 

* Increase voter confidence in voting technology. 

* Inform Manufacturers, election officials, and the EAC of issues 
associated with voting systems in a real-world environment. 

* Share information among jurisdictions that use similar voting 
systems. 

* Resolve problems associated with voting technology or manufacturing 
in a timely manner by involving manufacturers, election officials, and 
the EAC. 

* Provide feedback to the EAC and the Technical Guidelines Development 
Committee (TGDC) regarding issues that may need to be addressed through 
a revision to the Voluntary Voting System Guidelines. 

* Initiate an investigation when information suggests that 
Decertification is warranted. 

In addition to the three recommendations discussed above, GAO has 
issued a Matter for Congressional Consideration. The Congressional 
recommendation states: 

To address the potentially longstanding void in centrally facilitated 
problem identification and resolution for non-EAC certified voting 
systems, we are raising for congressional consideration amending HAVA 
to give EAC explicit responsibility for identifying, tracking, 
reporting and facilitating the resolution of problems that states and 
local jurisdictions experience with voting systems that are not covered 
by EAC certification, and providing EAC with the resources needed to 
accomplish this. 

GAO is recommending to Congress that it give "EAC explicit 
responsibility for identifying, tracking, reporting and facilitating 
the resolution of problems that states and local jurisdictions 
experience with voting systems that are not covered by EAC 
certification..." With this recommendation GAO is effectively 
requesting that EAC have the authority to regulate the voting systems 
that are currently in the field as well as the users of those systems. 

HAVA is explicit that both the voluntary voting system guidelines and 
the testing and certification program are voluntary. (42 U.S.C. ï¿½ï¿½ 
15361, 15362, and 15371(a)(2)). States must act to adopt the guidelines 
and participate in the program in order to require the use of EAC 
certified systems in their respective jurisdictions. Furthermore, the 
duties of the EAC as established by HAVA are prospective. HAVA tasks 
EAC with developing a new set of voting system testing standards (42 
U.S.C. ï¿½ 15322(1)) and developing a testing and certification program 
(42 U.S.C. ï¿½ 15371). In fact, HAVA recognizes and provides for a period 
of transition until these elements are in place: 

"(d) TRANSITION. ï¿½ Until such time as the Commission provides for the 
testing, certification, decertification, and recertification of voting 
system hardware and software by accredited laboratories under this 
section, the accreditation of laboratories and the procedures for 
testing, certification, decertification, and recertification of voting 
system hardware and software used as of the date of the enactment of 
this Act shall remain in effect." (42 U.S.C. ï¿½ 15371(d)). 

GAO's proposal would require making both the voluntary voting system 
guidelines and the testing and certification program to become 
mandatory. Furthermore, as written, the proposal would apply not only 
to systems that were previously tested under another program, but also 
those systems fielded by states that have chosen not to participate in 
EAC's testing and certification program. 

If Congress desires EAC to identify, track, report and facilitate the 
resolution of problems with voting systems that are currently in the 
field and which were tested and certified prior to the existence of 
EAC's and its testing and certification program and systems that have 
been fielded by states that chose not to participate in EAC's program, 
specific authority must be given to EAC to compel the manufacturers of 
these systems to provide those systems for testing, and to compel the 
users of those systems (states and local units of government) to report 
and resolve any identified problems. In order to accomplish this type 
of mandate, EAC would have to subject the fielded systems to testing 
against a set of standards. EAC would have to have a mechanism to force 
states and local governments that have these systems to correct any 
problems that were identified and make them report any future problems 
to EAC. 

Specifically, this would mean that EAC would have the authority to 
compel both voting system manufacturers and state and local government 
users of those systems to submit the systems for testing pursuant to 
EAC's testing and certification program. Systems would be tested 
against the existing voluntary voting system guidelines, 2005 VVSG. 
This is in stark contrast to the current authority given this 
Commission by HAVA, which is to operate a voluntary testing and 
certification program against voluntary testing standards. Similarly, 
EAC would have to be given the authority to regulate the users of these 
fielded systems so that EAC could compel those users to resolve 
identified problems and report any future problems with their voting 
systems. Under the current authorities granted by HAVA, EAC obtains the 
agreement of participating manufacturers to submit systems for testing 
and to report anomalies, problems, or issues with the operation of the 
systems that have been tested and certified through the EAC program. 

While EAC will implement any program or set of requirements that 
Congress desires, EAC must be given the appropriate authority to 
conduct those programs and the human capital and financial resources 
necessary to effectively implement any changes to its existing 
operations. GAO's proposal intends to affect a sea change in the way 
that EAC operates its testing and certification ï¿½ from voluntary to 
mandatory. GAO's proposed change to HAVA would place EAC in the 
position of acting in a regulatory capacity without the specific 
authority necessary to carry out the enumerated functions. 

The EAC thanks GAO for its work in assisting the Commission in its 
efforts to improve and develop the Voting System Testing and 
Certification Program. The EAC is committed to continuous improvement 
in all of it programs and will work hard to implement the 
recommendations made in this report. The EAC is focused on developing a 
world class testing and certification program to benefit election 
officials and the voting public. 

Sincerely,

Signed by: 

Thomas R. Wilkey: 
Executive Director: 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Randolph C. Hite, (202) 512-3439 or [email protected]: 

Staff Acknowledgments: 

In addition to the person named above, Paula Moore, Assistant Director; 
Mathew Bader; Neil Doherty; Nancy Glover; Dan Gordon; David Hinchman; 
Valerie Hopkins; Rebecca LaPaze; Jeanne Sung; and Shawn Ward made key 
contributions to this report. 

[End of section] 

Footnotes: 

[1] See, for example, GAO, Elections: All Levels of Government Are 
Needed to Address Electronic Voting System Challenges, GAO-07-576T 
(Washington, D.C.: Mar. 7, 2007); Elections: The Nation's Evolving 
Election System as Reflected in the November 2004 General Election, GAO-
06-450 (Washington, D.C.: June 6, 2006); Elections: Federal Efforts to 
Improve Security and Reliability of Electronic Voting Systems Are Under 
Way, but Key Activities Need to Be Completed, GAO-05-956 (Washington, 
D.C.: Sept. 21, 2005); Elections: Perspectives on Activities and 
Challenges Across the Nation, GAO-02-3 (Washington, D.C.: Oct. 15, 
2001); Elections: Status and Use of Federal Voting Equipment Standards, 
GAO-02-52 (Washington, D.C.: Oct. 15, 2001); and Elections: A Framework 
for Evaluating Reform Proposals, GAO-02-90 (Washington, D.C.: Oct. 15, 
2001). 

[2] GAO-02-52. 

[3] 42 U.S.C. ï¿½ 15371. 

[4] NASED is an independent, nongovernmental organization consisting of 
state election officials. It voluntarily formed the first national 
program to test and qualify voting systems to federal standards. 

[5] GAO-02-3. 

[6] GAO-02-3. 

[7] GAO-05-956. 

[8] GAO-05-956. 

[9] Help America Vote Act, Pub. L. No. 107-252 (Oct. 29, 2002). 

[10] GAO-04-975T. 

[11] The Technical Guidelines Development Committee was established 
under HAVA to assist EAC in developing the VVSG. 

[12] GAO, Elections: Federal Programs for Accrediting Laboratories That 
Test Voting Systems Need to Be Better Defined and Implemented, GAO-08-
770 (Washington, D.C.: Sept. 9, 2008). 

[13] The implementation statement provides a summary description of the 
voting system's capabilities, including identification of voting system 
hardware and software components, version numbers, and dates. It must 
also include a checklist identifying all of the VVSG requirements that 
the voting system implements. 

[14] A signature of a file or set of files is produced using a HASH 
algorithm. A file signature, sometimes called a HASH value, consists of 
a value that is computationally infeasible of being produced by two 
similar but different files. File signatures are used to verify that 
files are unmodified from their original versions. 

[15] Hardware is commonly verified by identifying the model and 
revision numbers of the system's printed wiring boards and major 
subunits, and comparing the system with detailed photographs of the 
printed wiring boards and internal construction of the unit that was 
tested and approved. Software is typically verified by using a self- 
booting compact disk or similar device to verify the file signatures of 
the system application files and of all nonvolatile files that the 
application files access during operation, and comparing against the 
file signatures of the original executable files that were placed in 
the software repository. 

[16] NIST, Conformance Testing and Certification Model for Software 
Specifications, Lisa Carnahan, Lynne Rosenthal, and Mark Skall, March 
1998; and ISO/IEC, Guide 65: 1996 (E) General Requirement for bodies 
operating product certification systems, First Edition; and Guide 60: 
2004 (E) Conformity Assessment - Code of good practice, Second Edition. 

[17] 42 U.S.C. ï¿½ 15371. 

[18] NIST, Conformance Testing and Certification Framework, Lisa 
Carnahan, Lynne Rosenthal, and Mark Skall, April 2001; Conformance 
Testing, Martha Gray, Alan Goldfine, Lynne Rosenthal, and Lisa 
Carnahan; and Conformance Testing and Certification Model; and ISO/IEC, 
Guide 65: 1996 (E); Guide 60: 2004 (E); and Guide 7: 1994 (E) 
Guidelines for drafting of standards suitable for use for conformity 
assessment, Second Edition. 

[19] EAC has not yet established a date for finalizing these 
guidelines. 

[20] According to the Program Director, the program has requested 
additional certification resources for fiscal year 2009. 

[21] In a May 2008 correspondence to state election officials, EAC 
cited several issues that have contributed to delays in certifying 
voting systems, such as manufacturers and VSTLs having to adjust to the 
demands of the commission's program and the lack of standardized test 
methods. 

[22] The trusted build produces a file signature of the executable code 
as well as file signatures of the installation disk(s). A file 
signature is produced using a HASH algorithm, sometimes called a HASH 
value, which creates a value that is computationally infeasible of 
being produced by two similar but different files. These file 
signatures are deposited into EAC's designated repository. Election 
jurisdictions can refer to the file signatures of the installation 
disk(s) to validate the software before installing it on the voting 
system. 

[23] According to Section 7.4.6 of Volume I of the 2005 VVSG, the 
verification process should be able to be performed using commercial 
off-the-shelf software and hardware available from sources other than 
the voting system manufacturer. If such tools are commercially 
available, then election officials will be able to independently 
acquire the tools needed to verify that their systems are unmodified. 

[24] ISO/IEC, Guide 65: 1996 (E); Guide 67: 2004 (E) Conformity 
assessment - Fundamentals of product certification, First Edition; 
Guide 28: 1982 (E) General rules for a model third-party certification 
system for products, First Edition; and Guide 53: 2005 (E) Conformity 
assessment - Guidance on the use of an organization's quality 
management system in product certification, Second Edition; and ISO, 
Guide 27: 1983 (E) Guidelines for corrective action to be taken by a 
certification body in the event of misuse of its mark of conformity, 
First Edition. 

[25] EAC is responsible for, among other things, serving as a national 
clearinghouse and resource for the compilation of information by, for 
example, carrying out duties related to the testing, certification, 
decertification, and recertification of voting system hardware and 
software. 42 U.S.C. ï¿½ 15322. 

[26] For fiscal year 2009, EAC requested $16,679,000 in appropriated 
funds for salaries and expenses. Of this amount, $4,202,000 was for 
personnel compensation providing for 37 full-time-equivalent (FTE) EAC 
staff salaries. The 37 FTEs include 29 full-time-permanent employees 
and 8 experts, consultants, contract employees, students, and other 
part-time employees. 

[27] According to 5 C.F.R. ï¿½ 304.103(c)(2)(i), an agency may reappoint 
an expert or consultant, with no limit on the number of reappointments, 
as long as the individual is paid for no more than 6 months of work in 
a service year. 

[28] According to HAVA, EAC shall serve as a national clearinghouse and 
resource for the compilation of information, including information on 
the experiences of state and local governments in, for example, 
operating voting systems. 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability.  

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates."  

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548:  

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061:  

To Report Fraud, Waste, and Abuse in Federal Programs:  

Contact:  

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: [email protected]: 
Automated answering system: (800) 424-5454 or (202) 512-7470:  

Congressional Relations:  

Ralph Dawn, Managing Director, [email protected]: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548:  

Public Affairs: 

Chuck Young, Managing Director, [email protected]: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: 

*** End of document. ***