Elections: Status of GAO's Review of Voting Equipment Used in
Florida's 13th Congressional District (03-AUG-07, GAO-07-1167T).
In November 2006, about 18,000 undervotes were reported in
Sarasota County in the race for Florida's 13th Congressional
District. Following the contesting of the election results in the
House of Representatives, the Task Force met and unanimously
voted to seek GAO's assistance in determining whether the voting
systems contributed to the large undervote in Sarasota County. On
June 14, 2007, we met with the Task Force and agreed upon an
engagement plan, which included the following review objectives:
(1) What voting systems and equipment were used in Sarasota
County and what processes governed their use? (2) What was the
scope of the undervote in Sarasota County in the general
election? (3) To what extent were tests conducted on the voting
systems in Sarasota County prior to the general election and what
were the results of those tests? and (4) Considering the tests
that were conducted on the voting systems from Sarasota County
after the general election, are additional tests needed to
determine whether the voting systems contributed to the
undervote? In our meetings with Sarasota County, we learned the
entire process of configuring the election, running the election,
and tallying the results, and about the testing the county
conducts on the voting systems, such as the logic and accuracy
testing. In our meetings with the Division of Elections, we
discussed the conduct of certification testing, in particular,
the testing conducted on the ES&S system used in Sarasota County,
and the conduct of the state audit--how decisions were made to
conduct the audit and the processes used to conduct the audit. In
addition, we have received and are reviewing and analyzing data
and documentation received from both sources, as well as the
submissions from the contestant and the contestee provided by the
Task Force.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-1167T
ACCNO: A73907
TITLE: Elections: Status of GAO's Review of Voting Equipment
Used in Florida's 13th Congressional District
DATE: 08/03/2007
SUBJECT: Elections
Operational testing
Strategic planning
Voting
Voting records
Electronic Voting System
Florida
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-1167T
* [1]Summary
* [2]GAO's Mission
* [3]Obtaining Copies of GAO Reports and Testimony
* [4]Order by Mail or Phone
* [5]To Report Fraud, Waste, and Abuse in Federal Programs
* [6]Congressional Relations
* [7]Public Affairs
Statement
Before the Task Force on Florida-13, Committee on House Administration,
House of Representatives
United States Government Accountability Office
GAO
For Release on Delivery
Expected at 10:00 a.m. EDT
Friday, August 3, 2007
ELECTIONS
Status of GAO's Review of Voting Equipment Used in Florida's 13th
Congressional District
Statement of Dr. Nabajyoti Barkakati
Senior-Level Technologist
Center for Technology and Engineering
Applied Research and Methods
GAO-07-1167T
Chairman Gonzalez, Ms. Lofgren, Mr. McCarthy,
I am pleased to appear before the Task Force today to update you on the
progress of our review of voting equipment used in Florida's 13th
Congressional District, which we are conducting in response to your
request of May 25, 2007. I want to thank the Task Force for its continued
support of our efforts. We have accomplished a lot in the past few weeks,
but we still have several work items to complete before we can formally
draw any conclusions.
In November 2006, about 18,000 undervotes were reported in Sarasota County
in the race for Florida's 13th Congressional District.1 Following the
contesting of the election results in the House of Representatives, the
Task Force met and unanimously voted to seek GAO's assistance in
determining whether the voting systems contributed to the large undervote
in Sarasota County. On June 14, 2007, we met with the Task Force and
agreed upon an engagement plan, which included the following review
objectives: (1) What voting systems and equipment were used in Sarasota
County and what processes governed their use? (2) What was the scope of
the undervote in Sarasota County in the general election? (3) To what
extent were tests conducted on the voting systems in Sarasota County prior
to the general election and what were the results of those tests? and (4)
Considering the tests that were conducted on the voting systems from
Sarasota County after the general election, are additional tests needed to
determine whether the voting systems contributed to the undervote?
To conduct our work, we visited Sarasota County twice, most recently 2
weeks ago, and we were in Tallahassee last week to meet with the Secretary
of State and the Division of Elections. While in Tallahassee, we were able
to execute a nondisclosure agreement that permitted us access to items
that the State of Florida and the manufacturer of the voting system,
Election Systems and Software (ES&S), considered proprietary, including
the proprietary appendixes of the Florida State University source code
review report; the technical data package, which includes items such as
the software specification; and the source code for the firmware installed
in the iVotronic touchscreen voting systems used in Sarasota County. We
are currently working on a separate nondisclosure agreement to access
technical and testing information from ES&S directly.
1Undervotes are votes for fewer choices than permitted. In this case, it
means ballots that did not record a selection for either candidate in the
congressional contest.
In our meetings with Sarasota County, we learned the entire process of
configuring the election, running the election, and tallying the results,
and about the testing the county conducts on the voting systems, such as
the logic and accuracy testing. In our meetings with the Division of
Elections, we discussed the conduct of certification testing, in
particular, the testing conducted on the ES&S system used in Sarasota
County, and the conduct of the state audit--how decisions were made to
conduct the audit and the processes used to conduct the audit. In
addition, we have received and are reviewing and analyzing data and
documentation received from both sources, as well as the submissions from
the contestant and the contestee provided by the Task Force.
Summary
We have identified the voting systems and equipment used in Sarasota
County and verified that the systems were approved for use by the Florida
Division of Elections. We know that nine different ballot styles were used
on the iVotronic touchscreen voting systems and have an understanding of
how the ballots were configured and loaded onto the machines. Further, it
was also explained to us how votes are tallied and certified, including
the conduct of the machine and manual recounts.
We have been analyzing the detailed ballot results from the election as
well as the incident and technician logs from Sarasota County to identify
patterns in the undervote. Specifically, we have examined the undervote by
machine, precinct, and ballot style. Patterns in the undervote could
provide us insight on specific conditions that could have caused the
undervote. However, we have not yet noticed any apparent patterns, but we
are continuing our analysis. From our analysis, we have been able to
verify that 1,499 iVotronic voting systems recorded votes in the 2006
general election and the vote counts for the contestant, contestee, and
undervotes match the vote totals for election day, early voting, and
provisional ballots in the Florida-13 race. A total of 17,846 undervotes
were recorded in the Florida-13 race out of the 119,919 ballots cast using
the iVotronic voting systems--corresponding to a 14.88 percent undervote
rate.2
2Because the absentee ballots were not cast using iVotronic voting
systems, we did not verify the absentee ballot counts. When absentee
ballots are included, a total of 142,532 ballots were cast and a total of
18,412 undervotes were recorded.
While we have not yet completed our review of all of the testing efforts
to determine whether they provide reasonable assurance that the machines
properly reflect in their totals the selections made when the ballot is
cast, there are some preliminary observations we can make.
A variety of testing is needed to obtain reasonable assurance that this
objective is accomplished, including ballot testing, load testing, and
environmental testing.3 As agreed with you, our efforts will review the
testing that has already been completed, including tests conducted by the
State of Florida (certification testing), Sarasota County (logic and
accuracy testing), and the equipment manufacturer. We are also reviewing
the tests conducted as a part of the state audit, including parallel
testing, the examination of Sarasota County's election practices, and the
Florida State University source code review. Once we complete our review
of the testing efforts, we will identify the potential benefits associated
with conducting any additional tests--how they will help us understand
whether the system contributed to the undervote issue--and the resources
needed to conduct such tests.
So far, we have focused our efforts on two types of tests--ballot testing
and load testing. With between 28 and 40 contests on the Sarasota County
ballots in the 2006 general election, the number of possible voting
combinations is over 100 trillion. Accordingly, it is unrealistic to
expect that all possible vote combinations can be tested.
We have also examined how the system allowed voters different ways to make
a selection in the Florida-13 race and recognized that these represented
different ways that the voters could indicate their intent in the race. By
taking into account these variations, our analysis has found at least 112
different ways a voter could make his or her selection and cast the ballot
in the Florida-13 race, assuming that it was the only race on the ballot.
Specifically, a voter could (1) initially select either candidate or
neither candidate (i.e. undervote), (2) change the vote on the initial
screen, and (3) use a combination of features to change or verify his or
her selection by using the page back and review screen options. We found
that the Florida certification tests and the Sarasota County logic and
accuracy tests verified 3 ways to select a candidate; and the Florida
parallel tests verified 10 ways to select a candidate--meaning that of the
112 ways, 13 have been tested. We have not yet assessed whether this is
significant.
3For the purposes of this review, ballot testing is a subset of the
functional testing that focuses on the vote selection and casting
functions. This includes testing the different ways in which a voter may
make selections on a ballot and then cast a ballot with the iVotronic
electronic voter interface. For example, the Florida Voting Systems
Standards require the system to allow the user (1) to make a selection for
each contest, and (2) to review the selections made and make any changes
prior to the vote being cast.
Load testing, for the purposes of this review, is the testing performed to
provide reasonable assurance that the voting system can properly handle
the expected volume of voters and ballots that are expected. Florida
certification tests include a test to verify that a precinct count system,
such as the iVotronic, can process at least 9,900 ballots.
According to the Florida Voting System Standards, environmental tests are
intended to simulate exposure to shock and vibration associated with
handling and transportation and to temperature conditions. For example,
voting systems in Florida are to be able to operate in temperature
conditions ranging between 40 and 100 degrees Fahrenheit.
[End footnote.]
A test to determine whether a system can handle the expected volume of
activity is commonly referred to as load testing. We found that ballots
used for load testing during the certification testing were
machine-generated using a testing program built into the iVotronic system,
i.e., users do not touch the screen to make a selection and cast a ballot.
Neither the Florida audit nor Sarasota County's logic and accuracy testing
performed load testing. We have not yet assessed whether this is
significant.
We have also been reviewing the Florida State University source code
review. As we mentioned, we obtained access to the source code last week
and we were able to verify for ourselves some of the items discussed in
its report. We have had prior discussions with the leader of the Florida
State review team and will be continuing our discussions with the review
team and the manufacturer to ensure our understanding of both the findings
of their review and the operations of the iVotronic system. One of the
items noted in the report was that the review team did not (1) convert the
source code to object code, and (2) compare the resulting object code to
the object code that was used to run the voting machines in Sarasota
County.4 We are still assessing the significance of this item.
As a part of our review of the state audit, we examined the selection of
samples for the parallel testing and the review of the Sarasota County
election practices. Our preliminary analysis has found that these sample
sizes are too small to support generalization of the results to the
overall population. For example, the generalization of the results from
the use of 10 machines for parallel testing cannot be supported because
the sample drawn was not random and the sample size was too small.
Similarly, we have little assurance that the examination of 6 machines'
firmware is adequate to conclude that the firmware was not compromised on
any of the machines. Our discussions with Florida officials indicate that
such limitations resulted from court-imposed restrictions on machine
access and resource considerations of performing the testing.
4According to the Institute of Electrical and Electronics Engineers,
source code contains computer instructions and data definitions expressed
in a form suitable for input to an assembler, compiler, or other
translator that generates the object code. Object code contains the
computer instructions and data definitions expressed in a form that can be
recognized by the processing unit of a computer.
It is important to bear in mind that these are just our preliminary
observations. It is not clear to us yet whether these are items we think
will need to be tested; but they are items we have noticed while we are
reviewing the previously completed test activities. As we previously
discussed, for any testing issues we identify, we plan to determine how
relevant and significant the issue is and the resources needed to conduct
such tests. Our identification of resources will include test personnel
and equipment, the voting systems and equipment to be tested, and the time
required to conduct such tests. For example, as we have discussed, one of
the issues we identified in the source code review is that the source code
was not converted to object code and compared to ensure that it
represented the code used in Sarasota County. Further, our preliminary
analysis has shown that we do not have reasonable assurance that the
firmware was not compromised on any of the iVotronic systems used during
the election. In order to determine whether these issues warrant further
testing, we still need to determine the potential significance of these
issues, as well as identify the test personnel and equipment, the voting
systems and equipment to be tested, and the time required to conduct such
tests. To identify these resources, it will also be important to determine
how such tests should be structured and executed.
Besides conducting such resource analyses, we still have several
activities to complete with regard to testing. First, we have not yet
evaluated the testing conducted by the system manufacturer, and second, we
are still in the process of identifying other appropriate tests that could
be used to determine whether the voting systems caused the undervote (for
example, the effects of provisional ballots and environmental conditions).
Mr. Chairman, this completes my prepared statement. I would be happy to
respond to any questions you or the other members of the Task Force may
have at this time.
For further information about this testimony, please contact Keith Rhodes,
Chief Technologist, at (202) 512-6412 or [8][email protected] , or Naba
Barkakati at (202) 512-4499 or [9][email protected] .
(460592)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. The published product may be reproduced
and distributed in its entirety without further permission from GAO.
However, because this work may contain copyrighted images or other
material, permission from the copyright holder may be necessary if you
wish to reproduce this material separately.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( [10]www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
[11]www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000
TDD: (202) 512-2537
Fax: (202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: [12]www.gao.gov/fraudnet/fraudnet.htm
E-mail: [13][email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [14][email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [15][email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
References
Visible links
8. mailto:[email protected]
9. mailto:[email protected]
10. http://www.gao.gov/
11. http://www.gao.gov/
12. http://www.gao.gov/fraudnet/fraudnet.htm
13. mailto:[email protected]
14. mailto:[email protected]
15. mailto:[email protected]
*** End of document. ***