What Happened in Sarasota County?

The cause(s) of the undervote in the 2006 congressional race in Sarasota County, Florida, are still a mystery.

On November 7, 2006, there was an electoral disaster in Sarasota County, Florida. Almost 18,000 people, about one in seven of the people who voted electronically, left the polls without recording a vote in the congressional race, the hottest race on the ballot. Most observers agree that few of these voters deliberately skipped voting in that race. Instead, they were either misled into not seeing that race or the voting machines somehow failed to record their votes. There is consensus on one point, however. Although Republican Vern Buchanan was certified the winner by only 369 out of more than 238,000 votes and is now representing the 13th Congressional District of Florida (CD13) in the U.S. House of Representatives, if the “missing” votes had been recorded, Democrat Christine Jennings would almost certainly have been elected (Stewart, 2006).

This election illustrates in dramatic fashion not only the complex problems that arise with the use of all-electronic voting systems, but also the deep concerns of computer scientists and security experts about total reliance on software to capture and count votes in public elections. A considerable amount of technical investigation has been done into the circumstances of this election, and many hypotheses have been eliminated. But to date (April 2007), the exact cause(s) are not known with complete certainty—indeed, they may never be known.

CD13 is comprised of four counties (Sarasota, Manatee, Hardee, and Desoto) and part of a fifth (Charlotte). The district as a whole has leaned Republican in recent years, although the most populous county, Sarasota, where the problem occurred, leans Democratic. There was no incumbent in the 2006 CD13 election, and both candidates for the seat had won very bitter primaries against multiple opponents. For these reasons, and because control of the House of Representatives was considered a toss-up between the two major parties, the race was hotly contested, and the candidates had blanketed the media with massive advertising. The race cost $13.1 million, including the costs of the primary contests, making it the single most expensive House race in U.S. history (Wallace, 2007).

Statistical Analysis
Sarasota County used iVotronic voting machines built by Election Systems and Software, the largest voting system vendor in the United States. The iVotronic, which has a touch-screen interface (Figure 1), is one of a class of voting machines, paperless, direct-recording electronic devices (DREs), that was widely adopted after the 2000 presidential election. Since then, paperless DREs have been the target of widespread criticism from technical experts, primarily because the vote counts they produce are not verifiable or auditable in any meaningful way.

Table 1 shows certified election returns from Sarasota County for the electronic votes cast in the three top races on the ballot—U.S. senator, U.S. representative, and governor. The table shows just the number of electronic votes for each race and each candidate (including early votes and those cast on election day), along with the number of undervotes (i.e., ballots that did not show a vote for any candidate in that race). Note that only the returns for Sarasota County are shown. When the other counties in the district are included, the final count shows Buchanan winning by 369 votes.

The key statistic in Table 1 is the percentage of electronic undervotes in the congressional race, shown in the far right column. The undervotes for the senate and gubernatorial races were 1.2 and 1.4 percent, respectively, which is typical for top-of-ballot races nationwide. The undervote in the congressional race was more than 10 times higher—14.9 percent of the electronic votes. The undervote rate, which varied from precinct to precinct, was almost 30 percent in some areas.

In a statistical analysis of the pattern of undervotes in Sarasota County, Professor Charles Stewart III of MIT estimated that the number of undervotes beyond the number expected was between 13,209 and 14,739. He further projected that, if those votes had been successfully cast and counted, Jennings would have won the election “by at least 739 votes, and possibly by as many as 825 votes” (Stewart, 2006). No one, not even Buchanan’s team, has disputed that conclusion, adding drama and political significance to what might otherwise be just a technical puzzle.

FIGURE 1 The iVotronic voting machine.

What Went Wrong?
Why did the number of electronic voters who did not record a vote in this race exceed expectations by about 18,000? Many hypotheses have been proposed, some of which can be discarded almost immediately. Election officials in Sarasota County initially suggested that most of those voters had deliberately not voted in the CD13 race, perhaps to protest the ugliness of the campaign or because they disliked both candidates. There is strong evidence, however, that this is not what happened.

If voters in general had been “turned off” by the campaigns or the candidates, we would expect that absentee and provisional voters (who voted on paper ballots) would have a similarly high undervote. But among absentee and provisional voters in Sarasota County the undervote rate was about 2.5 percent in the congressional race. The same was true for all voters in the other counties in CD13, who were exposed to the same campaigns. The undervote in those counties in the congressional race was in the normal range, less than 2 percent, nothing like the 14.9 percent in Sarasota County. These numbers strongly suggest that, whatever the problem was, it was specifically related to the votes cast in Sarasota County on the iVotronic machines.

Three primary hypotheses for the cause of the problem have been advanced:

1. Malicious Code: The software in the iVotronic machines might have contained logic that was deliberately designed not to record some votes in a way that would benefit Buchanan. (This would constitute election fraud.)
2. A Software Bug: The software (or hardware) in the iVotronic machines may have had a subtle bug of some kind that caused some votes to go unrecorded.
3. Ballot Layout: The layout of the ballot on the screen was misleading in a way that caused many voters to inadvertently skip the congressional race

The rest of this paper is devoted to reviewing the evidence for and against each of these hypotheses and to a general discussion of the underlying security issues in electronic voting illustrated by the Sarasota County election.

The Malicious Code Hypothesis
The most unsettling hypothesis is that malicious logic, which caused the machines not to record the missing votes, was somehow injected into the iVotronic code in Sarasota County. Security experts consider insider attacks in general, and malicious code in particular, a grave danger because of the potential damage of such attacks and the difficulty of detecting them, let alone repairing the damage. In this specific case, however, no evidence of malicious code has been found. In fact, several lines of evidence indicate that malicious code was probably not involved.

A malicious code attack designed to produce a large undervote is not very likely, because it would certainly be detected and would thus be less effective than a more direct, less noticeable attack, such as switching votes from one candidate to another. Still, the possibility of malicious code cannot be dismissed without further investigation.
____________________________

So far, no evidence of
malicious code has been
found in Sarasota County
voting machines.
___________________________

The Florida Department of State conducted a variation of a test protocol known as “parallel testing” first used in California that was designed specifically to detect malicious code in voting systems (FDOS, 2006). The test is based on the idea that the primary problem facing a would-be attacker is to design the attack code to cheat during a real election but avoid detection by conventional functionality testing during federal or state certifications of the voting system or during the immediate pre- and post-election “logic and accuracy” testing. Parallel testing involves putting a random sample of voting machines through a mock election that is so much like a real election, even down to the statistical pattern and timing of the votes cast, that the potentially malicious software has no possible cue it is undergoing a test. Thus the software is tempted to cheat as it would in a real election. A very large number of variables have to be carefully controlled to avoid giving the potentially malicious software a clue that it is being tested. But if, at the end of the test, the vote totals from the machines match the mock votes that were cast on them, simple malicious code attacks can probably be ruled out.

The results of parallel testing by the Florida Department of State were clear. No unexplained anomalies were found, and hence no evidence of malicious code was discovered, either in a sample of the machines used in the election (and kept under seal by court order) or in a sample of identical machines prepared for testing as they would have been for the real election. Although the test protocol has been criticized for some imperfections (Dill and Wallach, 2007), the test as conducted does set a lower bound on the sophistication of any malicious code that might have been responsible for the undervote problem.

The Software Bug Hypothesis
Voting machines have much more complex software than one might imagine given the apparent simplicity of their vote-collecting function. The iVotronic probably has on the order of 100,000 lines of code. Thus it is quite possible that a subtle program bug might have contributed to the undervote problem. For example, a bug might have caused the congressional race not to be presented on the screen to some voters; or it might have failed to notify voters on the summary screen that they had not voted in that race; or it may simply have failed to record some votes. Any reasonable hypothesis must, of course, be consistent with the facts observed in the election and must explain, for example, why only the congressional race appeared to be affected.

To investigate the possibility that software errors may have caused the problem, the Florida Department of State engaged the Security and Assurance in Information Technology (SAIT) Laboratory at Florida State University in Tallahassee. The lead investigator, Alec Yasinsac, a professor of computer science, assembled a panel of nationally prominent software, security, and election experts to study the hardware, software, audit logs, and other data of the CD13 race. On February 23, 2007, the panel issued a 67-page technical report in which numerous flaws, bugs, and vulnerabilities found in the software were discussed. The key summary paragraph of the report follows (Yasinsac et al., 2007):

The team’s unanimous opinion is that the iVotronic firmware, including faults that we identified, did not cause or contribute to the CD13 undervote. We base this opinion on hundreds of hours of manual code review complemented by automated static analysis and extensive study of the problem symptoms and the execution environment. We traced program execution from terminal initialization, through voter selection, to ballot image creation, to ballot image collection. We also investigated the possibility of asynchronous system faults not associated with any particular phase of voting. Our investigation provided no evidence that an iVotronic software malfunction caused or contributed to the CD13 undervote. [Emphasis in the original]

The panel acknowledges, of course, that no study could absolutely rule out the possibility that a very subtle bug had escaped notice. But extraordinary pains had been taken to look for all conceivable problems in the source code, and none had been found. The conclusion also indicates that no evidence of malicious code was found, which is consistent with the finding from the parallel testing.

Since the publication of the SAIT report, two other prominent computer scientists and election experts, Dan Wallach and David Dill, have published a paper detailing a number of limitations of the SAIT study and the parallel testing conducted by the Florida Department of State (Dill and Wallach, 2007). They call for further investigation to address a number of concerns, such as limitations in the parallel testing protocol, verification that the binary code actually loaded on the iVotronic machines used in the election was consistent with the source code examined by the SAIT team, and further examination of touch-screen calibration procedures and of a previously known bug in the “smoothing filter” of the iVotronic touch-screen driver. (The smoothing filter filters out anomalous touch-screen inputs, such as finger bounces and multi-finger touches.)

The Ballot Layout Hypothesis
The layout of the ballot, on paper or on screen, can greatly influence voter behavior. Candidates are well aware that whoever is listed first in a race has a measurable advantage. The format of the infamous “butterfly ballot” in Palm Beach County, Florida, in the 2000 election led many voters who intended to vote for Al Gore to mistakenly vote for Pat Buchanan. Various almost unknown candidates in the gubernatorial recall election in California in 2003 got an unexpectedly large number of votes because their names on the multi-column ballot happened to be next to that of Arnold Schwartzenegger.

In hindsight, the layout of the ballot in the Sarasota County election looks especially problematic. Figure 2 shows the first two pages, with the U.S. Senate race on the first page and the congressional and gubernatorial races on the second. To understand why this layout might be confusing, we must remember that even people who never miss an election only vote about once a year for about 10 minutes. Thus, from one election to the next, voters do not generally remember the layout and interaction conventions in a voting machine; they must relearn them each time.

FIGURE 2a First page of the Sarasota County CD13 ballot.

FIGURE 2b Second page of the Sarasota County CD13 ballot.

Furthermore, there is likely to be a certain amount of cognitive interference from point-and-click interfaces they may be familiar with. On the iVotronic screen, there are no menus, icons, windows, or scrolling—just active areas that are not even surrounded by strong borders. It is not even visually apparent whether the voter is supposed to touch the candidate’s name on the screen or touch the box to the right of the candidate’s name.

The voter also has to visually “parse” the screen to find the races and candidates. What visual cues indicate that one race has ended and another has begun? A page (screen) boundary? A big, background-colored heading? A thin horizontal line? Are races split over pages? After a few pages voters may begin to intuit the answers through experience. But in the first few races, before the patterns have been established, some voters may make mistakes.

Based on the first two pages of the ballot shown in Figure 2, many observers argue that the congressional race on page two appears, at first glance, not as a separate race at all, but as a continuation of the U.S. Senate race on the previous page. The congressional race is sandwiched between two major statewide races, both of which are introduced by large, colored headings. But the congressional race does not have such a heading. With careful reading, the ballot is clear. But voters nervously trying to learn the interface, or voters who are more visually than textually oriented, might not take the time to process all of the text and, therefore, might miss voting in the congressional race.

How can we determine for certain whether the ballot layout hypothesis is the explanation for the undervote? Unfortunately, there is no direct, controlled way to test the hypothesis. One could try conducting a number of mock elections with the same races laid out on the ballot in different ways using test voters demographically similar to the ones in Sarasota County and then compare the undervote rates. But if the test voters were from Sarasota County, they would probably guess what was being tested, which would completely undermine the experiment. If one controlled for that factor and chose test voters who were unfamiliar with the Sarasota County candidates and the outcome of the real election, the degree of motivation among Sarasota voters to vote in the CD13 race as a result of being exposed to the heated campaigns would not be replicated.
___________________________

Ballot layout now seems the
most likely explanation for
the undervote.
___________________________

In the absence of experimental verification of the ballot layout hypothesis, we are left in an unsatisfying situation. Although ballot layout now seems to be the most likely explanation for the CD13 undervote, this conclusion is based more on arguments of plausibility and the elimination of other hypotheses than on direct evidence in its favor.

Conclusion: What Should Be Done?
Some of the serious technical problems in all-
electronic voting systems are illustrated in the Florida CD13 election in Sarasota County. In the Sarasota County case, an enormous amount of time and technical expertise have been spent to determine that software errors were (probably) not responsible for the anomalous outcome. Clearly, an election system that requires this much work to resolve a disputed election is unsustainable. To avoid these problems in the future, voting systems must be transparent, and they must produce results that are verifiable, so that even in a close election, the results are indisputable and even the losers agree with their validity.

At least two technical conclusions can be drawn from the CD13 experience. First, much more attention must be paid to the human-computer interface of voting machines. Representatives of both parties had approved the ballot layout used in the CD13 election, and yet the problem occurred. As yet no systematic body of experience, conventions, or standards have been developed to ensure reliable human interaction with computerized voting machine interfaces. (This is also an especially serious problem with audio interfaces for the disabled.) This is an area ripe for research.

Second, and of deeper concern, is the intrinsic difficulty of building software that is verifiably correct, reliable, and secure for such profoundly important missions as public elections. We clearly need voting systems and procedures that can produce verifiably accurate election results even in the presence of software errors or malicious code.

One recent explication of this idea, the notion of “software independence,” is under consideration by the Technical Guidelines Development Committee, the panel that recommends technical standards for voting to the Election Assistance Commission. A voting system is software-independent “if an undetected change or error in its software cannot cause an undetectable change or error in an election outcome” (Rivest and Wack, 2006).

A software-independent system would not, in and of itself, have resolved the mystery of the CD13 election, which centers on votes that were not recorded. But it could prevent or resolve disputes in elections in which votes are alleged to be incorrectly recorded or counted.

Even with voting systems designed on the principle of software independence, however, we cannot rely solely on software to determine the outcome of an election. There must be an alternative, end-to-end data path, independent of software, such as a voter-verified paper audit trail, that can be used as a check on the validity of the electronic results.

References
Dill, D., and D. Wallach. 2007. Stones Unturned: Gaps in the Investigation of Sarasota’s Disputed Congressional Election. April 13, 2007. Available online at http://www.cs.rice.edu/~dwallach/pub/sarasota07.html.
FDOS (Florida Department of State). 2006. Parallel Test Summary for Sarasota County. December 18, 2006. Available online at http://election.dos.state.fl.us/pdf/parallelTestSumReprt12- 18-06.pdf.
Florida State University, February 23, 2007. Available online at http://election.dos.state.fl.us/pdf/FinalAudRepSAIT.pdf.
Rivest, R., and J. Wack. 2006. On the Notion of ‘Software Independence’ in Voting Systems. July 2006. Available online at http://vote.nist.gov/SI-in-voting.pdf.
Stewart, C. III. 2006. Declaration of Charles Stewart III on Excess Undervotes Cast in Sarasota County, Florida, for the 13th Congressional District Race, December 2006. Available online at http://www.heraldtribune.com/assets/pdf/SH81421120.PDF.
Wallace, J. 2007. District 13 was costliest race ever. The Herald Tribune, January 18, 2007. Available online at http://www.heraldtribune.com/apps/pbcs.dll/article?Aid= /20070118/NEWS/701180378/-1/NEWS0521.
Yasinsac, A., D. Wagner, M. Bishop, T. Baker, B. de Medeiros, G. Tyson, M. Shamos, and M. Burmester. 2007. Software Review and Security Analysis of the ES&S iVotronic 8.0.1.2 Voting Machine Firmware. Security and Assur-ance in Information Technology Laboratory, Florida State University, February 23, 2007. Available online at http://election.docs.state.fl.us/pdf/


TABLE 1 Electronic Vote Counts and Undervote Rates for Three Top Races in Sarasota County

	Total Electronic Votes	Electronic Undervote
United States Senator
Katherine Harris	47,899
Bill Nelson	68,376
4 Others plus write-ins	2,092
Undervotes	1,392
		1.2%
U.S. Representative
Vern Buchanan	47,509
Christine Jennings	54,439
Undervotes	17,811
		14.9%
Governor & Lieutenant Governor
Charlie Crist	62,825
Jim Davis	51,380
4 others plus wrtie-ins	3,889
Unvervotes	1,665
		1.4%
Total of electronic ballots cast	119,759