The importance of ballot design broke into the national conversation in the 2000 U.S. Presidential election, when the vote in Palm Beach County, FL, may have tilted a race that had been in limbo for 36 days after election night.

Many researchers believe that the oddity of the so-called “butterfly ballot” may have misled the intention of a few voters in that race and — potentially — affected the results of a county that ultimately chose the nation's president.

“A typical election ballot consists of many races, and the way you design such a ballot can influence the outcome of an election,” said Alessandro Arlotto, an associate professor at Duke University’s Fuqua School of Business.

Past research has proven the existence of “ballot cues” that influence the vote of some less informed voters, Arlotto said.

For example, there may be a bias for the candidate listed first in a race, or for the incumbent, or for some demographic affinities with the voter, such as the candidate's gender or perceived ethnicity of their name.

Similarly, the way you design the ballot across different races may play a role, he said. For example, the order of the candidates in the presidential race may induce some voters to choose specific “down-ballot” candidates that lack party labels, just because of an order bias.

“Suppose your party's candidate is listed second on the ballot in some of the races, but on the same ballot there are races that list candidates without party affiliation,” Arlotto said. “After you voted for your party’s candidate in the presidential race and other partisan races, which all happen to be listed second, who would you vote for, if you encounter a race where there is no party affiliation?”

Arlotto says the rules overseeing North Carolina elections in the last 20 years have created a mix of ballot-design changes that ended up with many instances of “flipped” candidate party orders.

In the working paper, “Ballot design and electoral outcomes: the role of candidate order and party affiliation,” Arlotto and his Fuqua colleagues Alexandre Belloni and Saša Pekeč — together with Fei Fang of Yale University — examined public North Carolina election data from 2004 through 2020.

Until 2016, the order of the US presidential candidates in the NC ballots followed different rules than the order of the candidates for some judicial races, Arlotto said, and many of the judicial races didn’t label the party affiliation of the candidates.

The researchers studied judicial races with and without party labels. Because the party affiliation of a judicial candidate can be inferred from external sources (such as public endorsements or public statements), the authors were able to exactly determine which judicial races had a party-order flip relative to the presidential race in contests with and without party labels. 

Their hypothesis was that the design of a ballot with many races matters and that the flipped party order may have misled some voters into voting for a candidate of the opposite party, Arlotto said.

If the percentage of voters who picked a Republican presidential candidate doesn’t align with the percentage of votes for a Republican judicial candidate (after accounting for the fact that some voters might naturally split tickets), chances are some of the judicial votes may have been a mistake, he said

To single out the unique effect of the flipped party order, the researchers had to neutralize — “control for” — other variables that influence the vote, Arlotto said.

“It is very hard to measure directly how much each voter knows about each race. Therefore, we consider more than six thousand potential variables,” he said.

The researchers used machine learning tools to help parse out their dataset of 69,161 observations of judicial vote outcomes for each party. 

Because ballot design rules changed many times between the election cycles of 2004 and 2016, the researchers had a substantial amount of information about different voting patterns within the same precinct. Arlotto said this allowed them to estimate the proportion of votes that can be attributed only to the flipped party order of candidates.

The researchers also used the technique of “double machine learning,” in which they compared vote share predictions with the actual outcomes and combined them with the NC election law changes to determine the causal effect of a party order flip. 

In the end, they estimated that a party order flip caused about 13% of the votes in the North Carolina judicial races between 2004 and 2016 to be mistakenly cast for the opposing party’s judicial candidate. 

Conversely, they also found that when party affiliation was introduced for judicial races in the 2016 and 2020 ballots (while maintaining party order flips), the effect disappeared.

“There is a detectable pattern that suggests that when there is a party order flip and there are no party labels, some people are going to assume that their party candidate is the one in the same position as the candidate in the presidential race,” Arlotto said.

Why ballot design is relevant

In the specific case of North Carolina, most of the voting mistakes from voters of opposing parties may have canceled each other out, Arlotto said, because of the two main parties being almost evenly split in statewide vote share.

“However, voting mistakes might not cancel out forever,” he said. ”If a ballot design can mislead approximately 13% of voters into mistakenly casting their vote for a candidate they did not intend to support, that factor could potentially influence the outcome of entire elections.”