A person approaches the table and picks up a 10-sided die. She rolls.
Four.
The next roller in line takes her turn.
Seven.
The unusual die, shaped like an elongated diamond, may seem a curiosity except to those who use it most: “board gamers and elections officials,” Jonathan Marks, Pennsylvania’s deputy secretary of state, said.
But the rollers, employees of the Pennsylvania Department of State, are not casting their dice to determine their next move in Dungeons & Dragons. Instead, they are generating a long, random number that will determine the course of Pennsylvania’s 2022 risk-limiting audit.
A risk-limiting audit is a type of post-election review designed to give statistical confidence that an election outcome was accurate. This year is the first when all 67 counties are required to participate in one of these audits before certifying their election results.
The math used to conduct the audit is available to the public, though practitioners agree it is hard to understand. Elections officials who have used it said the audit took time to understand but they now have confidence in it, and they hope it will give the public more confidence as well that election outcomes are accurate.
What is a risk-limiting audit?
As the result of a settlement in a lawsuit brought by Jill Stein, the 2016 Green Party presidential candidate, the state was required to implement risk-limited audits statewide by 2022. But despite the contentious origin, many administrators will now tell you they are a fan of the process.
“At the core, it is a test to confirm that the margin of a race was not overstated,” Marks said. “What we like about risk-limiting audits from an election administrator perspective is … number one it is efficient, and one of the other benefits is it is adaptable. And that is really the big key for me.”
Under Pennsylvania law, counties have long been required after each election to perform an audit of 2% of all ballots or 2,000 ballots, whichever is fewer.
But that style of audit has its limitations. For example, an election administrator could choose to only examine ballots from one precinct. While that review could uncover a problem with that specific precinct or the ballots cast there, it won’t reveal much about the election as a whole.
A risk-limiting audit, on the other hand, is specifically designed to check the margin of a race to ensure that the reported winner actually won. It is also “adaptable”: If a race has a closer margin, the algorithm requires that more ballots be checked to ensure the outcome was correct.
If the margin is wider, fewer ballots need to be checked.
The 2% audit is also only a local audit and not a statewide one, meaning each individual county is performing its own audit that has no bearing on other counties, while the risk-limiting audit is a statewide endeavor that looks at contests as a whole.
“I think county elections officials like it because it only requires them to do as much work as necessary to confirm the outcome of a contest, whereas with a fixed percentage you’re doing the same amount of work irrespective of the facts about that election,” Marks said. “In one county [2%] may be more than enough, and in another county, it may not be enough.”
One fan is Jeff Greenburg, Mercer County’s former longtime elections director and now senior adviser to The Voter Project, a Pennsylvania nonprofit focused on voting access. Greenburg ran one of the state’s first risk-limiting audits in 2019 as part of a pilot program.
“[Getting] a 95% confidence level might mean you only need to count several thousand ballots across the state,” Greenburg said of how the adaptive portion works. “Logically, it doesn’t make sense. That’s one of the questions I remember asking the state back in 2019. ‘How do I explain to people if I count 2,000 ballots versus 200, how do I explain to them that 200 ballots is actually giving me a better number than the 2,000?’ And that’s where the math comes in.”
After going through the process, Greenburg recognized its value, and the Department of State working group he was a part of recommended that the Pennsylvania legislature make the risk-limiting audit statutorily required. The legislature has yet to take up that recommendation.
Only 15 states currently perform risk-limiting audits, and only three of those statutorily mandate the process.
How does it work?
A random 20-digit number, called a “seed,” is generated by rolling 20 separate ten-sided dice numbered zero to nine. This was performed on a live webcast by Department of State employees on Nov. 17.
At this point, uniquely numbered batches of ballots from all counties have been cataloged in the state’s risk-limiting audit software. The seed is then entered into the risk-limiting audit algorithm to determine which ballots need to be checked.
Mark Lindeman is policy and strategy director of Verified Voting, a nonpartisan elections technology organization that helps states including Pennsylvania perform risk-limiting audits. He said while generating the seed number is completely random, once that number is known, anyone could double-check the work.
Since the algorithm is open-source, meaning it is available to the public, anyone could use that information and the seed to ensure that the batches the algorithm dictates be counted were actually the ones counted. Lindeman said this also prevents any administrator from knowing ahead of time, and possibly manipulating, which specific batch will be counted.
Once the batches are known, counties begin their part of the process: a hand count of the ballots in each designated batch to ensure the outcome was accurate. The number of ballots in a batch varies by county, depending on how administrators organize ballots.
“The audit math is tolerant of small discrepancies normally observed in hand count audits,” Lindeman said, like if a ballot is miscounted by a machine due to a stain on the paper. “Small errors in the machine count would have no bearing on the outcome, but large discrepancies that raise doubts about the outcome could make it necessary to conduct further auditing or potentially further investigation, including a full recount.”
This year, the Department of State selected the governor’s race for the audit. Since the margin of victory between Attorney General Josh Shapiro and state Sen. Doug Mastriano was very wide — nearly 15 percentage points — few batches had to be checked to ensure accuracy.
Nonetheless, all counties must be prepared to perform the audit, since it is unknown which batches will be selected.
A spokesperson for the Pennsylvania Department of State said that the seed dictated that 25 batches be selected, which accounted for 10,209 ballots. These batches were spread across seven counties.
In Chester County, one of the state’s largest, elections administrator Karen Barsoum recruited six teams of three auditors each, with at least one Democrat and one Republican on each team.
Barsoum organizes her ballot batches by precinct and vote type (mail-in, in-person, etc.) ahead of the post-election period to ensure a smooth audit process. The teams had seven different pieces of paper in front of them — each representing one of the choices in the gubernatorial race, including write-in votes and instances where voters selected zero or more than one candidate.
“At the beginning, it is unclear how long it will last,” she said of her system Friday morning, just moments before the audit was scheduled to start. “It depends on how many batches the seed tells us to grab.”
For Chester County, it didn’t last very long. No batches from the county were selected by the algorithm.
“We’re disappointed,” Barsoum said with a somewhat dejected but contently resigned smile. She had seemed to be eager to implement her well-organized, color-coded system.
“We were all ready to go.”
The counties that were selected have until Friday to upload their results.
Marks, the deputy secretary of state, understands that the risk-limiting audit will take time for people to understand, but he hopes it will provide Pennsylivians with at least some measure of greater confidence in the process.
“I’m not deluding myself into believing that doing [risk-limiting audits] is necessarily going to convince every skeptic,’’ he said. “But I am hopeful over time, as people see it work and people see it in action, that they’ll understand it. That is certainly my experience.”
Carter Walker is a reporter for Votebeat in partnership with Spotlight PA. Contact Carter at cwalker@votebeat.org.
