This story originally was published by Carolina Public Press.
The N.C. State Board of Elections will consider the results of testing a new kind of post-election audit in North Carolina this week.
The board is scheduled to meet Tuesday to certify the results of November’s municipal elections. As part of that process, the board reviews data and reports from the county and state election officials to make sure they ran they elections fairly and accurately.
This time, the board has an extra tool for reviewing results from the 17 counties that test-ran “risk-limiting audits,” or RLAs, last week. RLAs are “tabulation audits,” or the kind of post-election check using statistical analysis to make sure the machines that read ballots and count votes did so accurately.
The counties participating in the pilot are Beaufort, Brunswick, Buncombe, Carteret, Cleveland, Granville, Harnett, Henderson, Johnston, Mecklenburg, Scotland, Stokes, Transylvania, Union, Watauga, Wayne and Wilkes.
State and county election officials are looking to RLAs to increase public confidence in elections and to improve the efficiency of election administration, though county election officials told Carolina Public Press more work remains before RLAs are likely to achieve either goal in North Carolina.
The state board ran the audit pilots to find ways it can improve the process before moving forward, according to spokesperson Pat Gannon.
North Carolina follows in the footsteps of Colorado, Virginia and Rhode Island, which require RLAs after elections. Another dozen states are piloting RLAs or have made them an option. The audits are recommended by the Senate Intelligence Committee, the National Institute of Standards and Technology, the National Academies of Sciences, Engineering and Medicine, and a bevy of other state, federal and good-governance organizations.
Depending on the RLA report state election staff presents on Tuesday, the board could decide to run another round of pilots in more — or all — counties in the spring primary elections, Gannon said. Alternatively, the board could decide to take no further action.
Dueling audits
North Carolina already checked election results to confirm that vote-counting machines worked properly. By state law, county boards of elections must run these “sample audits,” which are simpler than RLAs but lack their most important features.
The sample audit randomly picks two precincts, which could include an early voting site or by-mail ballots. Then county elections staff members do a hand-eye recount of every ballot in those precincts to make sure they match the tabulated results.
The RLAs take a more holistic approach by randomly sampling ballots from every voting method.
“It does give you a sampling across the board of all your categories: absentee-by-mail, early voting, election day precincts; whereas our sample audit is basically making sure the machine counts correctly,” said Jeff Storey, elections director for Transylvania County, which audited its race for Brevard’s mayor.
Storey wants the state to pick one audit rather than requiring both, as it did for the pilot.
In every election, every county board of elections must run the sample audits before certifying the final election results. This November, the 17 selected counties ran RLAs on their results, in addition to the sample audits, between the time they certified their elections and when the state will vote on certification tomorrow.
The board can require counties to run RLAs in addition to the sample audits, but in order to use the RLAs in place of the sample audits, the legislature would have to change state law.
The board wanted to run these pilots to see how they performed before formally discussing the possibility with the legislature, Gannon said.
“If we’re going to risk-limiting audits, is that better than the currently statutorily required sample audit count, and would it be beneficial from a time-saving standpoint, from a public confidence standpoint and from a statistical standpoint?” Gannon said.
What they learned
Claire Jones, Harnett County’s elections director, said the process for running the RLA was simple. The math behind it was hard to understand.
North Carolina is pursuing RLAs, in part, to boost voter confidence in the administration and correct outcomes of the elections. There will need to be a voter education campaign to teach people about the math behind an RLA and why it is important, Jones said.
The pilot audit was her first time learning about RLAs, and Jones said she is not able to describe the statistics that drive the random sampling of the ballots or the decision that the results are likely accurate. Jones was not alone among election officials, indicating the voter education campaign would likely need to come from the State Board of Elections.
Tonya Burnette, Granville County’s elections director, said the main learning process was figuring out how to manage ballots and run the audit efficiently. Each of the four county election directors Carolina Public Press spoke with said they were concerned about replicating this process in a larger election, though each thought they could if needed.
For Storey in Transylvania County, the biggest lesson was finding potential problems with running both a sample audit and an RLA.
Elections that have a less than 1% margin have to be recounted. Storey said it is an open administrative question whether an RLA would still need to be done after that, or even when it could be done because recounts take so much time.
Election protests were also filed at the same time counties piloted the RLAs for the November election. The 2020 elections saw recounts and election protests across the state, most notably for the chief justice for the Supreme Court race and its razor-thin margin.
If the state board requires RLAs in addition to the legislatively required sample audits, it will need to account for each of these possibilities. It will also need to decide what it will do if an RLA finds a discrepancy that the sample audits do not.
Ready for March … or later
Election officials live in a state of flux, especially in North Carolina, where lawsuits regularly delay elections or call for do-overs.
The state board could decide as soon as Tuesday that all counties need to run RLAs in the March primaries, but the date could also be pushed back by the lawsuits challenging the legislature’s congressional and district maps.
Until RLAs move from a “pilot” status to being codified in a policy or state law, the state board could also change plans at any time. The state board first intended to run RLA pilots in March 2020, but COVID-19 closed election offices and stopped those plans. Then, the board planned to run RLAs for the October 2021 municipal primaries but decided to go with the November elections instead. The board first announced that 15 counties would participate but moved to 17 counties.
Whatever they’re called to do, they will be ready, said Kellie Hopkins, Beaufort County’s elections director.
“Elections professionals have stuff like this thrown at us constantly by the legislature or lawsuits, or a number of things that could cause us to have a workload increase or just a new process in general,” she said. “We’re a pretty flexible bunch, and we’ll get it done, especially if this helps voters feel more confident in the process.”
How to run a risk-limiting audit
A step-by-step guide to how counties ran the pilot RLAs.
- North Carolina uses an open-source tool called Arlo from VotingWorks, a nonprofit organization that makes open-source election technology. This tool guides the statistical analysis and general organization of the risk-limiting audits, taking the pressure of the highly technical process off county and state election officials.
- Arlo requires a random “seed number” and data about the election in order to randomly select ballots to be audited.
- State board staff rolls 10 dice to get a seed number.
- County board of elections staff members have to organize ballots into batches based on the voting method. For example, a county could organize ballots from one election day precinct into 10 stacks of 20 ballots. Each stack would be a “batch.”
- State board staff inputs election results and the “ballot manifest,” i.e., how the ballots are organized in the county.
- Arlo then takes the seed number and election data to randomly select ballots for the RLAs. Ballots are selected from each voting method, including by-mail, early in-person and election day voting.
- The size of this random sample depends on the margin of the vote. During the November pilots, Granville County counted 17 ballots, compared with Transylvania County, which had a much larger and closer race and had to count 679 ballots. These ballots are counted by hand.
- As county election officials pull ballots, they put a placeholder in the stack to keep their place. They label the pulled ballots to identify where they pulled them from.
- Election officials tally the results from the sampled ballots.
- They put the audit results in the Arlo tool, which then tells the state if the sample reflected the results of the contest with a high degree of mathematical confidence, called a “risk limit.”
- If the results don’t meet the risk limit, the Arlo tool tells the county to pull more randomly selected ballots. This happened in Harnett County during the November pilots. In the first round, Harnett counted 15 ballots out of a race with 119 ballots. The sample did not meet the risk-limit, so the county pulled seven more ballots which then met the risk limit.
- RLAs are responsive to election results, meaning that officials must count more ballots in very tight races. If the results don’t meet the risk limit on the first, smallest count of randomly sampled ballots, the sample size gets larger. This happens until the results meet the risk-limit or officials count the entire race by hand.
- The State Board of Elections used a risk limit of 15% for the pilot to make things easier. In a real application of RLAs, the risk limit is usually 10% or 5%, meaning election officials are even more confident the audits would catch any errors in the election results.
- Once the results meet the risk limit, the audit is complete. Officials put the ballots back into the same batches from which they pulled them and remove all labels, leaving the ballots in the condition they were before the audit.
Carolina Public Press is an independent, in-depth and investigative nonprofit news service for North Carolina.