Risk Limiting Audits Nuts, Bolts, and Paperclips

1. Risk Limiting AuditsNuts, Bolts, and Paperclips

2. Overview RLA Key Ingredients Preparation Pre-Election Election Day Post-Election The Audit Event Event Planning and Costs

3. RLA Key Ingredients Trustworthy Audit Trail (paper ballots) RLA Tool (software) Risk Limit; Target Race(s) and Results Ballot Manifest (record of how ballots are organized) Method for Selecting Random Seed # [Cast Vote Record – Comparison Audit only]

4. Pre-Election Preparation

5. Pre-Election Preparation Election Day Precincts: No Special Prep AV Counting Board: Get Organized Process and store ballots in batches no larger than individual precincts. Smaller batches (25-100) may be necessary for a Comparison Audit.

6. Election Day Precincts: Follow Existing Procedures Ballot Accounting and Chain of Custody General Security AV Counting Board: Execute

7. Post-Election Preparation Secure and clearly ID Provisional Envelope Ballots that are counted Determine Risk Limit Compile Results from Target Race(s) Create the Ballot Manifest [Get Cast Vote Record Data from County Clerk – Comparison Audit Only] Obtain Seed Selection Hardware

8. Post-Election Preparation Secure and clearly ID Provisional Envelope Ballots that are counted Determine Risk Limit Compile Results from Target Race(s) Create the Ballot Manifest [Get Cast Vote Record Data from County Clerk – Comparison Audit Only] Obtain Seed Selection Hardware

9. The Audit Event Seed Selection->Ballot Retrieval->Tallying

10. The Audit Event Seed Selection->Ballot Retrieval->Tallying Purpose: to create a randomly generated “seed number” so the RLA tool can randomly generate the list of ballots to be examined.

11. The Audit Event Seed Selection->Ballot Retrieval->Tallying Process: use a physical source of randomness such as rolling dice or drawing cards to create a 20 digit number. Example: randomly select 20 people from the audience to roll dice; the resulting 20-digit number becomes the seed number.

12. The Audit Event Seed Selection->Ballot Retrieval->Tallying Purpose: to retrieve the ballots to be examined.

13. The Audit Event Seed Selection->Ballot Retrieval->Tallying Process 1 – Countdown Method: using the list of randomly selected ballots created by the RLA tool, locate the ballot container, remove the ballots from the ballot container, place them in a stack on a table, and count down to the ballot(s) specified for that precinct.

14. The Audit Event Seed Selection->Ballot Retrieval->Tallying Process 2 – K-Cut Method: “Cut” the stack of ballots like a deck of cards for a predetermined number of times (5-6). The ballot that ends up on top is the ballot selected.

15. The Audit Event Seed Selection->Ballot Retrieval->Tallying Purpose: to document the results for the targeted race(s), and any other races being examined on the selected ballots, so those results can be entered into the RLA tool.

16. The Audit Event Seed Selection->Ballot Retrieval->Tallying Process: audit teams examine the ballots and hand-tally the votes according to the Michigan standards for interpreting ballot marks. The results are entered into the RLA tool, which compares the results of the sample to the reported election results and determines the risk-limit.

17. Event Planning and Costs Space for ballot retrieval and tallying Technology – computer(s), printer/copier, document camera, video displays Staffing – how many and for how long? Supplies – slip sheets, labels, tally sheets

18. 2018 Kalamazoo Pilot RLASpending Supplies <$50 Staffing $125 Food $450 Total $625

19. 2019 Projected RLA Spending Supplies <$50 Staffing $384 Food ~$75 Total $509

20. Future Considerations for Cost/Staffing Involvement/role of state RLA tool capabilities Impact of coordination by county Ability to estimate # of ballots to examine