1 / 61

EML Alchemy NIST Voting Data Formats Workshop Gaithersburg October, 2009

EML Alchemy NIST Voting Data Formats Workshop Gaithersburg October, 2009. David RR Webber Technical Editor OASIS Election Services TC E-mail: drrwebber@acm.org. Overview.

LeeJohn
Télécharger la présentation

EML Alchemy NIST Voting Data Formats Workshop Gaithersburg October, 2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EML AlchemyNIST Voting Data Formats WorkshopGaithersburgOctober, 2009 David RR Webber Technical Editor OASIS Election Services TC E-mail: drrwebber@acm.org

  2. Overview • Change is constant (even in the conservative voting world) - how do we make trusted, cost effective information technology solutions, that improve democratic processes? • How do we leverage and align standards based mechanisms and tools within our voting processes to make them more consistent, predictable, interoperable and verifiable? • How does this fit into modern software engineering techniques, familiar best practices and popular development environments? • Provide voting management staff the ability to vet and operate standard open processes without constant hand holding from technical staff.

  3. What makes something interoperable? • Clearly defined open processes and exchanges • Based on open standards and approach; not proprietary • Adaptive approach with consistent methods • This allows tailoring to specific local profiles and use patterns • When requirements change can be adjusted consistently • Fault tolerant and not brittle • Ability to ignore non-critical interchange items and especially not to fail for trivial technical reasons not related to actual information details • Leveraging XML capabilities to make self-describing transactions and exchanges possible • Support new uses without extensive reprogramming • Usage patterns set via external configuration allowing broad but controlled uses

  4. Agenda • EML Technical Overview • Introduction to transaction sets • Typical simple illustrative use path • OASIS interoperability event • Recent US related EML work • California Election Results Reporting • Pew/Google VIP Project Geo-districting • Massachusetts State Elections, IRV and EML • Rapid EML localization tools • Using OASIS CAM toolkit and EML templates • Quick look – EML templates and NIEM IEPD process • The Importance of Domain Dictionaries • Summary / Review

  5. EML Technical Overview Introduction to XML transaction sets Typical simple illustrative use path OASIS interoperability event

  6. The Voting Process Overview

  7. Why XML? • XML is multi-faceted – it can represent: • data (information exchanges) • rules • workflows and processes • forms and reports • configuration and control • semantics (metadata) • documents and contracts • multimedia and presentations • XML pervasive across software systems

  8. EML V6.0 What’s in the box? • OASIS EML transactions – XSD Schemas • EML XML dictionary – Excel spreadsheet • Formal specification document • EML templates – OASIS CAM templates* We will discuss EML transactions first… * CAM = Content Assembly Mechanism, OASIS standard template specification

  9. Category Overview of EML transactions • Pre-election • Election details (100) • Candidates (200) • Voters (300) • Ballot Options (600) • Election • Voting (400) • Post election • Results (500) • Analysis (500) • Audit (350, 400) One or more series of XML schemas are provided to support each general process area.

  10. Document Name Purpose EML 110 – election event Information about an election or set of elections. EML 150 – election boundaries Boundary geo-districting information and polling places. EML 210 – candidate nomination Used to nominate candidates or parties, consenting or withdrawing EML 230 – candidate list Contest and candidates details EML 310 – voter registration Used to register voters for an election EML 330 – voter election list Details of actual voters for an election EML 340 – polling information Notification to voter of an election, their eligibility and how to vote EML 410 – ballot Describes the actual ballot to be used for an election EML 420 – voter authentication Used for voter authentication during a voting process EML 440 – cast vote Actual record of vote cast Group of votes being transferred for counting EML 460 – votes group Documents access to voting records and reason EML 480 – audit log Results of election contest(s) and counts EML 510 - count Communicating specific result details on candidates and elections EML 520 – results reporting EML 530 – statistics reporting Statistics relating to the election results and votes cast EML V6.0 Election Transactions Family Standard record structures to hold election information, ballot details, voting records and services

  11. Document Name Purpose EML 130 – response Confirm receipt and processing of EML transaction. EML 220 – response Candidate nomination response and confirmation. EML 350 – series of messages General system level of messages for operational control EML 360 – channel in/out detail Voting channel device selection and use EML 445 – retrieve vote Access and audit control to vote records, reason, event EML 450 – vote confirmation Acknowledges receipt of vote from voting channel EML 470 – voting token log Voting token digital certificate channel use log EML 610 – options nomination Nomination of option proposals for referendum EML 620 – options receipt confirm Confirm receipt of option proposal nomination details EML 630 – list of proposal options List of accepted option proposals for referendum ballot form EML V6.0 – Supporting / Control transactions Used to manage security, audit control, access, system, setup and event tracking.

  12. Example - EML 230 Candidate List <EML Id="230" SchemaVersion="6.0"> <TransactionId>OK-2007-09-1</TransactionId> <CandidateList> <Election> <ElectionIdentifierId=“Oklahoma 2007" /> <Contest> <ContestIdentifierId="State Governor" /> <Candidate>Brad Henry</Candidate> <Candidate>John Wayne</Candidate> <Candidate>Bill Okapi</Candidate> <Candidate>Jane Smith</Candidate> </Contest> </Election> </CandidateList> </EML> Example of an EML artefact in XML

  13. Typical EML Process Path • EML 230 candidate list • EML 330 voter registration • EML 410 ballot layout • EML 440 cast votes • EML 460 vote group • EML 480 audit log activity • EML 510 vote counts • EML 520 election results Also consider • EML 360 voting channel use • EML 470 voting channel digital token log • EML 530 election statistics reporting Pre-Election configuration Polling activity Tallying and Reporting

  14. OASIS Demonstration – London, 2007 • EML 330 voter registration records • Voter selects voting channel preference • EML 440 retained by voting channels • EML 460 sent to IBM tabulator system in Brussels • Reconciled with EML 330 totals per channel • EML 510 count results • EML 520 voting results • Illustrates the reconciliation of the number of authorised voters with votes cast and votes counted by channel and in total. • White Paper available describing event: http://www.oasis-open.org/committees/download.php/25992/EML%20Interop%20Demo%20Report.pdf

  15. System interoperability testing XML Publish System Uses & Creates Templates Test Cases Structure Rules Context Examples Documentation Dictionary 2 Package 3 1 Validate XML Test Localization Content Hints Testing, Agreement / Alignment. Results Report XML 4 html Verify Result Outcomes Pass / Fail

  16. Technology Considerations Summary • Use of XML in software systems is pervasive • All platforms and all major developer environments support XML • Everything from FaceBook™ to Microsoft Office™ to Oracle™ database is XML capable • The OASIS EML transaction Schemas can be directly used by developer software tools to handle the associated XML transactions • The OASIS EML templates provide support for generating additional XML artifacts to suit localization needs

  17. Recent US related EML work California Election Results Reporting Pew/Google VIP Project Geo-districting Use of EML in State Elections with IRV

  18. California Election Results Reporting • Audience – news media channels • Developed in summer of 2007 • Enhanced EML 510/520 • New EML 530 statistics transaction • Joint effort between CA SoS staff and OASIS EML TC

  19. Outcomes • Extend EML logical geographic constructs to handle USA district / ward structures • Results for California 2006, 2007 and 2008 elections reported • Validated ability of EML to report US election results • Added extensible statistics reporting – same EML 530 now being used in Belgium as well

  20. Sample of California Election Statistics EML 530 in web browser EML 530 XML

  21. EML 150 - Pew / Google VIP Project • Audience – voters • District boundary, ballot and polling place information • Mapping tool support EML 150

  22. EML 150 – Lessons Learned • Re-usable components from EML allow rapid development of new transactions • EML V6.0 dictionary provides core enabler • EML 150 provides capabilities surpassing the original VIP XML layouts • Includes GML standard mapping capabilities • Polling place location, layout, access and facilities • Aligned to OASIS EDXL for emergency management support • OASIS EML technical committee able to rapidly respond to emerging needs and provide solutions • VIP team is able to transition their existing data to new EML 150 format and gain benefits of broader information access and reuse (currently each state is new project).

  23. Massachusetts State Elections, IRV, EML • Feasibility study on use of EML to support IRV for state-wide elections • Determine whether EML would be an appropriateway to convey partially aggregated ballots from an IRV (or other preferential ranking system) from a lower jurisdictional level up to a higher jurisdictional level where the ballots would be counted and theresult of the election would be determined • Currently changes being considered forMassachusetts law to use IRV for state-wide offices • OASIS EML TC members supporting investigation into using EML v6.0 • Possible enhancements envisioned during EML v6.0 review period to precisely cover identified needs

  24. Rapid EML Localization tools Using OASIS CAM toolkit and EML templates Quick look – Templates + NIEM IEPD process Importance of Domain Dictionaries

  25. XML Schema Localization Key Components 1  EML Schema XSD aligns voting information model, processes and domain dictionary  Documentation Readable information use details (HTML), EML cross-reference Spreadsheet 2 Key components (aka IEPD*)  XML artifacts The localized XSD Schema and picklist.xml 3  Examples & Test Cases Realistic test XML instances for conformance and interoperability testing 4 *IEPD – Information Exchange Package Documentation

  26. National Information Exchange Model Support for NIEM approach • Builds components for IEPD (Information Exchange Package Documentation) • Best practices - template evaluation for NDR compliance • http://www.niem.gov OASIS at NIEM training event: http://events.oasis-open.org/home/ei-summit/2009

  27. NIEM and OASIS work • OASIS Emergency Management TC - joint MoU between NIEM and OASIS • Development of NIEM “adaptor” for handling OASIS XSD schema and domain dictionary definitions within NIEM Core • Integrated into NIEM 2.1 (latest release) • Successful formula / election services overlaps • Opportunity to follow same path for OASIS EML localization for USA use • Potential benefits: NIEM is Federal, State, Local and Tribal facing - developers already using NIEM • Broad community resources for training/facilitation

  28. Localization Approach Steps EML Steps: • OASIS develops standard international XSD schema • Implements the voting process, management and information exchange needs • Ingest the standard XSD into base CAM template • Apply localizations to CAM template • Exclude unwanted optional components • Align local exchange needs to standard • Extend for local needs, add new components as needed • Generate new localized XSD schema that conform to EML standard • Build dictionary cross-reference of localization usage • Publish documentation and XML examples 4 Local XML XSD Generate Voting XML XSD Revised CAM Template Base CAM Template 1 2 3 localize develop ingest Exchange 5 Compare EML Dictionary Manage 6 Publish Documentation + Examples

  29. EML V6.0 provides base CAM templates* • Templates come pre-packaged in the EML V6 download * Open source Eclipse desktop tool: http://www.sourceforge.com/projects/camprocessor

  30. XML Schema Components CIQ xsd Namespace Common xsd Localization EML xsd CAM Template EML xml How it works in a nutshell OASIS EML XSD Collection XSLT tools process entire EML XSD collection Structure Rules INGEST Documentation CAM = Content Assembly Mechanism

  31. Example – Eclipse XML Structure Editor EML template is shown in the structure navigator panel for review and editing Template contains all the default content model and structure rules ingested from the XSD. All annotations and documentation from XSD also ingested (show as “paperclip” symbol). Code lists and typical content values inserted for easy visual reference.

  32. Using Editor to mark Exclude items Structure Editor Tools Select focus on item Invoke action menu (right mouse click) Pick action Tip: exclude rule display is context sensitive and only available on optional items Tip: use “Add New Rule” mode to specify different XPath for exclude (quick mode assumes “current path”)

  33. Extending the base EML schema • In addition to excluding existing optional components you may need to add new pieces • New pieces should be assigned their own extension namespace prefix – to identify them from the base EML components • Typically the OASIS EML committee will review new pieces as candidates for adding to future versions of the standard • Send comments and suggestions to election-comment@lists.oasis-open.org

  34. Adding New Domain Elements 1 Select root element node in structure; right mouse click for context menu; select “Add Namespace”; then enter prefix and URI Select element node in structure; right mouse click for context menu; select “Add Child Attribute / Element”; then enter prefix and name 2

  35. Generating your EML use schema • Once your localization details are completed you are ready to complete the remainder of the process • This starts with generating your own new subset schema • Then once schema details are OK – creating test cases, cross-reference spreadsheet and documentation

  36. Generating subset use schema File Menu Option Select Export CAM as XSD menu option Confirm the location and filename, and namespace mode. 1 Select ‘default’ for Mode to write a regular import set of XSD schema for each namespace 2

  37. Schema subset generated Set of XSD files with filename and namespace suffix Each namespace file is import for those specific type definitions Reviewing XSD results in a schema visual editor tool

  38. Information Restriction Considerations • The CAM template allows full use of XPath conditional expressions and a rich set of over 30 functions including: • setNumberRange(), setLength(), setLimit(), setDateMask(), setNumberMask(), restrictValues(), lookup() • Those that are compatible with XSD constraints will cause constraint schema assertions to be written out when exporting to schema • Also option to set cross field validations using XPath conditional rules

  39. Dictionary and Documentation Tools • Dictionary Tools • Create dictionary from CAM template • Merge dictionaries together • Compare to dictionary • Excel spreadsheet compatible • Documentation Tools • Web interactive layout • Business review tabular layout • Developer raw template view • Dictionary importance across process • Discovery, reuse, consistency • New development - library of components • XML Tool support

  40. EML v6.0 spreadsheet dictionary

  41. Test Case Generator Feature Summary • Make both Pass / Fail testing examples • Content hinting so examples use real not fake data • Test optional item logic with: all / random / none • Uses exclude() assertions so does not include those items – makes realistic examples of your use pattern • Can pass in seed value – use when adding and testing hints (each test case is labelled with its seed value) • Make hundreds of test cases without manual editing • Can link test case to XSD schema for structure tests • You can modify XSLT tool for own testing needs

  42. Run CAM Rules Check on Examples Run Menu Option 1 Pick Test Case Example to VALIDATE; click Finish to run validation rules Review validation results 2 3

  43. Localization Approach Recapitulation EML Steps: • OASIS develops standard international XSD schema • Implements the voting process, management and information exchange needs • Ingest the standard XSD into base CAM template • Apply localizations to CAM template • Exclude unwanted optional components • Align local exchange needs to standard • Extend for local needs, add new components as needed • Generate new localized XSD schema that conform to EML standard • Build dictionary cross-reference of localization usage • Publish documentation and XML examples 4 Local XML XSD Generate Voting XML XSD Revised CAM Template Base CAM Template 1 2 3 localize develop ingest Exchange 5 Compare EML Dictionary Manage 6 Publish Documentation + Examples

  44. Summary Review and take-aways

  45. EML V6.0 Technical Highlights • Set of standard transactions for core processes in elections • Extended control, event and audit tracking transactions • Domain dictionary of common components used in EML v6.0 • OASIS CAM templates allow rapid localizations • Set of IEPD artifacts generation • Compatible with NIEM.gov approach

  46. Questions?

  47. Localization SupportTools Quick Feature Review Sample Screenshots Tutorial Links

  48. Localization Support Tools • Provides automated generation of need XML artifacts: • Pick list development from template • Schema subsets generation • Dictionary cross-reference spreadsheet • Use and rule documentation reports • XML example instance and test cases • Quick summary screenshots here • Tutorial resources: • http://oasis-open.org/committees/download.php/34489/EML%20and%20jCAM%20tutorial%20and%20primer.pdf

  49. Test Case Generator Feature Summary • Make both Pass / Fail testing examples • Content hinting so examples use real not fake data • Test optional item logic with: all / random / none • Uses exclude() assertions so does not include those items – makes realistic examples of your use pattern • Can pass in seed value – use when adding and testing hints (each test case is labelled with its seed value) • Make hundreds of test cases without manual editing • Can link test case to XSD schema for structure tests • You can modify XSLT tool for own testing needs

  50. Test Case Generation Quick Start Default directory to write examples into File Menu Option Name to be used for the examples How many examples to create Repeating elements count for Quick Test – just click “OK” to use default settings

More Related