Describing and serializing structured data – A history and comparison of approaches

Describing and serializing structured data – A history and comparison of approaches John LarmouthITU-T and ISO/IEC ASN.1 Rapporteur j.larmouth@btinternet.com

Terminology has changed over time • Markup languages • Abstract Syntax and Concrete Syntax • Abstract syntax notation and encodings • Schema definition • Describing and serializing structured data • The terms all mean roughly the same thing! • That is the main message from this tutorial • In the 1980s, it was sexy to talk about Abstract Syntax and encodings" • Today it is sexy to talk about describing and serializing structure data

What is structured data? • There is a general recognition that we are talking about data that can be best described using basic primitives like integers and booleans and strings • And “structuring” (with various names) using sequence, sequence of (repetitions), and choice

ASN.1 has developed over time (1) • Developments to meet user demands • All added immense richness, but inevitably more to be learned if you need the richness • Description of structured data was extended to include information object classes, bringing with it much greater use of object identifiers and the need for the OID repository – so much so that OIDs have a life of their own, and a tree structure

ASN.1 has developed over time (2) • Description of structured data was extended to include addition of constraints, which were visible for encoding rules (bringing ASN.1 more into line with programming languages • Parameterization of the basic notation was introduced (but not heavily used)

ASN.1 has developed over time (3) • More encoding rules were added, some becoming very popular (e.g. PER) • Encoding control notation was introduced (a substantial new topic, but not much used (?) ) • Object identifier resolution using DNS was introduced – still in its gestation period in 2012, but very important

ASN.1 has developed over time (4) • Fast web services and Fast Infoset were introduced, targeting the “XML heart-land” • Competition with EXI became an issue – still not fully resolved. • Today there is increasing competition on the notation to be used for structured data definition and its serialization/encodings – discussion later

The bottom line • Through-out all of this, ASN.1 has always been, and still is, the ITU-T Recommendation for the description and serialization of structured data

Acronyms and standard names – you cannot avoid them (1)! • SGML • ASN.1 • TLV • HTML, XML • XSD, Relax NG • PER, XER • OIDs

Acronyms and standard names – you cannot avoid them (2)! • JSON (RFC 4627) and BSON • Google Protocol Buffers • RFC 3072, RFC 4506 • YAML • Thrift • ETCH • Hadoop

When will I stop? • Some of you may be leaving the room! • I just want to do a little historical stuff, and then to invite some discussion on things today • I know that some groups are investigating other notations and encodings • I believe that in most cases they have decided to stay with ASN.1. I would like to invite comments shortly.

The 1960s to early 1980s • People used typewriters! • Physical cut and past (I have still scissors from those days!) was the way standards were developed! (some others in this room also did!) • Markup languages were developed (Tex and Latex, for example) • Immensely important as the basis for future work

The 1980s – a heady time (1) • Led into IBM’s seminal work on SGML, which led to HTML from CERN, and in due course to XML • These were all essentially mark-ups of content (encoding rules), not structured data definition. • OSI badly needed a way of describing the structured data in protocols.

The 1980s – a heady time (2) • I don’t want to waste your time too much on history, but I have to mention Doug Steedman and Jim White (see other sources of description • ASN.1 became established – there was no real competitor! • OSI badly needed a way of describing the structured data in protocols. ASN.1 emerged from Courier work in Rank Xerox.

The 1980s – a heady time (3) • Papers at that time were addressing “OSI versus SNA”. • They later changed to OSI versus TCP/IP. • We all know which won! • But ASN.1 remained as the notation of choice for the definition of structured data

The 1990s – stability? (maybe not!) • The XML people introduced XSD. • Standardise a mapping from XSD to ASN.1? Or vice-versa? Or a round-trip mapping? • “You map from a competitor notation to yours, not vice-versa” – bad advice which led to people using XSD as the schema definition of choice, relying on the mapping to ASN.1 for ASN.1 binary encodings. • XML encoding rules for ASN.1 (XER)

And to the 2000s – it is up in the air again! • We have a plethora of new options: • JSON and BSON • Google Protocol Buffers • Candle markup • Bencode • YAML • Hadoop • Thrift • What have I missed?

ASN.1 remains the choice for … • Big Data • Satellie (Europe’s Galileo project) • Aviation (3GPPP 4G, LTE • Advanced online gaming • Wind turbines • Personal health records • Smart highways • Electrical smart grid

Discussion invited please! I am done!

Describing and serializing structured data – A history and comparison of approaches