1 / 62

Open Data Protocol (OData) Deep Dive

Open Data Protocol (OData) Deep Dive. A common, open, RESTful Web protocol for querying and updating data that provides a uniform way to unlock data and free it from silos that exist in applications today. OData. = REST Resource-oriented Entities modeled as URI-addressable Resources

mimis
Télécharger la présentation

Open Data Protocol (OData) Deep Dive

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Open Data Protocol (OData)Deep Dive A common, open, RESTful Web protocol for querying and updating data that provides a uniform way to unlock data and free it from silos that exist in applications today.

  2. OData = REST Resource-oriented Entities modeled as URI-addressable Resources Relationships modeled as links (URIs) CRUD = POST, GET, PUT/PATCH, DELETE Hypermedia-driven Navigate from Service Document to Sets to Members to related items to… Links in Payload for editing, navigation, operations, etc. + Data Model URLs, operations, namespaces, derived from declarative model + Common Conventions Common query string options Representation within Formats + Common Formats ATOM, JSON

  3. USE of HTTP • HTTP Methods • GET, POST, PUT/PATCH, DELETE • HTTP Headers • Content Negotiation • Accept, Accept-Charset, Accept-Encoding, Accept-Language • Content-Type, Content-Length, Content-Encoding, Content-Language • Concurrency • Etags, If-Match, If-None-Match • Etc… • HTTP Result Codes • 2xx Success Codes • 3xx Redirection • 4xx Client Error Messages

  4. OData Feature Areas • Versioning • Service Metadata • Data Requests • Data Modification • Relationships • Content Types • Extensibility

  5. Versioning

  6. Protocol Versioning • DataServiceVersion header • Describes the protocol version according to which the request/response should be interpreted • MaxDataServiceVersion Request Header • The maximum protocol version that the client can accept • MinDataServiceVersion Request Header • The minimum protocol version the client can accept -Clients SHOULD specify a MaxDataService -Services return a response formatted according to the latest protocol version the service supports that is less than MaxDataServiceVersion but at least MinDataServiceVersion

  7. Schema Versioning • When a data model or other breaking change is made to the service, the service URL is generally versioned. For example: • odata.netflix.com/v1/Catalog • odata.netflix.com/v2/Catalog

  8. Service Metadata

  9. Service Document • Root of service exposes a “Service Document” • Describes collections of Entities GET /odata.netflix.com/v2/Catalog HTTP/1.1 <servicexmlns="http://www.w3.org/2007/app"xmlns:app="http://www.w3.org/2007/app" xmlns:atom="http://www.w3.org/2005/Atom"xml:base="http://odata.netflix.com/v2/Catalog/"> <workspace> <atom:title>Default</atom:title>-<collectionhref="Genres"> <atom:title>Genres</atom:title> </collection>-<collectionhref="Titles"> <atom:title>Titles</atom:title> </collection>-<collectionhref="TitleAudioFormats"> <atom:title>TitleAudioFormats</atom:title> </collection>-<collectionhref="TitleAwards"> <atom:title>TitleAwards</atom:title> </collection>-<collectionhref="People"> <atom:title>People</atom:title> </collection>-<collectionhref="TitleScreenFormats"> <atom:title>TitleScreenFormats</atom:title> </collection>-<collectionhref="Languages"> <atom:title>Languages</atom:title> </collection> </workspace> </service>

  10. Entity Data Model • Exposed on /$metadata • Describes Entity Model of Data Service • Entity Types • Complex Types • Association Types • EntityContainers GET /odata.netflix.com/v2/Catalog/$metadata HTTP/1.1 <edmx:Edmxxmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx"Version="1.0"> <edmx:DataServicesm:DataServiceVersion="1.0" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"> <Schemaxmlns="http://schemas.microsoft.com/ado/2008/09/edm" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" Namespace="Netflix.Catalog.v2"> +<EntityTypeName="Genre"> +<EntityTypeName="Title"m:HasStream="true"> +<ComplexTypeName="BoxArt"> +<AssociationName="Genre_Titles"> +<EntityContainerName="NetflixCatalog"m:IsDefaultEntityContainer="true"> </Schema> </edmx:DataServices> </edmx:Edmx>

  11. Entity Type • Named structures with keys • Support Inheritance • Properties • String, Integer, Boolean, DateTime, Spatial datatypes • Collections, Complex Types • Relationship Properties • May be marked as "Open" • May return "dynamic" properties not described in $metadata <EntityTypeName="Genre"> <Key> <PropertyRefName="Name"/> </Key> <PropertyName="Name"Type="Edm.String"/> <NavigationPropertyName="Titles"Relationship="Self.Genre_Titles" ToRole="Genre_Titles_Target" FromRole="Genre_Titles_Source"/> </EntityType>

  12. Complex Type • Named Structures w/o keys • Support Inheritance • Don't support relationships • Because they don't have an independent ID <ComplexTypeName="BoxArt"> <PropertyName="SmallUrl"Type="Edm.String"/> <PropertyName="MediumUrl"Type="Edm.String"/> <PropertyName="LargeUrl"Type="Edm.String"/> </ComplexType>

  13. Associations • Define relationships (navigation paths) between Entities • Bi-directional • Specify Cardinality of each end • Can also impose integrity constraints <AssociationName="Genre_Titles"> <EndType="Self.Genre"Multiplicity="*" Role="Genre_Titles_Source"/> <EndType="Self.Title"Multiplicity="*"Role="Genre_Titles_Target"/> </Association>

  14. Entity Container • Contains • EntitySets – collections of Entity Instances • AssociationSets – relationships between entities • FunctionImports – Actions/Functions exposed by service <EntityContainerName="NetflixCatalog" m:IsDefaultEntityContainer="true"> <EntitySetName="Genres"EntityType="Self.Genre"/> <EntitySetName="Titles"EntityType="Self.Title"/> <AssociationSetName="Genre_Titles"Association="Self.Genre_Titles"> <EndRole="Genre_Titles_Source"EntitySet="Genres"/> <EndRole="Genre_Titles_Target"EntitySet="Titles"/> </AssociationSet> </EntityContainer>

  15. Data Requests

  16. Querying Data • Querying a Set • GET /v2/Catalog/Genres HTTP/1.1 • Requesting an Individual Entity by ID • GET /v2/Catalog/Genres('Adventures') HTTP/1.1 • Requesting an Individual Property • GET /v2/Catalog/Genres('Adventures')/Name HTTP/1.1 • Requesting an Individual Property Raw Value • GET /v2/Catalog/Genres('Adventures')/Name/$value HTTP/1.1

  17. $Filter • Basic predicates, built-in functions • GET /v2/Catalog/Titles?$filter=Name eq 'The Sting' HTTP/1.1 • Filter on Properties of a derived type • Only returns entities of that type that match the predicate • GET /v2/Catalog/Awards?$filter=Netflix.AcademyAward/Category HTTP/1.1 • Query based on collection Membership using Any/All • GET /v2/Catalog/Titles?$filter=Genres/Any(g:g/Name eq 'Adventure') HTTP/1.1

  18. Operators • Logical Operators • eq, ne, gt, ge, lt, le, and, or, not • Mathematic Operators • add, sub, mult, div, mod • Grouping Operator • () • NULL literal • null

  19. Built-in Functions • String Functions • substringof, endswith, startswith, length, indexof, substring, tolower, toupper, trim, concat • DateTime Functions • year(), month(), day(), hour(), minute(), second() • Math Functions • round(), floor(), ceiling() • Type Functions • isof() • Casting • Insert type in path • Geo Functions • geo.length, geo.intersects, geo.distance

  20. $Orderby • Comma separated list of properties, expressions • GET /v2/Catalog/Titles?$orderby=AverageRating,ReleaseYearHTTP/1.1 • Can specify asc (default) or desc • GET /v2/Catalog/Titles?$orderby=AverageRatingdesc HTTP/1.1

  21. $top/$skip • Enables Client-side paging through large result sets • GET /v2/Catalog/Titles?$top=10&$orderby=AverageRatingdesc HTTP/1.1 • GET /v2/Catalog/Titles?$top=10&$skip=10 &$orderby=AverageRatingdescHTTP/1.1 • If no $orderby, server guarantees stable ordering

  22. $expand • Specify comma separated list of navigation property paths to include in response • GET /v2/Catalog/Titles?$expand=Cast,Awards HTTP/1.1 • Result are returned inline according to the appropriate format

  23. $select • Narrow the set of fields returned • GET /v2/Catalog/Titles?$select=Name,Synopsis,Rating HTTP/1.1 • Related properties must be specified in $expand • GET /v2/Catalog/Titles?$select=Name,Cast/Name&$expand=Cast HTTP/1.1 • Can include all properties using * • Does not include navigation properties • GET /v2/Catalog/Titles?$select=* HTTP/1.1

  24. $count/$inlinecount • Get just the count • Can include additional query operators • GET /v2/Catalog/Titles/$count HTTP/1.1 • Include the count with the results • $inlinecount ignores $top/$skip but includes $filter • GET /v2/Catalog/Titles?$inlinecount=allpages&$top=10 HTTP/1.1

  25. $FORMAT • Specify a particular format • "atom" • application/atom+xml, • application/atomsvc+xml for service document • "json" • application/json • "xml" • application/xml • Overrides content type specified in headers

  26. Server Driven Paging • Server controls the maximum number of records to return at a time • Each "page" of results includes a "next link" for retrieving the next page of results • Pages can vary in size; some may be blank • i.e., return records as computed or across a federation

  27. Data Modification

  28. Insert • POST entity to the EntitySet collection • Returns inserted entity • Use PREFER header to request no content returned • Media Resources have special rules according to AtomPub: • POST media resource to the collection • Returns Media Link Entry • Update the Media Link Entry

  29. Update • PUT to the edit-link to replace • Unspecified values are set to null or default • PATCH to the edit-link to affect only specified values • Unspecified values are left unchanged • Use PREFER header to request content returned • Use entity etag in if-match for concurrency control • Obtained from header or payload

  30. Delete • DELETE to the edit-link to delete • Use entity etag in if-match for concurrency control • Obtained from header or payload

  31. Batch Requests • "Batch" multiple statements in a single multi-part mime request • Results for each statement returned in a multi-part mime response • Multiple Data Modification statements may be grouped in a single "ChangeSet" • ChangeSets are atomic • Reference the results of other statements in the changeset • i.e., add a child to an added parent • May have multiple ChangeSets per Batch

  32. Relationships

  33. Requesting Relationship Links • Use $links to request the relationship links for a particular navigation property • Results are returned as an array of URIs • GET /v2/Catalog/Titles/Genres('Adventures')/$links/Titles HTTP/1.1

  34. Creating Links to New Entities • Create a new Entity with a new related entity by POSTing the entity with the related entity as inline content ("Deep Inserts") • Create a new entity related to an existing entity by POSTing to the relationship collection of the existing entity • POST /v2/Catalog/Titles/Genres('Adventures')/$links/Titles HTTP/1.1 <entry>…</entry>

  35. Creating Links to Existing Entities • Create a relationship between two existing entities by POSTing the URL of the one entity to the appropriate $links collection of the other entity • POST /v2/Catalog/Titles/Genres('Adventures')/$links/Titles HTTP/1.1 <uri>http://odata.netflix.com/v2/Catalog/Titles('13kbf')</uri>

  36. DELETING Links • Remove a relationship by sending a DELETE request to the $links collection specifying the url to remove • DELETE /v2/Catalog/Titles/Genres('Adventures')/$links/Titles HTTP/1.1 <uri>http://odata.netflix.com/v2/Catalog/Titles('13kbf')</uri>

  37. Content Types (Formats)

  38. Atom Format • Multiple entities represented as a <feed> • Count optionally returned in <metadata:count> element • Next link returned as <link> element • Each entity represented as an <entry> • <id> is unique id • <category> specifies type • Self, edit <link>s • <link> element for Relationships • Inline content as child <metadata:inline> element containing feed or element • <link> element for editing media • Properties nested in <metadata:properties> element • Individual properties namespace qualified with data namespace • For Media Resources, <content> has a src property to the media stream • <metadata:properties> is a sibling of <content> • For non-Media Resources, <metadata:properties> is a child of <content> • Functions/Actions returned as <metadata:function> and <metadata:action> elements

  39. Atom Payload - Example <?xmlversion="1.0"encoding="utf-8" ?> <feedxml:base="http://odata.netflix.com/v2/Catalog/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom"> <titletype="text">Genres</title> <id>http://odata.netflix.com/v2/Catalog/Genres/</id> <updated>2012-07-26T18:37:18Z</updated> <linkrel="self"title="Genres"href="Genres" /> <m:count>326</m:count> <entry> <id>http://odata.netflix.com/v2/Catalog/Genres('Adventures')</id> <titletype="text">Adventures</title> <updated>2012-07-26T19:08:35Z</updated> <author> <name/> </author> <linkrel="edit"title="Genre"href="Genres('Adventures')" /> <linkrel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Titles" type="application/atom+xml;type=feed"title="Titles" href="Genres('Adventures')/Titles" /> <categoryterm="Netflix.Catalog.v2.Genre" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" /> <contenttype="application/xml"> <m:properties> <d:Name>Adventures</d:Name> </m:properties> </content> </entry> <linkrel="next"href="Genres/?$skiptoken='Adventures'" /> </feed>

  40. JSON Verbose Format • Results wrapped in a "d" wrapper • Multiple entities returned as JSON Array property named "Result" • Optionally has __Count property for number of elements in query result (across all pages) • Optionally has "__next" property for next link • Each Entity has • A "__metadata" property containing: • URI • Type • Media Link, edit-media link, content type, etag for media resources • A "__deferred" property for each un-expanded relationship containing the URL for the related entity/collection • Properties a Name/Value pairs • Each Complex Type has a "__metadata" property containing the Type

  41. JSON PAYLOAD - Example { "d" : { "results": [ { "__metadata": { "uri":"http://odata.netflix.com/Genres('Adventures')", "type": "Netflix.Catalog.v2.Genre" }, "Name": "Adventures", "Titles": { "__deferred": { "uri":"http://odata.netflix.com/Genres('Adventures')/Titles"} } } ], "__count": "326", "__next": " Genres/?$skiptoken='Adventures'" } }

  42. New JSON OData Format • Effort to simplify JSON • More readable JSON format • Reduce size • Preserve OData-ness • Still Hypermedia-driven • Uses Namespacing Mechanism • Removes almost all metadata from payloads • Calculates values from conventions or templates in metadata • 80-90% more efficient than Atom • Future contribution to TC

  43. NEW JSON PAYLOAD - Example { "odata.metadata":"http://odata.netflix.com/$metadata#Netflix/Genres", "count":"830", "value": [ { "Name": "Adventures", } ], "nextLink":" http://odata.netflix.com/Genres('Adventures')/Titles " }

  44. Extensibility

  45. Custom Actions • Side-effecting operations that may or may not return a value • May be exposed on instances (Hypermedia-driven) • Use entity etag in if-match for concurrency control • Obtained from header or payload

  46. Custom Functions • Extend query language • May be exposed on instances (Hypermedia-driven) • Use entity etag in if-match for concurrency control • Obtained from header or payload

  47. Vocabularies An OData Vocabulary defines a shareable set of Annotations that may be applied to metadata or instance data • Extended Property Information • Units of measurement • Read-only, read-write • Ontologies • This entity represents the common concept of a person • Validation • This integer property is between 10 and 20 • Display hints • Caption field, grouping, navigation • Service Capabilities • Query limitations, etc. • Extension • Analytics

  48. Annotations Consist of: • A Target: The construct against which the annotation is applied • May be any Model Construct (type, property, etc.) • A Term: The global name of the annotation • For example, org.odata.display.Title • A Value: The value for the annotation • May be a static value, path, or expression A Vocabulary is a set of terms in a common namespace

  49. Annotations may… • Be defined in a common format • Machine readable (i.e., for validation) • Be embedded in a model • To define extended metadata • Be applied through an external annotation application file • Externally annotate existing services • Be applied to Instance Data • Information that may vary per instance

  50. Types of Annotations • ValueTerm • A single annotation term applied to a type, property, association, etc. • Complex TypeTerm • Generally used to describe an is-a relationship • Relationships may be modeled as properties of individual or collections of other complex type terms • Entity TypeTerm • Most prescriptive; define keys, relationships

More Related