ChangeSummary / DAS Requirements (SDO 3.0 Virtual F2F)

ChangeSummary / DAS Requirements(SDO 3.0 Virtual F2F) Christophe Boutard (christophe.boutard@datadirect.com) François Huaulmé (francois.huaulme@datadirect.com) Date: February 2009

Agenda SDO + DAS current states ChangeSummary scope ChangeSummary operations clarification New ChangeSummary format

Today SDO… • … provides a ChangeSummary tracking modifications in the scope of a containment tree • Changes done on objects that are not in the data graph closure are not taken into account • Containment relationships match XML oriented Data Models • … considers creations/deletions/modifications in scope of the data graph (containment tree) • … has the XML ChangeSummary format implying data graph XML serialization • … will introduce the “Orphans” concept in v3.0: • Orphans allow dealing with non-closed data graphs (shared instances, huge business oriented models…) • Orphans only exist in the XML (serialization artifacts)

The upcoming DAS… • … will provide data oriented services for reading SDOs and applying back changes to multiple backends (RDBMS, XML, services…) • … defines an « applyChanges » operation relying on the SDO ChangeSummary content • DAS can be seen as the main consumer for ChangeSummary • … will be able to deal with non-closed data graphs • Complex Business/Data models support • Relationships between multiple « read » services results • Non-closed data graphs changes committed in a single transaction

Objective Extend the XML ChangeSummary scope • Put the « Orphan » objects changes in the scope of the XML ChangeSummary • Allow DASes to apply changes done on non-closed data graphs • Complete SDO 3.0 non-closed graphs support • 50% already completed (« orphans » data objects XML serialization) • 50% TO BE DONE (« orphans » changes) • Avoid breaking changes with SDO 2.1.x

How does it work? (1/2 ) 3 scenarios: • Orphans outside the scope of a ChangeSummary continue to be serialized without changes as described in the first CD • Orphans in the scope of a ChangeSummary having « logging=false » follow the rule above even if the main containment tree tracks changes • Orphans in the scope of a ChangeSummary having « logging=true » are serialized and their changes included into the XML ChangeSummary • DAS « read » services return data graphs with a logging=true ChangeSummary

How does it work? (2/2) • In-memory orphans are just objects that are outside the containment tree to serialize • At XML serialization time, orphan objects are added to orphan properties and then belong to the containment tree • Orphans having changes are included in the XML ChangeSummary and can be referenced like any DataObject in SDO 2.1.x

XML ChangeSummary: Model example employeeOfTheMonth 1 employees * Company project Employee address 1 1 Project Address country containment 1 non-containment Country

XML ChangeSummary: SDO 2.1.x scope employeeOfTheMonth 1 employees * Company project Employee address 1 1 Project Address country containment 1 non-containment Country

XML ChangeSummary: SDO 3.0 scope employeeOfTheMonth 1 employees * Company project Employee address 1 1 addressOrphans Project Address projectOrphans country containment 1 non-containment Country

XML ChangeSummary: proposed scope employeeOfTheMonth 1 employees * Company project Employee address 1 1 addressOrphans Project Address projectOrphans country containment 1 non-containment Country

Example DataObject megaCorp; megaCorp = das.readCompanyByName(“MegaCorp”); List employees = megaCorp.getList(“employees”); DataObject john =employees.get(0); DataObject mary = employees.get(1); DataObject jane = employees.get(2); DataObject address = john.getDataObject(“address”); DataObject country; country = address.createDataObject(“country”); country.setString(“name”, “UK”); DataObject project = mary.getDataObject(“project”); project.setString(“description”, “NEW description of PRJ002”);

Example: XML Schema <xsd:complexType name="CompanyType"> <xsd:sequence> <xsd:element name="employees” type="company:EmployeeType" maxOccurs="unbounded"/> <xsd:element name="addressOrphans“ type="company:AddressType“ minOccurs="0“ maxOccurs="unbounded” sdox:orphanHolder="true"/> <xsd:element name="projectOrphans" type="company:ProjectType" minOccurs="0" maxOccurs="unbounded“ sdox:orphanHolder="true"/> </xsd:sequence> <xsd:attribute name="name" type="xsd:string"/> <xsd:attribute name="employeeOfTheMonth" type="xsd:string" sdo:propertyType="company:EmployeeType"/> </xsd:complexType>

Example: XML data graph <company:company name="MegaCorp" employeeOfTheMonth="#/sdo:datagraph/company:company[1]/employees[2]"> <employees name="John Jones" SN="E0001" address="#/sdo:datagraph/company:company[1]/addressOrphans[1]“ project="#/sdo:datagraph/company:company[1]/projectOrphans[1]"/> <employees name="Mary Smith" SN="E0002" address="#/sdo:datagraph/company:company[1]/addressOrphans[1]“ project="#/sdo:datagraph/company:company[1]/projectOrphans[2]"/> <employees name="Jane Doe" SN="E0003" address="#/sdo:datagraph/company:company[1]/addressOrphans[2]“ project="#/sdo:datagraph/company:company[1]/projectOrphans[2]"/> <addressOrphans number="123" street="main street"> <country name="UK"/> </addressOrphans> <addressOrphans number="456" street="16th street"> <country name="US"/> </addressOrphans> <projectOrphans code="PRJ001" description="description of PRJ001"/> <projectOrphans code="PRJ002" description="NEW description of PRJ002"/> </company:company>

Example: XML ChangeSummary <changeSummary delete="" create="#/sdo:datagraph/company:company[1]/addressOrphans[1]/country[1]"> <company:companysdo:ref="#/sdo:datagraph/company:company[1]" employeeOfTheMonth="#/sdo:datagraph/company:company[1]/employees[1]"> <addressOrphans number="123" street="main street" sdo:unset="country"/> <addressOrphans number="456" street="16th street"> <country name="US"/> </addressOrphans> <projectOrphans code="PRJ001" description="description of PRJ001"/> <projectOrphans code="PRJ002" description="description of PRJ002"/> </company:company> </changeSummary>

Objectives • Make events recorded in the ChangeSummary more easily usable by a DAS • Objects lifecycle should be independent of the data graph • « Creations » and « Deletions » should describe the state of objects with a bigger scope than a data graph • Distinguish « moving » an object from « deleting » it

Extending the DataObjectlifecycle SDO 3.0 and DAS tend to consider DataObjects outside data graphs • The SDO 3.0 Keys lead to extend the DataObjects lifecycle • DataObjects can • be identified independently from a data graph • appear in multiple data graphs • In SDO + DAS architecture objects uniqueness must be ensured at the backend level • isCreated in a data graph does not always mean isCreated at the backend level

Creations (1/2) • In DAS architectures DataObjects can be instantiated by: • User (client side): insert new instances into the backend • DAS (server side): “read” operations results • Today “ChangeSummary.isCreated()” means “is added to a data graph” • This does not allow differencing objects instantiated by users (really created) from objects returned by a DAS • SDO must not solve DAS issues but should provide mechanisms that make things possible…

Creations (2/2) • Containment relationships would continue following the SDO 2.1.x rules • Adding an object on a containment relationships would be tracked as a creation without considering previous actions done on the object (e.g. DataObject.delete()) • Non containment relationships to orphan objects need a new rule • An object added to a non containment relationship will not be considered as a creation if it was not in the scope of a ChangeSummary at the time ChangeSummary.beginLogging() was called

Delete vs. Move • Today SDO does not allow distinguishing “moves” from “deletes” • From a DAS (backend) point of view • “Delete” must produce a delete order (e.g. SQL delete for RDBMS) • “Move” must produce a reference modification • Explicit deletions (DataObject.delete()) must always be tracked as delete orders without considering the containment information • The needed clarification only affects “moves”…

How to track “Moves”? • Containment relationships • Moving a contained DataObject should continue to be tracked as a deletion in the context of the old container • Then setting the moved object to another relationship would follow the rules defined for “creations” • Non containment relationships to “orphan” objects • A “nullification” must be tracked for the old owner of the relationship • The moved object is not marked as deleted

Objectives – Questions Objectives: • Provide a more efficient XML ChangeSummary format • Provide a format that can be more easily used by a DAS • SDO Keys mapped to a backend identity • Optimistic Concurrency Control (OCC) based on old values + Keys Questions: • Should SDO 3 allow a ChangeSummary to be sent without anything but only the changes? • Avoid data graph XML serialization (less verbose). • Currently XML ChangeSummary only stores old values. New values are serialized in the data graph. • A more efficient XML ChangeSummary format could leverage SDO 3 Keys • Objects identity can move from technical state to a business representation • Use of identity in sdo:ref?

How to represent the object identity? Use SDO 3 identity in the ChangeSummary. But how? • A canonical representation of the key? • General format: • {type_name, idProperty0=idProperty0Value, …, idPropertyN=idPropertyNValue} • Examples • sdo:ref=”{abc:Type, id=2, id2=3, objectId={abc:ObjectType, objId=10}}” • sdo:ref=”{abc:Type, id=2, id2=3}” • A new reprensentation of the “sdo:ref”? • XML elements containing Keys • Something else?

Improve SDO ChangeSummary information • SDO 2.1: ChangeSummary is the “copy” of a data graph when beginLogging() was called. • Should ChangeSummary be… • An action log? A list of actions (new, move, delete, set…) • Can highly reduce the old values size when dealing with many=true properties. • An optimized action log? • A log that prunes unnecessary recorded actions.

QUESTIONs ?

ChangeSummary / DAS Requirements (SDO 3.0 Virtual F2F)