200 likes | 316 Vues
Grid DAIS: Database Access and Integration Services. Greg Riccardi Florida State University riccardi@cs.fsu.edu. Overview of Presentation. Goals of DAIS Conceptual model of Grid database access Examples of client-service interactions Discovery and creation of services
E N D
Grid DAIS:Database Access and Integration Services Greg Riccardi Florida State University riccardi@cs.fsu.edu
Overview of Presentation • Goals of DAIS • Conceptual model of Grid database access • Examples of client-service interactions • Discovery and creation of services • Asynchronous query processing and datasets • Updating from datasets • Representing Sky Query in DAIS • Other topics/issues
Goals of DAIS • The group seeks to promote standards for the development of grid database services, focusing principally on providing consistent access to existing, autonomously managed databases. • Provide service-based access to existing data management systems. • Accommodate several widely used data management paradigms (e.g., relational, object, XML) within a consistent framework. • Provide sufficient information about itself to allow the service to be used given the specification of the service and the metadata provided by the service. • Peacefully coexist with other Web and Grid Service standards. • Be orthogonal to Grid authentication and authorization mechanisms. • Support higher-level information-integration and federation services.
Desirable Properties of DAIS Systems • OGSI/A compliant • Letter and Spirit • Plugability/Extensibility • Different kinds of data resources • Many access mechanisms • Evolvable • Easy to understand and apply • Existing standards/designs • Tooling • GridServices and WebServices applicable • Supports current technology • Access AND integration • Integration of different models at the data level • Implementable • Integrateable into customer scenarios • Technology independent
The Model – External Artifacts External data resource manager External data resource External data set DBMS DB Resultset External = external to the OGSI compliant grid
The Model – Logical Artifacts data resource manager DBMS DB data resource data activity session data request data set Resultset
Data Resource Manager • External data resource manager (edrm) • A data management system such as a relational database management system or a file system • Data resource manager (drm) • A grid service that represents the external data resource manager • Binds to an existing edrm • Supports management operations such as start and stop • Mainly out of scope of DAIS. A place holder for interaction with other working groups
Data Resources • External Data Resource (edr) • A data construct managed by the external data resource manager, for example, a database or a directory structure. • An external data resource manager may manage many external data resources • Data Resource (dr) • A grid service that represents an external data resource • Represents the point of contact to the data structures managed by the edrm. • Exposes meta-data about the structure of the edr • Defines the the target for queries across the edr • Can act as a notification source for notifications associated with the edr • Is bound to existing or newly created edr • Has similarities with a data set. More of which later.
Data Sets • External Data Set (eds) • Data logically separated from an external data resource manager • Could be a snapshot (query) of a relational database or data generated by some process prior to being inserted into a database • Will be typed and identifiable • Data Set (ds) • A service wrapper for the eds • Exposes meta data about the type, description, format of the eds • Immutable • Exposes simple data access operations depending on the type of data. • getAllData, createIterator, getTuple, getFile, getByte, etc. • Can be moved while maintaining its handle and data identity • Can be copied or replicated while maintaining its data identity • Can be delivered to a data manager for persistence • Query and update could be supported
Putting It Together Logical Artifact = Service External world edrm create edr eds bind/ create create bind bind/ create drm create dr create das ds data request DAIS world locate requester access data
Exploiting The Logical Artifacts: Data Sets edr edr GSH GSH move copy dr ds ds ds dr create reference launch launch reference create reference create das das move service copy service query insert/update target details target details Analyst1
Client-Server Interaction Patterns Update/Insert Retrieve Pipeline 1. 4. Q + U Q 7. Q1 G = P A G A G S S + R S1 I A U/R Q2 + D U 5. 2. Q + D I P S2 G = C A Q + D G S A G R S C Q1 + D G = P 8. 6. 3. Q P U S1 A A U/R G Q Q2 S A G D C S R S2 G = C
Examples of client-service interactions • Discovery and creation of services
Examples of client-service interactions • Asynchronous query processing and datasets
Examples of client-service interactions • Updating from datasets
Other topics and issues for DAIS • Data provenance management • Transaction management • Fault tolerance • Security, logging, auditing • Supporting many concurrent users • Establishing the identity and provenance of datasets • Creating pipelines and other workflows • Querying streams of data