David De Roure University of Southampton semanticgrid

1. David De Roure University of Southampton www.semanticgrid.org The Semantic Grid:Enabling e-Science

2. Outline • The ambition • Enabling Technologies • Grid • Semantic Web • Semantic Grid • Are we there yet? • The Future ISGC

3. Outline • The ambition • Enabling Technologies • Grid • Semantic Web • Semantic Grid • Are we there yet? • The Future ISGC

4. Vision: e-Science e-Science is about global collaboration in key areas of science and the next generation of [computing] infrastructure that will enable it. e-Science will change the dynamic of the way science is undertaken. John Taylor, Director General of UK Research Councils in 2001 ISGC

5. Vision: Grid Grid computing has emerged as an important new field, distinguished from conventional distributed computing by its focus on large-scale resource sharing, innovative applications, and, in some cases, high-performance orientation...we [define] the "Grid problem”…as flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources - what we refer to as virtual organizations From "The Anatomy of the Grid: Enabling Scalable Virtual Organizations" by Foster, Kesselman and Tuecke ISGC

6. Requirements These visions require an infrastructure for flexible, coordinated resource sharing – they are fundamentally about joining resources up, automatically, in order to do things that weren’t possible before ISGC

7. myGrid Combechem • Wish to reuse • Data • Services • Knowledge • Software • Practice • Anticipated use • Unanticipated use ISGC

8. Outline • The ambition • Enabling Technologies • Grid • Semantic Web • Semantic Grid • Are we there yet? • The Future ISGC

9. On demand transparentlyconstructed multi-organisational federations of distributed services Distributed computing middleware Computational Integration • An automatically processable, machine understandable web • Distributed knowledge and information management • Information integration Carole Goble ISGC

10. ISGC

11. Origins of the Semantic Web The Semantic Web is an extension of the current Web in which information is given a well-defined meaning, better enabling computers and people to work in cooperation. It is the idea of having data on the Web defined and linked in a way that it can be used for more effective discovery, automation, integration and reuse across various applications. The Web can reach its full potential if it becomes a place where data can be processed by automated tools as well as people. W3C Activity Statement ISGC

12. Layers of Languages You are here ISGC

13. Web vs Semantic Web Web page Any Web Resource <a href= URI> HTML <a href=“http://…”> URI URI URI RDF is like the web! RDF Hendler ISGC

14. Making Knowledge Explicit Ontology Inference Layer DAML OIL RDF DAML+OIL All influenced by RDF OWL Lite (thesaurus) OWL DL (reason-able) OWL Full (anything goes) OWL RDF Resource Description Framework OWL Web Ontology Language ISGC

15. Applications connected by concepts ISGC

16. Outline • The ambition • Enabling Technologies • Grid • Semantic Web • Semantic Grid • Are we there yet? • The Future ISGC

17. So what is the Semantic Grid? • The Grid vision is about coordinated resource sharing – virtual organisations • Fundamentally, this means that information and knowledge in and about the system must be ‘machine processable’ • The full richness of the Grid ambition depends upon realising the Semantic Grid ISGC

18. The Semantic Grid Report 2001 • At this time, there are a number of grid applications being developed and there is a whole raft of computer technologies that provide fragments of the necessary functionality. • However there is currently a major gap between these endeavours and the vision of e-Science in which there is a high degree of easy-to-use and seamless automation and in which there are flexible collaborations and computations on a global scale. www.semanticgrid.org NB Report updated – March 2005 issue of Proceedings of the IEEE ISGC

19. Building bridges ISGC

20. Semantic Grid SemanticWeb SemanticGrid Scale of Interoperability ClassicalWeb ClassicalGrid Scale of data and computation Based on an idea by Norman Paton ISGC

21. Semantics in and on the Grid The Semantic Grid is an extension of the current Grid in which information and services are given well-defined meaning, better enabling computers and peopleto work in cooperation ISGC

22. Grid Computing The Semantic Grid The Semantic Web Web Services Semantics in and on the Grid ISGC

23. Grid is metadata based middleware Astronomy Sky Survey Data Grid 1. Portals and Workbenches 2.Knowledge & Resource Management Bulk Data Analysis Metadata View Data View Catalog Analysis 3. Standard APIs and Protocols Concept space 4.Grid Security Caching Replication Backup Scheduling Information Discovery Metadata delivery Data Discovery Data Delivery 5. Standard Metadata format, Data model, Wire format Catalog Mediator 6. Data mediator Catalog/Image Specific Access 7. Compute Resources Catalogs Data Archives Derived Collections ISGC

24. For example… Annotations of results, workflows and database entries could be represented by RDF graphs using controlled vocabularies described in RDF Schema and DAML+OIL Personal notes can be XML documents annotated with metadata or RDF graphs linked to results or experimental plans Exporting results as RDF makes them available to be reasoned over RDF graphs can be the “glue” that associates all the components (literature, notes, code, databases, intermediate results, sketches, images, workflows, the person doing the experiment, the lab they are in, the final paper) The provenance trails that keep a record of how a collection of services were orchestrated so they can be replicated or replayed, or act as evidence At the data/computation layer: classification of computational and data resources, performance metrics, job control, management of physical and logical resources At the information layer: schema integration, workflow descriptions, provenance trail At the knowledge layer: problem solving selection, intelligent portals Governance of the Grid, for example access rights to databases, personal profiles and security groupings Charging infrastructure, computational economy, support for negotiation; e.g. through auction model Represent the syntactic data types of e-Science objects using XML Schema data types Represent domain ontologies for the semantic mediation between database schema, an application’s inputs and outputs, and workflow work items Represent domain ontologies and rules for parameters of machines or algorithms to reason over allowed configurations Use reasoning over execution plans, workflows and other combinations of services to ensure the semantic validity of the composition Use RDF as a common data model for merging results drawn from different resources or instruments Capture the structure of messages that are exchanged between components ISGC

25. Semantics in e-Science • RDF-based service and data registries • RDF-based metadata for experimental components • RDF-based provenance graphs • OWL based controlled vocabularies for database content • OWL based integration ISGC

26. CombeChem pilot project Video Simulation Properties Analysis StructuresDatabase Diffractometer X-Raye-Lab Propertiese-Lab Grid Middleware ISGC

27. FloodNet The “inner loop” subscribers Simulated Nodes users Peer-to-peer computing GIS gateway broker GPRS Flood prediction Sensor Nodes Live data informs predictions grid Predictions influence sampling and reporting rates Meteorological data The “outer loop” ISGC

28. GGF9 Semantic Grid Workshop • The Role of Concepts in myGrid Carole Goble • Planning and Metadata on the Computational Grid Jim Blythe • Semantic support for Grid-Enabled Design Search in Engineering Simon Cox • Knowledge Discovery and Ontology-based services on the Grid Mario Cannataro • Attaching semantic annotations to service descriptions Luc Moreau • Semantic Matching of Grid Resource Description Frameworks John Brooke • Interoperability challenges in Grid for Industrial Applications Mike Surridge • Semantic Grid and Pervasive Computing David De Roure ISGC

29. E-Science Special Issue • IEEE Intelligent Issue Special Issue on E-Science, Jan-Feb 2004 • De Roure, Gil, Hendler • Challenges: • Realising the network effect • Moving beyond centralised stores • Automated assembly • Collaboration tools ISGC

30. GGF11 Semantic Grid Workshop • Using the Semantic Grid to Build Bridges between Museums and Indigenous Communities Schroeter • Collaborative Tools in the Semantic Grid De Roure • The Integration of Peer-to-peer and the Grid to Support Scientific Collaboration • OWL-Based Resource Discovery for Inter-Cluster Resource Borrowing Yoshida • Semantic Annotation of Computational Components Vanderbilt • Interoperability and Transformability through Semantic Annotation of a Job Description Language Hau • Engineering semantics: Costs and Benefits Cox • Designing Ontologies and Distributed Resource Discovery Services for an Earthquake Simulation Grid Pierce • Exploring Williams-Beuren Syndrome Using myGrid Goble • Distributed Data Management and Integration Framework: The Mobius Project Hastings • eBank UK - Linking Research Data, Scholarly Communication and Learning De Roure ISGC

31. Dagstuhl Semantic Grid Workshop semantic web applications semantic web p2p agents logic applications grids grids hybrid hybrid ISGC

32. Next Generation Grids Reports NGG3 – 2005 Future for European Grids: GRIDs and Service Oriented Knowledge Utilities Vision and Research Directions 2010 and Beyond Main source of inspiration for FP6 Grid Research and beyond NGG2 – 2004 Requirementsand Optionsfor European Grids Research 2005-2010 and Beyond NGG1 – 2003 European Grid Research2005 – 2010 http://www.cordis.lu/ist/grids

33. NGG2 Grid Research Projects in FP6 (Call 2) EU Funding: 52 MILLION Started: SUMMER 2004 Grid@Asia Towards EU-Asian Co-operation GridCoordBuilding the ERA in Grid research K-WF GridKnowledge basedworkflow & collaboration inteliGRIDSemantic Grid based virtual organisations Grid-based generic enabling application technologies to facilitate solution of industrial problemsSIMDAT OntoGridKnowledge Services for the semantic Grid UniGridSExtended OGSAImplementation based on UNICORE EU-driven Grid services architecture for businesS and industry NextGRID Mobile Grid architecture and services for dynamic virtual organisations Akogrimo DataminingGridDataminingtools & services HPC4UFault tolerance,dependabilityfor Grid European-wide virtual laboratory for longer term Grid research-creating the foundation for next generation Grids CoreGRID ProvenanceTrust and provenance for Grids Specific support action Integrated project Network of excellence Specific targeted research project ISGC

34. Application 1 Application N Security Optimization Data OGSA Execution Management Semantic-OGSA Semantic Services Resource management Information Management Infrastructure Services ISGC

35. Semantic Provisioning Services Semantic binding Knowledge Metadata Ontology Annotation Reasoning Application 1 Application N Security Optimization Data OGSA Execution Management Semantic-OGSA Semantic Services Resource management Information Management Infrastructure Services ISGC

36. Outline • The ambition • Enabling Technologies • Grid • Semantic Web • Semantic Grid • Are we there yet? • The Future ISGC

37. Virtual Learning Environment Reprints Peer-Reviewed Journal & Conference Papers Technical Reports LocalWeb Preprints & Metadata Institutional Archive Publisher Holdings Certified Experimental Results & Analyses Data, Metadata & Ontologies eBank Undergraduate Students Digital Library Graduate Students E-Scientists E-Scientists E-Scientists Grid Entire E-Science CycleEncompassing experimentation, analysis, publication, research, learning E-Experimentation ISGC

38. CombeChem Smart Tea www.smarttea.org ISGC

39. Annotation at Source ISGC

40. ISGC

41. CombeChem Semantic Datagrid • A social experiment! • Existing datastores linked by triplestores • 3 stores, with 70 million triples • Chemists appreciate powerful queries and flexibility • Metadata infrastructure is in place ISGC

42. BuddySpace BuddySpace Meeting Replay Jabber Server Compendium Compendium I-X Process panels I-X Process panels KMI, AIAI ISGC

43. NASA Scenario 1. Astronauts debrief on EVA Compendium maps from trained compendium astronaut Remote Science Team (RST) on earth e.g. geologists Mars Video and Science Data Plan for next Day’s EVA 2. Virtual meeting of RST using CoAKTinG tools ISGC

44. Image from NASA ISGC

45. Outline • The ambition • Enabling Technologies • Grid • Semantic Web • Semantic Grid • Are we there yet? • The Future ISGC

46. Research agenda March 2005 • Automated Virtual Organisation Formation and Management • Service Negotiation and Contracts • Security, Trust and Provenance • Metadata and Annotation • Content Processing and Curation • Knowledge Technologies • Design and Deploy • Interaction • Collaboration • Pervasive Computing ISGC

47. Service-Oriented Knowledge Utility NGG3 The architecture comprisesservices which may be instantiated and assembled dynamically, hence the structure, behaviour and location of software is changing at run-time A utility is a directly and immediately useable service with established functionality, performance and dependability, illustrating the emphasis on user needs and issues such as trust Services are knowledge-assisted (‘semantic’) to facilitate automation and advanced functionality, the knowledge aspect reinforced by the emphasis on delivering high level services to the user ISGC

48. Web and Web Services NGG3 Methodologies Service Oriented Architecture Grid Stateful Service Utility Agent Technologies Autonomic Stateful Service Utility Semantics Societal Autonomic Stateful Service Utility Heuristics Knowledge-aware Societal Autonomic Stateful Service Utility Formal Languages Reliable Knowledge-aware Societal Autonomic Stateful Service Utility = SOKU Next Generation Grids and SOKU ISGC

49. Service-Oriented Knowledge Utility NGG3 The primary difference to earlier approaches is a switch from a prescribed layered view to a multi-dimensional mesh of concepts, applying the same mechanisms along each dimension across the traditional layers. ISGC

50. NGG3 Scenarios • Enterprise • illustrates the power of the virtualisation and interoperability provided by Grids and SOKU within the enterprise context • End-user • shows the role of Grid in delivering public information services (‘knowledge utilities’) which respect ownership and privacy issues • Manufacturing/Industrial • shows how these approaches benefit collaborative processes within industry ISGC