Management of a Cooperative Cache in Grids

1. Management of a Cooperative Cache in Grids Yonny Cardenas, Jean-Marc Pierson & Lionel Brunie LIRIS – INSA de Lyon/CNRS FRANCE

2. Outline • Introduction • Grid Cache Model • Cache Operations • Cache Information Model • Cache Service Implementation • Conclusion -

3. DATA DATA DATA DATA Data in Grids • Grids need to operate and control large and numerous data in a distributed way. • Grid data dissemination requires important network and storage resources (data replica) • A significant quantity of these data are used only for a limited period of time (temporary data). • Without global coordination, these resources are used in an inefficient way. -

4. Coordinated Cache CACHE • Caching has been demonstrated effective to manage temporary data. • Collaborative caches permit to reuse and to share data in distributed environments. • Multiple cache management mechanisms. •  Need to integrate such isolated caches into a global and coordinated grid service CACHE CACHE -

5. Grid Caching • Grids need high flexibility to change dynamically the parameters and schemes of collaborative caches • We propose a software infrastructure for managing coordinated caches • Operates and controls dynamically different cache mechanisms and cache schemes in grid. • Collaborative cache capacities are extended to manage distributed data CACHE CACHE CACHE CACHE -

6. Coordination Cooperation Operations Control Storage Cache Model -

7. Coordination Cooperation Operations Control Storage Storage Layer • Interacts with the storage resources used by the cache • Abstraction layer that allows isolated and heterogeneous storage mechanisms interoperate. -

8. Coordination Cooperation Operations Control Storage Control Layer • Controls and monitors the requests to the data and transmits them to the storage layer • Individually applies particular control methods and policies for internal cache management -

9. Coordination Cooperation Operations Control Storage Cooperation Layer • Supports intercache “classical” data requests. • Extends cache interactions to aspects related to monitoring and coordination of caches • Exposes its operations in a standard way accessible by a wide variety of clients  Cache grid Service -

10. Coordination Layer • Operates a coordinated management of multiple caches • Configures and establishes different mechanisms of collaboration (replication, fragmentation, migration…) • Collects and provides content and monitoring information : • description of cache content • data activity (accesses, transfers, etc) • cache activity (requests, replacements, etc) Coordination Cooperation Operations Control Storage -

11. Access OperationsPublish Data data organization cache organization PUT ? cache PUT PUT ? cache PUT ? GRID cache data data organization -

12. data cache organization data cache GET ? GET ? cache GRID GET ? cache data organization Access OperationsRetrieve Data -

13. organization cache data organization cache GRID cache Collective Coordinator cache data organization Monitoring and Management Operations Monitoring operations • data description • processed requests • transfers • replacements • used storage resource • Etc. Management operations • Set replacement method • Set default time to live • Set cache group • Set cache coordinator • Set collective work mode • Etc. -

14. Information Model • Storage information • storage resource • storage use • … • Cache information • Cache group • Cache coordinator • Cache replacements • Cache requests • … • Data information • data actions • data permissions • metadata • … • Transfers information • … • Defines the information to be exchanged between the components of the system • Represented as an XML DTD -

15. DATA DATA LCS DATA DATA Cache Service Implementation (1/2) • Local Cache Service (LCS) implemented as a grid service • Supports cache information exchange (XML) • Supports inter cache operations • Access Operations • Collective data retrieval • Collective data publication • Data content description • Transfers using GridFTP -

16. LCS LCS LCS Cache Service Implementation (2/2) • Monitoring Operations • Cache capacity • Data requests • Management operations • Dynamic configuration of replacement method • Dynamic configuration of data time to live • Dynamic configuration of cache groups • WSRF compliant • Developed using Globus Toolkit 4.1 • Deployed in three French laboratories -

17. Conclusion • Grids manipulate large volumes of large data. • Collaborative cache can operate and organize such data for reuse and sharing. • In this work, a basic infrastructure is proposed for the coordinated management of collaborative caches. • This infrastructure operates and dynamically controls multiple cache mechanisms and cache schemes disseminated over a grid. • The main components of the proposed infrastructure are: a management reference model, an information cache model, a set of generic cache operations. • A set of tests are under progress to evaluate the volume of resources consumed by the system and the performance benefits. -

18. Management of Cooperative Cache in Grids Thank you !