Multi-Grid Resource Usage Service in LCG

1. Multi-Grid Resource Usage Service in LCG X. Chen, A. Khan Brunel University

2. Outline • Introduction to Grid accounting • Accounting in LCG • LCG-RUS • Architecture • Performance Test • Issues • Next-Step • OGSA-DAI RUS • Standardisation • Summary

3. PART IGrid Accounting

4. Grid Accounting Overview What is Grid accounting • Providing Grid-wide view of resource usage; • Enabling usage auditing, accounting and economic model;

5. Key Concepts • Usage Metrics • Job ownership (who runs what job at what time); • Job submission (Where and When) the job is submitted); • Consumption Properties (What and how many resources consumed during the job execution); • OGF Usage Record Working Group1 • Resource Usage Service • Common operations on usage records; • Extraction of usage records; • Populate usage records; • OGF Resource Usage Service Working Group2 1http://www.psc.edu/~lfm/PSC/Grid/UR-WG/ 2https://forge.gridforum.org/projects/rus-wg

6. Usage Record (UID) Record Identity (UID) Job Identity Job Ownership Resource Consumption Properties (CPU, Storage, Memory…) Normative Usage Record Information Model

7. Web Service Container Service Interface RUS WS Application ACL DB Resource Usage Service • Web Service endpoint • Common usage data operation; • Insert into usage records; • Extraction usage records for Grid entities; • Access Control is the main concern;

8. PART IILCG Accounting

9. Accounting in LCG Accounting in LCG to obtain usage information from three Grid resources Resource Sharing From three Grid infrastructures

10. Requirements • Data Interoperability • Uniform data sharing mechanism; • Common usage data format; • Accounting for various grid entities (user, VO, site…); • Job and aggregate/summarised accounting; e.g. The total usage information of jobs belonging to a particular VO during a period of time (month, week, day…); • Performance • On-demand data processing on huge amount of data (over 1900 job usage records / hour); • Security • Data privacy (e.g. user DN); • Role-based access control; • Accommodate cross-domain access control policies; Records in the storage by this week Over 17 Million Records Since 2006

11. HLR HLR SGAS GOC RAL Aggregation periodically LCG schema UR XML UR summary UR Site UR APEL aggregate and email spreadsheets DGAS Gratia UR XML UR Landscape Relational DB

12. Unresolved Issues • Heterogeneous usage representation • Relational against XML; • Standardization against LCG-specific schema; • Lack of interoperability • Between various accounting systems; • Web portal is not good enough for interoperability to high-level applications (economic model for example); • Aggregate Accounting only • No job usage information exhibited at front end; • No user-level accounting; • Not access control with consequences in • OSG only share aggregate/summarised usage information; • NorduGrid does not share usage information at all;

13. PART IIILCG Resource Usage Service

14. LCG-RUS Overview • An “RUS-like” implementation because of • RUS specification was still under development; • usage data model is based on LCG-specific schema; (http://goc.grid-support.ac.uk/gridsite/accounting/interoperate.html) • But… • Provide dynamic data sharing model; • Enable server-side aggregation; • Realise Job and aggregate accounting as a whole and • Therefore support accounting for various grid entities (including user/vo/site-level accounting); • Simple role-based access control for data privacy; • All these aspects are within OGF-RUS scope;

15. RUS Service Endpoint Aggregate Usage Record PUSH insert deploy Job Usage Record aggregate usage record stream Site Manager PULL Job usage record stream Data Sharing • LCG-RUS support two data sharing models; • Push Model • Contribute aggregate usage records to LCG-RUS via “insert” service interface; • Pull Model • Contribute job usage records to LCG-RUS via “deploy” service interface; • The deployment only writes remote/site database access information into LCG-RUS; • RUS stores data source access information into an XML file, known as data source container; • A back-end thread called “aggregator” periodically access the data source container and aggregate remote job usage information into LCG-RUS storage; Aggregate usage storage Data Source Container aggregator

16. <?xml version=“1.0” ?> <DataSources> <DataSource> <provider>MySQL</provider> <url>jdbc:mysql://localhost:3306</url> <storage>tableName</storage> <account>lcgrus</account> <passphrase>lcgrus</passphrase> </DataSource> …… Server-side Aggregation • Aggregate accounting ensures • Maintenance of relative small number of usage records; • Serving user request on the fly; • Data privacy; • Server-side aggregation • Remote usage storage deployed is known as a data source; • The deployment operation will write a block of access information for that data source into registry file (an xml file as the example); • The service runtime spawns a set of aggregator threads, each corresponds to a single data source entry; • each records summarise usage record as the granularity of per user, per VO, per site during a period of time configured at service deployment stage; Job Records Since 2004

17. Access Control • Roles • Five predefined roles proposed by EGEE; • Role-Based Access Control • Simple XML access control file with “user DN” and role mapping; • Hard-coded access control mechanisms in the implementation; <lcgrus:roles> <lcgrus:SiteManager DN=“/c=uk/o=eScience/ou=Brunel/L=ECE/CN=xiaoyu chen” SiteName=“BNL” /> …… </lcgrus:roles>

18. Workflow Example • User/VO level Accounting • HTTPS based authentication; • Every authenticated user automatically takes the role of Grid user; • The user who claimed to be the role of “VO manager” has to be authorised by query access control file; • Get aggregated usage information via “extraction” service interface; • On-demand job usage tracing through “tracing” service interface on deployed usage storages at sites; Deployed usage storages

19. Performance Test • Test Platform • RUS server specification • A Client toolkit is used to spawn a number of threads, each of which simulates a site of the grid; • The toolkit is installed in three PCs to simulate three Grid infrastructures of LCG • Observations • Aggregation time • The time elapsed for summarisation of usage information maintain by RUS; • Aggregation time with the number of usage records of a single record data source; • Aggregation time with the number of record data sources; • Tracing Time • The time elapsed for finding a particular job usage records; • Tracing time with the number of usage records in a single record data source; • Tracing time with the number of record data sources;

20. Test Result (Granularity concept) Data Source Granularity=number of data sources deployed (In Milliseconds) (In Milliseconds) 80000 Conclusion: Keeps job usage records distribute give better performance of RUS service in LCG.

21. Publications • X.Chen and A. Khan, “LCG-RUS: Aggregative accounting service enabling economic modelling for commercial Grid”, Proceedings of Science, Grid in Finance 2006; • X. Chen and A. Khan, “Design and Performance Analysis of Resource Usage Service for LCG”, IEEE NSS2006;

22. PART IVFuture Work

23. Next Step • Fully OGF-RUS compatible • Record in full OGF-UR compatible; • Normative Service Interface Definitions; • Flexible Access Control instead of hard-coded; • Interoperability to other RUS implementations

24. Challenges Ahead • Usage Representation • Even UR compatible, custom property extensions are also possible; • No standard aggregation representation schema; • RUS Implementation • Distribute usage record sharing; • In turn, distribute “update” transaction; • Usage record sharing via third-party RUS implementations; • Access control policies from various communities;

25. Proposed Solution • Standardization • Aggregate Usage Representation (https://forge.gridforum.org/sf/go/doc14336?nav=1) • Resource Usage Service • OGF-RUS WS-I core specification v1.9 (https://forge.gridforum.org/projects/rus-wg); • OGF-RUS WS-I advanced specification (April, 2007) • Implementation • OGSA-DAI as distribute data management; • OGSA-DAI Extension for third-party RUS; • XACML-based access control • Authorisation policies from different layers (data sharer, RUS core, OGSA-DAI and etc); • On-request authorisation based on combined policies;

26. Our Scope OGF-RUS Roadmap

27. XML Job UR XML Job UR Site Site OGSA-DAI RUS architecture These properties are refreshed each time new data inserted in order to ensure efficient query RUS Response Request RUS Implementation XACML Policy authorization OGSA-DAI Request Wrapper OGSA-DAI Response OGSA-DAI Request OGSA-DAI service Server-side Aggregation Extensions Extensions for RUS Implementations as usage Data Sources Properties for User/vo/resource … accounts RUS policy Authorizer Other RUS Implementations

28. RUS Logic Site manager RUS manager Data provider User OGSA-DAI Site Sharing Policies RUS Composite Policies RUS Core Policies DB Access Policies Third-party RUS Implementations OGSA-DAI RUS Authorization Runtime Policy Enforcement Point (transfer user request into XACML request context) Policy Decision Point (authorization decision making RUS AC Module Request Context WS Request PEP PDP Forward Request

29. Summary • Why Grid accounting in LCG • Healthy resource sharing and economic modelling; • How Accounting in Multi-Grid • LCG-RUS (dynamic data sharing model); • Introduction Aggregate Accounting • High performance (responsiveness); • Less maintenance task; • Why Access Control • Data privacy; • Towards “nonymous” (User/VO-level) accounting; • Next Goal • Normalisation; • Flexibility and Extensibility;

30. The End Thanks!

31. Backup slides

32. create Simple RUS Implementation LCG-RUS Implementation OGSA-DAI RUS Implementation Third-party RUS Implementations Possible Future Architecture (backup) • Multiple Implementation Deployment • Initialize appropriate RUS implementation at the deployment time; • Simple RUS implementation provides solution for single usage storage; • LCG-RUS provides solutions for multiple usage storages; • OGSA-DAI RUS extends from LCG-RUS with interoperability to third-party RUS implementations (Gratia, SGAS, APEL, DGAS…); Deployment descriptor Factory class Init ( ) Default RUS Implementation Factory Framework