1 / 24

Production Grid Challenges in Hungary

This presentation explores the complexities of grid initiatives in Hungary, focusing on the ClusterGrid project. It addresses significant challenges in a production environment, including the need for clear objectives, system transparency, security, compatibility, and user support. The discussion outlines management issues related to computing resources, storage, and user engagement, as well as monitoring strategies. A look into future challenges highlights the increasing demand for reliable computing and data storage infrastructures, emphasizing the importance of interoperability and international standards in grid systems.

leane
Télécharger la présentation

Production Grid Challenges in Hungary

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Production Grid Challenges in Hungary Péter Stefán Ferenc Szalai Gábor Vitéz NIIF/HUNGARNET

  2. Agenda • Brief introduction • Grid initiatives - ClusterGrid • Challenges in a production environment • Generic ClusterGrid operation model • Management issues • User support • Monitoring • ClusterGrid future challenges • Conclusions

  3. collaborative infrastructure computing infrastructure middleware infrastructure networking infrastructure Brief NIIF Introduction • Hungarian NREN. Videoconference, central HA cluster GRID supercomputing VPNs, VoIP, directory service IP, IPv6, MPLS, lambda etc 10G backbone, ~600.000 users, ~750 institutions

  4. Supercomputers • Consists of 2 SUN E15Ks and 2 SUN 10Ks located at two universities, including 276 CPUs, 300 GB of memory. • Used to be in the top 500. • In production since 2001. • Serves more than 200 users, and 100 scientific projects.

  5. Hungarian grid initiatives, MGKK • Hungarian grid initiatives can be classified into grid infrastructure and grid system development projects. • Key role-players formulate grid collaboration: Hungarian Grid Competence Center (MGKK) involving BUTE, ELUB, MTA-SZTAKI, NIIF/HUNGARNET, KFKI, University of Veszprém. • Intensive participation in many national and European grid initiatives: EGEE, NorduGrid, SEE-GRID, etc.

  6. ClusterGrid initiative • It is a pool of 1400 PC nodes throughout the country involving more than 26 clusters. • Production infrastructure since July 2002. • Supercomputer clusters are planned to be involved too. • A rough measurement on the total compute capacity is about 600 Gflops. • Even though it is much smaller than regional, continental grids, in complexity it is at the same range.

  7. Challenges in production environment • Grid definition - set clear objectives what to build • Simplicity - keep the system transparent, usable • Completeness - cover not only application level • Security - using computer networking methods (MPLS, VLAN technologies) • Compatibility - other grids (X509, LDAP) • Manageability - easy-to-maintain • Robustness - fault tolerant behavior • Usability - cover many job classes, user support • Platform independency - to be able to execute on MS

  8. ClusterGrid architecture

  9. Some new ideas… • MPLS, VLAN connected resources • Web-transaction based resource broker • Dynamic, separated run-time environment

  10. Generic production service model

  11. Challenges in production … cont’d … • Management • physical compute resources (supercomputers, clusters), • virtual resources (virtual clusters), • storage nodes, • users, • services • User support • Grid architecture monitoring

  12. Compute-cluster management

  13. Virtual compute-cluster management

  14. Virtual compute-cluster management

  15. Storage management • Low level management of disks and volumes, file systems (cost efficient storage solutions by using ATA over Ethernet - AoE). • Medium level access management (gridFTP, FTPS). • High level data brokering (extended SRM model).

  16. User management • User personal data is kept in an LDAP based directory service separately from authentication data. • Aided by a web registration interface. • Authentication: • X509 certificates, • LDAP based authentication. • No authorization yet.

  17. Service management (experimental) • Relatively new direction. • It is a special service. • It is based on well-established authorization. • Basically helps to start, stop, (re)configure grid services.

  18. User support • Grid service provider gives user support covering: • consultation about the benefits of grid usage, • code porting and optimization, • partial aid in code implementation, • job formation and execution, • generic grid usage. • Not yet covered: • model creation, • formal description, • algorithm creation.

  19. ClusterGrid monitoring • Fluctuation of grid cluster resources between the day-shift and night-shift operation. • Blue line – total; Green area – occupied. • 2-layer hierarchical monitoring system.

  20. ClusterGrid monitoring

  21. ClusterGrid monitoring

  22. Future ClusterGrid (?) challenges • Continuously growing demands for reliable compute and data storage infrastructure. • Grid systems should conform to international standards and MUST interoperate with one another. • Platform-independency is not an issue yet, but will be. • LEGO-based principles are of increasing importance. • Threats: solutions that prevent development; erosion of the belief in the power of “grid”.

  23. Conclusions • One of the first production-level grids have been shown in a nutshell. • With special emphasis on operation, management and user support issues. • Management generally covers grid resource, grid user management and monitoring. • Some remarks regarding future development were also done.

  24. Thanks for your attention! www.clustergrid.hu www.mgkk.hu grid-tech@niif.hu

More Related