1 / 11

Enhancing StoRM: Activities and Improvements from TEG DM & SM Workshop at Nikhef

This document summarizes the activities and improvements discussed during the TEG DM & SM workshop held at Nikhef, Amsterdam, from January 24-25, 2011. Focus areas include short and medium-term strategies for StoRM's maintenance, development, robustness, security, and compliance with standards. Key topics covered are service monitoring, fault tolerance, integration with Argus for security enhancements, and support for various data access protocols, including cloud storage solutions. The goal is to improve the overall quality and sustainability of the StoRM product in the European Grid Infrastructure context.

alexa
Télécharger la présentation

Enhancing StoRM: Activities and Improvements from TEG DM & SM Workshop at Nikhef

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Riccardo Zappi INFN-CNAF StoRM until 2013 and beyond TEG DM & SM, NIKHEF, Amsterdam January 24-25, 2011

  2. Activities January 24-25, 2011, TEG DM & SM, NIKHEF, Amsterdam • Short/Medium term activities: • defined within the EMI context • maintenance and development • Other activities are based on requests and suggestions coming from: • Customers (via EGI and EMI, or directly) • Working groups like TEG

  3. Activities and Improvements (1/3) January 24-25, 2011, TEG DM & SM, NIKHEF, Amsterdam • Robustness: • reactive interventions • bug fixes • user support • proactive activities: • test suites and unit tests • standard adoption • e.g. File System Hierarchy Standard (FHS) compliance • improving product quality

  4. Activities and Improvements (2/3) January 24-25, 2011, TEG DM & SM, NIKHEF, Amsterdam • High Availability, Fault Tolerance • Twin-BE • Service Monitoring • Heartbeat enhancement • Ganglia module • Responsiveness • Synchronous PtG • Performance on SRM call executions

  5. Activities and Improvements (3/3) January 24-25, 2011, TEG DM & SM, NIKHEF, Amsterdam • Security • Argus Integration: • Ban list on DN and VOMS group • Storage Area protection: • Evaluating to use Argus • Space Accounting • StAR (Storage Accounting Record) implementation • Catalogue synchronization mechanism • As defined in EMI

  6. Storage Systems support January 24-25, 2011, TEG DM & SM, NIKHEF, Amsterdam • StoRM relies on underlying file systems • It works with any POSIX FS with ACL and EA enabled • StoRM product is committed to : • support all the wished data access/transfer protocol • http(s), (s)ftp, gsiftp, dav, xrd, torrent, ... • support all the most used, promising and required FS • HDFS, NFSv4.1 • investigate on how to integrate cloud storage solution • AWS-S3, Google CloudStorage

  7. Issues and Solutions to support new SS January 24-25, 2011, TEG DM & SM, NIKHEF, Amsterdam • No ACL support (e.g. AWS-S3, HDFS) • No more Direct Access support (security issue) • Solutions: • No file protocol support: use it as a second tier in ahierarchical configuration (like a MSS) • Adapt StoRM to use only standard permission (minor flexibility) • No EA support • No more default store for checksum values • No more StoRM-system metadata with files • Solutions: • Use of hidden files (like SVN) • short term DB

  8. Supported Protocols January 24-25, 2011, TEG DM & SM, NIKHEF, Amsterdam • File protocol (file:// ) support: • file protocol is supported by design since StoRM was born • to take advantage from Cluster-FS (GPFS and Lustre) • "copy-to-WN" paradigm is used, then other data "access" protocol are (or will be) supported (rfio, xrd) • http e https support: • already supported both in read and write mode • require a SRM interaction (srmPtG and srmPtP with specified http(s) as desired protocol) • gsiftp, rfio and root

  9. Protocols under evaluation January 24-25, 2011, TEG DM & SM, NIKHEF, Amsterdam • WebDAV support • work in progress • It will available in 6-9 months • pNFS support • NFSv4.1 doesn't support EA • Checksum issue • Integration tests are expected : • on dCacheand/or BlueArc(upon availability) • GPFS v3.5: Active File Management (AFM) • Tests are ongoing at CNAF

  10. StoRM sustainability January 24-25, 2011, TEG DM & SM, NIKHEF, Amsterdam • StoRM is currently developed at CNAF • it is highly integrate in Tier-1 storage solution • StoRM is included into the portfolio of middleware products of IGI: • IGI is the Italian NGI (National Grid Initiative) and is part of the European Grid Infrastructure (EGI) • StoRM is developed in the context of EMI, ending in May 2013. Main goals are: • Improve robustness and quality • Harmonization (e.g. Argus integration) • Standard adoption (e.g. http(s), file://, ) • The sustainability is guaranteed by the context

  11. Questions ? January 24-25, 2011, TEG DM & SM, NIKHEF, Amsterdam

More Related