1 / 4

Transitioning ATLAS from RFIO to XRootD for Enhanced File Access in UK DPM Sites

The ATLAS collaboration is transitioning from RFIO to XRootD for file access in UK DPM sites. This change responds to performance issues and a desire to retire RFIO, supported by the WLCG Storage Interfaces group. Testing of XRootD has been ongoing since last June, and while some initial problems were noted, the system has stabilized on the DPM server side. A working configuration for xrdcp is in place, and direct access needs to be optimized. Future discussions will focus on the rollout strategy and addressing concerns like HTTP copying and FTS transfers.

kareem
Télécharger la présentation

Transitioning ATLAS from RFIO to XRootD for Enhanced File Access in UK DPM Sites

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Rfio -> Xrootd for ATLAS on DPM • All UK sites use rfio for local file access • Perceived performance issues • Only used in copy mode (for atlas) not direct • Sometimes there are issues with client libs (e.g. EMI 32bit ones; link to libshift.so; CMS many issues) • DPM developers want to move away • WLCG “Storage interfaces” group supported decision to “retire” rfio. • CMS requested / required it in WLCG ops mtg • Noone objected / some people supported moving (ATLAS) in GridPP storage meeting

  2. Testing… • We’ve been testing xrootd in UK since last Jun (!) • Initially some issues but everything stable on the DPM server side for a while now. • (The issues seen on FAX tests are N2N / Fax specific and shouldn’t affect local xrd ops) • (Though also our HammerCloudsat Glasgow didn’t show any performance gain) • Basically only schedconfigbattles remain • I agree with others (Alessandra, Ewan etc.) that moving into production is possible/ preferable.

  3. SchedConfig changes https://twiki.cern.ch/twiki/bin/viewauth/Atlas/DPMFaxXrootdSchedconfig Seem to have a working config for xrdcp now. (Direct-access one works for analysis) Reconfirm on ECDF_8CORE queue this week Try ECDF and OX prod queues next Monday? (Manchester test queue too) Then roll-out slowly after - volunteer sites first

  4. A few debateables • Direct or copy-to-scratch: push copy first but I would like direct access to work well • Should we push http copy at some sites?   'copytoolin' : curl “works” As further steps can also use xrdcp for • Stage-out (is lfc registering taken care of elsewhere? What about SpaceTokens ). • FTS transfers (soon in FTS3 (again possible ST issue))

More Related