1 / 7

dPROOF Implementation on the Grid: Insights and Progress Update

This document presents an overview of the ongoing development of dPROOF, which integrates PROOF with AliEn for interactive parallel analysis on distributed clusters. The system operates using a multi-master architecture across various sites, including CERN, GSI, and CNAF. The process involves a series of interconnected ProxyClients and ProxyServers utilizing Xrootd. Key features include seamless user workspace integration, stress testing with a peak processing rate of 400MB/s, and testing at multiple Grid sites. Continuous improvements and adaptations for the ALICE collaboration are also discussed.

cree
Télécharger la présentation

dPROOF Implementation on the Grid: Insights and Progress Update

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PROOF on the Grid Marco MEONI CERN - Offline Week – 28.10.2009

  2. dPROOF distributed PROOF is an ongoing development A combination of PROOF and AliEn…… to achieve interactive parallel analysis on distributed clusters exploiting data locality Grid Site C Grid Site A Grid Site B

  3. A ProxyServer starts Xrootd and PROOF master • JAs are sent to the VOs and start ProxyClients • A ProxyClient starts an Xrootd server and registers to the ProxyServer • A ProxyServer keeps the list of all the partecipating nodes • A user starts a dPROOF session connecting the topMaster 1 User workspace 2 AliEn-LCG Grid 3 Grid API ROOT AliEn-LCG WMS 4 5 5 PROOFtopMaster 1 VO-box 1 VO-box n VO-box m ProxyServer Xrootdmanager PROOFlocalMaster 4 2 3 ProxyClient JA WN1 WN2 WNn Xrootd server PROOFworker WN workspace Site workspace

  4. Execution (CERN)

  5. Execution (GSI)

  6. Participating Sites http://pcalimonitor/dproof/ • CNAF (by the end of the week) • Kosice • NIHAM 4 steps to enable dPROOF on a VObox http://cern.ch/mmeoni/dproof/

  7. Overview ALICE is adapting PROOF to the Grid The framework is under test at 3 sites 3 sites will most likely join within the next week The multi-master PROOF architecture is under test and improvement There is room for changing/improving the policy of JA submission Workgroup meets twice a week since a month Stress test per-site works steadily for hours and keep the PROOF workers busy Test with up to 100 workers on a single site (CERN), processing rate peak 400MB/s Acknowledgement Adam, Anar, Andreas, Andrei, Axel, Bellenot, Costin, Cvetan, Jan Fiete, Latchezar, Federico, Mariella, Matevz, Pablo, Peter

More Related