1 / 14

Remote Online Farms

. 0. Remote Online Farms. 01 11 010 001 1101 1110 11001 01011 110110 001101 1111111 0111000 11101010 01001110 110111001 000101101 1111010001 0101111100 111101001111 010110000101. Sander Klous. On behalf of the Remote Online Farms Working Group. TDAQ. 25 April.

abdalla
Télécharger la présentation

Remote Online Farms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 0 Remote Online Farms 01 11 010 001 1101 1110 11001 01011 110110 001101 1111111 0111000 11101010 01001110 110111001 000101101 1111010001 0101111100 111101001111 010110000101 Sander Klous On behalf of the Remote Online Farms Working Group TDAQ 25 April ACAT 2007 W H Z t

  2. Large Hadron Collider and ATLAS

  3. Data processing nightmare • There is no way to store all the info produced by ATLAS 40 million events per second x 1.5 MB/event = 60 TB per second • In fact: 99.9995% of the data is thrown away • So…The data processing nightmare is all about storage Unfortunately… No • Rigorous multilevel trigger system • First level in hardware • Higher levels in software • But what if your favoritechannel is not in the 0.0005%?

  4. So…The data processing nightmare is all about CPU Well, that’s a problem… Online bottleneck Scarce CPU resources LVL1 HLT

  5. Detector calibration Physics selection Some are more equal than others… Networking enables us to prioritize these activities • So… • The data processingnightmare is all aboutnetworking

  6. Data Acquisition 40 MHz Level 1 Level 2 Level 3 Amsterdam Accept 1 in 500 NIKHEF/SARA Network switch Accept 1 in 50 Accept 1 in 10 Economics In fact, it is about balance Maximize performance  Minimize costs Computing grid

  7. Is it worth the effort? Gary Stix, editor of Scientific American January 2001

  8. RF RF RF RF EF SFO This can be difficult… The basics Copenhagen Edmonton Krakow Manchester Amsterdam North Area ATLAS Detectors Level 1 Trigger Remote Event Processing Farms ROB ROB ROB ROB Data Collection Network Packet Switched (GEANT) SFI SFI SFI lightpath Back End Network L2PU Event Filter Bdlg. 513 Mass storage Switch Local Farm The “Magni” Cluster

  9. Event (Partial) Event (Partial) Event Event (Partial) Event LVL1 Info LVL1 Info LVL1 Info LVL1 Info LVL1 Info Ath / PESA RoutingTag RoutingTag RoutingTag RoutingTag RoutingTag StreamTag 2 StreamTag StreamTag 2 StreamTag 1 StreamTag 1 Ath / CALid PT Ath / CALIB Ath / CalStr PT Event LVL1 Info Stream n Stream 1 Stream 2 Stream implementation Remote From LVL1 LVL2 SFO RoiB Ros/Robin pRos Stream selection Add to StreamTag1 EFD L2sv Input Create RoutingTag Routing DFM B ExtPT L2pu SFI (partially) build Routing Add to RoutingTag Create StreamTag D C Trash ExtPT Add to StreamTag2 Duplicating Output Output Stripping

  10. SFI SFI SFI PT PT PT PT PT PT PT PT PT PT PT PT Grid/Proxy implementation UI Broker EFD Dispatcher Buffer EFD Buffer Infrastructure monitoring ProxyPT HEP VO Database PT PT PT PT PT int.eu.grid HEP VO PT Application Monitoring Local PT Farm CE CE Events Remote PTs Worker Nodes Worker Nodes

  11. Open issues • Data management • Software management • Database access • Authentication, Authorization and Accounting • Performance and reliability • Looking for PhD student… • If you are interested: • Mail to sander@nikhef.nl

  12. Conclusion • Remote online farms are interesting • From a physics perspective • From a computer science perspective • From an organizational perspective • Infrastructure is put in place • Many open questions • More news next year • PhD candidates: mail to sander@nikhef.nl

More Related