1 / 29

HyperDAQ

HyperDAQ. Where Data Acquisition Meets the World Wide Web. Johannes Gutleber, CERN Dept. PH/CMD. Outline. What is HyperDAQ? Why is it done? What is it good for? How is it done? Where is it used? Summary. The Web is all about linking documents. HyperDAQ

oliver-nichols
Télécharger la présentation

HyperDAQ

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HyperDAQ Where Data Acquisition Meets the World Wide Web Johannes Gutleber, CERN Dept. PH/CMD

  2. Outline • What is HyperDAQ? • Why is it done? • What is it good for? • How is it done? • Where is it used? • Summary Johannes.Gutleber@cern.ch

  3. The Web is all about linking documents Johannes.Gutleber@cern.ch

  4. HyperDAQ is all about linking applications Johannes.Gutleber@cern.ch

  5. Each application in a distributed system • is directly browsable • can discover others • can link to the others Johannes.Gutleber@cern.ch

  6. A way to provide access to distributed data acquisition systems through World Wide Web + Peer to Peer What is HyperDAQ? Johannes.Gutleber@cern.ch

  7. Why is it done? To realize the potential of distributed data acquisition systems, access to information must be simple. See system as single entity from anywhere “drill into the system” Johannes.Gutleber@cern.ch

  8. What is it good for? • Monitor distributed applications • Control distributed applications • Help distributed development Johannes.Gutleber@cern.ch

  9. HTTP engine Applications serve Web pages link How is it done? Contents are linked together Applications can be discovered Address applications by URN Johannes.Gutleber@cern.ch

  10. URN • Uniform Resource Name • Identifies resources on a computer • Applications • Services • Data sources urn:xdaq-application:service=monitor Johannes.Gutleber@cern.ch

  11. URL • Uniform Resource Location • Gives context to URN • Calls operations on URN http://host:port/ urn:xdaq-application:service=monitor/ retrieve?flash=cpuUsage Johannes.Gutleber@cern.ch

  12. Toolkit for distributed data acquisition • C++, cross-platform • Enablers for high-performance operation • Zero copy message passing • Concurrent use of multiple transports • Memory pools • Run-time extensions • Application components • Peer-transport modules Johannes.Gutleber@cern.ch

  13. HyperDAQ Programming • Incoming HTTP requests call handlers • Input stream • Output stream • Libraries for creating Web content • Cgicc (GNU) • Mimetic (GPL) • XGI (XDAQ) Johannes.Gutleber@cern.ch

  14. Service Discovery • Abstract discovery interface • Pluggable implementations • SLP: Service Location Protocol • UPnP: Universal Plug and Play • JXTA Optional – but simplifies configuration Johannes.Gutleber@cern.ch

  15. Modular Approach Core libraries Data serializers Logging HyperDAQ core Monitoring tools Control tools Security modules Discovery services Peer transports Event builder DAQ monitoring Hardware access Johannes.Gutleber@cern.ch

  16. Where is it used? Johannes.Gutleber@cern.ch

  17. CMS - Distributed DAQ Readout Units Buffer event fragments Event fragments: Event data fragments are stored in separated physical memory systems Event Manager Interfaces between RU, BU and trigger Full events: Full event data are stored in a single physical memory system associated to a processing unit Builder Units Assemble event fragments Requirements: L1 trigger: 100 kHz (@2KB), ev-size 1MB, 200 MB/s in AND out per RU, 200 MB/s in AND 66 MB/s out per BU Johannes.Gutleber@cern.ch

  18. Control XAct is a job controller. It is a HyperDAQ application Operations Link to the mainapplication for each computer Johannes.Gutleber@cern.ch

  19. Monitor Data collected from computers Widget Collected data rendered graphically Johannes.Gutleber@cern.ch

  20. Direct Access Johannes.Gutleber@cern.ch

  21. Johannes.Gutleber@cern.ch

  22. Simplify Integration Loosely coupled development collaborations write programs with HyperDAQ interfaces link together their programs Monitoring Controller 1 Controller 2 Johannes.Gutleber@cern.ch

  23. Summary • simplify access to distributed systems • give direct control to any application • decreaseconfiguration complexity • enable loosely coupled development By linking applications we Johannes.Gutleber@cern.ch

  24. Information http://xdaqwiki.cern.ch http://www.sourceforge.net/projects/xdaq Johannes.Gutleber@cern.ch Johannes.Gutleber@cern.ch

  25. Backup Johannes.Gutleber@cern.ch

  26. Security • XAccess Module • Basic Web authentication • IP based filtering • Combination of both • SSL • For server and client authentication • Custom policy • Any policy can be “plugged-in” Johannes.Gutleber@cern.ch

  27. Clients • Web browser • LabView • MS Excel Johannes.Gutleber@cern.ch

  28. Installations • Magnet test cluster in Cessy • 64 computers • Adopted by subdetectors • Used for commissioning • DAQ Monitoring • Event builder (8 slices, 64x64) • Front End readout link (512) Johannes.Gutleber@cern.ch

  29. Monitorable Entities • 512 FRL • 512 RU (64 * 8) • 512 BU (64 * 8) • 50 FRL Controllers (32 partitions, at most 2 hierarchy levels) • 4 FMM Controllers • 8 EVM (1 per partition) • 200 DCS • 16 Myrinet Switches (8 Fed builder, 8 RU builder) • 2 Ethernet Switches • 8 Run Control hosts (1 per partition) • 64 Filter Subfarm Controllers • 4096 Filter nodes (512 subfarms with 8 nodes each) • 5984 Total • From 6000 nodes, 1 MByte/s (monitored data at 1 HZ) Johannes.Gutleber@cern.ch

More Related