1 / 23

Enhancing Monitoring Systems at CERN: Progress and Future Directions

This progress report discusses recent developments in monitoring systems at CERN, including the deployment of a new monitoring sensor agent (MSA) with smart reconfiguration and self-monitoring capabilities. It also covers the ongoing work on alarm display, database structure, transport mechanisms, and upcoming releases. The report outlines the challenges faced, such as data loss and security issues, along with proposed solutions. Additionally, it highlights the need for clarifying implementation choices and integration plans for future enhancements.

malachi-cooley
Télécharger la présentation

Enhancing Monitoring Systems at CERN: Progress and Future Directions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WP4 - Monitoring Progress report sylvain.chapeland@cern.ch WP4 Workshop - CERN

  2. Overview • Work completed • New monitoring sensor agent (MSA) • Work in progress • Alarm display • Repository : database and server • Transport • Work to be done WP4 Workshop - CERN

  3. MSA • Release 1.3 • Deployed on ~325 nodes at CERN • New features: • Smart reconfiguration • Multiple transport mechanism supported • Self-monitoring capabilities • Tree-like configuration • Works on Solaris WP4 Workshop - CERN

  4. Alarm display To be released in July WP4 Workshop - CERN

  5. Repository • Multithreaded server being implemented • Currently, data still stored in flat files. • Two relational databases being tested : • MySQL • Oracle WP4 Workshop - CERN

  6. Repository : database structure • Database aimed at storing samples: • Metric identifier WHAT • Target identifier WHERE • Timestamp WHEN • Value • Value is very different from one metric to the other. WP4 Workshop - CERN

  7. Repository : database structure One table per metric WP4 Workshop - CERN

  8. Repository : database structure CPU load WP4 Workshop - CERN

  9. Repository : database structure Network traffic WP4 Workshop - CERN

  10. Repository management • Not clear what to use for subscription mechanism • A simple ASCII protocol might be used • Server side already implemented, missing client implementation of C API and subscription. WP4 Workshop - CERN

  11. Transport • UDP transport easy to use and deploy, but: • Data loss : 0.01% to 10% • Currently no security • Lack of scalability • New transport to be released in July. WP4 Workshop - CERN

  12. Transport • TCP: direct connection to server not scalable. • Possible solution: use of proxies • Proposed transport mechanism: pull data, file transfer, proxy collector. WP4 Workshop - CERN

  13. Transport Repository Proxy Proxy MSA MSA MSA MSA MSA MSA MSA MSA WP4 Workshop - CERN

  14. Transport Repository Proxy MSA B FILE B1 MSA A FILE A1 FILE A2 FILE B2 WP4 Workshop - CERN

  15. Transport Repository Proxy File P1: FILE A1 ; FILE A2 GET * MSA B FILE B1 MSA A FILE A1 FILE A2 FILE B2 WP4 Workshop - CERN

  16. Transport Repository Proxy File P1: FILE A1 ; FILE A2 ; FILE B1 ; FILE B2 GET * MSA B FILE B1 MSA A FILE A1 FILE A2 FILE B2 WP4 Workshop - CERN

  17. Transport Repository File P1: FILE A1 ; FILE A2 ; FILE B1 ; FILE B2 GET * Proxy File P1: FILE A1 ; FILE A2 ; FILE B1 ; FILE B2 MSA B FILE B1 MSA A FILE A1 FILE A2 FILE B2 WP4 Workshop - CERN

  18. Transport Repository File P1: FILE A1 ; FILE A2 ; FILE B1 ; FILE B2 DEL P1 Proxy File P1: FILE A1 ; FILE A2 ; FILE B1 ; FILE B2 MSA B FILE B1 MSA A FILE A1 FILE A2 FILE B2 WP4 Workshop - CERN

  19. Transport Repository File P1: FILE A1 ; FILE A2 ; FILE B1 ; FILE B2 Proxy DEL A1 A2 DEL B1 B2 MSA B FILE B1 MSA A FILE A1 FILE A2 FILE B2 WP4 Workshop - CERN

  20. Transport Repository File P1: FILE A1 ; FILE A2 ; FILE B1 ; FILE B2 Parse and store Proxy MSA B MSA A WP4 Workshop - CERN

  21. Transport • Transport safe • Duplication of transport possible • Need a file index. Repository only keeps highest index. File already inserted in DB are rejected. • Can be implemented with FTP. • But then polling required. • Push/Pull possibles WP4 Workshop - CERN

  22. Transport • Open issue : security • Possible use of Grid-FTP • Other solutions also easy to implement • Tunneling, or use of wide spread libraries • Anyway, key distribution? Are there any preferred method? WP4 Workshop - CERN

  23. Work to be done • Complete work in progress • Architecture defined, but implementation choices to be tested and clarified • Integration: • Lcfg object • Sensors : what do we need? WP4 Workshop - CERN

More Related