SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Real-Time Monitoring for Grid Activity

etreadwell avatar
etreadwell
17 diapositives · 202 Views · Jan 07, 2020

The RTM (Real-Time Monitoring) tool allows for monitoring and visualization of the grid activity, providing real-time plots, historical data, and detailed analysis. It supports monitoring of multiple resource brokers and various Grid projects. Users can view plots stacked by VO or CE, embed graphs in web pages, and access job states data.

Real-Time Monitoring for Grid Activity - slide 1
01 / 17Real-Time Monitoring for Grid Activity
1 / 17

Télécharger la présentation

Real-Time Monitoring for Grid Activity
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. RTM for monitoringhttp://gridportal.hep.ph.ic.ac.uk/rtm/O. van der Aao.vanderaa@imperial.ac.uke-Science, HEP, Imperial College London On behalf of D. Colling, G. Moont, M. Aggarwal

  2. Changes in the RTM • Big changes in underlying design allowing for more flexibility • 51 Resource Brokers now monitored • Other EGEE Grid Projects have requested to be monitored; EUMED, EUCHINA, EELA • Historical data available and taken by several groups • Real Time data being visualised in new ways RTM for monitoring – o. van der Aa

  3. The original form of the Monitor - popular as a demo Problem in users are unaware of full capabilities via clickingin the Key; selection by VO and/or RB RTM, the Applet RTM for monitoring – o. van der Aa

  4. RTM, Google earth • Static view ofthe grid • Shows a plotof runningjobs for each site you clickon. RTM for monitoring – o. van der Aa

  5. RTM, real time plots • The RTM keeps all job states in a Postgresql database • Round-robin archives are then produced to allow real time plotting of the number of jobs in any given state. • Good for real time monitoring of the Grid activity RTM for monitoring – o. van der Aa

  6. How does it look like • See https://gfe03.hep.ph.ic.ac.uk:4175 • Select a set of VO and CE and the time period for the plot • One plot stacked by VO • On plot stacked by CE RTM for monitoring – o. van der Aa

  7. RTM, running jobs 1month back Last month, running jobs for the whole Grid lhcb cms atlas alice biomed RTM for monitoring – o. van der Aa

  8. View per country UK France Italy swiss RTM for monitoring – o. van der Aa

  9. Embedding graphs in your web pages • https://gfe03.hep.ph.ic.ac.uk:4175/cgi-bin/googlegraph.cgi? • Arguments are • ce=[yource1]&ce=[yource2] • If no ce is given all the existing ones are plotted • If filter=[country] is used only the ce in that country are shown • Date=-1w • W=800 (width) • H=400 (height) • Examples: • Googlegraph.cgi?ce=gw39.ph.ic.ac.uk&date=-1w&w=800&h=400 • Googlegraph.cgi?filter=uk&date=-1w&w=800&h=400 RTM for monitoring – o. van der Aa

  10. RTM for detailed analysis • Round robin is fast to render real time data view over long periods • It contains averages of the number of job in a given state • For more detailed analysis we need the full data on a per job basis (jobid) • Use root to store the timings of the job state transitions • Also store all the states the job went in RTM for monitoring – o. van der Aa

  11. Where to find the root and ascii data • http://gridportal.hep.ph.ic.ac.uk/rtm/resource-brokers/reports/ascii_report_data_2006-05-01.dat • http://gridportal.hep.ph.ic.ac.uk/rtm/resource-brokers/reports/root_report_data_2006-05-01.root • The daily data is that of jobs which are considered as "finished" by the RTM within a 24 hour period (local time UK midnight-midnight). Finished means either they were CLEARED by a user, or had been sitting in a DONE / ABORTED / CANCELLED state for over 2 hours. RTM for monitoring – o. van der Aa

  12. Job states data • http://gridportal.hep.ph.ic.ac.uk/rtm/resource-brokers/all.dat • http://gridportal.hep.ph.ic.ac.uk/rtm/resource-brokers/update.dat • Their format is (Java code snippet) - the all.dat does NOT have the rtm_timestamp ; • println( rbAddress + "\t" + jobid + "\t" + status + "\t" + state_entered + "\t" + registered + "\t" + ui + "\t" + ce + "\t" + queue + "\t" + wn + "\t" + vo + "\t" + rtm_timestamp ) ; • By reading the all.dat, and rereading the update.dat exactly once a minute afterwards, you should be able to maintain a current view. RTM for monitoring – o. van der Aa

  13. Examples (jan-june data) • Fractional useful time for atlas • Total Succesful Hours/Total Hours RTM for monitoring – o. van der Aa

  14. More examles: Fractional usefull time per vo Fractional useful time RTM for monitoring – o. van der Aa

  15. Job scheduling (Match Time) versus load (mean number of jobs/sec during the matching) ExampleWMS monitoring RTM for monitoring – o. van der Aa

  16. Conclusion • RTM is more than the applet • It can provide rrd archives for real time plotting • Number of job in a given state. • Per CE view • Per VO view • Could measure abort rate and trigger alarms • It also provides root files for detailed historical analysis • Timing analysis of job cycles • WMS monitoring • Efficiency (Usefull Time) • Please fell free to use the root/ascii and round robin data RTM for monitoring – o. van der Aa

  17. URLS • http://gridportal.hep.ph.ic.ac.uk/rtm/ • https://gfe03.hep.ph.ic.ac.uk:4175 • The historical data • http://gridportal.hep.ph.ic.ac.uk/rtm/resource-brokers/reports/ascii_report_data_2006-05-01.dat • http://gridportal.hep.ph.ic.ac.uk/rtm/resource-brokers/reports/root_report_data_2006-05-01.root • The real time data (job states,ce, rb, etc) • http://gridportal.hep.ph.ic.ac.uk/rtm/resource-brokers/all.dat • http://gridportal.hep.ph.ic.ac.uk/rtm/resource-brokers/update.dat RTM for monitoring – o. van der Aa