1 / 11

GPGPU Accounting

GPGPU Accounting. John Gordon STFC. Accounting Overview. Parse Batch Logs Get Identity Information Join and cut UR Publish to APEL APEL receives UR into database APEL Summarises Portal Visualises from Summaries. Parse Batch Logs. Today for CPU Accounting we support LSF GE Torque

bowie
Télécharger la présentation

GPGPU Accounting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GPGPU Accounting John Gordon STFC EGI CF – Accounting and Billing

  2. Accounting Overview • Parse Batch Logs • Get Identity Information • Join and cut UR • Publish to APEL • APEL receives UR into database • APEL Summarises • Portal Visualises from Summaries Business Models & Sustainability – EGI TF Lyon - Sept 2011

  3. Parse Batch Logs • Today for CPU Accounting we support • LSF • GE • Torque • SLURM (in EMI 3.1) • Do these batch systems report GPUs in their batch logs? • Lots of caveats. Still to be fully confirmed. Business Models & Sustainability – EGI TF Lyon - Sept 2011

  4. Get Identity Information • No change • Done by BLAH in CREAM Business Models & Sustainability – EGI TF Lyon - Sept 2011

  5. Join and cut UR • Extend UR to include second type of CPU • Do this in conjunction with OGF UR-WG. • UR 2.0 doesn’t support multiple CPU types in a job or attached processors. • nb Intel Phi is an attached processor(s) so we will need to address this anyway. Business Models & Sustainability – EGI TF Lyon - Sept 2011

  6. Publish to APEL • SSM ignores the content of the records it transports Business Models & Sustainability – EGI TF Lyon - Sept 2011

  7. APEL receives UR into database • APEL would need to extend the job record database to add extra columns • Simple for a test system but a mammoth task to migrate the production database. Business Models & Sustainability – EGI TF Lyon - Sept 2011

  8. APEL Summarises • Aggregating total GPU Use for a User, VO, Site, Country, NGI, etc is straightforward • Correlations like GPU/CPU, GPU/wallclock, etc are more complicated • Requirements? Business Models & Sustainability – EGI TF Lyon - Sept 2011

  9. Portal Visualises from Summaries • Straightforward to add other variables GPU, normalised GPU to the existing ones. • Portal displays one variable (njobs, totcpu, etc) as function of two others • Site vs VO • Region vs Date • Etc. Business Models & Sustainability – EGI TF Lyon - Sept 2011

  10. Summary • Most things possible once the major condition is met. • i.e. Batch logs report reliably and understandably on GPU identified with a batch job • Still a lot of work to be done to understand this • See John Walsh’s experiences • PRACE reported that they don’t. Business Models & Sustainability – EGI TF Lyon - Sept 2011

  11. What Can We Do? • The APEL Team could implement a test server receiving records from a hacked apel client once the research had been done on retrieving the relevant infor from batch records • After the end of EMI and with a full roadmap for 2013 we don’t have enough effort to do this research or to do the development in extending the relevant parsers for all batch systems. Business Models & Sustainability – EGI TF Lyon - Sept 2011

More Related