1 / 44

CMS Computing TDR (e non solo…)

CMS Computing TDR (e non solo…). Cosa descrive il C-TDR (e cosa no) Computing Model rivisitato Cosa e’ cambiato I “servizi” e l’operazione del Computing Qualche risultato. This document is primarily concerned with the preparations for the first full year of LHC running, expected to be 2008.

lamar-good
Télécharger la présentation

CMS Computing TDR (e non solo…)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CMS Computing TDR(e non solo…) Cosa descrive il C-TDR (e cosa no) Computing Model rivisitato Cosa e’ cambiato I “servizi” e l’operazione del Computing Qualche risultato This document is primarily concerned with the preparations for the first full year of LHC running, expected to be 2008.

  2. C-TDR: cos’e’ e cosa non e’ • Descrizione di una possibile implementazione del Computing (la parte “C” di CMS-CPT) • Valutazioni per il primo “full year” di LHC (nominale 2008) • Con il necessario ramp-up prima e dopo • Il “Software” (in dettaglio…) sara’ nel Physics TDR vol. 1 (~fine 2005) • La parte “P” sara’ nel Physics TDR vol. 1 e vol. 2 (~Aprile 2006) • Non e’ un “blueprint” del Computing System • Ma una descrizione di cosa puo’ essere “baseline” e come procedere oltre questa • Va di “pari passo” con il TDR di LCG (WLCG nel documento) • Servizi, Tiers, Organizzazione, Etc. The CMS computing environment is a distributed system of computing services and resources that interact with each other as Grid services. The set of services and their behavior together provide the CMS computing system as part of the Worldwide LHC Computing Grid. Together they comprise the computing, storage and connectivity resources that CMS uses to do data processing, data archiving, event generation, and all kinds of computing related activities.

  3. Computing Model: variazioni • CERN CAF (CERN Analysis Facility) • Al posto del Tier1 e del Tier2 al CERN • Integrazione con Grid (few flavours) con interfaccia verso le applicazioni “comune” • Coinvolgimento nelle specifiche • Adattamento del “software-tools” di CMS • Infrastruttura distribuita (Computing System) basata su (W)LCG • Incremento delle necessita’ di nastro ai Tier1 • Definizione dei Tier3 • WAN bandwidth less of a factor 2 at Tiers • Efficienza di uso tolta per consitenza con gli altri Exps • Identificazione dei Tier1 e dei Tier2 insieme a LCG

  4. Tiered Architecture Not all connections are shown - for example flow of MC data from Tier-2’s to Tier-1’s or peer-to-peer connections between Tier-1’s.

  5. Tier-3 Centers • Functionality • User interface to the computing system • Final-stage interactive analysis, code development, testing • Opportunistic Monte Carlo generation • Responsibility • Most institutes; desktop machines up to group cluster • Use by CMS • Not part of the baseline computing system • Uses distributed computing services, does not often provide them • Not subject to formal agreements • Resources • Not specified; very wide range, though usually small • Desktop machines -> University-wide batch system • But: integrated worldwide, can provide significant resources to CMS on best-effort basis

  6. CMS-CAF • Functionality • CERN Analysis Facility: development of the CERN Tier-1 / Tier-2 • Integrates services associated with Tier-1/2 centers • Primary: provide latency-critical services not possible elsewhere • Detector studies required for efficient operation (e.g. trigger) • Prompt calibration ; ‘hot’ channels • Secondary: provide additional analysis capability at CERN • Responsibility • CERN IT Division • Use by CMS • The CMS-CAF is open to all CMS users (As are Tier-1 centers) • But: the use of the CAF is primarily for urgent (mission-critical) tasks • Resources • Approx. 1 ‘nominal’ Tier-1 (less MSS due to Tier-0)+ 2 ‘nominal’ Tier-2 • CPU 4.8MSI2K; Disk 1.5PB; MSS 1.9PB; WAN >10Gb/s • NB: CAF cannot arbitrarily access all RAW&RECO data during running • Though in principle can access ‘any single event’ rapidly.

  7. Project Organization

  8. Services Overview

  9. The CTDR has served to converge on a basic architectural blueprint for a baseline system. We are now beginning the detailed technical design of the components It should be possible to bring up such a system over the next 6­9 months for the cosmic challenge and then CSA 2006 Basic Distributed Workflow

  10. Data Management • Data organization • ‘Event collection’: the smallest unit larger than one event • Events clearly reside in files, but CMS DM will track collections of files (aka blocks) (Though physicists can work with individual files) • ‘Dataset’: a group of event collections that ‘belong together’ • Defined centrally or by users • Data management services • Data book-keeping system (DBS) : “what data exist?” • NB: Can have global or local scope (e.g. on your laptop) • Contains references to parameter, lumi, data quality info. • Data location service (DLS) : “where are the data located?” • Data placement system (PhEDEx) • Making use of underlying Baseline Service transfer systems • Site local services: • Local file catalogues • Data storage systems

  11. Workload Management • Running jobs on CPUs… • Rely on Grid workload management, which must • Allow submission at a reasonable rate: O(1000) jobs in a few sec • Be reliable: 24/7, > 95% job success rate • Understand job inter-dependencies (DAG handling) • Respect priorities between CMS sub-groups • Priority changes implemented within a day • Allow monitoring of job submission, progress • Provide properly configured environment for CMS jobs • Beyond the baseline • Introduce ‘hierarchical task queue’ concept • CMS ‘agent’ job occupies a resource, then determines its task • I.e. the work is ‘pulled’, rather than ‘pushed’. • Allows rapid implementation of priorities, diagnosis of problems

  12. Integration Program • This Activity is a recognition that the program of work for Testing, Deploying, and Integrating components has different priorities than either the development of components or the operations of computing systems. • The Technical Program is responsible for implementing new functionality, design choices, technology choices, etc. • Operations is responsible for running a stable system that meets the needs of the experiment • Production is the most visible operations task, but analysis and data serving is growing. • Event reconstruction will follow • Integration Program is responsible for installing components in evaluation environments, integrating individual components to function as a system, performing evaluations at scale and documenting results. • The Integration Activity is not a new set of people nor is it independent of either the Technical Program or the Operations Program • Integration will rely on a lot of existing effort

  13. Cosmic SC3-4 Commissioning P-TDR CSA-06 2005 2006 2007 2008 2009 Project Phases • Computing support for Physics TDR, -> Spring ‘06 • Core software framework, large scale production & analysis • Cosmic Challenge (Autumn ‘05 -> Spring ‘06) • First test of data-taking workflows • Data management, non-event data handling • Service Challenges (2005 - 06) • Exercise computing services together with WLCG + centres • System scale: 50% of single experiment’s needs in 2007 • Computing, Software, Analysis (CSA) Challenge (2006) • Ensure readiness of software + computing systems for data • 10M’s of events through the entire system (incl. T2) • Commissioning of computing system (2006 - 2009) • Steady ramp up of computing system to full-lumi running.

  14. Detector-PRS D.Acosta P.Sphicas Analysis-PRS A.DeRoeck P.Sphicas ECAL/e-g C. Seez Y. Sirois Heavy Ions B. Wyslouch TRACKER/b-t I.Tomalin F. Palla Higgs S. Nikitenko HCAL/JetMET J.Rohlf C.Tully Standard Model J. Mnich Muons N. Neumeister U.Gasparini SUSY & BSM L. Pape M. Spiropulu Online Selection S. Dasu C. Leonidopoulos Generator Tools F. Moortgat S. Slabospitsky CPT organization: Today Project Office V.Innocente L.Taylor Project Manager P.Sphicas Computing L. Bauerdick D.Stickland Software L.Silvestris A.Yagil Technical Program P.Elmer/S.Lacaprara Framework L. Sexton EvF/DQM E. Meschi Integration Program S. Belforte/I. Fisk Reconstruction T. Boccali Analysis Tools L. Lista ORCA for PTDR S. Wynhoff Operations Program L. Barone Calibr/alignment O. Buchmuller L. Lueking Fast Simulation P. Janot Simulation M. Stavrianakou Facilities and Infrastructure N. Sinanis Software Devel. Tools S. Argiro’

  15. Analisi: CMS-CRAB Monitor Submitted jobs Submitted from • Via LCG-Grid • Solo parte dei jobs di CMS • Per es. Simulazione, trasferimenti, validazione, etc. • L.Faina, L.Servoli, D.Spiga, H.Riahi (PG) Destination of jobs

  16. # of jobs September August July week 18 Sep 31 Jul 10 Jul 17 Jul 21 Aug 04 Sep 11 Sep 07 Aug 24 Jul 28 Aug 14 Aug Jobs di analisi (CMS-CRAB via Grid) ~100000 CRAB-jobs / ~3 months By Daniele Spiga

  17. CMS-CRAB Monitor: analisi dei dati # of submissions # of jobs per task By Daniele Spiga

  18. CMS-CRAB Monitor: analisi dei dati By Daniele Spiga

  19. LCG Service Challenge 3 (‘SC3’) • Data transfer and data serving in real use-cases • Two phases: • Jul05: SC3 “throughput” phase • Tiers simultaneous import/export, MSS involved: move real files, store on real hw • >Sep05: SC3 “service” phase • small scale replica of the overall system • modest throughput, main focus on testing a quite complete environment

  20. SC3 rates

  21. C-TDR resources and INFN Tier1 for CMS CMS Tier2s pledged = Chris Eck tables Sept05 “should be” = CMS C-TDR Italian expected contribution

  22. Tier1-CNAF (CMS only) CPU Dischi Nastri

  23. Tier2s CMS Italy CPU Dischi

  24. Richieste 2006 CMS Calcolo

  25. Milestones 2005: specifiche • CMS Computing TDR (e TDR-LCG) [Luglio 2005] • Definizione delle risorse del Tier1 e dei Tier2 che partecipano al Computing Model [Febbraio 2005] • Definizione dei servizi (dati, software, LCG) disponibili in Italia [Febbraio 2005] • Definizione della partecipazione di CMS Italia a LCG e INFN Grid [Maggio 2005] • Produzione ed analisi dei dati per il P-TDR [Gennaio 2006]Gruppi di fisica di responsabilità e interesse italiano: b-tau, muon, e-gamma, Higgs, SYSY, SM • Partecipano il Tier1 e almeno metà dei Tier2/3 [Luglio 2005] • Produzione di ~2 M eventi/mese [Gennaio 2005] • Analisi preliminare di almeno 4 canali (es. H->WW->2mu2nu) [Febbraio 2005] • Partecipano il Tier1, tutti i Tier2 e piu’ della metà dei Tier3 [Ottobre 2005] • Produzione di ~ 20 M eventi [Dicembre 2005] • Analisi di almeno 8 canali per il P-TDR [Dicembre 2005] • Deployment di un prototipo di sistema di analisi distribuito su LCG [Aprile 2005] • Definizione delle componenti [Gennaio 2005] • Definizione e implementazione dell’infrastruttura organizzativa italiana [Febbraio 2005] • Data Challenge 05 completato (~20% INFN) [Dicembre 2005] Done On progress : ~50% Done ===========================================

  26. Milestones 2006 • June 2006. LCG Service Challenge 4 (SC4) start, being the software and computing support for the Cosmic Challenge ready:includes Tier1-CNAF+ at least 1/2 of CMS-Italy Tier2s. • October-November 2006. Computing, Software and Analysis Challenge (CSA-2006): includes Tier1-CNAF, all CMS-Italy Tier2s and some CMS-Italy Tier3s. • December 2006. Integration of Computing systems atTier1s and Tier2s ready for testing:includes all Italian Tiers for CMS.

  27. Additional slides

  28. Occupazione farm (Apr-Sept 05) CPU time totale wallclock time totale http://tier1.cnaf.infn.it/monitor/LSF/plots/acct/

  29. KspecInt2k (Maggio-Agosto)

  30. Accounting storage NOTE • per BABAR inclusi anche ~32 TB e per CDF ~ 10 TB • 200 raw TB da fine Settembre, phase-out di hw vecchio (~ 20 TB) • * = accesso a CASTOR configurato ma non ancora usato

  31. RUNNING PENDING Total nb. jobs Max nb. slots T1 farm occupancy T1 power failure T1 switch problem Upgrade to LCG260 (ramp-down, upgrade, LCG certification…) May be capable to address the needs of experiments Focused effort needed on: • reliability and 24/7 support • scheduled interventions • troubleshooting • Underwent some migration seasons: • OS: RH  SLC v.3.0.4 • mw:  LCG v.2.6.0 • ~90% WN migrated, running LCG cert • install WNs/servers: Quattor • batch scheduler: LSF v.6.1 [ all jobs at the T1 ] + LCG interf.

  32. Technical Program • job configuration and scheduling services, including policies, prioritisation of workloads, and ensuring scalability of job scheduling; • dataset placement and data transfer services, • dataset bookkeeping services, including physics metadata services; • instrumentation services and user interfaces, including a CMS dashboard, monitoring, job tracking; • CMS storage dataset access services, e.g., SRM, Castor, POOL/PubDB++/local file catalogs; • CMS workflows support, for the data taking and processing, MC production, and calibration workflows; • CMS VO services.

  33. Integration Program • developing and maintaining the CMS Computing Integration Plan; • preparing for and running of series of data challenges, and service challenges; • taking responsibility for validation, releases, and deployment; • providing workflow integration for production, dataset publishing, distributed analysis, data taking and processing; • providing component integration into a coherent and functional computing environment; • releasing and delivering the integrated computing environment of CMS computing services and components into the CMS production Grid environment, working with the WLCG; • liaising on a very practical level with CERN-IT,the CMS regional centres, the LCG and Grid projects: EGEE, OSG, and NorduGrid.

  34. Operation Program • developing and maintaining the CMS Computing operations model and plan, working with the Computing Management Team, • MC production operations, • database system operations, • calibration workflow support, • data-taking operations and data validation; and • user support.

  35. Facilities Program This covers services that are a shared need and responsibility of the full CMS Collaboration, such as the support for the CMS common computing and software environment, software process services, and core support for production operations. In preparation for data taking, CMS will need to keep close liaison and coordination with its Tier-1 centres that provide the bulk of computing resources required for data processing and data analysis. Together, these form the CMS Computing Facilities program, thus consisting of • CMS Tier-0 coordinator • CMS CAF coordinator • CMS common services and infrastructure coordinator • CMS Tier-1 technical contacts

  36. CPT L1 and Computing L2 Milestones V34.2

  37. Average Numbers: 6 Tier1s, 22.5 Tier2s

  38. Data Tiers • RAW • Detector data + L1, HLT results after online formatting • Includes factors for poor understanding of detector, compression, etc • 1.5MB/evt @ <200Hz; ~ 5.0PB/year (two copies) • RECO • Reconstructed objects with their associated hits • 250kB/evt; ~2.1PB/year (incl. 3 reproc versions) • AOD • The main analysis format; objects + minimal hit info • 50kB/evt; ~2.6PB/year - whole copy at each Tier-1 • TAG • High level physics objects, run info (event directory); <10kB/evt • Plus MC in ~ 1:1 ratio with data

  39. Data Flow • Prioritization will be important • In 2007/8, computing system efficiency may not be 100% • Cope with potential reconstruction backlogs without delaying critical data • Reserve possibility of ‘prompt calibration’ using low-latency data • Also important after first reco, and throughout system • E.g. for data distribution, ‘prompt’ analysis • Streaming • Classifying events early allows prioritization • Crudest example: ‘express stream’ of hot / calib. events • Propose O(50) ‘primary datasets’, O(10) ‘online streams’ • Primary datasets are immutable, but • Can have overlap (assume ~ 10%) • Analysis can draw upon subsets and supersets of primary datasets

  40. Tier-0 Center • Functionality • Prompt first-pass reconstruction • NB: Not all HI reco can take place at Tier-0 • Secure storage of RAW&RECO, distribution of second copy to Tier-1 • Responsibility • CERN IT Division provides guaranteed service to CMS • Cast iron 24/7 • Covered by formal Service Level Agreement • Use by CMS • Purely scheduled reconstruction use; no ‘user’ access • Resources • CPU 4.6MSI2K; Disk 0.4PB; MSS 4.9PB; WAN 5Gb/s

  41. Tier-1 Centers • Functionality • Secure storage of RAW&RECO, and subsequently produced data • Later-pass reconstruction, AOD extraction, skimming, analysis • Require rapid, scheduled, access to large data volumes or RAW • Support and data serving / storage for Tier-2 • Responsibility • Large CMS institutes / national labs • Firm sites: ASCC, CCIN2P3, FNAL, GridKA, INFN-CNAF, PIC, RAL • Tier-1 commitments covered by WLCG MoU • Use by CMS • Access possible by all CMS users (via standard WLCG services) • Subject to policies, priorities, common sense, … • ‘Local’ use possible (co-located Tier-2), but no interference • Resources • Require six ‘nominal’ Tier-1 centers; will likely have more physical sites • CPU 2.5MSI2K; Disk 1.2PB; MSS 2.8PB; WAN >10Gb/s

  42. Tier-2 Centers • Functionality • The ‘visible face’ of the system; most users do analysis here • Monte Carlo generation • ‘Specialized CPU-intensive tasks, possibly requiring RAW data • Responsibility • Typically, CMS institutes; Tier-2 can be run with moderate effort • We expect (and encourage) federated / distributed Tier-2’s • Use by CMS • ‘Local community’ use: some fraction free for private use • ‘CMS controlled’ use: e.g., host analysis group with ‘common resources’ • Agreed with ‘owners’, and with ‘buy in’ and interest from local community • ‘Opportunistic’ use: soaking up of spare capacity by any CMS user • Resources • CMS requires ~25 ‘nominal’ Tier-2; likely to be more physical sites • CPU 0.9MSI2K; Disk 200TB; No MSS; WAN > 1Gb/s • Some Tier-2 will have specialized functionality / greater network cap

  43. Resource Evolution

More Related