ProdAgent

1. UCSD Caltech Open Science Grid (OSG) Mission CMS Operations on OSG OSG Usage : CMS vs Others • OSG is a US based distributed computing grid for data-intensive research. • OSG goal is to promote collaboration in data-intensive research by providing a computing facility and services that integrate distributed, reliable and shared resources to support computation at all scales. • The computing and storage resources from campuses and research communities into a common, shared grid infrastructure via a common set of middleware. • Resources are owned and managed by the contributing members of OSG. • Among others, CMS T2 (and T3) sites in the US are members of the OSG. CMS (T1 + T2 + T3) CMS Usage of the Open Science Grid and the US Tier-2 CentersAjit Mohapatra, University of Wisconsin, Madison(On Behalf of CMS Offline and Computing Projects) CMS MC Production in OSG Routing CMS Jobs to OSG Running/Idle Job Monitoring • Centralised installation/maintainance of CMS Software package (CMSSW) at the OSG sites that participate in MC production. • MC production is handled by CMS specific “ProdAgent” software and managed by several teams based at CERN and Wisconsin in LCG and OSG regions, respectively. • Overall management of MC production and data transfer are performed by the CMS DataOps group. • Production in OSG (including 7 CMS T2s in the US) are handled by ProdAgent servers based at Wisconsin. • Utilization of various monitoring/debugging tools alongwith timely/effective communication with site admins has enabled us to run production efficiently. • ProdAgent creates and hands over jobs to Condor. Condor Schedd routing policy ProdAgent Job JobRouter for OSG site X max Jobs max Idle site Y max Jobs max Idle … Condor submit Job Routed to site X GRAM Site X Site Y Site Z Production Stats(LCG & OSG) Contribution of US (CMS) T2s Job Quality : US (CMS) T2s Jan’ 08 - Feb’ 09 Summer08/Fall08/Winter09 Summer08 / Fall08 / Winter09 & Part CSA07 / CSA08 T2_Region_FNAL (OSG) ~ 1B evts ~ 2B evts Jan’ 08 - Feb’ 09 CMS Analysis @ US T2s Analysis Jobs @ the T2s Summary & Plan • Terminated jobs stats according to CMS dashboard • Data hosted by the T2s • User Analysis : • CRAB(CMS Remote (Analysis Builder) Jobs • Local Analysis Jobs • Private (user) production • OSG provides packaged, tested and supported collections of software for the CEs and SEs for uniform operation across all the participating sites. • OSG has already been used successfully for CMS MC production for last 2 years. • Effective utilization of OSG (USCMS T2s) for MC production. All the US T2s have been very productive, stable and highly reliable. • While OSG itself provides supports to sites for grid related operations, effective communication between sites and CMS production operators are important to address production issues at sites in timely manner. • Expand the scope of MC production to CMS T3s (T3s are already used for user analysis) and other opportunistic resources which are also part of OSG. June’ 08 - Feb’ 09 Caltech (176816) UCSD-b (261475) Florida (159696) Wisc (434008) UCSD (312131) MIT (44353) • Effective and reliable usage of site resources depends strongly on the support and timely response at/from the site. Nebraska (65711) Purdue(198128) Computing in High Energy Physics, Prague (Czech Republic), March 2009