SAM-Grid Status Update: Monte Carlo Production Enhancements and Future Goals
This addendum provides an overview of the SAM-Grid status and recent developments in Monte Carlo (MC) production for the DØ experiment. Iain Bertram outlines the successful integration of DØ production executables with SAM-Grid, highlighting that over 50% of DØ production MC has been generated in the UK. Future goals include running MC production on SAM-Grid in the UK and developing a standard reconstruction interface. Key challenges, particularly in metadata complexity, are discussed, along with lessons learned and a focus on improving user experience in accessing datasets.
SAM-Grid Status Update: Monte Carlo Production Enhancements and Future Goals
E N D
Presentation Transcript
SAM-Grid DØ Status • Addendum to SAM-Grid Demo • Monte Carlo Production Iain Bertram http://d0db.fnal.gov/sam
Status • Working SAM-Grid (previous) • Met all deliverables for GridPP • GridFTP standard for DØ data transfers • >50% DØ Production MC generated in UK • Near Term Goal MC Production Running on SAM-Grid in UK • 3.6.6 Prototype SAM Monte Carlo Production system using Resource Broker 31-Mar-03 • 3.6.9 Pilot SAM Grid for D0 users. 30-Sep-03 Iain Bertram
MC Production • Runjob Package to integrate DØ production executables with SAM-Grid • Also used by FNAL-CMS Group • maintained and developed within UK • Used by all remote DØ production centres. • Being Considered as standard reconstruction interface for DØ. • Controls Metadata production • Uses macro script to control job. Iain Bertram
MC Production Outline • User Requests MC • Describes a virtual data set using keyword:value pairs • Stores Request in DØ database • <<<< Grid Here >>>> • Production Site • Gets next MC set • produces events and delivers to user Iain Bertram
MC Production on SAM-Grid • Plan to run MC in the UK on a SAM-Grid by March • ICL, Lancs, Mancs, and RAL • Use Resource Broker to advertise MC site (Pure Condor) • Runjob controls execution on remote sites • store all output files using the Grid • Extensions • Automatic code installation • Data Reconstruction • Long Term: Generic User Job Iain Bertram
Lessons • Biggest Problem at DØ • Defining user datasets • Metadata is very complex and requires very good database table design • This complexity is necessary so users can find the data they need • Once have list of file all OK • Conclusion • rapid Progress, full steam ahead Iain Bertram
