lyneth
Uploaded by
5 SLIDES
191 VUES
50LIKES

Enhancing Data Accessibility in DDM: Insights from the BNL ATLAS User Feedback Workshop

DESCRIPTION

On January 29, 2007, during the BNL ATLAS DDM Workshop, Hong Ma presented crucial feedback on the Direct Data Management (DDM) system. Key discussions included typical uses of DDM, challenges in accessing validation datasets, running reconstructions, and the importance of obtaining AOD and RDO datasets for physics analyses. Insights into calibration data accessibility and dataset subscriptions highlighted operational difficulties. The focus remained on improving user experience and streamlining data access processes for the physics analysis community.

1 / 5

Télécharger la présentation

Enhancing Data Accessibility in DDM: Insights from the BNL ATLAS User Feedback Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

Playing audio...

  1. DDM Ops User Feedback Hong Ma BNL ATLAS DDM Workshop Jan 29,2007

  2. Typical Uses of DDM • Validation • Quick access to data from production • Access to ESD/AOD datasets, validate the software/performance • Or, be able to run reconstruction on RDOs • Performance studies • Running reconstructions on RDOs • Private code, development releases • Need to have access to RDO datasets • Physics Analysis • Proocessing large number of AOD datasets • Access to AODs • Either copy AODs, or run distributed analysis • Commissioning analysis • Access to commissioning data from detector • Some calibration data are also in DDM. SM Meeting, Hong Ma

  3. Experiences • Great effort to make the calibrations samples RDO available in BNL and Lyon • See Claire’s talk in Nov, stress test of the system, largely a success • Reconstruction benefited from the availability of the data • 12.0.4 AOD datasets were not promptly accessible for validation • Difficulty of accessing data does not match validation needs • Coupled to Production System • Dataset subscription • Very useful for dataset distribution • Some subscriptions halt in the middle • Requires too many expert intervention • Subscription before consolidation does not work well • Hard to figure out which site actually has the data • No distinction for the primary data source • Subscription to a “good” source works better, but hard to know which one is good. • Ideally user would like the system to figure out the best way to get the data • dq2 client tool (dq2_get, etc) • Very useful and essential, good documentation • Site specific instruction is very useful. SM Meeting, Hong Ma

  4. CSC11 AODs at BNL • We subscribed many CSC11 AOD datasets for BNL • Submitted tickets for some of the problems, but did not follow all. • Need to do better than this for csc12. SM Meeting, Hong Ma

  5. Outlook • We aim to support large physics analysis community at BNL • Data accessibility is always a hot issue • Past: Copy data by experts (DC2/Rome) Using DQ2 Subscription+Complaints (csc11) • Future: data come to us automatically? • csc12 AOD availability is essential for CSC studies • DDM Op’s plan of AOD replication to Tier-1 seems sound • Monitoring the data transfer, don’t wait for users’ complaint. • Expect subscriptions of other datasets (ESDs and RDOs) • Many of the SampleA RDOs • Support is greatly appreciated • Thanks for help. SM Meeting, Hong Ma

More Related