1 / 5

LCG Gridview / LCG SAM use cases

LCG Gridview / LCG SAM use cases. Miguel Anjo 8 th July 2008 Database Developers’ Workshop. Scenario. Summarization and Monitoring of Grid Sites and Services Large schemas (>400GB each) 30GB new data/month Big raw data tables (>300M rows in Gridview case) Need to partition tables

fleta
Télécharger la présentation

LCG Gridview / LCG SAM use cases

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LCG Gridview / LCG SAM use cases Miguel Anjo 8th July 2008 Database Developers’ Workshop

  2. Scenario • Summarization and Monitoring of Grid Sites and Services • Large schemas (>400GB each) • 30GB new data/month • Big raw data tables (>300M rows in Gridview case) • Need to partition tables • Used RANGE partition on a timestamp column Presentation title - 2

  3. LCG Grid View example • RAW results table • Monthly partitions • Drop old partitions after three months • Summarized table • Big composite (multi-column) indexes to help performance of aggregations • Drawback: space used by indexes >= space used by table • Big merge queries • Not easy to tune • Working to transform in PL/SQL bulk operations Presentation title - 3

  4. LCG SAM example • Raw data table • includes CLOB column with details • CLOB can’t be easily deleted • Interest on keep >1 year raw data (except CLOB) • But no access to data >3 months • Current solution • Partitioned raw data table • Created monthly partitioned “History” table • No indexes, no CLOB column • Move data >3 months to “History” table • Gain of 80% space per month Presentation title - 4

  5. Lessons learned • Necessary to think on the growth • Partition is not (yet) self managed • Necessary to manually drop/create partitions • User responsibility to manage partitions • Operations with partitions are bit tricky • But difficult to test with few data • Still not good solution for keeping old, not accessed, data • User would like a DVD with it  • History table, no indexes, read only tablespace Presentation title - 5

More Related