1 / 13

From the ChannelArchiver to the B est E ver A rchive Ut ility, Y et

From the ChannelArchiver to the B est E ver A rchive Ut ility, Y et. kasemirk@ornl.gov October 2009. Channel Archiver. IOC. “ASCII” Config. Channel Access. History ~2000: Started by Bob Dalesio ~2003: Index Tools, Data Server ~2007: CSS Client. Archive Engine. config.xml.

jcarbaugh
Télécharger la présentation

From the ChannelArchiver to the B est E ver A rchive Ut ility, Y et

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. From theChannelArchiverto theBest Ever ArchiveUtility, Yet kasemirk@ornl.gov October 2009

  2. Channel Archiver IOC “ASCII” Config. Channel Access History ~2000:Started byBob Dalesio ~2003:Index Tools,Data Server ~2007:CSS Client Archive Engine config.xml Binary Data Files Data Server XML-RPC CSS-based OPI

  3. Problems with old Channel Archiver • Data file format optimized to write many samples quickly • More than 40000/second • .. but we only used maybe 1000/sec • .. and had many ill-configured or duplicate channels • Headaches with data maintenance: • Scripts to restart engines, copy data, update indices. • Index time grows with data • Stuck when index files reach 2GB • SNS Users faced with ~80 sub-archives • No clue what needs fixing after network/power problems • No idea who contributes how many samples • No way to remove selected channels or time ranges • Improving on this means implementing an RDB

  4. New Setup IOC Channel Access ArchiveEngine Samples Config. config.xml RDB(Oracle/MySQL) Other tools for config & samples EngineConfig -Import CSS-based OPI

  5. CSS Data Browser Handles Both • New URL • Just one‘RDB’sub-archive • Old and newdata can becombined inone plot

  6. Web Interface to Engine Config • Tomcat/JSP/Servlets to view and edit • Part of reporting system; somewhat specific to Oracle and SNS

  7. Stats • Host that runs sample engines: • CPU load 45%, zero disk I/O wait,very responsive • Oracle Setup • Cluster • Sample tables partitioned by day • 8000 samples/sec peak in write tests • Operationally maybe ¼ of that • Better configuration would likely have fewer samples/sec

  8. Web Config View: Channel Stats

  9. Web Config View: Sample Stats

  10. Viewing the Data Almost every software technology can access data in RDB • Automated weekly performance plots • Custom reports for- and done by - various SNS groups(SNS Beam Instrumentation: Mariano Padilla, D.J. Edwardson)

  11. Main User Interface: CSS Data Browser • Automatic switch frommin/max/average to ‘raw’ data • This operation is currently quite slow • Old “Network Data Server” ran min/max/avg. on server side • RDB code for now in client. Working on stored procedure. Zoom in…

  12. RDB: Perfect except for retrieval speed • Fermilab’s archiver, Jlab’s MyA • Wrapper code around MySQL to create ‘cluster’, or to store data compressed into BLOBs, i.e. access is not plain RDB. • BNL Ideas: • Gabriele Carcassi mentioned RDDTool • Mauro Giacchini investigates Hypertable All faster than plain RDB for basic read/write. New sample engine and Data Browser could be adapted to them. Still, both loose advantages of plain RDB access to data and configuration

  13. Summary, Status • At SNS, BEAUtY replaced Channel Archiver in July • Before, parallel operation for ~2 month • About a year of testing, many Oracle setup issues • Oracle cluster setup • Updated partitioning • Next • Stored procedure for retrieval • Data reduction: Replace Oracle partitions of old data with reduced channel/sample count

More Related