Current Mass Storage Status/Plans at CERN

1. Current Mass Storage Status/Plans at CERN HEPIX 3 November 2000 H.Renshall PDP/IT Current Mass Storage Status/Plans at CERN

2. Devices and Media at CERN • Data centre: high performance, quality and cost, higher number of write/read passes • IBM3590 series (Magstar) • STK Redwood • STK 9840 • STK 99xx series - currently under field test at CERN • Intermediate: medium to high performance, quality and cost • Linear Tape Open (LTO - IBM, Seagate, HP) - IBM LTO in Annaconda robot currently under field test at CERN • Commodity: lower performance, quality and cost, lower number of write/read passes • Quantum DLT series • Exabyte 8200/8500 • HP Dat • CERN strategy is to use data centre quality for raw and processed physics data and expect commodity for import export. Placing of LTO not yet clear (especially media costs) but seen to be successor to DLT for small labs and import/export. Current Mass Storage Status/Plans at CERN

3. Drive/Media Comparison • Cartridge Cartridge capacity GB Cost per GB CHF Data Rate • (Uncompressed) (Uncompressed) (Uncompressed) • Drives over 30k$: • IBM 3590 10 4 10 • IBM 3590E 40 2 14 • STK Redwood 50 1 10 • STK 9840 20 5 10 • STK 99xx (60) (2) (10) • Drives under 10K$: • LTO 100 2-3 15 • DLT 2000 10 5 1 • DLT 7000 35 3 5 Current Mass Storage Status/Plans at CERN

4. Current Tape Drive and Robotics Hardware • A cluster of four STK silos in bat.513 machine room containing: • 24 STK Redwood tape drives (physics) • 10 STK 9840 tape drives (Legato+physics) • 4 IBM 3590E tape drives (adsm + HPSS) • A cluster of three STK silos in bat.513 basement containing: (Was two at 2 March Focus) • 8 STK Redwood tape drives (physics) • 10 STK 9840 tape drives (physics) • 2 IBM 3590 tape drives (physics) • 4 IBM 3590E tape drives (adsm + HPSS) • A new single STK powderhorn silo in bat 513 basement being used for STK 99xx field test and possibly to become part of LHC testbed • Physics+adsm IBM 3494 robot has now been stopped (drives moved to STK robot) • second (HPSS) IBM 3494 robot to be stopped this year • STK Timberwolf robots reconfigured (more slots + DLT 7000 drives) • TL820 DLT robot to be stopped this year Current Mass Storage Status/Plans at CERN

5. Share of (none-HPSS) tape mounts by drive type Current Mass Storage Status/Plans at CERN

6. Tape Mounts and data Transfer by Group in 2000 Current Mass Storage Status/Plans at CERN

7. Objectives in Pre LHC Era • Provide adequate bandwidth and storage capacity for pre-LHC experiments and testbeams • Encourage a competitive situation for the supply of tape drives and media • Provide a second, physically separate robotic library to provide users and major applications with the ability to plan sensible disaster avoidance • Rationalise the robotics infrastructure (no real need for competition here) to reduce maintenance costs and complexity (hence manpower) • Try to support all reasonable media for small scale data import/export requirements • Migrate away from Redwood technology as soon as possible • Deploy Cern Advanced Storage Manager (CASTOR) and enhance it for LHC while keeping HPSS as a fall-back solution. Current Mass Storage Status/Plans at CERN

8. CASTOR and HPSS • HPSS stores about 20TB in 350000 files used in 2000 for: • CDR for NA57 • LHC Test beams • “User tapes” • DELPHI simulation data • CMS (ORCA included) • ALICE Data Challenge • HPSS in 2001 new data only in “User Tapes” class • CASTOR used in 2000 for: • Tape movers (SHIFT and CASTOR tapes, CDR) • DELPHI stager • ALICE Data Challenge • COMPASS and L3C production • Interface to HPSS • CASTOR in 2001 for all new data except “User Tapes” Current Mass Storage Status/Plans at CERN

9. Software Plans • CASTOR is the proposed solution for LHC. Estimate 2-3 man years further development required. • HPSS kept for some time as a possible fall-back solution. Keep it alive using some modestl data class. • FATMEN/HEPDB kept only as long as required by old experiments. Migration to CASTOR encouraged but not all older experiments will succeed. • No realistic hope for a new common HEP solution (Interstage) • But Data Grid WP5 will provide: • Common API (RFIO is a proposed solution) • Definition of an exchange format for data and metadata Current Mass Storage Status/Plans at CERN

10. Migration to CASTOR • Existing SHIFT tapes can be transparently imported into CASTOR name space • Either the tapes will be scanned and names generated automatically • Or the user will provide the description of the tapes • Then the user could stage a file giving • either the tape vid and file sequence number • or the fully qualified filename • To get a unique name space, data taking experiments can ask to transfer existing data off HPSS to CASTOR. This is recommended but no systematic transfer is currently planned • The copy of data from HPSS to CASTOR must be done while CERN has an HPSS license • The copy will be done with stagein from HPSS and stagewrt to CASTOR name space • Users will only see their top level directory change from /hpss to /castor Current Mass Storage Status/Plans at CERN

11. Other Plans • Move second STK silo complex from bat 513 to new bat 613 (Autumn 2000) • move 3 silos and add fourth/fifth for 2002 data. • Tender for LHC testbed robot/drive capacity - 20+ drives and up to 2000 slots • if powderhorn robot populate with STK 99xx or IBM 3590E • robot (IBM Annaconda or STK L-series) with LTO drives (HP or IBM) is an option • Migrate HPSS data from Redwood and 3590 to 3590E • runs as background job, transparent to users • Leave old 3590 physics data read only else transparently copy to Castor managed storage (mostly data of L3 and Nomad) Current Mass Storage Status/Plans at CERN

12. Migration from Redwood Technology • STK have announced end of service (no repair service) of Redwood for end 2002. Our contract guarantees support of Redwood for 10 years but after 5 years (i.e. April 2002) out fixed maintenance price period ends. They also have many operational problems and realistically we should plan to get out of Redwoods now and have started negotiations with STK. • All new experiment data to go into tape-efficient Castor managed storage on IBM 3590E or STK 99xx. Need to discuss bookeeping issues with a few experiments and media cost issues should be discussed in CERN Cocotime committee. • Migrate existing ‘active’ Redwood data to Castor managed storage (probably to STK 99xx) to stop or reduce to negligible Redwood use by end 2001. • Redwood migration to be transparent to end users. Current stagein and use of Vid mapping (already used to copy from 3480 to Redwood 3 years ago) to continue to work as now but option to move to using Castor name space will be encouraged (could be coupled with a move out of FATMEN). Current Mass Storage Status/Plans at CERN

13. End of Direct Tape Usage • Experiments/users should no longer purchase tapes at CERN except for export. IT will work out a charging algorithm based on volume stored and activity (some older experiments will still buy tapes for Monte-Carlo data in 2001). • Castor and HPSS managed experiment data to be charged at FS 2/GB now as an interim solution. This price is the equivalent of buying your own tapes but deployed as managed storage. • We are consulting directly with the major experiments on the issues that affect them. Current Mass Storage Status/Plans at CERN