1 / 46

Computing at NCMRWF for Short to Seasonal Range Forecast Guidance   A K Bohra NCMRWF, New Delhi

Computing at NCMRWF for Short to Seasonal Range Forecast Guidance   A K Bohra NCMRWF, New Delhi CAS2K3,ANNECY, France: September 10, 2003. OVERVIEW. Introduction Objectives Activities Computing Resources Future Plans Conclusions. ORGANISATIONAL SET UP OF NCMRWF.

ianthe
Télécharger la présentation

Computing at NCMRWF for Short to Seasonal Range Forecast Guidance   A K Bohra NCMRWF, New Delhi

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computing at NCMRWF for Short to Seasonal Range Forecast Guidance   A K Bohra NCMRWF, New Delhi CAS2K3,ANNECY, France: September 10, 2003 CAS2K3,ANNECY,SEPT 10,2003

  2. OVERVIEW Introduction Objectives Activities Computing Resources Future Plans Conclusions CAS2K3,ANNECY,SEPT 10,2003

  3. ORGANISATIONAL SET UP OF NCMRWF DEPARTMENTOF SCIENCE & TECHNOLOGY PROJECT IMPLEMENTATION COMMITTEE (PIC) SCIENTIFIC ADVISORY COMMITTEE (SAC) N C M R W F IMD CELL ICAR CELL COMPUTER & NETWORKING DIVISION RESEARCH DIVISION APPLICATION DIVISION ADMINISTRATION DIVISION CAS2K3,ANNECY,SEPT 10,2003

  4. OBJECTIVES • Development/Implementation of Global and Regional scale NWP models • Set-up a state-of-the-art supercomputing infrastructure • Set-up agro-meteorological advisory service(AAS) units, in each of the 127 agroclimatic zones of the country and provide weather based guidance • Set-up a stable/fast dedicated communication network with AAS units Objective of NCMRWF

  5. NETWORK OF AGROMET ADVISORY SERVICE (AAS) UNITS OF NCMRWF CAS2K3,ANNECY,SEPT 10,2003

  6. NWP Models at NCMRWF • Global Models • T80 L18(150 x 150) km resolution [operational] • T170 L28 resolution [semi-operational] • Regional Spectral Model • 50 x 50 km resolution • Mesoscale Models • MM5 (Nested 90, 30, 10 km) [operational] • ETA (48 x 48) km [operational]

  7. SPECIAL/ CUSTOMIZED FORECASTS/ PRODUCTS Forecasts for Defence Operations and Exercises. •  Launch of Space Vehicles • Weekly weather forecasts • Forecasts of Onset/Advancement/Withdrawal of Monsoon to IMD • Forecasts on special events • Forecasts for Avalanche prediction • Forecasts for tourism and adventure • Organization of field experiments •  Wind fields for Ocean State Forecast •  Agrinet •  Antarctica Expedition •  Govt. of Andhra Pradesh Disaster Mitigation Project •  Initial conditions for Climate Experiments, Extended Range forecasting and Meso-scale modeling. • Merged Gauge and Satellite data sets. CAS2K3,ANNECY,SEPT 10,2003

  8. THE NCMRWF WEATHER FORECASTING SYSTEM CAS2K3,ANNECY,SEPT 10,2003

  9. CURRENT OBSERVATIONAL PROGRAM • Indian Climate Research Program (ICRP)with sub components like • I The Bay of Bengal Monsoon Experiment (BOBMEX) • BOBMEX- Pilot 1-3 November 1998 • BOBMEX- Field Phase July-August 1999 • Understand the processes governing the variability of organized • convection over the Bay of Bengal. • II The Arabian Sea Monsoon Experiment (ARMEX) • 1. ARMEX I (May- August 2002) • Off-shore trough Experiment (OST) • Aimed at probing the mesoscale convective episodes (high intensity rainfall events ) occurring along west coast of India •       2. ARMEX II (April- June 2003)Warm Pool Experiment(WP) • Aimed at studying evolution, maintenance and collapse of Arabian Sea warm pool observed in summer months over SE Arabian Sea CAS2K3,ANNECY,SEPT 10,2003

  10. Computing Resources Dec-Alpha Cluster (Anupam Alpha) Origin Cluster Cray SV1 Param CAS2K3,ANNECY,SEPT 10,2003

  11. ANUPAM-ALPHA System Indigenously developed Parallel Processing Computer. 2 Nos. DEC-ALPHA Servers AS4100 @600MHz each with Memory of 1GB each 8MB Cache. 9 Nos. DEC-ALPHA Work Stations @600MHz each with Memory of 512MB each and 4MB Cache Inter-Node communication: Gigabit Ethernet Smart Switch Router. CAS2K3,ANNECY,SEPT 10,2003

  12. Anupam-Alpha system CAS2K3,ANNECY,SEPT 10,2003

  13. Cray SV1 24-Processors      1.2 GFlops per processor      8 GB Main Memory      800GB Disk. CAS2K3,ANNECY,SEPT 10,2003

  14. Softwares available on Cray • CF90 Programming • Environment for unlimited users. • C++ • DMF • IMSL LIBRARY(fortran 77/90 and C) CAS2K3,ANNECY,SEPT 10,2003

  15. Origin Cluster Configuration 2 Servers(4-cpu each) Origin200@255MHz , Memory 1 GB each 3 Servers (1cpu) Origin200@270MHz, Memory 512 MB each 1 Server (1cpu) Origin200@180MHz, Memory 512 MB each 4 Workstations O2@200MHz, Memory 512 MB each CAS2K3,ANNECY,SEPT 10,2003

  16. IBM 3490 CRAY SV1 ADMINISTRATION WORK STATION AUTOLOADER DLT craysgi1_fddi 1 CPU ORIGIN 200 craysgi2_fddi 1 CPU ORIGIN 200 craysgi3_fddi 1 CPU ORIGIN 200 9TRACK SPOOLER TAPE DRIVE AUTOLOADER DLT CRAYSGI\2 1 CPU ORIGIN 200 CRAYSGI13 1 CPU ORIGIN200 RAID BOX CRAYSGI1 1 CPU ORIGIN 200 AUTOLOADER DLT LINK TO LAN O200-1 1 CPU ORIGIN 200 O200-3 4 CPU ORIGIN 200 O200-2 4 CPU ORIGIN 200 ORIGIN VAULT CAS2K3,ANNECY,SEPT 10,2003

  17. PARAM SYSTEM Model is PARAM 10000 2 Sun Ultra Sparc-II Servers (4-CPU each) @300 MHz Memory 1 GB each MYRINET switch CAS2K3,ANNECY,SEPT 10,2003

  18. Parallel Processing at NCMRWF Latitude Parallel Strategy followed Performances were compared objectively Actual Timings noted are no longer relevant now. The Exercise showed that indigenously developed clusters could be used for NWP CAS2K3,ANNECY,SEPT 10,2003

  19. T80 on XMP Origin Alpha PARAM T126 on SV1 8 CPU 12 CPU 16 CPU 24 CPU CAS2K3,ANNECY,SEPT 10,2003

  20. LOAD IMBALANCE IN WEATHER MODELS … Land-Sea: In a given Latitude Circle, Number of Land or Sea Points are not the same (LSP) Same is true for Mountains also !!! … Convection: Convection conditions are satisfied only in Tropics, Near the Pole, there is No Convection !!! … Clouds: Cloud Formation, Type, etc. can be different in different latitude circles It is very difficult to achieve Load Balance in Weather Models CAS2K3,ANNECY,SEPT 10,2003

  21. Optimized Convection (one call to convection scheme) tropics mid-lat polar Mixed latitudes NCMRWF T80 Model CAS2K3,ANNECY,SEPT 10,2003

  22. Network Servers Modem 64 kbps Leased Line WWW PDNS FTP Mail SDNS Servers - Dell PowerEdge 1300 V.35 Firewall Proxy CAS2K3,ANNECY,SEPT 10,2003

  23. T80 T170 MM5 ETA MISC 643.6 MB 936.8 MB 233 MB 70.1 MB 500MB Archival Details/DAY CAS2K3,ANNECY,SEPT 10,2003

  24. MEDIA DETAILS • DAT,DLT,CDROM,EXABYTE drives are available in the center. • Data is archived in various Media. • For operational purpose usually DLT media is used. • When a specific data set is prepared for an outside agency, data is sent in DAT / DLT/ CDROM. • Data is also sent using the FTP public outgoing area, were it can be accessed by the outside agency. CAS2K3,ANNECY,SEPT 10,2003

  25. Types of data distributed • Post processed model output • Monsoon Data • Observation ,Analysis and Forecast CAS2K3,ANNECY,SEPT 10,2003

  26. Data Visualisation Tools • MAGICS • GrADS • GMT • GIS • NCAR Graphics • Other Public Domain Software CAS2K3,ANNECY,SEPT 10,2003

  27. ARMEX ATMOSPHERIC DATA CENTRE •   NCMRWF - IITD - IMD • Objectives of Data Centre • Processing, quality control and archiving conventional and • non-conventional data of    atmospheric and oceanographic • observations collected during ARMEX period • Storing gridded analysis prepared by Operational Analysis • Forecasting system of NCMRWF, gridded satellite data (viz • INSAT, METEOSAT, SSMI, TRMM) during ARMEX • period • Assimilation of ARMEX observations using NCMRWF • Global T170 Model to generate reanalysis at 70 X 70 Km • resolution CAS2K3,ANNECY,SEPT 10,2003

  28. Future Road Map of Modeling Activity at NCMRWF Model Resolution T340L60 Ensemble Runs Meso-Scale 10km over the entire Indian Region 1 km models for Clouds Use of more Satellite Data Direct Radiance Assimilation Real-time Seasonal Prediction Climate Variability and Change AMIP-type Runs CAS2K3,ANNECY,SEPT 10,2003

  29. Infrastructure upgradation Plans Procurement of Param Padma Upgradation of Cray SV1 Upgradation of Anupam Increase Storage to 20TB Create a SAN solution Upgradation ILL capacity CAS2K3,ANNECY,SEPT 10,2003

  30. PARAM Padma • Member of the Top500 HPC list • Supports applications based on professional Unix and Linux • At present it is certified for AIX and Linux • By supporting above Operating Systems, PARAM technology becomes more or less Vendor independent • Supports Parallel Development tools and Parallel File System (PFS) • Proven Platform running applications ranging from Scientific to Databases to Financial Modeling CAS2K3,ANNECY,SEPT 10,2003

  31. PARAM Padma Compute Cluster Layout PARAM Padma HPC Cluster based on 4 way Power 4 + Systems 1 8 Compute Nodes connected through PARAM Padma System Area Network and Gigabit Ethernet PARAM Padma System Area Network using PARAMNet II 16 port Switches + Gigabit Network LinuxConsole 9 16 1 4 SAN Switch SAN Switch Graphics Workstations Storage Array BackupLibrary 5 TB 10 TB Gigabit Ethernet FC-AL CAS2K3,ANNECY,SEPT 10,2003

  32. CRAY SV1 ex A 500 MHz clock Low-latency high bandwidth cache A 32 Gbyte internal SSD SDRAM twice the bandwidth of SV1 DRAM Vector processing rate of 2 Gflops per CPU Scalar processing rate of 500MIPS CAS2K3,ANNECY,SEPT 10,2003

  33. ANUPAM-Xeon/128 • Processor :64 Dual Xeon Servers • Speed :2.4 GHz • Peak Performance :202 Gflops • Memory/processor :2GB • Interconnect Tech. : Scalable Coherent Interface(SCI) • OS : LINUX CAS2K3,ANNECY,SEPT 10,2003

  34. Anupam-Xeon/128 CAS2K3,ANNECY,SEPT 10,2003

  35. We began with … CAS2K3,ANNECY,SEPT 10,2003

  36. We have….. CAS2K3,ANNECY,SEPT 10,2003

  37. Near Future Plans… CAS2K3,ANNECY,SEPT 10,2003

  38. Conclusion • Computing Resources at NCMRWF are moderate • Plans to enhance resources are underway CAS2K3,ANNECY,SEPT 10,2003

  39. Thank You CAS2K3,ANNECY,SEPT 10,2003

  40. CAS2K3,ANNECY,SEPT 10,2003

  41. CAS2K3,ANNECY,SEPT 10,2003

  42. Proposed PARAM Padma Configuration to NCMRWF • Proposed 16 Node, 4 CPU / Node based PARAM Padma running on AIX Operating System • Based on the high performance Power 4 plus Processor technology of IBM • Dual PARAMNet interconnectivity for Fail over • Parallel software development environment • Fiber channel based network centric architecture with 5 TB Storage with Parallel File System(PFS) support and 10 TB High Speed Tape Backup • Provides balanced performance and I/O for high data intensive applications. • Applications like T80, T170, T126 and MM5 Weather codes are running successfully CAS2K3,ANNECY,SEPT 10,2003

  43. ANUPAM-Xeon/128 • Processor :64 Dual Xeon Servers • Speed :2.4 GHz • Peak Performance :202 Gflops • Memory/processor :2GB • Interconnect Tech. : Scalable Coherent Interface(SCI) • OS : LINUX CAS2K3,ANNECY,SEPT 10,2003

  44. Why more computing power? • Computing Requirement increases as more data • are ingested in data assimilation system • Assimilation of Direct Radiance data • from Satellites • 4-D Variational Technique needs most compute • intensive running of Adjoint Models • (as at ECMWF and Meteo-France)

  45. Parallel Processing Software Indigenously developed ANULIB. In ANULIB Message Passing Communication Calls are divided into 4 categories a. Initialisation calls b. Termination calls c. Communication calls d. Miscellaneous calls ANUPAM-ALPHA System runs at NCMRWF as Master-Slave Parallel System with maximum up to 1+8 nodes. CAS2K3,ANNECY,SEPT 10,2003

  46. Local Area Network Details • All LAN servers are DELL PowerEdge 1300 Machines. • Red Hat Linux 8.0 • Pheonix Firewall • Cabletron Smart switch 2200 CAS2K3,ANNECY,SEPT 10,2003

More Related