1 / 21

(TIGGE and) the EU Funded BRIDGE project

(TIGGE and) the EU Funded BRIDGE project. Baudouin Raoult Head of Data and Services Section ECMWF. THORPEX was established in May 2003 by the 14th WMO Congress as a ten-year international global atmospheric research and development programme

dobry
Télécharger la présentation

(TIGGE and) the EU Funded BRIDGE project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. (TIGGE and) the EU Funded BRIDGE project Baudouin Raoult Head of Data and Services Section ECMWF GO-ESSP Paris. June 2007

  2. THORPEX was established in May 2003 by the 14th WMO Congress as a ten-year international global atmospheric research and development programme • THORPEX is under the auspices of the WMO Commission for Atmospheric Sciences (CAS) • THORPEX Goals: • To reduce and mitigate natural disasters • To fully realise the societal and economic benefits of improved weather forecasts GO-ESSP Paris. June 2007

  3. TIGGE • THORPEX Interactive Grand Global Ensemble • Key objectives of TIGGE: • Enhanced collaboration on development of ensemble prediction, internationally and between operational centres and universities • New methods of combining ensembles from different sources and of correcting for systematic errors • A deeper understanding of the feasibility of interactive ensemble system responding dynamically to changing uncertainty • The development of a prototype future Global Interactive Forecasting System (GIFS) GO-ESSP Paris. June 2007

  4. The TIGGE core dataset • Global ensemble forecasts to around 14 days generated routinely at different centres around the world • Outputs collected in near real time and stored in a common format for access by the research community • Easy access to long series of data is necessary for applications such as bias correction and the optimal combination of ensembles from different sources GO-ESSP Paris. June 2007

  5. Homogeneity of the TIGGE database • Homogeneity is paramount for TIGGE to succeed • The more consistent the archive the easier it will be to develop applications • There are three aspects to homogeneity: • Common terminology (parameters names, file names,…) • Common data format • Definition of an agreed list of products (Parameters, Steps, levels, …) GO-ESSP Paris. June 2007

  6. Completeness • The objective is to have 100% complete datasets at the Archive Centres • Completeness may not be achieved for two reasons: • The transfer of the data to the Archive Centre fails • Operational activities at a data provider are interrupted and back filling past runs is impractical • Incomplete datasets are often very difficult to use • Most of the current tools used for ensemble forecasts assume a fixed number of members from day to day • These tools will have to be rewritten GO-ESSP Paris. June 2007

  7. Strong governance • Precise definition of: • Which products: list of parameters, levels, steps, units,… • Which format: GRIB2 • Which transport protocol: UNIDATA’s LDM • Which naming convention: WMO file name convention • Only exception: the grid and resolution • Choice of the data provider • Best possible model output • One must do what one requires from others • Sample dataset available • Various GRIB2 tools, “tigge_check” validator, … • Scripts that implement exchange protocol GO-ESSP Paris. June 2007

  8. ECMWF NCAR CMA Phased implementation of the archive • Phase 1: multiple instances, low development effort Phase 1 Archive Centres GO-ESSP Paris. June 2007

  9. Building the TIGGE database • Three archive centres: CMA, NCAR and ECMWF • Ten data providers: • Already sending data routinely: ECMWF, JMA (Japan), UK Met Office (UK), CMA (China), NCEP (USA) • Coming soon: BOM (Australia), CPTEC (Brazil), KMA (Korea), MSC (Canada), Météo-France (France) • Exchanges using UNIDATA LDM, HTTP and FTP • Firewall issues with China • Operational since 1st of October 2006 • 43 TB, growing by ~ 1 TB/week GO-ESSP Paris. June 2007

  10. Quality Assurance GO-ESSP Paris. June 2007

  11. Accessing TIGGE • Portals at NCAR and ECMWF. CMA portal to be developed • ECMWF portal offers: • Access to offline data • Aggregation along any axis (date, level, parameter, origin, ensemble, …) • Provision of multi-model data on a single grid (regridding to any lat/lon grid) • Sub-area selection • Reduces volumes to be downloaded by many order of magnitude GO-ESSP Paris. June 2007

  12. Phased implementation of the archive • Phase 2: distributed approach, higher development effort GO-ESSP Paris. June 2007

  13. BRIDGE • Bridge is a 2 years project funded by the EC under the FP6-IST programme. • It will demonstrate the benefits of GRID technology for international cooperation, in particular between Europe and China • This work focuses on the development of interoperable Grid infrastructures (CNGrid, GRIA) • Three applications: Aircraft Design, Meteorology, Drug Design GO-ESSP Paris. June 2007

  14. Meteo scenario: exploring TIGGE phase 2 • Distributed processing on distributed data, across the two GRID middleware • Each site hosts only part of the data • Each site offers basic operations on the data (e.g. computing an average) • Strategy: minimize data transfers • Run operations at data location • Decompose operations in simpler ones • Most of the time intermediate results are much smaller GO-ESSP Paris. June 2007

  15. EPS Products • Examples • Ensemble mean • Standard deviation • Clustering • Probability of weather events • Extreme Forecast Index • EPSgram • Some products can be decomposed in simpler operations on a subset of members • Ensemble means • Some products need all the members • Clustering GO-ESSP Paris. June 2007

  16. Example 1: ensemble mean • Data requests describes 10 fields • 6 are available from ECMWF • 4 are available from CMA • S1 = sum(6 ECMWF fields) performed at ECMWF • S2 = sum(4 CMA fields) performed at CMA • Intermediate results and associated metadata moved to site were user invoked request: (S1,6) and (S2,4) • Final result A=(S1 + S2)/(6+4) computed locally and returned to user GO-ESSP Paris. June 2007

  17. Example 2: EPSGram • Send lat/lon location at each sites • Receive list of values back • Compute distributions • Generate plot GO-ESSP Paris. June 2007

  18. Non-decomposable operation GO-ESSP Paris. June 2007

  19. Decomposable operation GO-ESSP Paris. June 2007

  20. Operations as Workflows • Study a selection of EPS products • Classify them into decomposable and non-decomposable • For decomposable operations • Implement sub-operations as web services • Deploy several instances of these web services • Describe full operation as a workflow • User will not see the workflow, but the high level operation • There may be several ways to execute a workflow • Because the same sub-operations are offered by several sites • Choose the “cheapest” workflow • Cost of a workflow based on amount of data moved GO-ESSP Paris. June 2007

  21. Conclusion • TIGGE Phase 1 is progressing well • Strong governance • Very good working relationship between CMA, NCAR and ECMWF • BRIDGE project gives us the funding to explore how to implement TIGGE Phase 2 GO-ESSP Paris. June 2007

More Related