Earth Science GRID on Demand & e-collaboration

Earth Science GRID on Demand & e-collaboration The ESRIN GRID team - luigi.fusco@esa.int “Grid and e-collaboration for the Space Community” ESA-ESRIN, 2 February 2005

Summary • Earth Science and GRID • Earth science and e-collaboration • Present infrastructure at ESRIN • Medium term vision • Example of applications

Earth Science and Grid > 1 Terabyte data processed – 1 Km res. September 2004 June 2004 August 2004 May 2004 July 2004 October 2004 • GRID for Earth Science means: • Single stop shop for data access • Number-crunching, e.g EO data reprocessing • e-collaboration, e.g. access and sharing of data sources, tools, means, models, algorithms … MERIS global composite

e-collaboration technologies Earth Science community analysis • The Internet and improved communication • Web-Services • Grid • Semantic Technologies and Ontologies • Sensor Web • Wireless Technologies • Workflow Management Technologies • Virtual Reality or Augmented Reality • Standards (OGC, W3C, …)

Reference infrastructure Client Applications OGC Web Services SOAP AMS ESA Data Archive ESA EO operational environment http://www.eoportal.org Part of operational ESA EO environment WCS WFS / WMS Client Applications (JAVA / Linux / Windows) SOAP Catalogue (CSS) GRID Engine MUIS ESA Catalogue Web Portal GLOBUS+? LCG + ? DATAGRID EGEE LCG + ? ESA EO planned operational environment European Grid European Grid Storage Elements Storage Elements Storage Elements Storage Elements Storage Elements MEGALAB ESA EO GRID ComputingElements ComputingElements ComputingElements ComputingElements ComputingElements Local Grid GRID & EO Web Services @ ESA geant

ESA actions and 2y vision • Mid term vision – 2 years • Target: mid –2007 • EO missions Ground Segment operations • Satellites data routine acquisition, processing, archiving, dissemination • Earth Science User environment • Science and commercial applications

EO Grid infrastructure G/S vision • Transfer to operation of present GRID capabilities: • Support large scale reprocessing requirements • Demonstrated already for L1  L2 GOMOS • Planned for GOMOS / MERIS in 2005 • New EO processors based on Linux • Enable new classes of large-scale applications (e.g. processing of global MERIS Level 3) • Innovative solution for the data access and exchange • Increased exploitation of available resources • GRID interconnection of ESA and other operational facilities • Reference infrastructure for new missions G/S studies • Take advantage of EC e-infrastructure initiative

Earth Science User Segment Vision • Set up of the EO Science Grid User Segment • Include support to science data processing / analysis based on Envisat tools (e.g. BEAM) • Platform for e-collaboration in Earth Science (e.g. multi data source CAL/VAL…) • Goal 1: Move algorithms and analysis tools to data • data assimilation … integration in models… • Support applications within the ENVISAT AO, e.g. • MERIS data assimilation for chlorophyll forecast… • Goal 2: extend approach to wide science user community • Wish to define a dedicated AO for next year • Contribute to the definition of GMES User segment

Status of applications in Grid environment • Ready for operations • On Demand: Ozone GOME validation, Global and regional MERIS L3 products (MGVI, Chlorophyll …) • Validation of MGVI with Science users • Under development in house • Preparation of ENVISAT (GOMOS, MERIS ) reprocessing • ASAR Global Monitoring Antarctica (sea ice, icebergs, …) • AATSR fire detection and SST; RA2 Lake and river altimetry • Direct access to NASA archives (CEOS) • Under development with external partners • e-collaboration for ozone (GOMOS, SCIAMACHY, MIPAS, …) • Objective analysis SST and chlorophyll in the Med (AATSR, MODIS, MERIS…) • MERIS applications: case 2 waters, data assimilation for chlorophyll forecast, … • CNR – integration of MSG for fire detection; • Uni Tor Vergata – reprocessing of ERS-2 SAR for oil spill statistics

Portal Grid Services • Job selection, chaining, launching and live status • Direct result visualization in WMS (using open source quickWMS client) • Join personal data folder as a WMS

Chlorophyll BEAM processing July 2004 May 2004 June 2004 Input Data: 198.81Gb in 862 files Output Files: 700 Kb (JPEG) 28 Mb (TIFF) 36 Mb (DIM-MAP) Input Data: 158.31Gb in 1643 files Output Files: 700 Kb (JPEG) 28 Mb (TIFF) 36 Mb (DIM-MAP) Input Data: 178.92 Gb in 1828 files Output Files: 700 Kb (JPEG) 28 Mb (TIFF) 36 Mb (DIM-MAP)

AGGREGATION FUNCTIONAssembles the resulting files from the binning processes DATA REGISTRY FUNCTIONRegistries the data files and the corresponding metadata in the defined storage element COLLECT FUNCTION Searches and retrieves the requested files from the storage elements BINNING FUNCTIONProjects the Envisat files to a given datum and projection and according to the requested geographical bounding box Earth Observation Visualization FunctionGeneric functions for image composition and format conversion (JPEG/TIFF/…) RETRIVE FUNCTIONGeneric function for file transfer (FTP, SFTP, gridFTP,…) Chlorophyll processing on GRID BEAM code is embedded here Task chaining in the “Science Grid on Demand” environment. Based on L2 products

Alternate MERIS MGVI algorithms Arithmetic Mean Algorithm fapar (JRC) Algorithm Testing different temporal aggregation methods for MERIS MGVI

MERIS ASAR Global mode mapping July 04 May 04 June 04 Extension of Sea Ice PackDisplacement of icebergs

Participation to external projects • GRID – EC funded: • Participation to EGEE – NA4 • DILIGENT– Digital Libraries in GRID environ’t • started in Sept 04 • ESA in charge of Environmental User scenario • Very interesting technology for Long Term Data and Knowledge Preservation • e-collaboration – EC funded: • BRAINBRIDGE – to KO in Mar 05 • define road map for e-collaboration initiative in FP7 • Infrastructure – Regional funding: • MEGALAB • GRID infrastructure across Frascati-Tor Vergata-Rome area

Grazie!

Earth Science GRID on Demand & e-collaboration