1 / 66

NA4: Environmental VO Overview and Planning

NA4: Environmental VO Overview and Planning. PSC05 Meeting, Dubrovnik, 09-11 September, 2009. Environmental VO applications. ENV VO support structure. Helpdesk – VO support group How to get membership: https://voms.ipp.acad.bg:8443/voms/env.see-grid-sci.eu/

jesusahall
Télécharger la présentation

NA4: Environmental VO Overview and Planning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NA4: Environmental VO Overview and Planning PSC05 Meeting, Dubrovnik, 09-11 September, 2009 The SEE-GRID-SCI initiative is co-funded by the EC under the FP7 Research Infrastructures contract no. 211338

  2. Environmental VO applications PSC04 meeting, Sofia, 6-8 May, 2009 2/51

  3. ENV VO support structure Helpdesk – VO support group How to get membership: https://voms.ipp.acad.bg:8443/voms/env.see-grid-sci.eu/ UI are available at each of the supporting sites, contact site administrators General Information: http://wiki.egee-see.org/index.php/SG_Environmental_VO Configuration information for ENV VO support: http://wiki.egee-see.org/index.php/Environmental_VO_Support PSC05 meeting, Dubrovnik, 9-11 Sept, 2009 3/51

  4. MCSAES MCSAES – Monte Carlo Sensitivity Analysis for Environmental Systems Environmental VO, SEE-GRID-SCI, May 2009 Aneta Karaivanova, E. Atanassov, T. Gurov, Institute for Parallel Processing, BAS PSC05 meeting, Dubrovnik, 9-11 Sept, 2009 4/51

  5. Application Summary • Application problem description • The aim of the application is to develop an efficient Grid implementation of a Monte Carlo technique for sensitivity studies in the domains of Environmental modelling and Environmental security. The developed application will be applied for studying the damaging effects that can be caused by high pollution levels (especially effects on human health), when the main tool will be the Danish Eulerian Model (DEM). • Inputs • Grid-bound, in perspective - real data from Bulgarian hydrometeorology institute. The input format is standardized. • Outputs • The output of the Grid jobs are concentrations of pollutants, but the application deals with sensitivity analysis of these data with respect to some chemical constants. Thus the output of the whole application are coefficients that measure this sensitivity.

  6. MCSAES Implementation Scheme • The green modules are ported to the Grid: • Monte Carlo integral computational kernels with MPI on the Grid. • Execution of the DEM model as a grid job. • Integration of the RBI QRBG generator. • The modules in grey are now in testing grid mode – proceeds according to plan.

  7. M18 – Quasi-Monte Carlo methods with scrambling development (some codes developed at the new NVIDIA GTX295 WNs with a total of about 7 teraflops in single precisions) M18-M24 – Deployed at production-level: Application fully integrating MC and QMC methods with DEM - running in full scale. MCSAES time plan

  8. MCSAES deployment and support The application software is a relatively small distribution (<1GB), that will be deployed at the shared software area of the supporting sites. The supporting sites must support MPI. One application run will consist of high number of MPI jobs, all of which will have to complete in order to obtain the final results. The RBI QRGB should be available for grid users as a source of random numbers. The users will interact with the JTS and UPM services in order to optimize the execution of the MPI jobs. Performance data related to the execution will be gathered.

  9. MCSAES Results Scientific publications: E. Atanassov, A. Karaivanova, T. Gurov, S. Ivanovska, M. Durchova “Using Sobol’ Sequence in Grid Environment”, Accepted for Proceedings of MIPRO32 2009/GVS. E. Atanassov, S. Ivanovska and A. Karaivanova “Tuning the Generation of Sobol Sequence with Owen Scrambling”, Accepted for Proceedings of LSSC09, 2009, June 3-8.

  10. MSCAM (BG) MSACM– Multi-Scale Atmospheric Composition Modelling PSC05 Meeting, Dubrovnik, 09-11 September, 2009 Kostadin Ganev, Geophysical Institute, BAS

  11. Application Summary • Application problem description • Atmospheric composition directly affect many aspects of life. AQ studies are fundamental for the future orientation of national, regional and Europe’s Sustainable Development strategy. • Inputs • Physiographic data (topography and land-use) – USGS data base • Meteorologicalbackground - US NCEP Global Analysis data • Emission data - UBA emission database for Europe • Outputs • Meteorological fields: wind, temperature, turbulent coefficients, precipitation, cloud cover • Standard CMAQ output (concentrations, columnar values, depositions). The number of compounds depends on chemical mechanism chosen, but is within the range 50-100 • Additional – derivatives from the concentrations, according to the EU Directive defined criteria (like AOT40, NOD60)

  12. Application Summary (cont’d) • Expected results and their consequences • high quality scientifically robust assessments of the air pollution and its origin from urban thruogh local to regional (Balkan) scales • Determination of the main pathways and processes that lead to atmospheric composition formation in different scales • Possible scientific impact • achieving multi-scale model operating proficiency and skills, model validation • improved scientific understanding of processes and situations behind specific pollution episodes in the country • development and implementation of interdisciplinary approaches by applying up-to-date IT and grid technologies to AQ studies

  13. Application Summary (cont’d) Possible social impact EU compatible tools and AQ monitoring strategy AQ information and forecasts as a basis for sound decision making (short-term and strategic pollution abatement strategies) General public and target groups awareness Potential or existing user or beneficiary community Existing: environmental scientists, relevant governmental bodies (National Executive Environmental Agency), international AQ monitoring bodies; size ~20 - 100person Potential: municipal authorities, other stakeholders (industry), media (general public) ; size – up to General population

  14. Collaborations • Collaborations already set • National EEA will supply the project activities with the necessary administrative information and will ensure, when necessary for the project needs, collaboration from other governmental, regional or municipal authorities • COST Actions 728 and ES0602 – data exchange, model tuning and validation, joint exercises for model comparison and cross-comparison • Research groups in Albania (MetOffice), Greece (AUTH) and Romania (NMA)

  15. MSACM implementation scheme

  16. MSACM progress Deployment of regional to local scale simulations grid-mode of MSACM application: A set of numerical experiment scenarios is defined and the corresponding scripts are written, which makes it possible by running a large number of uniformly designed jobs on the grid to obtain and accumulate model outputs, which can be used for: Evaluation of country-to-country pollution budgets Evaluation of the contribution of different sources, or different SNAP categories to the regional and local scale air pollution Evaluation of the mutual influence of different sources/SNAP categories/countries on the overall pollution pattern or on different processes that form it Scripts have been offered to AL partner to participate in the optimisation of the grid execution. Discussing the involvement of MD partner (training needed). GUI under development.

  17. MSACM current status Deployed at production-level. In order to be able to utilize this application on the Grid, the following work was performed: Model validation tests for episodes of very high PM10 concentrations over Germany in February and March of 2003. The Process Rate Analysis option of SMAQ is used to clarify the contribution of different processes and mechanisms to the formation of PM10 concentration fields over Germany; Local scale simulations for Maritza TPP; The US EPA Models 3 system is installed on the computer cluster of GPhI Software for emission speciation and for introducing emission temporal profiles is created; A procedure for calculating biogenic VOC emissions by using the SMOKE abilities is developed; The US EPA models WRF and WRF-Chem are installed as a GRID application; Interface (Fortran code) for transferring ALADEN output into input for MM5 and/or WRF is created; Interface (Fortran code) for downscaling the CMAQ simulations from Balkan to national scale – extracting boundary conditions from AIRTESS (air quality simulation system operated by the Aristotel University of Thessaloniki, Greece) 10-km Grid2 CAMx average files (3 successful days). The actions performed by the interface are: Interpolation of CAMx concentrations in BgDomain boundary points; Vertical interpolation and species redistribution, NetCDF file creation. Organization (respective scripts are created) of the whole technological chain for Air Quality simulations for the country into an operational system which: Downscales the meteorological fields and performs meteorological pre-processing for CMAQ; Calculates natural and anthropogenic emissions for Bulgaria; Extracts boundary conditions for the CMAQ simulations from larger scale air pollution fields; Performs high resolution Air Quality simulations for the country, accounting for 78 pollutants. The application can be run currently on the Grid. The high storage requirements (100 GB output per job) are a problem. Filters have been developed to decrease the data to be retained.

  18. MSACM time plan M13/M18 – Continue using grid-mode (1) – Regional to local scale simulations; M13/M21 – Using grid-mode (2) – Simulation for selected "hot spots"; M13/M24 – Identifying end users, terms, ways and formats of parsing the output, elaborating tools for output converting; Development proceeds according to plan.

  19. MSACM deployment and support The input data, related to meteorology can be computed once using Grid jobs and deployed at the storage elements (this computation for 2000 year was accomplished at end of April in just 5 days). All input data are estimated to take around 3 TB for computations related to one year. The applications consist of a collection of independent MPI jobs. The supporting sites must have sufficient storage resources in order to accommodate the input data. Each execution needs between 100 – 200 GB temporary storage space depending on the scenario and generates in the order of 100 GB output data, which is stored at the storage element. This data will be filtered in order to decrease the storage requirements for full-scale utilization. The estimate is that about 18 TB will be necessary for computations related to one year and 10 different scenarios.

  20. MSACM scientific results • D. Syrakov, K. Ganev, M. Prodanova, N. Miloshev, G. Jordanov, “Background Pollution Forecast over Bulgaria” 2009, to appear in Lirkov, S.Margenov, and J.Wasniewski (Eds.), LSSCD2009, Lecture Notes in Computer Sciences (accepted, in press) • D. Syrakov, M. Prodanova, K. Ganev, N. Miloshev, E. Atanasov, T. Gurov and A. Karaivanova, Grid computing for multi-scale atmospheric composition modelling for the Balkan region, 18th International Symposium ECOLOGY & SAFETY, June 8 - 12, 2009, Sunny Beach, Bulgaria (published on a CD), ISSN: 1313-2563 • Kostadin Ganev, Dimiter Syrakov, Maria Prodanova, Emanouil Atanasov, Todor Gurov, Aneta Karaivanova, Nikolai Miloshev, “Grid Computing for Air Quality and Environmental Studies in Bulgaria”, 23rd EnviroInfo 2009 Conference - Environmental Informatics and Industrial Environmental Protection: Concepts, Methods and Tools, Berlin, September 9th - 11th 2009, (accepted, in press) PSC04 meeting, Sofia, 6-8 May, 2009 20/51

  21. MSERRHSA (BG) MSERRHSA– Modelling System for Emergency Response to the Release of Harmful Substances in the Atmosphere PSC05 Meeting, Dubrovnik, 09-11 September, 2009 Kostadin Ganev, Geophysical Institute, BAS

  22. Application Summary • Application problem description • High quality scientific and technical information is critical for proper emergency management • High requirements for air pollution modelling for emergency response purposes • extensive preliminary studies, model tuning and validation, risk assessments, evacuation plans, emergency response exercises • Inputs • Physiographic data (topography and land-use) – USGS data base • Demographic data – source still not identified • Meteorologicalinput - US NCEP Global Analysis data/NIMH current forecast • Routine emission data - UBA emission database for Europe • Emergency emission scenarios – National Civil Protection Agency

  23. Application Summary (cont’d) Outputs Meteorological fields: wind, temperature, turbulent coefficients, precipitation, cloud cover Harmful pollutants concentrations Human exposure/health impact assessments Risk assessment maps (in the system preparedness mode) Depositions and environmental impact maps (in the system recovery mode)

  24. Application Summary (cont’d) • Expected results and their consequences • Operational multi-mode emergency response system • More adequate fast-decisions in case of emergency • Improved preparedness for emergency situations • Possible scientific impact • Applying sophysticated meteorological and pollution transport models with high spatial resolution for the very complex terrain of the Balkan Peninsula • Following the accidentally released toxic gases from local through regional to European scale, accounting for the mesoscale dynamic phenomena

  25. Application Summary (cont’d) Possible social impact Better preparedness for emergency situations Better emergency response: mitigation of casualties Formulation of long-term strategic measures and activities for abatement of the caused damages and gradual restoration of the environment Potential or existing user or beneficiary community Existing: governmental bodies - State Agency “Civil Protection”, international emergency response bodies; size 5-20 person Potential: municipal authorities, media (general public) ; size – up to General population

  26. Collaborations • Collaborations already set • State Agency “Civil Protection” will provide information of possible hazardous objects and respective emergency release scenarios • Balkan region collaboration with National Meteorological Administration, Romania, Laboratory of Atmospheric Physics, Aristotle University of Thessaloniki, Greece, Hydro-meteorological Institute, Albania in the frame of NATO ESP.EAP.SFPP 981393 project – data exchange, model tuning and validation, joint exercises • Collaborations planned for the future • It is planned the system to be incorporated in the ENSEMBLE project and/or some of the ENSEMBLE tools to be applied for emergency forecast in local to regional scale. This will add some European dimensions to the project results implementation

  27. MSERRHSA Implementation Scheme

  28. MSERRHSA current status Testing Grid-mode: Database of industrial sites in Bulgaria, which, by the definitions of Directives 2003/105/ЕС and 96/82/ЕС can be a source of harmful substances in the atmosphere is created. Database with some scenarios of accidental harmful substances releases in the atmosphere is created. Current maintaining and upgrading of the already created databases: The two new chemical mechanisms for Hazardous Air Pollutants that are introduced in Version 4.6 of CMAQ - SAPRC99 and CB05CLTX are carefully studied and adopted. The Version 4.6 of CMAQ with CB05CLTX chemical mechanism is installed and compiled on the computer cluster of GPhI. Risk analysis simulations for accidental Chlorine release scenario started on the computer cluster of GPhI are going on – the meteorological pre-processing (downscaling the meteorological fields from the US NCEP Global Analysis data to a fine 1km resolution, using the WRF model) is completed.

  29. MSERRHSA current status Deployment of preparedness system function (risk analysis and assessment for selected “hot spots”) as a grid-mode of MSERRHSA application: grid mode simulations for a typical summer and winter month for day-time and nocturnal accidental releases of 500 t of Cl2 (molecular chlorine) in “VEREJA-HIM” JAMBOL are completed. Development of cluster mode real time forecast of propagation of tracers released twice a day from selected industrial sites – the most dangerous potential sources of accidental toxic gas releases in the atmosphere started.

  30. MSERRHSA Time Plan M13-M15 – Testing grid-mode (1) - preparedness system mode: risk analysis and assessment for selected "hot spots"; M16-M18 – Testing grid-mode (2) - fast decision system mode: emergency response exercises; M19-M24 – Deployed at production-level and Presentable - "recovery" system mode: application fully deployed and presentable.

  31. MSERRHSA deployment and support For this application the input databases have to be deployed at the supporting Grid sites. Parts of the input data (boundary conditions) are downloaded from Greece – a process that can be optimized using Grid protocols, since the collaborating partners from Aristotle University of Thessaloniki also have Grid access. The usability of the application and especially the visualization part will need some deficiencies in the network infrastructure at the partner institutes to be resolved. Using the application for real incidents will impose stronger QoS requirements for job execution and file transfers. On one side, they will be addressed with a MoU between the providers of infrastructure and the users of the application. Technically they will be resolved by the SEE-GRID-SCI application specific operational tools, especially JTS and UPM (implementation of QoS and follow-up with usage and performance information). It is estimated that approximately 1.9 TB will be necessary for computations related to one “hot-spot” and 1 year. The realistic scientific case needs computations related to 10 years, i.e. 19 TB. It is expected that during the second year of the project new resources will be made available and will cover these requirements. The usage of this application in forecasting mode will have more modest requirements – approximately 5 GB for 24 hours

  32. MSERRHSA scientific publications Scientific presentation and publications: A. Todorova, G. Gadzhev, G. Jordanov, D. Syrakov, K. Ganev, N. Miloshev, M. Prodanova, “Numerical Study of Some High Pm10 Levels Episodes”, to appear in Lirkov, S.Margenov, and J.Wasniewski (Eds.), LSSCD2009, Lecture Notes in Computer Sciences (accepted, in press) A. Misev and E. Atanassov, “User Level Grid Quality of Service”, to appear in LNCS, 7th LSSC09, Sozopol, 4-8 June, 2009.

  33. GREEN WIEW GreenView – Refinement of surface and vegetation parameters in SEE region based on satellite images PSC05 Meeting, Dubrovnik, 09-11 September, 2009 Dorian Gorgan Computer Science Department Technical University of Cluj-Napoca, Romania dorian.gorgan@cs.utcluj.ro

  34. Application Summary • Application problem description • The aim of the GreeView pilot application is a refinement of surface and vegetation parameters in SEE region based on satellite images. In the frame of this project, construction, usage and comparison of diverse satellite datasets will be performed. High resolution satellite measurements can be used for numerous environmental studies (climate-related or air pollution modeling). • Using the sophisticated environmental data the change of the vegetation distribution in the Carpathian Basin and its climate-related causes will be investigated. It may also be used to extend the urban climate related research, in order to study the impact of urban environment on the vegetation.

  35. Application Summary • Application problem description (cont’d) • The dataset may also contribute to the development of air pollution models, which describe the dispersion of tracers or their exchange between the surface and the atmosphere. These transport and deposition/exchange models require detailed input fields about surface and vegetation (for example vegetation types, leaf- area index, albedo). Satellite data could be an appropriate source to create these input datasets for the regional models. • Other potential pilot applications will be investigated, as for example water detection and water area supervision, which may be of interest for flooding events monitoring.

  36. Application Summary • Inputs • Satellite image datasets MODIS (MODerate Resolution Imaging Sounder) products. • Meteorological data • Field measurements Eddy covariance measurements for a particular geographical area. • Outputs • Vegetation parameters • NDVI-FPAR/LAI correlation function • Satellite images classified by vegetation indices • Statistics on output data • Performance evaluation on data processing

  37. Contributors Technical University of Cluj-Napoca (RO) Eötvös Loránd University (HU) Research and Educational Networking Association of Moldova (MD) National Center for Information Technology of University "Politehnica" Bucharest (RO) Computer Science Department of West University of Timisoara (RO), State Hydrometeorological Service (MD) Faculty of Radioelectronics and Telecommunications of Technical University of Moldova (MD)

  38. GreenView current status Developing Grid-mode. Actually the GreenView application has the following achievements in Y1 of the project: Coarse to Fine interpolation Fine to Coarse interpolation is under work Monte Carlo based model calibration (in the test phase) Develop the GreenView functionality on the ESIP Platform Experiment the GreenView functionality over the Grid testbed under gLite middleware Develop the graphical user interface for Web application First experiments over the See-Grid infrastructure (access through WMS at ICI and UVT, the effective execution over the node ce.ngcc.acad.bg) a/ Initial temperature measurements in coarse resolution satellite image. b/ Computed temperature in fine resolution satellite image.

  39. GreenView time plan M11 – Implement and test the GreenView application over the testbed and gLite; M12 – Perform the evaluation of the application correctness and performance. Deployment of the first version over the SEE-GRID testbed infrastructure. Deliverable DNA4.2; M14 - Production tests over the SEE-GRID testbed infrastructure. First version of the GreenView application is available in SEE-GRID for production tests. The end user access the GreenView application as Web application; Completed succesfully. M17 – Perform the production tests. First version of the GreenView application is available in SEE-GRID at 30.09.2009 – proceeds according to plan. M20 – ESIP based enhancement. Extend the GreenView application according with new version of the ESIP platform, data and the observations arising from the production tests. Deliverable DNA4.3; M21 – Testing the new version; M22 – Measurements and evaluations of GreenView performance; deployment of the application in SEE-GRID, available to user community.

  40. Development Plan Current development status: Developing grid-mode

  41. GreenView deployment and support GreenView is a web application, and the user accesses by Internet the computation resources and environmental data that are provided by the Grid infrastructure. The GreenView application is based on the ESIP platform and related methodology, which are going to be provided by JRA1 of the SEE-GRID-SCI project. ESIP supports the development and the execution of the Grid based applications concerning particularly with the processing of satellite images and generally with environmental related processing and studies. ESIP is based on the gProcess platform that provides the user with the possibility to explore the optimal solutions for Grid processing and information searching in the multispectral bands of the satellite images. The gProcess platform is an interactive toolset supporting the flexible description, instantiation, scheduling and execution of the Grid processing. The gProcess platform provides a flexible diagrammatical description solution for image processing workflows in the Earth Observation. At the conceptual level the algorithms are described by processing acyclic graphs, in which the nodes represent operators, services, subgraphs and input data (e.g. satellite image bandwidths), and the arcs represent the execution dependencies between nodes.

  42. GreenView Results Papers have been prepared and submitted to the following conferences: Gorgan, D., Stefanut, T., Bacu, V., Mihon, D., Grid based Environment Application Development Methodology, SCICOM09 - 7th International Conference on "Large-Scale Scientific Computations", 4-8 June, Sozopol, Bulgaria (2009) Bacu V., Stefanut T., Rodila D., Gorgan D., Process Description Graph Composition by gProcess Platform. HiPerGRID - 3rd International Workshop on High Performance Grid Middleware, 28 May, Bucharest, (2009)

  43. Lizza-PAKP Lizza-PAKP Environmental VO, SEE-GRID-SCI, May 2009 Milos IvanovicResearch and Development Center for Bioengineering, Kragujevac, Serbia(CSASA at SEEGRID-SCI)‏ e-mail: mivanovic@kg.ac.yu internet: http://www.csk.kg.ac.yu

  44. Application Summary • Application problem description • Lizza-PAKP is a groundwater flow simulation system containing solver based on finite element method (PAKP) and user interface (Lizza). It was developed in cooperation between Institute for Water Resources "Jaroslav Cerni", Belgrade and CSASA Kragujevac. It provides full 3D modeling capabilities, stationary and non-stationary simulations, saturated and non-saturated environment calculation, as well as mass and heat transport handling. PAKP module includes solver based on MPI, making its presence a grid site requirement. The application also provides TCP binder interface, which helps hiding grid complexity from the end user

  45. Application Summary • Inputs • User forms arbitrary shaped 3D model specifying data in a way usual in hydrological practice (terrain contours, bottom surface contours, layer material characteristics, characteristics of river-beds). Boundary conditions contain specified levels, flow rates and wells (tube and Ranney kinds). Input data format is simple app. specific CSV, while background maps and textures are standard BMPs. • Outputs • PAKP output contains file(s) stored at grid Storage Element which complies with standard UNV format commonly used in FE analysis. Despite this fact, Lizza itself seems to be the best post-processing, visualization and graphing tool to answer these needs.

  46. Application on the Grid SE withGFAL API support PAK-P instance 1 LIZZAUser interfacepre- and post-processing&visualization WorkloadManagementSystem(WMS) PAK-P instance 2 PAK-P instance N Implementation on the Grid infrastructure gives this application totally new dimension, level of reality the user can deal with increased significantly, as well as the computational speed. Since it is based on a highly demanding finite element engine, grid processing and storage power are a natural ambience for Lizza-PAKP By employing grid resources, it is now possible to perform parametric study easy and quickly

  47. Collaborations • Collaborations already set • Institute for Water Resources "Jaroslav Cerni", Belgrade, Serbia, majority of the requirements regarding Lizza-PAKP features came from their side. They actively support application management, development and usage • RCUB Belgrade, Serbia (UOB at SEEGRID-SCI) regarding coopearation in TCP Binder interface development • Collaborations planned for the future • water supply companies and institutes collaborating with “Jaroslav Cerni” institute

  48. Current status and Time plan Thanks to recent changes regarding various improvements of Work Binder service incorporation and in-deep performance analysis performed, the authors' opinion is that Lizza-PAKP is now fully presentable.

  49. Deployment and Support Being a part of Environmental VO, Lizza-PAKP provides per site installation scripts enabling members of ENV, SEEGRID and AEGIS VO to use it without manual transfer of executables/libraries. The application runs in 2 modes - (1) using pure WMProxy and (2) using Work Binder to get sufficient number of WNs. In pure WMProxy mode, MPI can be enabled or disabled, according to specific site support. In Work Binder mode, MPI is disabled by default, but requires an instance of Work Binder supporting Lizza-PAKP application inside the configuration. Application usage monitoring is achieved by addition of a special VO group. User support is currently limited to mailing lists with correspondents from CSANU, UOB and

  50. CHERS (Al) CHERS – Study of Charges of Environment with Remote Sensing PSC05 Meeting, Dubrovnik, 09-11 September, 2009 Neki Frasheri, Polytechnic University of Tirana, Faculty of Information Technology, nfra@fti.upt.al

More Related