1 / 25

e-Science and Datacentric Frameworks

e-Science refers to the global collaboration in key scientific areas, supported by next-generation infrastructures. This evolving approach is revolutionizing how scientific work is conducted, emphasizing shared resources and collective problem-solving. The rapid growth of network and computing technologies enables unprecedented data analysis and international cooperation. Applications include Earth observations, climate monitoring, and biomedical research. Examples such as NEESgrid for earthquake simulation and NASA's IPG for airspace simulation illustrate the diverse capabilities of e-Science in addressing complex global challenges.

wauna
Télécharger la présentation

e-Science and Datacentric Frameworks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. e-Science and Datacentric Frameworks Hyunseung Choo Sungkyunkwan University http://monet.skku.ac.krchoo@skku.ac.kr

  2. e-Science and its examples

  3. e-Science ‘e-Science’is about global collaboration in key areas of science,and the next generation of infrastructure that will enable it. ‘e-Science’will change the dynamic of the way scienceisundertaken. Director General of Research Councils Office of Science and Technology John Taylor

  4. GRID vs. e-Science <KIPS Review, May, 2003>

  5. From Networking to Grid Computing • Exponential Growth of Network Technology • Network vs. Computer Performance • Computer speed doubles every 18 months • Network speed doubles every 9 months • Difference = order of magnitude per 5 years • 1986 to 2000 • Computers: x 500 • Networks: x 340,000 • 2001 to 2010 • Computers: x 60 • Networks: x 4000

  6. The Driver for e-Science • More and more data • Instrument resolution doubling / 12 months • Instrument and telemetry speeds increasing • Mobile sensors & radio digital networks • Storage capacity doubling / 12 months • More and more computation • Computations available doubling / 18 months • Faster networks can change methods • Raw bandwidth doubling / 9 months • These integrate and enable • More interplay between computation and data • More collaboration: scientists, medics, engineers, etc. • More international collaboration

  7. The New Behavior • Shared Infrastructure • Intrinsically distributed • Intrinsically multi-organizational • Multiple uses interwoven • Shared Software • A new attempt at making distributed computing economic, dependable and accessible • Scientists from all disciplines share in its design and use • Shared & Automated System Administration • Replicated farms of replicated systems • Autonomic management • Immediate Benefits • Faster transfer of ideas and techniques between disciplines • Amortization of development, operation and education

  8. Examples on e-Science Earth Observation Systems severe weather predictions, climate variations, flood monitoring, earthquakes, and tsunami (a tidal wave) Virtual Observatories Robotic Telescopes Bioinformatics / Functional genomics Collaborative Engineering Medical / Healthcare informatics TeleMicroscopy, and so on

  9. Example 1 – Earthquake Simulation NEESgrid National infrastructure to couple earthquake engineers with experimental facilities, databases, computers, & each other. Argonne, Michigan, NCSA, UIUC, USC

  10. Example 2 – Airspace Simulation NASA Information Power Grid (IPG) Aircraft, flight paths, airport operations and the environment are combined to get a virtual national airspace

  11. e-Science (USA) • Cyber infrastructure program like “e-Science community” for federal offices, supercomputing centers, and research institutes • Budget in 2003 : U$ 1.1 billion • e-Science Cases • Telescience Portal : X-ray related applications including Microbioanalysis • NASA IPG (Information Power Grid) : Aircraft simulation and analysis to reduce the design processing time • BIRN(Biomedical Informatics Research Network) : Study on human and animal brains for the new era in medical science

  12. BIRN(Biomedical Informatics Research Network) • Processing Pipelines for Morphometric Analysis • Medical Applications for HPC • non-linear registrations • biomechanical simulations • statistical analysis of large populations

  13. AccessGridalways-on video walls e-Science Centre (UK)

  14. e-Science Pilot Project (UK) (1/2) • Many to one project • Particle Physics and Astronomy Research Council (PPARC) • GridPP: A prototype Grid infrastructure for the CERN Large Hadron collider • AstroGrid: A Grid based Virtual Observatory • Biotechnology and Biological Sciences Research Council (BBSRC) • Medical Research Council (MRC) • Natural Environment Research Council (NERC) • Grid for Environmental Systems Diagnostics and Visualization • Climateprediction.com: Distributed computing for global climate research • Environment from the Molecular Level: Modeling the atomistic processes involved in environmental issues

  15. e-Science Pilot Project (UK) (2/2) • Economic Social Research Council (ESRC) • Engineering and Physical Sciences Research Council (EPSRC) • The Reality Grid: a tool for investigating condensed matter and materials • Comb-e-chem: Structure-Property Mapping: Combinatorial Chemistry and the Grid • DAME: Distributed Aircraft Maintenance Environment • GEODISE: Grid Enabled Optimization and Design Search for Engineering • Discovery Net: An e-Science Testbed for High Throughput Informatics • MyGrid: Directly Supporting the e-Scientist • Council for the Central Laboratory of the Research Councils (CLRC)

  16. e-Science (JP) • IT-based laboratory (ITBL), Grid based fundamental Informatics (A05), 100 Teraflop high performance computing (NAREGI) • All led by Ministry of Education, Culture, Sports, Science, and Technology (문부과학성) • e-Science Cases • ITBL : Project for virtual research environments • A05 : Grid computing project • NAREGI : Integrating distributed computing resources by high performance networks for 100 Teraflop HPC

  17. ITBL (IT-Based Laboratory) • 6 Organizations at ITBL • Japan Atomic Energy Research Institute (JAERI) 일본원자력 연구소 • RIKEN (The Institute of Physical and Chemical Research) 이화학연구소 • National Institute for Materials Science (NIMS) 재료 물질 연구 기구 • National Aerospace Laboratory of Japan (NAL) 항공우주기술연구소 • National Research Institute for National Research Institute for Earth Science and Disaster Prevention (NIED) 방재과학기술연구소 • Japan Science and Technology Corporation (JST) 과학진흥 사업단 • Massive collaborative research environment for remote researchers by SuperSINET based on IT infrastructure

  18. e-Science (CN) • Grid Projects in China (2002-2005) • The Ministry of Science & Technology 863 Grid Project • Grid Enabling Cluster (>4 Tflop/s) • Grid Nodes (Total 6-10 Tflop/s) • Grid Software (Grid OS, Developer and User Environment) • Grid Applications in Science, Manufacturing, Service industry, and Environment/Resource sector • The “Next Internet” Project (led by Chinese NSF) • Upgrade network infrastructure • Basic research in computing, data and access grids • The Chinese Academy of Sciences e-Science Grid • The Beijing City Manufacturing Grid

  19. Datacentric Applications

  20. Three different kinds of grids • Computational grids • These represent the natural extension of large parallel and distributed systems, and exist to provide high-performance computing • Access grids • This requires managing access to many specific, small resources that are actually located inside large, complex, organizational computer systems and networks • Data grids • These exist in order to allow large datasets to be stored in repositories and moved about with the same ease that small public files can be moved today ☼ Datacentric grids

  21. Facts about online data • They are big and growing fast • Data stored online quadruples every 18 months. • Process power ‘only’ doubles every 18 months. • They are naturally distributed • Data is captured via multiple channels • Operating systems struggle to handle files larger than a few GB • They are hard to move • Pragmatics: Few sites have enough swap space to handle the arrival of a terabyte dataset for temporary use • Performance • Politics: Data about individuals cannot be moved out of jurisdictions with strong privacy rules

  22. Implications of datasets that are large, distributed, and immovable • It’s much more effective to divide programs into separated pieces and send them to data • This requires a datacentric view of computation, rather than the conventional processor-centric view. • A new programming model is needed • Applications must be decomposable • The results of (partial) computations must be small enough to move around • These condensed forms are worth keeping • Execution nodes must be able to provide both computing cycles and high-performance data access.

  23. Some properties • Users can be productiveeven from a thin client • Applications require only thin pipes within the internet • Code mobility is essential • The format and content of a data repository will often be unknown to an application until it actually starts accessing it • Applications will tend to be standardized • Applications will often be built from templates, perhaps even expressed using a query language • Re-execution of an application on a different or updated dataset will be common • There will be increased sensitivity about information leakage

  24. A typical datacentric application

  25. Conclusion • e-Science and datacentric grid are strongly coupled • Meteorology data require dataqcentric grid computing in the future • Typical e-Science characteristics • Huge data size • Poor data site accessibility • Experts are spread over the country/world • Basically all are based on reliable networks • Exact computing on network probabilistic connectivity (one aspect of reliability measures) is theoretically hard • Fast approaches and good enough approximation algorithm are developed (will be published)

More Related