1 / 15

Scalarm: Scalable Platform for Data Farming

Scalarm is an innovative, self-scalable platform designed for data farming, enhancing the computational efficiency of experiments by adapting to the size of the simulation and type of analysis required. By leveraging heterogeneous computational infrastructures such as grids and cloud environments, it supports online analysis of partial results and integrates seamlessly with various systems. This platform simplifies the process of conducting large-scale simulations, enabling researchers to generate and monitor experiments effectively. Join us to explore how Scalarm can optimize your data farming needs.

vala
Télécharger la présentation

Scalarm: Scalable Platform for Data Farming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 1 Scalarm: Scalable Platform for Data Farming D. Król, Ł. Dutka, M. Wrzeszcz, B. Kryza, R. Słota and J. Kitowski ACC Cyfronet AGH KU KDM, Zakopane, 2013

  2. Agenda 2 About Us Problem statement Solution Contact

  3. Whoare we ? 3 Computer Systems Group AGH http://www.icsr.agh.edu.pl/ Knowledge in Grids Team http://www.icsr.agh.edu.pl/index.php/knowledge-in-grids-team Close collaboration with ACC Cyfronet AGH

  4. Whatare we doing ? 4 • Knowledge supported systems for Virtual Organizations • Framework for Intelligent Virtual Organizations • Grid Organizational Memory • X2R • Data monitoring and management • Service Level Agreement Monitoring • QStorMan • Self-scalable systems • Scalarm

  5. A short introduction to data farming 5 Processinitialization

  6. What problem do we address with Scalarm ? 6 Data farming experiments with an exploratory approach Parameter space generation with support of design of experiment methods Accessing heterogeneous computational infrastructure Self-scalability of the management part

  7. Typical work flow 7 2. A workstation or an institution server 3. National Grid environment 1. Local development 4. Cloud

  8. Whereis the problem ? 8 What about fault tolerence? What about collecting results ? 3. Shell + scheduler (gLite, QCG ...) What about parameter space ? 2. Shell Integrated DevelopmentEnvironment 4. Shell + proprietary software

  9. What is our solution ? 9 Specify application to run Execute simulation

  10. Scalarm overview 10 Self-scalable platform adapting to experiment size and simulation type Exploratory approach for conducting experiments Supporting online analysis of experiment partial results Integrates with clusters, Grids, Clouds

  11. Use case – Complete platform for data farming 11

  12. Use case – Simulation-as-a-Service 12 Nativeapp Execute simulation API

  13. But, doesitscale ? 13 Experiment evaluation: • Experiment size [#simulations]: 100 000, 200 000, 500 000, 1 000 000, 2 000 000 • Computational resources [#servers]: 2, 4, 8, 16 • #Clients: 240 * (#servers / 2)

  14. Results 14

  15. Do you want to now more ? 15 Talk to us during the conference Contact us -> dkrol@agh.edu.pl Visit a website -> http://www.scalarm.com

More Related