Introduction to Grid Computing

1. Introduction to Grid Computing Nectarios Koziris Assistant Professor National Technical University of Athens nkoziris@cslab.ece.ntua.gr

2. The Past – Electric Power (circa 1890) • Electricity was invented and distributed in cities • Local power generators • One generator for each area/city • Each city/county had their own generators

3. Problems: • Expensive to generate - use • Created lots of Pollution • Unreliable - Frequent Blackouts

4. and then…The power GRID came along… • Decouples power generation-consumption

5. Similarly: 1990-2000geographically distributed resources • People working geographically separated • Remote Supercomputers • Remote Personal Computers • Client-server technology

6. Problem: There is no easy way to collectively coordinate and control several distributed computing and communication resources

7. Resources? • Computing power • Data • Storage Capacity • Electronic Instruments (scientific: e.g. sensors medical: e.g. microscopes, etc) • e-Services!!! Who can provide with seamless access to distributed resources?

8. Exponential Growth

9. Solution: The GRID

10. A popular example: SETI at HOME • Search for extraterrestrial intelligence • Users Download a Screensaver • Kicks off when the computer is idle • Downloads data • Processes data • Sendsresults back • Coordinate geographically spread computers to solve a common problem

11. Protein Folding @ HomeFight AIDS @ Home • Harvest the computing power of remote computers

12. What is Grid Computing • A Virtual Organization is: • People from different institutions working to solve a common goal • Sharing distributed processing and data resources • Grid infrastructure enables virtual organizations What is GRID Computing: “Grid computing is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations” (Ian Foster)

13. GRID (πλέγμα) «Secure Accessandseamless, ubiquitous provisioning of distributed services, based on homogeneous sharing of various electronic resources «Ασφαλής Πρόσβαση και ομοιόμορφη, πανταχού παρούσα, παροχή κατανεμημένων υπηρεσιών που βασίζεται στην από κοινού διάθεση, ομοιογενώς, ποικίλων ηλεκτρονικών πόρων»

14. Grids vs Distributed Computing • Distributed applications already exist, but they tend to be specialized systems intended for a single purpose or closed user group • Grids go further and take into account: • Different kinds of resources • Not always the same hardware, data and applications • Different kinds of interactions • User groups or applications want to interact with Grids in different ways • Dynamicnature Resources and users added / removed / changed frequently

15. What are the characteristics of a Grid System ? Numerous Resources Connected by Heterogeneous, Multi-Level Networks Ownership by mutually distrustful Organizations and individuals Different Resource Management Policies Different Security Requirements and Policy Required Geographically Separated Potentially Faulty Resources Resources are Heterogeneous

16. Application Application Internet Protocol Architecture “Coordinating multiple resources”: ubiquitous infrastructure services, app-specific distributed services Collective “Sharing single resources”: negotiating access, controlling use Resource “Talking to things”: communication (Internet protocols) & security Connectivity Transport Internet “Controlling things locally”: Access to, & control of, resources Fabric Link Layered Grid Architecture(By Analogy to Internet Architecture)

17. Protocols, Services,and APIs Occur at Each Level Applications Languages/Frameworks Collective Service APIs and SDKs Collective Service Protocols Collective Services Resource APIs and SDKs Resource Service Protocols Resource Services Connectivity APIs Connectivity Protocols Local Access APIs and Protocols Fabric Layer

18. Why use Grids for Science? • Scale of the problems • Science increasingly done through distributed global collaborations enabled by the internet • Grids provide access to: • Very large data collections • Terascale computing resources • High performance visualisation • Connected by high-bandwidth networks • e-Science is more than Grid Technology

19. Today challenges – particle physics

20. Challenges • Must share data between thousands of scientists with multiple interests • Must ensure that all data is accessible anywhere, anytime • Must be scalable and remain reliable for more than a decade (open heart surgery – keep the patient alive regardless) • Must cope with different access policies • Must ensure data security

21. The Grid Vision The Grid: networked data processing centres and “middleware” software as the “glue” of resources. Researchers perform their activities regardless geographical location, interact with colleagues, share and access data Scientific instruments and experiments provide huge amount of data

22. Science Opportunities • Pre-GRID era: • Isolated Research Groups • Perform experiments on own equipment • Little sharing – little collaboration • Underutilization of • Equipment • Human potential

23. Science Opportunities • Post GRID era • Scientists will have access and control of: • Remote supercomputers • Scientific Databases • The digital sky • Genome data • Scientific instruments • Particle accelerator • Sharing – Collaboration • The GRID enables us to do SCIENCE that was not possible before

24. Grid Applications(apart from particle physics) • Medical/Healthcare(imaging, diagnosis and treatment ) • Bioinformatics(study of the human genome and proteome to understand genetic diseases) • Nanotechnology(design of new materials from the molecular scale) • Engineering(design optimization, simulation, failure analysis and remote Instrument access and control) • Natural Resources and the Environment(weather forecasting, earth observation, modeling and prediction of complex systems)

25. Health: the National Digital Mammography Archive • On-line Database of Mammography images: • Scan mammograms into database • Compare mammograms to previous ones of the same patient • Compare it with similar cases of other patients • Data mining over millions of mammograms for research • Several such projects around the world: • The National Digital Mammography Archive (USA) • eDiamond (UK)

26. Connecting People: Access Grid

27. Business Opportunities • A new generation of companies and divisions • Provide GRID facilities • Consume GRID resources

28. Business Opportunities: • PC-based computing platform • Life and Material Sciences • Deliver Best Science in the time allowed • Financial Services • Portfolio optimization • Scenario analysis • Risk analysis • Energy • Seismic data analysis • Search for oil • Industrial Manufacturing • Rendering - Simulation

29. Examples with local interest • Civil protection (forest fires, monitoring earth activity-distributed sensors, weather forecast models, atmospheric pollution etc) • Tourism Sector • Medical Sector • e-government (composite e-services) www.topoftheweb.net: e-Europe, 12 basic services for the citizens… • education, distribution of content and services

30. Grid Projects

31. More information • Globus: open source GRID toolkit • http://www.globus.org/ • Global GRID Forum: Standardization • http://www.gridforum.org/ • The GRID 2: • Blueprint for a new computing infrastructure

32. Greek sites: • www.hellasgrid.gr • www.ebusinessforum.gr (Ομάδα εργασίας Z-6)  Τεχνολογίες Πλέγματος (Grid): Προοπτικές και Εφαρμογές στο Η-Επιχειρείν

33. Summary • The GRID provides sharing and coordination of remote resources • Provides new scientific oportunities • Life sciences, HEP, astronomy • Provides new business oportunities • Finance, life sciences, industrial manufacturing, energy