Advance Reservations of Grid Resources for GEMSS Applications

1. Advance Reservations of Grid Resources for GEMSS Applications Junwei Cao Falk Zimmermann Guy Lonsdale C&C Research Laboratories NEC Europe Ltd., Sankt Augustin, Germany http://www.ccrl-nece.de/ PSNC POZNAŃ 2003

2. Outline • GEMSS Overview • GEMSS Applications • GEMSS Architecture • COSY Resource Management • COSY Queue Scheduling • COSY Advance Reservations • Future Work PSNC POZNAŃ 2003

3. GEMSS Project • GEMSS: Grid-Enabled Medical Simulation Services • EC/IST FP5 Grid Project (1.9.2002-28.2.2005) • Grid middleware initiative within medical application setting. http://www.gemss.de/. • GEMSS Consortium: 10 partners from industry & academia including University clinics: NEC Europe Ltd, MPI Leipzig, ISS Vienna, CFX Ltd, CRID FUNDP, IT Innovation, USFD, IDAC Ireland Ltd., ASD, IBMTP Vienna. PSNC POZNAŃ 2003

4. GEMSS Objectives • Demonstrate that the grid can improve pre-operative planning & near- real-time surgical support by providing access to advanced simulation and image-processing services. • Build middleware on existing or developing grid technology standards to provide support for authorization, workflow, security and Quality of Service aspects. • Develop, evaluate and validate a test-bed for the GEMSS system, including its deployment in the end-user’s working environment. • Anticipate privacy, security and other legal concerns by examining and incorporating into its grid services the latest laws and EU regulations related to providing medical services over the Internet. PSNC POZNAŃ 2003

5. GEMSS Applications PSNC POZNAŃ 2003

6. Pre- and post surgery Courtesy Dr. Dr. Th. Hierl, University Clinic Leipzig. Maxillo-facial Surgery Simulation • Provide a virtual try-out space for the pre-operative planning of maxillo-facial surgery. • Based on image processing, meshing & HPC Finite Element simulation. • Grid scenario: client-server, doctor uses a client machine to access remote HPC services. Modeling the distraction procedure PSNC POZNAŃ 2003

7. An Example Implementation PSNC POZNAŃ 2003

8. GEMSS Architecture • Service-oriented architecture • Client-server topology • Based on Web Services technologies • Client-side pluggable component framework • Protocol-independent client-side APIs for service invocation • Support for complex process models: • Business process workflows • QoS negotiation workflows • Application workflows PSNC POZNAŃ 2003

9. Client-side Architecture PSNC POZNAŃ 2003

10. Server-side Architecture PSNC POZNAŃ 2003

11. Resource Management Requirements From application services: • Start a batch job • Kill a job • Job status query From QoS negotiation services: • Book a reservation • Confirm a reservation • Release a reservation • Reservation status query PSNC POZNAŃ 2003

12. Users / Applications Generic User Interfaces (submit, release, queue, confirm,…) Specific User Interfaces (mpirun …) Job Scheduling Log Management Configurations Resource Monitoring Database Management Job Execution Operating Systems, MPI Environments (NEC, SCore …), … Plugin Scripts Resources COSY Structure PSNC POZNAŃ 2003

13. COSY Job Scheduling • Queue Scheduling: • First-come-first-served algorithm • Aggressive backfilling • Multiple queue support • Queue and user priority • Resource limitation • Access control list • Batch and interactive support • Advance Reservations: • Exact start time • Latest end time • Two phase commitment PSNC POZNAŃ 2003

14. When Scheduling with ARs … • Mean queue wait time increases and resource utilization decreases. • Schedulable only with limit percentage of ARs. • How to prevent ARs from taking start time advantages over queued jobs? • How to satisfy ARs as far as possible without sacrificing queue efficiency? PSNC POZNAŃ 2003

15. COSY Policies for ARs … • A mandatory shortest notice time for ARs is defined • Using current mean wait time of queued jobs • Using times of mean wait time of queued jobs PSNC POZNAŃ 2003

16. Experimental Results • Queue wait time • Resource utilization PSNC POZNAŃ 2003

17. Summary • COSY takes the current mean wait time of queued jobs as the mandatory AR shortest notice time so that ARs cannot take start time advantages. • The mandatory AR shortest notice time is applied in COSY as 1 time of current mean wait time of queued jobs and increases to 4 linearly as the AR percentage increases from 0% (not inclusive) to 15% so that ARs can be satisfied as far as possible without increasing the queue wait time. • When the AR percentage is more than 15%, COSY will stop accepting ARs temporarily in order to guarantee a proper queue scheduling. • Advantage: adaptive to queue workload. • Disadvantage: additional calculation of statistics. PSNC POZNAŃ 2003

18. Future Work • Implementation of GEMSS grid architecture (9.2003 – 11.2003 – 2.2004 – 8.2004) • Integration of GEMSS applications with GEMSS grid infrastructure (11.2003 – 8.2004) • Extension of COSY with standard languages and protocols (e.g. GRAAP) • Cost estimation of ARs over queue jobs • Co-reservation and co-allocation PSNC POZNAŃ 2003