Frédéric Gava

1. Parallel I/O In Bulk-Synchronous Parallel ML Frédéric Gava PAPP 2004 Gava

2. Outline • Introduction • The BSP model • The BSML language • External Memory in BSML • Cost model • Problems and solutions • Conclusion and Future Work PAPP 2004 Gava

3. Introduction PAPP 2004 Gava

4. Bulk Synchronous Parallelism + Functional Programming = BSML • Advantages of the BSP model: • Portability • Scalability, deadlock free • Simple cost model  Performance prediction • Advantages of functional programming: • High level features (higher order functions, pattern-matching, concrete types, etc…) • Savety of the environment • Programs Proofs PAPP 2004 Gava

5. The Caraml Project • Funds by the ACI Grid program (French National Grid program) • Tools and applications • Organized in 3 phases: • First phase: safety • Second phase: multiprogramming • Third phase: extensions for Grid computing PAPP 2004 Gava

6. The BSP model 0 1 2 3 p-1 Proc. T(s) = (max0i<p wi) + hg + L

7. The BSML language • Library for the « Objective Caml » language (called BSMLlib) • Operations on a parallel data structure called vector:par • Operations to access to the BSP parameters : • 4 Operations on a parallel vectors PAPP 2004 Gava

8. Global Conditional b0 b1 … true … bp-1 if vec at n then … else … = if at n then e1 else e2 n  e1

9. External Memory PAPP 2004 Gava

10. Model • We have: • M = Size of the main memory • D = Number of disks • B = Size of one block in a disk • G = Time to read/write in parallel B blocks (D*B data)

11. Problem « local side effects  » => modification of the global environment let bug = mkpar (fun pid -> if pid=0 then open_write « toto.dat » else NOTHING) in open_read « toto.dat » PAPP 2004 Gava

12. Solutions • Two file systems • local files => one files system on each process • global files : • a shared files system • or replicate local files on a different directory) • New primitives for the differents files • Confluence of the semantics • Compositional cost model

13. Example Scan_list : scan_list (+) <[0;1], [2;3], [4] > < [0;0+1], [0+1+2;0+1+2+3], [0+1+2+3+4] > Read/write values in blocks using tempory files PAPP 2004 Gava

14. Benchmark

15. Conclusion • BSML = BSP + ML • External Memory in BSML • New cost model • New Primitives • Confluence • Compositional cost model PAPP 2004 Gava

16. Future Work • Polymorphic type system for BSML with I/O • Implementation of « big » applications • Add to BSML : • Parallel composition • Exceptions • Pattern – matching of parallel values

17. PAPP 2004 Gava