Fortress

1. John Burgess and Richard Chang CS691W University of Massachusetts Amherst Fortress

2. Overview Sun’s DARPA HPCS project High productivity a goal “Fortress” derived from intent to produce “secure Fortran” Not backwards compatible with Fortran Design decisions motivated by desire to make scientists/programmer’s job as easy as possible Mathematical syntax Implicit/explicit parallelism, handled by libraries Decisions still being made, no compiler currently available

3. Design Goals High Performance Operations for parallel execution and distribution of large data structures Architecture-specific library implementations High Productivity Type safety Type inference used to reconstruct types Extensive Libraries Transactional memory Syntax emulates standard mathematical notation Management of APIs/Components

4. Overview • Fortress Syntax • Objects and Traits • Variables • APIs and Components • Parallelism in Fortress

5. “Advances” in Syntax Syntax resembling mathematical notation Dot product (Ascii version): dotProd[\T extends Field\] (left_vec:T[n],right_vec:T[n],foo_bar: T)= Dot product (Unicode version):

6. Objects and Traits • Traits • Like Java interfaces with code, classes without fields • Objects • Consist of fields and methods • Fortress also has top-level functions

7. Traits • Declare methods • Abstract methods have only headers • Concrete methods have definitions • May extends other Traits • Allow new types to be defined

8. Objects • Consist of fields and methods • Each object has a set of Traits • Multiple inheritance of code through traits • Inherits concrete methods of its Traits • May override inherited methods • Must include definition of all abstract methods of its traits

9. List Trait trait List{\T\] extends Object first:()->T rest:()->List[\T\] cons:T->List[\T\] append:List[\T\])->List[\T\] …(* Definitions of Object trait methods*) end

10. The Object Trait • Root of trait hierarchy • Every object in Fortress has trait Object • Every object has implementations of these methods (directly or inherits them from a trait)

11. EmptyList Object object Empty traits {List} first() = throw Error rest() = throw Error cons(x) = Cons(x, self) append(xs) = xs end • Implements methods of List trait • Defines an empty list object • Cons is another object that implements cons method for lists

12. Variables in Fortress • May be declared to be immutable or mutable • Variable declarations may not have explicit type declarations • Examples: • myList:List[\Integer\] := Empty.cons(13) (* mutable/explicit type declaration*) • myList2 := Empty.cons(10) (* mutable/inferred type*) • maxValue = 99 (* immutable/inferred type *)

13. Components and APIs • Components • Similar to modules in ML, scheme • Export and import APIs • APIs • Define types in traits, objects, and functions implemented by components

14. Hello World component Hello import print from io export executable run(args) = print “Hello world” end api io print:String -> () end api executable run:(args:String…) -> () end

15. Parallelism in Fortress • Spawn expressions • Transactional Memory • Implicit Parallelism • Generators • Distributions • Reductions • Recursive Decomposition • Matrix unpasting syntax

16. Spawn Expressions • Used to explicitly run sections of code in new thread executing in parallel • Syntax: v = spawn do expressions end • Synchronization methods • v.val() • returns value computed by expressions • Will block until v has completed • v.wait() • Blocks until v completes but doesn’t return value • v.ready() • Returns true if v has completed, returns false otherwise

17. Transactional Memory • Synchronization in Fortress done using atomic blocks • Code inside block appears to run atomically • Multiple threads accessing shared data will have access serialized • IO cannot be performed inside atomic block

18. Atomic arraySum • In Fortress for loops are implicitly parallel • atomic modifier serializes updates to sum

19. Implicit Parallelism • Implicit threads created by Fortress runtime for: • Tuple expressions, for loops, summations, etc • Implicit barrier at the end of implicitly parallel construct

20. Generators • Used to express parallel iteration • For loops parallel by default: • Body of iterations run in separate implicit threads • Example generators: • 0:9 • a.indices() • sequential(a.indices())

21. Regions • Fortress programs execute within set of regions • Abstractly describe structure of a machine • Organized as a tree • Every Fortress object resides in a region • Library writers have control over how objects/threads are distributed to regions • Programmers perspective: shared global address space • Can specify how objects/threads are distributed using distributions

22. Tree Abstraction

23. Distributions • Attempt to separate task of data distribution from ensuring program correctness • Architecture/platform specific library will provide default distributions for organizing data produced by generators and allocating and distributing arrays • sequential(g) applies local distribution to generator g

24. Distributions Cont. • Built in distributions: • Default - machine/platform specific • Sequential - Arrays allocated to one piece of (local) memory • Par - Blocked into chucks of size 1 • Blocked - Blocked into roughly equal sized chunks • Plus others… • Library writers can provide other distributions • Programmers can specify which distribution array and generators should use: • a = blocked.array(xSize, ySize)

25. Reductions • Fortress provides reductions based on those of OpenMP • Reductions defined to have identity value • arraySum example: • Variable sum is reduced • Sum assigned to identify of + at beginning of each iteration • At end of iterations, original value of sum is combined with final values of sum from each iteration using the reduction (+) in arbitrary order

26. Recursive Subdivision • Recursively divide problems into smaller problems • On-the-fly load balancing • Computation independent of machine size • Generators provided by library do this internally • Can be used explicitly by programmers

27. Recursive Sum • Recursively subdivide generator until it is small enough that runtime returns sequential generator

28. Matrix Unpasting • Shorthand for breaking up matrix into parts • Useful for recursive subdivision • Assume M is a matrix: • [topM bottomM ] = M • [leftM rightM ] = M • Concise, eliminates corner case errors

29. Matrix Multiplication

30. Conclusion • Fortress philosophy: • Use abstractions to hide details of parallel execution as much as possible • Extensive libraries to handle data and thread distribution • Productivity improvements • Mathematical syntax • Component/APIs to organize large projects • Type Checking