1 / 36

Parallel Programming in Undergraduate Education: A View from the Ground

Parallel Programming in Undergraduate Education: A View from the Ground. Dan Grossman University of Washington Workshop on Directions in Multicore Programming Education March 8, 2009. A few hats. Researcher in programming languages Semantics for transactional memory, etc.

whitney
Télécharger la présentation

Parallel Programming in Undergraduate Education: A View from the Ground

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Parallel Programming in Undergraduate Education: A View from the Ground Dan Grossman University of Washington Workshop on Directions in Multicore Programming Education March 8, 2009

  2. A few hats • Researcher in programming languages • Semantics for transactional memory, etc. • Teacher at University of Washington • Freshmen through advanced graduate courses • Added 10% concurrency/parallelism to some courses • Current co-chair of department core-curriculum revision • Updating a 20-year-old course structure • Largely juggling 40 brilliant people’s perspectives Today’s opinions, all my own, informed by all • But focus on experience from (3) Dan Grossman: Undergraduate Parallel Programming

  3. Trade-offs Do I believe every computer-science undergraduate needs to be prepared for the multicore era? Absolutely. But rather than “preaching to the choir”… What about: Web programming MapReduce Software security Software process UI design Machine learning Embedded systems Technical writing Business Co-op experience(s) Foreign language Shakespeare, Freud, Darwin, … And of course everything we already require? Dan Grossman: Undergraduate Parallel Programming

  4. First things first So before what/how/where to teach multicore programming… What does an undergraduate curriculum actually look like? • The outdated one we’re changing • Where we’re probably heading • Similar to many of our peers • Where concurrency/parallelism currently appears Dan Grossman: Undergraduate Parallel Programming

  5. A few details Each University has particular constraints University of Washington: • Top-ten department at a large public university • Very selective admission to the major • Quarter system: 10-week courses • Community-college transfers 2+2 years (in theory) • Arcane credit-hour rules and such None of this matters… except it does Walk through an undergraduate career… Dan Grossman: Undergraduate Parallel Programming

  6. Curriculum overview Procedural programming in Java • 1800+ students a year • 88% won’t become CS majors • What should they see • Massive logistical infrastructure CS1 Dan Grossman: Undergraduate Parallel Programming

  7. Curriculum overview CS1 CS2 ADTs, lists, trees, recursion, inheritance • 900+ students a year • 75% won’t become CS majors Dan Grossman: Undergraduate Parallel Programming

  8. Curriculum overview CS1 “300-level” 8 required courses • back to this in a minute • what we are trimming / modernizing CS2 Dan Grossman: Undergraduate Parallel Programming

  9. Curriculum overview CS1 “300-level” “400-level” CS2  20 choices • students take  5 •  1 per “area” • almost no sequences Dan Grossman: Undergraduate Parallel Programming

  10. Curriculum overview CS1 “300-level” “400-level” CS2 synthesis experience • several options • widely loved capstone design Dan Grossman: Undergraduate Parallel Programming

  11. 300-level today (8 required courses) Dan Grossman: Undergraduate Parallel Programming

  12. 300-level today (8 required courses) discrete structures logic, proofs, sets, combinatorics, probability, … Dan Grossman: Undergraduate Parallel Programming

  13. 300-level today (8 required courses) discrete structures formal models finite automata, regexps, context-free languages, Turing machines Dan Grossman: Undergraduate Parallel Programming

  14. 300-level today (8 required courses) discrete structures formal models data structures big-O, balanced trees, heaps, hashing, sorting, graph algorithms, NP-completeness Dan Grossman: Undergraduate Parallel Programming

  15. 300-level today (8 required courses) programming languages functional programming, static vs. dynamic typing, modularity, ML/Haskell, Scheme, Ruby discrete structures formal models data structures Dan Grossman: Undergraduate Parallel Programming

  16. 300-level today (8 required courses) programming languages digital design boolean algebra, gates, binary numbers, finite automata, ALUs discrete structures formal models data structures Dan Grossman: Undergraduate Parallel Programming

  17. 300-level today (8 required courses) programming languages digital design “303” discrete structures C, tools, “ethics”, everything else formal models data structures Dan Grossman: Undergraduate Parallel Programming

  18. 300-level today (8 required courses) programming languages digital design “303” discrete structures formal models architecture data structures assembly programming, CPU design, caching, pipelining Dan Grossman: Undergraduate Parallel Programming

  19. 300-level today (8 required courses) programming languages digital design “303” discrete structures formal models architecture statistics data structures probability distributions, regression, … Dan Grossman: Undergraduate Parallel Programming

  20. programming languages digital design “303” discrete structures formal models architecture statistics data structures What’s the problem Too many requirements • Field now too large for everyone to take everything Outdated • In overall focus and details Have innovated within courses, especially parallelism (more later) • Me: 1 week C threads and shared-memory in “303” • Luis Ceze: 1 week in architecture • Note: Digital design inherently parallel too Dan Grossman: Undergraduate Parallel Programming

  21. Where we’re going Revision in progress and highly tentative, but broad strokes are: • Fewer requirements • Cannot require the entire “embarrassed list” • Opportunity for earlier specialization • (cf. Berkeley, Cornell, Stanford, Texas, many others) Only touching 300-level for now • Free up room to modernize 400-level later • Little parallelism at 400-level today • Synchronization mechanisms in O/S • New elective course on MapReduce computing Dan Grossman: Undergraduate Parallel Programming

  22. Required core probability for CS math foundations data structures + ?? modern software design hw/sw interface Dan Grossman: Undergraduate Parallel Programming

  23. Optional core (50-90%) data mgmt C/C++ programming net-centric computing hardware design ?? ?? Dan Grossman: Undergraduate Parallel Programming

  24. Where are we • Done: A top-down view of a curriculum • Now: How might we prepare students for multicore • Reorient entire curriculum • unlikely, risky • Add concurrency and parallelism courses • yes, but “we” don’t mean the same thing by that • Integrate concurrency and parallelism where relevant • my preference • some personal experience Dan Grossman: Undergraduate Parallel Programming

  25. Being radical I’m not opposed to parallel-from-day-one • After all, I’m a functional programmer at heart • And we can avoid so much unnecessary sequential thinking But I think it’s unrealistic • What can most instructors teach freshmen in 10 weeks? Even though you can do it: “Educational experiments are doomed to succeed” Albert Shanker, former Pres. Amer. Federation of Teachers (Reminds me of “my” programming-languages course.) Dan Grossman: Undergraduate Parallel Programming

  26. Senior-level courses • Yesteryear: Threads + synchronization mechanisms in O/S • Historical accident: only course that needed it • But: “a course on multicore stuff” means too many things • Architecture / very-low-level software • Concurrency* • primary challenge: responsiveness to external events • Example: O/S • Parallelism* • primary challenge: use resources to increase throughput • Example: scientific computing * other words we can agree on welcome Dan Grossman: Undergraduate Parallel Programming

  27. Some great, very different things … Dan Grossman: Undergraduate Parallel Programming

  28. To be clear • These are all great and important topics • Not just saying that because you are here • (But probably a dozen more books from people not here) • But they’re also really different • Topics, paradigms, emphases, levels of abstraction • Overlap < 30% (?) (probably seems even less to students) • Do we mean students should have: • One of these? • All of these? • “My way” • … Dan Grossman: Undergraduate Parallel Programming

  29. My preference Add 1-2 weeks of parallelism to an existing core-topic course • Data structures / algorithms • Introductory architecture* • Programming languages • Compilers and run-time systems • C programming* (or Java, doesn’t matter) Briefly consider each of these, with more details on (5) • Something I’ve done 3 times with 2nd-year students *Actually done at Washington Dan Grossman: Undergraduate Parallel Programming

  30. Data structures / algorithms • Amdahl’s Law • Span • The notion of dependencies • Thread-local versioning (caching)? Basically what Guy B. suggested Dan Grossman: Undergraduate Parallel Programming

  31. Architecture Topics mentioned “at least in lecture” • Amdahl’s Law • Synchronization primitives (CAS, LLSC, etc.) • Mutual exclusion • Need for cache coherence • SIMD Dan Grossman: Undergraduate Parallel Programming

  32. Programming languages • Threading and synchronization primitives • An interpreter supporting concurrent threads • Data races vs. application-level races • Data-race freedom Similar to coverage in Michael’s book chapter, but less is okay Note: In graduate courses, I teach Concurrent ML Dan Grossman: Undergraduate Parallel Programming

  33. Compilers / run-time systems • “Threads Cannot Be Implemented as a Library” [Boehm 05] • Why nearly every compiler optimization violates sequential consistency • Maybe why concurrent and/or parallel garbage collection is hard Separate/related note: Would like more focus on “how Java is implemented” than on “how is C converted to assembly” Dan Grossman: Undergraduate Parallel Programming

  34. C programming For students with 20 weeks of Java + 3 of C, I do this in 2 hours: • Race condition on a file via bash scripts • Thread basics (multiple call stacks, creation in C and Java) • Why use threads? Any one of: • Performance, Failure isolation, Responsiveness, Natural for the algorithm • Shared memory; the need to synchronize • Data races vs. higher-level races • Fork/join parallelism (in C and Java) • Bank-account example • First with atomic blocks (via Eclipse debugger plug-in) • Then with locks (discussing deadlock and false sharing) • One slide warning them against sequential consistency • So I’m not lying Dan Grossman: Undergraduate Parallel Programming

  35. Why this methodology Some “director’s commentary” on this material • Using atomic blocks for pedagogy nicely separates: • Determining application’s critical sections (hard!) • Finding a good locking protocol (also hard!) • I heavily emphasize basic methodology: • Immutable data isn’t racy (remember, I’m a functional guy) • Thread-local data isn’t racy • Rarely acquire a lock while holding one • Obviously there is more to do, such as: • Communication beyond shared memory • Actually achieving speed-up or responsiveness About exposure to the issues, not (yet) about broad competence Dan Grossman: Undergraduate Parallel Programming

  36. Conclusions • Computing undergoing a multicore revolution • But it’s okay for education to undergo a multicore evolution • Research should be radical and contrarian • Teaching experiments are doomed to succeed • Replace 10% of your course with parallelism and/or concurrency • Go ahead and teach parallelism to freshmen if you want to • Also add focused senior-level courses • Teach good habits even when assumption is sequential • High-level operations without unnecessary dependencies • Mostly functional • Don’t despair: always better to have too much to teach! Dan Grossman: Undergraduate Parallel Programming

More Related