Multi-core Programming

1. Multi-core Programming Introduction Topics

2. Topics • General Ideas • Moore’s Law • Amdahl's Law • Processes and Threads • Concurrency vs. Parallelism

3. General Ideas “Andy giveth, and Bill taketh away.” No mater how fast hardware gets, software quickly expands of overwhelm the new hardware performance. “Unfortunately, many programs are so big that there is no one individual who really knows all the pieces, and so the amount of code sharing you get isn't as great. Also, the opportunity to go back and really rewrite something isn't quite as great, because there's always a new set of features that you're adding on to the same program.” “Technology happens, it's not good, it's not bad. Is steel good or bad?”

4. General Ideas The major processor manufacturers and architectures, from Intel and AMD to Sparc and PowerPC, have run out of room with most of their traditional approaches to boosting CPU performance. Silicon manufacturers are turning to hyperthreading and multicore architectures.

5. General Ideas Ramifications of this change will hit hard in software development. • Typical software solutions can no longer suffice, relying on hardware for speed increases. • Software will need to adapt to the new hardware platforms. • Traditional programming models break down in n-core processor environments.

6. General Ideas • Intel CPU Introductions over time: • Clock Speed increases until ~2003 then levels off. • Number of transistors in a processor form continues to increase. Quad Core I7 2011 1,160,000,000 8 Core Itanium 2012 3,100,000,000 62 Core Xeon 2012 5,000,000,000 GK110 2012 7,100,000,000

7. General Ideas Over the past 30 years, CPU designers have achieved performance gains through: • Clock Speed : Doing the same work faster. • Execution Optimization : Doing more work per clock cycle. • Cache : Stay away from RAM as much as possible. 1 : CPU 0 , 2 : Cache L2 CPU 0, 3 : CPU 1, 4 : Cache L2 CPU 15 : System Request Interface, Crossbar Switch, Memory Controller, Hypertransport

8. General Ideas The previously mentioned “speedup” methods are concurrency agnostic methods – i.e. they will work with any sequential code base. CPU Performance increase hit a wall in 2003 from a traditional approach.

9. General Ideas It is becoming increasingly difficult to exploit faster CPU speeds. Key physical limitations are standing in the way: • Heat • Power Consumption • Current Leakage

10. Moore’s Law From: Electronics Magazine – April 1965 “The complexity for minimum component costs has increased at a rate of roughly a factor of two per year ... Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years. That means by 1975, the number of components per integrated circuit for minimum cost will be 65,000. I believe that such a large circuit can be built on a single wafer.”

11. Moore’s Law Worded differently: The number of transistors on a chip will double about every two years. Basically, Moore’s Law became a self fulfilling prophesy.

12. Moore’s Law Heat, power consumption, and current leakage are limiting the sheer “transistor” size for creation of a single CPU. Consumer demands are for smaller, more portable, yet feature rich digital devices.

13. Moore’s Law Moore’s predictions still hold – transistor counts continue to rise… … however, performance gains are going to be accomplished in fundamentally different ways. Most current applications will no longer benefit from the free “clock cycle” ride without significant redesign.

14. Moore’s Law Near future performance gains will be achieved through: • Hyperthreading: Running two or more threads in parallel inside a single CPU. • Multicore: Running two or more CPUs on a single silicon form factor (CPU chip). • Cache: Bringing instructions and data across the memory access bus in blocks and storing “on-die” for fast access. 1 : CPU 0 , 2 : Cache L2 CPU 0, 3 : CPU 1, 4 : Cache L2 CPU 15 : System Request Interface, Crossbar Switch, Memory Controller, Hypertransport

15. Amdahl’s Law Amdahl's law states that the overall speedup of applying the improvement will be: P - proportion of a computation where the improvement has a speedup of S. Example: • If an improvement can speed up 30% of the computation, P will be 0.3 • If the improvement makes the portion affected twice as fast, S will be 2. • Adding a second processor to a multithreaded application sets S to 2. The speedup of a program using multiple processors in parallel computing is limited by the sequential fraction of the program.

16. Amdahl’s Law To get rich applications running on shrinking form factors, Moore’s Law and Amdahl’s Law become increasingly important in the design of software based systems. Moore’s law dictates continued growth of CPU systems – driven toward multicore because of current physical limitations. Amdahl’s law dictates the extent to which multiple cores will increase software system performance.

17. Stack Code segment Data segment Stack Stack thread thread … Processes and Threads • Modern operating systems load programs as processes • Resource holder • Execution • A process starts executing at its entry point as a thread • Threads can create other threads within the process • Each thread gets its own stack • All threads within a process share code & data segments thread main()

18. Thread 1 Thread 2 Thread 1 Thread 2 Concurrency vs. Parallelism • Concurrency: two or more threads are in progress at the same time: • Parallelism: two or more threads are executing at the same time • Multiple cores needed

19. Trends • Multicore programming is here to stay • Multicore programming is more than multithreaded programming • A poorly written multithreaded program can cause performance decrease in a multicore environment • Software must be carefully designed to execute properly in a multicore environment • Technology “pundits” are claiming multicore programming is the “new OO”. • Multicore programming is being touted as “a more important leap in software development than that of OO”.