Microsoft Visual Studio: Bringing Out The Best In Multicore Systems

1. TL19 Microsoft Visual Studio:Bringing Out The Best In Multicore Systems  HazimShafi Principal Architect Microsoft Corporation

2. Outline • Motivation • Parallel application development • The basics • The role of tools • Exploiting parallelism • Current and future programming models • Parallel performance tuning • New Visual Studio tools • Demos throughout

3. Outline • Visual Studio 2010 includes many parallel programming features • Reduce programming complexity • Managed and native • Debugging support • Concurrency analysis tools • Causes of inefficiency • Opportunities for parallelism • Want more details? • References to related PDC talks and symposia

4. Motivation • Multicore processors are here • Single thread performance is ~flat • Performance burden shifted to software • Enables new user experiences • Major concern for Windows developers • Performance suffers with new features • Need help parallelizing applications • Parallel Computing Platform team • Ease of developing, debugging, and tuning parallel applications • Help is on its way in Visual Studio 2010

5. Visual Studio 2010Tools/programming models/runtimes Tools Programming models PLINQ Parallel Pattern library Agents library Parallel Debugger Toolwindows Task Parallel library Data structures Data structures Concurrency runtime Profiler concurrency analysis Task scheduler Thread pool Task scheduler Resource manager Resource manager Operating system Threads Key: Managed library Native library Tools

6. Parallel Application DevelopmentThe basics • Parallelism is about performance • Step 1: Understand your goals • Step 2: Measure existing performance • Step 3: Performance tuning • Starts with sequential version • Step 4: Identify opportunities for parallelism • Hide latency • Speed up CPU bound phases • Step 5: Express parallelism • Step 6: Tune

7. Goals And Sequential Tuning • Know your users • Use realistic datasets • Know your ecosystem • Parallelize only if necessary, but PLAN for it • Introduces bugs • Increases complexity • Visual Studio features reduce complexity • I/O, memory latency and algorithmic optimizations • Often sufficient to meet goals

8. demo Visual Studio Profiling Tools

9. Visual Studio Profiling ToolsDemo recap • Profiler • Sampling based – for CPU-bound phases • Lowest overhead • Analyze time and/or hardware performance counters • Instrumentation based – not limited to CPU-bound phases • Detailed, higher runtime overhead • Managed memory usage analysis • TL26: Improving .NET Application Performance And Scalability • Steve Carroll and Ed Glas • TL24: 1:15 PM in 153 (Wednesday)

10. Visual Studio 2010Tools/programming models/runtimes Tools Programming models PLINQ Parallel Pattern library Agents library Parallel Debugger Toolwindows Task Parallel library Data structures Data structures Concurrency runtime Profiler concurrency analysis Task scheduler Thread pool Task scheduler Resource manager Resource manager Operating system Threads Key: Managed library Native library Tools

11. Parallel Application DevelopmentMeasuring performance • Compare actual performance against goals • Requires instrumentation • Divide and conquer • Identify scenario phases • Tackle biggest contributors first • Amdahl’s Law • Instrumentation • Integrated support in Visual Studio 2010 performance tools

12. Parallel Application DevelopmentInstrumentation • Managed and native • Instrumentation provides • Time ∆, and possibly some resource usage information • Nesting support • Optional Tracing data emitted for post-run analysis • Integration with Visual Studio 2010 Trace Visualization Tools MeasurementBlockmyMB = new MeasurementBlock(); myMB.Begin(0, “Phase 1”); // Phase 1 work happens here myMB.End();

13. Parallel Application DevelopmentOpportunities for parallelism • Attempted performance tuning, but can’t meet goals • Identify opportunities for parallelism • Speed-up CPU-bound phases • Overlap/hide delays from user experience • Where and how? • Visual Studio 2010 Concurrency Analysis Tools • Managed and native • 32-bit and 64-bit • Windows 7, Windows Server 2008, and Windows Vista

14. Visual Studio 2010 Concurrency Analysis Tools • CPU utilization analysis • How it varies with time across phases • Identify opportunities for improvement • Drill into a range of execution for details • Thread blocking analysis • Analyzes each thread’s execution • When a thread is not executing, why? • Link to source code responsible • Disk I/O dependencies with filenames/delays • Core execution analysis • Thread migration or affinity issues

15. demo Visual Studio Concurrency Analysis Tools

16. Visual Studio 2010 Concurrency Analysis ToolsWhat I haven't shown • Thread synchronization dependencies • Which thread was I waiting for? • What was that thread doing? • Multi-process scenarios • My program spawns other processes • My program requests services from other processes • More statistics and guidance • Point out known issues • Provide references to relevant improvement techniques

17. Exploiting Parallelism • Forms of parallelism • Data, task, pipelining, latency hiding, etc. • Legacy programming patterns • Hand-coded thread creation, scheduling • Thread pools, I/O completion ports, asynchronous APIs etc. • New programming paradigms • Relieve developer from managing threads • Express parallelism using simple abstractions • Software stack handles the rest

18. Visual Studio 2010Tools/programming models/runtimes Tools Programming models PLINQ Parallel Pattern library Agents library Parallel Debugger Toolwindows Task Parallel library Data structures Data structures Concurrency runtime Profiler concurrency analysis Task scheduler Thread pool Task scheduler Resource manager Resource manager Operating system Threads Key: Managed library Native library Tools

19. Parallelism In .NET Framework 4.0Parallel Extensions • PLINQ – Parallel Language Integrated Query • Parallel.For, Parallel.ForEach • Parallel.Invoke { stmt1, stmt2,…} • Want to learn more? • Parallel Programming For Managed Developers With The Next Version Of Microsoft Visual Studio • Daniel Moth • TL26: 10:30 AM in Petree Hall CD (Wednesday)  

20. Visual Studio 2010Tools/programming models/runtimes Tools Programming models PLINQ Parallel Pattern library Agents library Parallel Debugger Toolwindows Task Parallel library Data structures Data structures Concurrency runtime Profiler concurrency analysis Task scheduler Thread pool Task scheduler Resource manager Resource manager Operating system Threads Key: Managed library Native library Tools

21. Parallel C++ ApplicationsConcurrency runtime and PPL • Parallel Pattern Library • parallel_for – demo later • Parallel task invocations • Asynchronous agents • Concurrency runtime void quicksort(int * a, int n) { if (n <= 1) return; int s = partition(a,n); quicksort(a,s);quicksort(a+s,n-s); return;} task_group g;g.run([&]{quicksort(a,s);});g.run([&]{quicksort(a+s,n-s);});g.wait();

22. Parallel C++ ApplicationsWant to learn more? • Parallel Programming For C++ Developers In The Next Version Of Microsoft Visual Studio • Rick Molloy •  TL25: 3:30 PM in 408B (TODAY) • Concurrency Runtime Deep Dive: How To Harvest Multicore Computing Resources • NiklasGustafsson • TL22: 1:15 PM in 408B (Wednesday)

23. Expressing ParallelismRemaining developer tasks • Ensure that parallel tasks can execute concurrently • Without violating order or data dependencies • Example: parallel_for loop iterations have to be independent • Synchronization • Identify races and ordering constraints • Add appropriate synchronization primitives

24. demo Visual Studio Concurrent Programming And Tool Support

25. Call To ActionConcurrency in Visual Studio 2010 • Many new features for parallel programmers • Explore these in the Visual Studio CTP • Give us your feedback! • Programming constructs to reduce program complexity • Managed and native • Concurrency runtime • Task Parallel library • Parallel Pattern library • Concurrency analysis tools • Analyze concurrency in your application • Understand sources of inefficiency • Support for TPL and PPL

26. Questions? Concurrency Runtime: TL22: 1:15 PM in 408B (Wednesday) .NET Performance: TL24: 1:15 PM in 153 (Wednesday) Native Parallel Programming: TL25: 3:30 PM in 408B (Today) Managed And Debugging: TL26: 10:30 AM in Petree Hall (Wednesday) Parallel Computing Symposium: 8:30 AM – 1:30 PM 515A (Thursday) http://blogs.microsoft.com/hshafi

27. Evals & Recordings Please fill out your evaluation for this session at: This session will be available as a recording at: www.microsoftpdc.com

28. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.