Parallelism in Programming: Loop Optimization with OpenMP
This lecture focuses on loop parallelism as a crucial technique for optimizing program performance through concurrent execution of iterations. It highlights the necessity of restructuring code to accommodate parallel execution and discusses the evolution of OpenMP, a standard for shared memory programming. OpenMP allows programmers to create parallel loops and sections efficiently while managing shared and private variables across threads. Key concepts covered include spawning threads, work-sharing constructs, and distinctions between OpenMP and Pthreads.
Parallelism in Programming: Loop Optimization with OpenMP
E N D
Presentation Transcript
Loop Parallelism and OpenMPCS433Spring 2001 Laxmikant Kale
Loop parallelism • Observation: • Most programs that need to be parallelized derive their complexity (in terms of the number of operations executed) from iterative behavior, expressed in loops • If we can execute multiple iterations of a loop in parallel… • We can often, indeed, do that • What are the conditions? • Sometimes, the code needs a bit of restructuring • Several research efforts • Also, automating the parallelization of loops via compilers • But that is not our focus in this lecture
OpenMP evolution • Standardization effort: • Included shared memory programming as well • I.e. subsumes Pthreads • Involves many vendors: • Intel, SGI, HP, DEC/Compaq, .. • Language bindings: • Fortran, C, C++
Overview • Basic view: • The program is single threaded, but it spawns parallel subtasks (threads) once in a while • Note the distinction with Pthreads model • There, each thread could be executing completely different code • But typically, it doesn’t • Parallel threads • Worksharing: • parallel loops • sections • Shared variables: • shared, private, firstprivate, lastprivate, ..
Spawning threads • Executes a section of code, by spawning multiple threads to execute it. • Each thread gets a copy of all private variables • Separate from the “main program”’s copy • (so, there are k+1 copies if there are k threads fired) • Threads may uses locks and barriers • calls are different than PThreads double A[1000]; omp_set_num_threads(4); #pragma omp parallel { int ID = omp_thread_num(); pooh(ID,A); } Some Example/s from sc99 Tutorial at OpenMP web site.
Work-sharing: for construct • Do all the iterations of a loop in parallel • We will use Prof. Padua’s slides for this • Not just the syntax and semantics of the parallel loop construct • But how to write code, by transforming sequential loops, so they are parallel • Slides 20-45 (Section 4.1 to 4.4), and 55-59 (4.7) • Also read the OpenMP C manual • http://polaris.cs.uiuc.edu/~padua/cs320 • http://polaris.cs.uiuc.edu/~padua/cs320/guidec.pdf #pragma omp parallel #pragma omp for schedule(static) for(i=0;I<N;i++) { a[i] = a[i] + b[i];}
Work-sharing: Sections • Parallel Loops allow identical sections of code (with different loop indices) to execute in parallel • Sections allow different sections of code to be run by parallel threads #pragma omp parallel #pragma omp sections { X_calculation(); #pragma omp section y_calculation(); #pragma omp section z_calculation(); }
