1 / 16

Lecture 2: Parallel computational models

Lecture 2: Parallel computational models. Control Unit (contains algorithms) . Data. Memory. CPU. (unlimited size). Sequential computational models. Turing machine RAM ( Figure ) Logic circuit model RAM (Random Access Machine) Operations supposed to be executed in one unit time

haines
Télécharger la présentation

Lecture 2: Parallel computational models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 2: Parallel computational models

  2. Control Unit (contains algorithms) Data Memory CPU (unlimited size) Sequential computational models • Turing machine • RAM (Figure ) • Logic circuit model • RAM (Random Access Machine) Operations supposed to be executed in one unit time • (1)Control operations such asif，goto • （for and while can be realized by for and goto. ） • (2)I/O operations such asprint • (3)Substitution operations such asa = b • (4)Arithmetic and logic operations such as +, -, AND.

  3. O-notation for computing complexity • Definition • Assume that f(n) is a positive function. If there are two positive constants c, n0 such that • f(n) ≦ c g(n) for all n ≧ n0, • then we say • f(n) = O( g(n) ). • For example, • 3n2-5n = O(n2) • n log n + n = O(n log n) • 45 = O(1) • （The item which grows most quickly)

  4. Algorithm analysis for sequential and parallel algorithms • Sequential algorithmsParallel algorithms • Models RAM Many types • Data division Not necessary Most important • AnalysisComputing time Computing time • Memory size Communicating time • Number of processors

  5. RAM 1 RAM 2 Shared Memory RAM m PRAM (Parallel RAM) model • PRAM consists ofa number of RAM (Random Access Machine) and a shared memory. Each RAM has a unique processor number. • Processors act synchronously. • Processor execute the same program. • （According to the condition fork based on processor numbers, it is • possible to executed different operations.) • Data communication between processors (RAMs) are held through the shared common memory. • Each processor can write data to and read data from one memory cell in O(1) time.

  6. Features of PRAM • Merits • Parallelism of problems can be considered essentially. • Algorithms an be easily designed. • Algorithms can be changed easily to ones on other parallel computational models. • Demerits • Communicational cost is not considered. (It is not realistic that one synchronized reading and writing can be done in one unit time.) • Distributed memory is not considered. In the following, We use PRAM to discuss parallel algorithms.

  7. Analysis of parallel algorithms on PRAM model • Computing time T(n) • Number of processors P(n) • Cost P(n) × T(n) • Speed-up Ts(n)/T(n) • Ts(n): Computing time of the optimal sequential algorithm） • Cost optimal parallel algorithms • The cost is the same as the computing time of the optimal sequential • algorithm, i.e., speed-up is the same as the number of processors. • 2. Time optimal parallel algorithms • Fastest when using polynomial number of processors. • 3. Optimal parallel algorithms • Cost and time optimal.

  8. Analysis of parallel algorithms on PRAM model NC-class and P-class • World of sequential computation • P problems：the class of problems which can be solved in polynomial time (O(n )). • NP problems：the class of problems which can be solved non-determinately in polynomial time. • NP-complete problems： the class of NP problems which can be reduced to each other. • P = NP ? • World of parallel computation • NC Problems: the class of problems which can be solved in log-polynomial time • (O(lg n) ). • P-complete problems：the class of problems which are not NC problems and can be reduced to each other. • Similarly, NC = P ? k k

  9. An Example of PRAM Algorithms • Problem：Find the sum of n integers (x1, x2, ... , xn) • - Assume that n integers are put in array A[1..n] on the shared memory. - To simplify the problem, let n = 2k (k is an integer). main () { • for (h=1; h≦log n; h++) • { • if (index of processor i≦ n/2h) processor i do • { • a = A[2i-1]; /* Reading from the shared memory*/ • b = A[2i]; /* Reading from the shared memory*/ • c = a + b; • A[i] = c; /* Writing to the shared memory　*/ • } • } • if (the number of processor == 1) printf("%d¥n", c); • }

  10. P P P i 1 2 A[n] An Example of PRAM Algorithms • Processor Pi reads A[2i-1], A[2i] from the shared memory, then writes their summation to A[i] of the shared memory. A[1] A[2] A[3] A[4] A[2i-1] A[i] A[2i]

  11. Output P Step 3 １ Parallel algorithm Step 2 P P １ ２ P P P P Step 1 4 3 1 2 x x x x x x x x Input 1 2 3 4 5 6 7 8 Sequential algorithm An Example of PRAM Algorithms Find the summation of 8 integers (x1, x2, ... , x8).

  12. An Example of PRAM Algorithms • Analysis of the algorithm • Computing time：for loop is repeated log n times, each loop can be executed in O(1) time →O(log n) time • Number of processors：not larger than n/2 →n/2 processors • Cost:O(n log n) It is not cost optimal since the optimal sequential algorithm run in Θ(n) time.

  13. Classification of PRAM by the access restriction • EREW (Exclusive read exclusive write) PRAM • Both concurrent reading and concurrent writing are prohibited. • CREW (Concurrent Read Exclusive write) PRAM • Concurrent reading is allowed, but concurrent writing is prohibited.

  14. Classification of PRAM by the access restriction • CRCW (Concurrent Read Concurrent write) PRAM • Both concurrent reading and concurrent writing are allowed. • It is classified furthermore: • - common CRCW PRAM • Concurrent writing happens is only if the writing data are the same. • - arbitrary CRCW PRAM • An arbitrary data is written. • - priority CRCW PRAM • The processor with the smallest number writes the data.

  15. Algorithms on different PRAM models Algorithms for calculating and of n bits (Input is put in array A[1..n]) • Algorithm on EREW PRAM model main (){ • for (h=1; h≦log n; h++) { • if (index of processor i ≦ n/2h) processor i do { • a = A[2i-1]; • b = A[2i]; • if ((a==1) and (b==1)) a[i] = 1; • }}} • Algorithm on common CRCW PRAM model main (){ • if (A[index of processor i] == 1) processor i do • A[1] = 1; • } O(log n) time n/2 processors O(1) time n processors Abilities of PRAM models: EREW < CREW < CRCW

  16. Exercise • 1. Suppose nxn matrix A and matrix B are saved in two dimension arrays. Design a PRAM algorithm for A+B using n and nxn processors, respectively. Answer the following questions: • What PRAM models that you use in your algorithms? • What are the runingstime? • Are you algorithms cost optimal? • Are your algorithms time optimal? • 2. Design a PRAM algorithm for A+B using k (k <= nxn processors). Answer the same questions.

More Related