1 / 7

Parallel computation models

Parallel computation models. Taxonomy: SIMD MIMD Single Program Multiple Data (SPMD) Communication Models Shared variable communication Shared Memory: PRAM Mode Message passing communication Systolic Array: Regularly connected, special hardware

ojal
Télécharger la présentation

Parallel computation models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel computation models • Taxonomy: • SIMD • MIMD • Single Program Multiple Data (SPMD) • Communication Models • Shared variable communication • Shared Memory: PRAM Mode • Message passing communication • Systolic Array: Regularly connected, special hardware • for specific problem. Input is pipelined one by one and synchronized with clock. • Interconnection networks • Bus, crossbar, tree, Multistage networks, • Hypercube, DeBruin's Graph, Cube Connected Cycles • Convergence: • Logically shared memory, physically interconnection network

  2. Timing Issues • Synchronous: • Every step is synchronized (central clock) • execution results predictable • PRAM is a special case • Asynchronous (no central clock) • execution results : unpredictable • example: distributed algorithms • Partially Synchronous • BSP: Bulk Synchronous Programming • LogP model: • Latency, Overhead, Gap, and P

  3. LogP Model(Karp93) • L: upperbound on the latency incurred in sending a message of a word • o: overhead, the length of time that a PE is engaged in the transmission of • each message; during this time, the PE cannot perform other operations. • g: gap, minimum time interval between consecutive message transmissionor • reception. 1/g is per PE communication bandwidth. • P: the number of PE/memory modules. • Length W messgae: time to reach o+gW+L • process the reception: o+gW • Network has a finite capacity, at most L/g messages can be in transit from • one PE to any other PE. • Example of braodcasting 1 to n: • PRAM Concurrent Read Model: d unit of time, where d is delay of memory access • PRAM Exclusive Read Model: O(d*log_2 n) time • LogP Model: O(d*log_d n) time, d=L+o+g

  4. CRCW model variations • Same Value (if multiple PE attempt to write, their value should be the same) • Priority (highest priority) • Random (any can be written)

  5. Example of Shared Memory Algorithm • Adding n numbers • Max of n numbers • O(n) • O(logn) • O(1) algorithm (CRCW) 1. initially r[i] = 0, 1<= i <= n 2. for all i,j PE[i,j] read a[i] and a[j] 3. for all i,j PE[i,j] set r[i] = 1 if a[i] < a[j] 4. for all i, PE[i] do {if r[i] = 0 them max=a[i]} Applications: Boolean matrix multiplication O(1) time using O(n3) PEs

  6. Work optimal maximum N data, P PEs • partition N data so that each PE has N/P data items. • find max of each partition • find the maximum among PEs.

  7. Simulation of CRCW model using EREW Theorem: each step of Priority CRCW can be simulated by EREW PRAM in O(log p) steps. Proof: Each CRCW step can be simulated by a tournament (of EREW) in O(logn) time.

More Related