1 / 78

Practical Concerns for Scalable Synchronization

Practical Concerns for Scalable Synchronization. Jonathan Walpole (PSU) Paul McKenney (IBM) Tom Hart (University of Toronto). The problem. “i++” is dangerous if “i” is global. CPU 0. CPU 1. load r1,i. inc r1. store r1,i. i. The problem. “i++” is dangerous if “i” is global. CPU 0

ora-hudson
Télécharger la présentation

Practical Concerns for Scalable Synchronization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Practical Concerns for Scalable Synchronization Jonathan Walpole (PSU) Paul McKenney (IBM) Tom Hart (University of Toronto)

  2. The problem • “i++” is dangerous if “i” is global CPU 0 CPU 1 load r1,i inc r1 store r1,i i

  3. The problem • “i++” is dangerous if “i” is global CPU 0 load r1,i CPU 1 load r1,i load r1,i inc r1 i i store r1,i i

  4. The problem • “i++” is dangerous if “i” is global CPU 0 inc r1 CPU 1 inc r1 load r1,i inc r1 i+1 i+1 store r1,i i

  5. The problem • “i++” is dangerous if “i” is global CPU 0 store r1,i CPU 1 store r1,i load r1,i inc r1 i+1 i+1 store r1,i i+1

  6. Question • What is this problem called?

  7. Question • What is this problem called? • What solution could we apply?

  8. The solution – critical sections • Classic multiprocessor solution: spinlocks • CPU 1 waits for CPU 0 to release the lock • Counts are accurate, but locks have overhead! spin_lock(&mylock); i++; spin_unlock(&mylock);

  9. Question • What are spinlocks built from?

  10. Critical-section efficiency Lock Acquisition (Ta ) Critical Section (Tc ) Lock Release (Tr ) Tc Critical-section efficiency = Tc+Ta+Tr Ignoring lock contention and cache conflicts in the critical section

  11. Critical section efficiency Critical Section Size

  12. Performance of normal instructions

  13. Question • Have synchronization instructions got faster? • Relative to normal instructions? • In absolute terms?

  14. Questions • Have synchronization instructions got faster? • Relative to normal instructions? • In absolute terms? • What are the implications of this for the performance of operating systems?

  15. Questions • Have synchronization instructions got faster? • Relative to normal instructions? • In absolute terms? • What are the implications of this for the performance of operating systems? • Can we fix this problem by adding more CPUs?

  16. What’s going on? • Taller memory hierarchies • Memory speeds have not kept up with CPU speeds • 1984: no caches needed, since instructions were slower than memory accesses • 2005: 3-4 level cache hierarchies, since instructions are orders of magnitude faster than memory accesses

  17. Why does this matter?

  18. Why does this matter? • Synchronization implies sharing data across CPUs • normal instructions tend to hit in top-level cache • synchronization operations tend to miss • Synchronization requires a consistent view of data • between cache and memory • across multiple CPUs • requires CPU-CPU communication • Synchronization instructions see memory latency!

  19. … but that’s not all! • Longer pipelines • 1984: Many clock cycles per instruction • 2005: Many instructions per clock cycle • 20-stage pipelines • Out of order execution • Keeps the pipelines full • Must not reorder the critical section before its lock! • Synchronization instructions stall the pipeline!

  20. Reordering means weak memory consistency • Memory barriers • - Additional synchronization • instructions are needed to • manage reordering

  21. What is the cost of all this? Instruction Cost 1.45 GHz 3.06GHz IBM POWER4 Intel Xeon • Normal Instruction • 1.0 • 1.0

  22. Atomic increment Instruction Cost 1.45 GHz 3.06GHz IBM POWER4 Intel Xeon • Normal Instruction • Atomic Increment • 1.0 • 183.1 • 1.0 • 402.3

  23. Memory barriers Instruction Cost 1.45 GHz 3.06GHz IBM POWER4 Intel Xeon • Normal Instruction • Atomic Increment • SMP Write Memory Barrier • Read Memory Barrier • Write Memory Barrier • 1.0 • 183.1 • 328.6 • 328.9 • 400.9 • 1.0 • 402.3 • 0.0 • 402.3 • 0.0

  24. Lock acquisition/release with LL/SC Instruction Cost 1.45 GHz 3.06GHz IBM POWER4 Intel Xeon • Normal Instruction • Atomic Increment • SMP Write Memory Barrier • Read Memory Barrier • Write Memory Barrier • Local Lock Round Trip • 1.0 • 183.1 • 328.6 • 328.9 • 400.9 • 1057.5 • 1.0 • 402.3 • 0.0 • 402.3 • 0 • 1138.8

  25. Compare & swap unknown values (NBS) Instruction Cost 1.45 GHz 3.06GHz IBM POWER4 Intel Xeon • Normal Instruction • Atomic Increment • SMP Write Memory Barrier • Read Memory Barrier • Write Memory Barrier • Local Lock Round Trip • CAS Cache Transfer & Invalidate • 1.0 • 183.1 • 328.6 • 328.9 • 400.9 • 1057.5 • 247.1 • 1.0 • 402.3 • 0.0 • 402.3 • 0 • 1138.8 • 847.1

  26. Compare & swap known values (spinlocks) Instruction Cost 1.45 GHz 3.06GHz IBM POWER4 Intel Xeon • Normal Instruction • Atomic Increment • SMP Write Memory Barrier • Read Memory Barrier • Write Memory Barrier • Local Lock Round Trip • CAS Cache Transfer & Invalidate • CAS Blind Cache Transfer • 1.0 • 183.1 • 328.6 • 328.9 • 400.9 • 1057.5 • 247.1 • 257.1 • 1.0 • 402.3 • 0.0 • 402.3 • 0 • 1138.8 • 847.1 • 993.9

  27. The net result? • 1984: Lock contention was the main issue • 2005: Critical section efficiency is a key issue • Even if the lock is always free when you try to acquire it, performance can still suck!

  28. How has this affected OS design? • Multiprocessor OS designers search for “scalable” synchronization strategies • reader-writer locking instead of global locking • data locking and partitioning • Per-CPU reader-writer locking • Non-blocking synchronization • The “common case” is read-mostly access to linked lists and hash-tables • asymmetric strategies favouring readers are good

  29. Review - Global locking • A symmetric approach (also called “code locking”) • A critical section of code is guarded by a lock • Only one thread at a time can hold the lock • Examples include • Monitors • Java “synchronized” on global object • Linux spin_lock() on global spinlock_t • What is the problem with global locking?

  30. Review - Global locking • A symmetric approach (also called “code locking”) • A critical section of code is guarded by a lock • Only one thread at a time can hold the lock • Examples include • Monitors • Java “synchronized” on global object • Linux spin_lock() on global spinlock_t • Global locking doesn’t scale due to lock contention!

  31. Review - Reader-writer locking • Many readers can concurrently hold the lock • Writers exclude readers and other writers • The result? • No lock contention in read-mostly scenarios • So it should scale well, right?

  32. Review - Reader-writer locking • Many readers can concurrently hold the lock • Writers exclude readers and other writers • The result? • No lock contention in read-mostly scenarios • So it should scale well, right? • … wrong!

  33. CPU 0 critical section read-acquire memory barrier lock read-acquire memory barrier read-acquire memory barrier critical section CPU 1 Scalability of reader/writer locking Reader/writer locking does not scale due to critical section efficiency!

  34. Review - Data locking • A lock per data item instead of one per collection • Per-hash-bucket locking for hash tables • CPUs acquire locks for different hash chains in parallel • CPUs incur memory-latency and pipeline-flush overheads in parallel • Data locking improves scalability by executing critical section “overhead” in parallel

  35. Review - Per-CPU reader-writer locking • One lock per CPU (called brlock in Linux) • Readers acquire their own CPU’s lock • Writers acquire all CPU’s locks • In read-only workloads CPUs never exchange locks • no memory latency is incurred • Per-CPU R/W locking improves scalability by removing memory latency from read-lock acquisition for read-mostly scenarios

  36. Scalability comparison • Expected scalability on read-mostly workloads • Global locking – poor due to lock contention • R/W locking – poor due to critical section efficiency • Data locking – better? • R/W data locking – better still? • Per-CPU R/W locking – the best we can do?

  37. Actual scalability Scalability of locking strategies using read-only workloads in a hash-table benchmark Measurements taken on a 4-CPU 700 MHz P-III system Similar results are obtained on more recent CPUs

  38. Scalability on 1.45 GHz POWER4 CPUs

  39. Performance at different update fractions on 8 1.45 GHz POWER4 CPUs

  40. What are the lessons so far?

  41. What are the lessons so far? • Avoid lock contention ! • Avoid synchronization instructions ! • … especially in the read-path !

  42. How about non-blocking synchronization? • Basic idea – copy & flip pointer (no locks!) • Read a pointer to a data item • Create a private copy of the item to update in place • Swap the old item for the new one using an atomic compare & swap (CAS) instruction on its pointer • CAS fails if current pointer not equal to initial value • Retry on failure • NBS should enable fast reads … in theory!

  43. Problems with NBS in practice • Reusing memory causes problems • Readers holding references can be hijacked during data structure traversals when memory is reclaimed • Readers see inconsistent data structures when memory is reused • How and when should memory be reclaimed?

  44. Immediate reclamation?

  45. Immediate reclamation? • In practice, readers must either • Use LL/SC to test if pointers have changed, or • Verify that version numbers associated with data structures have not changed (2 memory barriers) • Synchronization instructions slow NBS readers!

  46. Reader-friendly solutions • Never reclaim memory ? • Type-stable memory ? • Needs free pool per data structure type • Readers can still be hijacked to the free pool • Exposes OS to denial of service attacks • Ideally, defer reclaiming memory until its safe! • Defer reclamation of a data item until references to it are no longer held by any thread

  47. How should we defer reclamation? • Wait for a while then delete? • … but how long should you wait? • Maintain reference counts or per-CPU “hazard pointers” on data that is in use?

  48. How should we defer reclamation? • Wait for a while then delete? • … but how long should you wait? • Maintain reference counts or per-CPU “hazard pointers” on data that is in use? • Requires synchronization in read path! • Challenge – deferring destruction without using synchronization instructions in the read path

  49. Quiescent-state-based reclamation • Coding convention: • Don’t allow a quiescent state to occur in a read-side critical section • Reclamation strategy: • Only reclaim data after all CPUs in the system have passed through a quiescent state • Example quiescent states: • Context switch in non-preemptive kernel • Yield in preemptive kernel • Return from system call …

  50. Coding conventions for readers • Delineate read-side critical section • rcu_read_lock() and rcu_read_unlock() primitives • may compile to nothing on most architectures • Don’t hold references outside critical sections • Re-traverse data structure to pick up reference • Don’t yield the CPU during critical sections • Don’t voluntarily yield • Don’t block, don’t leave the kernel …

More Related