1. Performance Pathologies in Hardware Transactional Memory Jayaram Bobba, Kevin E. Moore, Haris Volos, Luke Yen, Mark D. Hill, Michael M. Swift, and David A. Wood Multifacet Project (www.cs.wisc.edu/multifacet) Computer Sciences Dept., University of Wisconsin Madison

2. Executive Summary (1/2) • Multiple Hardware Transactional Memory Designs • Best Performance ? • Depends ! Livelock University of Wisconsin Madison

3. pa●thol●o●gy: any deviation from a normal, healthy or efficient condition Executive Summary (2/2) • Why? • Performance Pathologies • What to do? • Better Conflict Resolution Livelock Performance PathologiesPerformance University of Wisconsin Madison

4. Talk Outline • Motivation • Base HTM Results • Performance Pathologies • Enhanced HTM Systems University of Wisconsin Madison

5. Motivation • Which HTM system performs best? • Different Assumptions • Hardware • Broadcast vs Directory Based • Software: • Continuous Transactions vs Critical Section Transactions • How to compare? University of Wisconsin Madison

6. HTM Design Space Stanford TCC Illinois Bulk No HTMs (yet) LL Herlihy/Moss TM MIT LTM Intel/Brown VTM MIT UTM Wisconsin LogTM EE EL Conflict Resolution: Adhoc “Better By Design” ? University of Wisconsin Madison

7. Experimental Setup • Base HTM Systems • Common Hardware Platform • 32 core CMP • Directory-Based Coherence • Common TM Programming Model • Critical Section Based Transactions • Common Workloads • 5 SPLASH + 2 Microbenchmarks (Btree, LFUCache) LL EL EE University of Wisconsin Madison

8. And the winner is… • Low Contention: Similar • High Contention: Depends Performance Pathologies Livelock University of Wisconsin Madison

9. Talk Outline • Motivation • Base HTM Results • Performance Pathologies • Enhanced HTM Systems University of Wisconsin Madison

10. LL EL EE Performance Pathologies • StarvingElder • SerializedCommit • RestartConvoy • FriendlyFire • DuelingUpgrades • FutileStall • StarvingWriter University of Wisconsin Madison

11. System LL • Conflict Detection: Lazy Detect conflicts at commit time (Validation) • Version Management: Lazy New value elsewhere, Update on commit + Abort - Commit • Conflict Resolution: Committer Wins compare with TCC, Bulk University of Wisconsin Madison

12. T1 T2 T3 load A store A ABORT load A store A ABORT load A store A ABORT COMMITING STALLED ABORTING StarvingElder • CAUSE • Committer Wins • EFFECT(S) • Starvation • Load Imbalance • FIX • Elder gets Priority ? Time University of Wisconsin Madison

13. T3 T2 T1 T4 SerializedCommit • CAUSE • Serial Validation • EFFECT(S) • Unnecessary Stall • FIX • Parallel Validation ? Time University of Wisconsin Madison

14. load A load A load A load A store A ABORT ABORT ABORT load A load A load A load A store A ABORT ABORT ABORT load A load A load A RestartConvoy • CAUSE • Fast Abort + • Immediate Restart • EFFECT(S) • Resource Contention • Serialized Execution • ≈ Lock Convoys • FIX • Stagger Restarts ? T1 T2 T4 T3 University of Wisconsin Madison

15. Pathologies LL • Pathology (% ExecutionTime) • Enhancement • Linear Backoff on Abort • Eliminate RestartConvoys LL LL_B University of Wisconsin Madison

16. Enhanced LL ( ) LL_B • Pathology (% ExecutionTime) • Performance LL_B LL University of Wisconsin Madison

17. Performance Pathologies • StarvingElder • SerializedCommit • RestartConvoy • FriendlyFire • DuelingUpgrades • FutileStall • StarvingWriter LL EL EE University of Wisconsin Madison

18. System EL • Conflict Detection: Eager • Detect conflicts immediately • Version Management: Lazy • New Value elsewhere, Update on commit + Abort - Commit • Conflict Resolution: • Requester Wins • Exponential Backoff on Abort compare with HMTM, LTM University of Wisconsin Madison

19. T1 T2 T3 load A store A ABORT load A ABORT store A load A ABORT store A ABORT load A ABORT store A FriendlyFire • CAUSE • Requester Wins • EFFECT(S) • Unnecessary Aborts • Livelock • FIX • Priority Based Conflict Resolution ? University of Wisconsin Madison

20. Pathologies EL • Pathology (% ExecutionTime) • Enhancement • Timestamp Based Conflict Resolution • Eliminate FriendlyFire EL EL_T University of Wisconsin Madison

21. Enhanced EL ( ) EL_T • Pathology (% ExecutionTime) • Performance EL EL_T University of Wisconsin Madison

22. Performance Pathologies • StarvingElder • SerializedCommit • RestartConvoy • FriendlyFire • DuelingUpgrades • FutileStall • StarvingWriter LL EL EE University of Wisconsin Madison

23. System EE • Conflict Detection: Eager • Detect conflicts immediately • Version Management: Eager • New Value in-place, Restore on Abort - Abort + Commit • Conflict Resolution • Stall Requester • Abort Requester on possible deadlock (Conservative Deadlock Avoidance) compare with LogTM University of Wisconsin Madison

24. T1 T2 load A load A store A STALL store A ABORT load A store A DuelingUpgrades • CAUSE • Conservative Deadlock Avoidance • EFFECT(S) • Unnecessary Stall • Unnecessary Abort • FIX • Acquire Store Permissions Early ? University of Wisconsin Madison

25. FutileStall • CAUSE • Stall Requester • EFFECT(S) • Unnecessary Stall • FIX • No simple fix ? T1 T2 load A store A STALL ABORT University of Wisconsin Madison

26. T1 T2 T3 T4 load A store A load A load A load A load A StarvingWriter • CAUSE • Stall Requester • EFFECT(S) • Starvation • Load Imbalance • ≈ Reader/Writer Locks • FIX • Writer Priority ? University of Wisconsin Madison

27. Pathologies EE • Pathology (% ExecutionTime) • Enhancements • DuelingUpgrades: Store-Set Predictor • StarvingWriter: Writer Aborts Readers immediately EE_HP EE University of Wisconsin Madison

28. Enhanced EE ( ) EE_HP • Pathology (% ExecutionTime) • Performance EE_HP EE University of Wisconsin Madison

29. Caveats / FutureWork LL_B • High Contention Scenarios • Better Conflict Resolution • Unidentified Pathologies • Better TM Workloads LL University of Wisconsin Madison

30. Executive Summary • Performance Pathologies • Enhanced HTM systems Livelock University of Wisconsin Madison

31. FutileStall DuelingUpgrades FriendlyFire RestartConvoy StarvingWriter StarvingElder SerializedCommit