Flash in a DBMS: Where and How? Manos Athanassoulis , Anastasia Ailamaki , Shimin Chen, Phillip B. Gibbons, Radu

1. Flash in a DBMS: Where and How? Manos Athanassoulis , Anastasia Ailamaki, Shimin Chen, Phillip B. Gibbons, RaduStoica Intel Labs Pittsburgh Presented by , PadmaHarikaPottamsetty 4279995

2. introduction: Solid state storage technologies like flash have very different characteristics compared to HDDs The different components of DBMS - query processing, query optimization, query evaluation are tuned for decades with the HDD characteristics in mind. Main Challenge : To Integrate such technologies into data intensive systems, such as database management systems (DBMS)

3. One Question that the paper Answers: Where and how will flash be exploited in a DBMS? This paper proposes some techniques for effective use of flash in three contexts: As a log device for transaction processing on memory-resident data Flash-friendly data layout in flash-resident OLTP systems As an update cache for HDD-resident data warehouses

4. Performance limitations of HDD • HDD’s are made of magnetic Rotating platters • Seeks and rotational latencies dominated by mechanical delays • Throughputs : Random accesses much lower than sequential accesses • Interference between multiple concurrent workloads • For decades HDD performance has being falling behind • Increase in CPU performance • Increase in DRAM memory bandwidth • Increase in Larger and deeper cache hierarchies

5. Performance limitations of HDD…….. • Since 1980: • HDDs’ random access latency and bandwidth have improved by only 3.5X • Sequential bandwidth lags far behind their capacity growth • Ratio of sequential to random access throughput has increased 19 fold

6. Solid State Storage Technologies- Flash and PCM • The absence of mechanical parts : • Flash is not limited by any seek or rotational delays • Does not suffer mechanical failure • Consumes less power than HDDs • Flash-based solid state drives (SSDs) fill in the latency and bandwidth gap left by HDDs. • Ratio between sequential and random access throughput – Low • The time to scan the entire device- Low • Disadvantage : Random Writes .

7. 1) Using Flash Devices for Online Transaction Processing (OLTP),(Transactional Logging) • Synchronous Transaction logging • Central Mechanism for ensuring Data Persistency and Recoverability • Log Records are forced to Stable Media before Transaction Commits. • Synchronous logging generates small sequential I/O writes • BUT, Traditional HDD’s cannot be used ,because • Incur a full rotational Delay between each write, to append the next log entry • Flash supports Sequential Writes ,So the Paper proposes • Flash Logging – Exploits Flash for Synchronous Logging.

8. FLASHLOGGING DESIGN A synchronous logging solution that exploits multiple (USB) flash drives FlashLogging architecture: exploiting an array of flash devices and an archival HDD for faster logging and recovery.

9. Performance Evaluation MySQL with InnoDB as the backend storage engine and serve TPCC workload FlashLogging : 5.7X improvement 98.6% of the ideal performance (ii) The optional archival HDD brings significant benefits. (iii) High-end SSD: Improves throughput by 3X Conclusion : When compared to the high-end SSD, multiple low-end USB flash drives achieve comparable or better performance at much lower price

10. 2) Flash-Friendly Data layout Using Flash as persistent storage medium for a DBMS often leads to excessive random writes, which impact flash performance

11. Append/Pack (A/P) Design. A new Flash aware data layout which stabilizes device performance By transforming all random writes to sequential writes at the expense of additional overhead If the additional overhead is low => random reads perform similarly to sequential ones. How A/P Design works : Treats the entire flash device as a log. 2) The log-structure data layout algorithm buffers the dirty pages and writes them in blocks equal to the erase block size in order to optimize write performance.

12. 3) A second log-structured region of the device ,used for the packing part of the algorithm • Some logical pages are appended more than once • Invalidate the old entry of the page in the in-memory data structure, in order to keep track of the most recent version of the data. • When the log is close to full with appended pages (Space low), begin packing, that is, move valid pages from the beginning of the log to the second log-structure assuming these pages are not updated for a long period (cold pages). • Ensures good sequential performance

13. Experimentation and Evaluation. • Implemented A/P as a standalone dynamic linked library • We experimented with a PCIe Fusion ioDrive card and serving a TPCC-like workload. • Observation: • Maintain stable performance (in a read/write mix) • 400MB/s throughput as a result of combining reads and writes • The performance of TPCC speedups up to 9X with A/P data layout

14. 3) Flash-enabled Online Updates in Data Warehouses • Flash : performance booster for data warehouses stored primarily on HDDs. • Traditional DWs : Allow only Offline Updates • But concurrent online updates increasingly desirable • The conventional approach of performing updates in place, slows TPCH queries • Differential update approach: • The basic idea is • To cache incoming updates in an in-memory buffer • To take the cached updates into account on-the-fly during query processing, so that queries see fresh data • To migrate the cached updates to the main data when the buffer is full

15. Differential update approach: • Advantage : Improved performance • Disadvantage : • Required large in-memory buffers – To avoid frequent, expensive update migrations. • Degrades query operator performance • Exploiting Flash to Cache Updates • Differential Update Approach • Instead of large in-memory Buffers , exploits SSDs (and a small amount of memory) to cache incoming updates. • Algorithm materialized sort-merge (MaSM). • Merging operation is similar to an outer join between the main data residing on HDDs and the updates cached on SSDs. • To facilitate the merging, we sort the cached updates in the same order as the main data • This requires M =√ ∥SSD∥ pages of memory to sort ∥SSD∥ pages of cached updates

16. MaSM migrates updates to the main HDD-resident data store by sequentially writing back the merging outcome of a full table scan. • Also by, attaching the commit timestamp to every update record, MaSM can correctly support concurrent queries, migration, and incoming updates

17. MaSM Evaluation

18. Conclusion • 1) Flash devices have better performance than HDD • 2) Flash can fit into the DBMS without making changes to it. • The use of flash in transactional logging in memory-resident OLTPsystems • Flash-friendly data layout in flash-resident OLTP systems • Flash as an update cache in HDD-resident data warehousing systems. Future Research • Flash-Only Data Warehouses • Exploiting Flash Beyond Traditional RDBMS. • Alternative Memory Hierarchies with Flash. • Emerging Byte-Addressable Non-Volatile Memory Technologies,

19. References S. Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD, 2009. R. Stoica, M. Athanassoulis, R. Johnson, and A. Ailamaki. Evaluating and repairing write performance on flash devices. DaMoN, 2009. M. Athanassoulis, S. Chen, A. Ailamaki, P. B. Gibbons, and R. Stoica. Towards Efficient Concurrent Updates andQueries in Data Warehousing. Technical Report, EPFL-REPORT-152941, 2010.