Caching and Buffering in HDF5

1. Caching and Buffering in HDF5 Tutorial Part V LCI Tutorial

2. Software stack and the “magic box” • Life cycle: What happens to data when it is transferred from application buffer to HDF5 file? Application Data buffer Object API H5Dwrite Library internals Magic box Virtual file I/O Unbuffered I/O File or other “storage” Data in a file LCI Tutorial

3. Inside the magic box • Understanding of what is happening to data inside the magic box will help to write efficient applications • HDF5 library has mechanisms to control behavior inside the magic box • Goals of this talk: • Describe some basic operations and data structures and explain how they affect performance and storage sizes • Give some “recipes” for how to improve performance LCI Tutorial

4. Topics • Dataset metadata and array data storage layouts • Types of dataset storage layouts • Factors affecting I/O performance • I/O with compact datasets • I/O with contiguous datasets • I/O with chunked datasets • Variable length data and I/O LCI Tutorial

5. HDF5 dataset metadata and array data storage layouts LCI Tutorial

6. HDF5 Dataset • Data array • Ordered collection of identically typed data items distinguished by their indices • Metadata • Dataspace: Rank, dimensions of dataset array • Datatype: Information on how to interpret data • Storage Properties: How array is organized on disk • Attributes: User-defined metadata (optional) LCI Tutorial

7. Metadata Data Dataspace Rank Dimensions 3 Dim_1 = 4 Dim_2 = 5 Dim_3 = 7 Datatype IEEE 32-bit float Attributes Storage info Time = 32.4 Chunked Pressure = 987 Compressed Temp = 56 HDF5 Dataset LCI Tutorial

8. Metadata cache and array data • Dataset array data typically kept in application memory • Dataset header in separate space – metadata cache Application memory Dataset array data Metadata cache Dataset header …………. Datatype Dataspace …………. Attributes … Dataset array data HDF5 metadata File LCI Tutorial

9. Metadata and metadata cache • HDF5 metadata • Information about HDF5 objects used by the library • Examples: object headers, B-tree nodes for group, B-Tree nodes for chunks, heaps, super-block, etc. • Usually small compared to raw data sizes (KB vs. MB-GB) • Metadata cache • Space allocated to handle pieces of the HDF5 metadata • Allocated by the HDF5 library in application’s memory space • Cache behavior affects overall performance • Metadata cache implementation prior to HDF5 1.6.5 could cause performance degradation for some applications LCI Tutorial

10. Types of data storage layouts LCI Tutorial

11. HDF5 datasets storage layouts • Contiguous • Chunked • Compact LCI Tutorial

12. Contiguous storage layout • Metadata header separate from raw data • Raw data stored in one contiguous block on disk Application memory Dataset array data Metadata cache Dataset header …………. Datatype Dataspace …………. Attributes … File LCI Tutorial

13. Chunked storage • Chunking – storage layout where a dataset is partitioned in fixed-size multi-dimensional tiles or chunks • Used for extendible datasets and datasets with filters applied (checksum, compression) • HDF5 library treats each chunk as atomic object • Greatly affects performance and file sizes LCI Tutorial

14. Chunked storage layout • Raw data divided into equal sized blocks (chunks). • Each chunk stored separately as a contiguous block on disk Application memory Dataset array data Metadata cache Dataset header B C D A …………. Datatype Dataspace …………. Chunkindex Attributes … header Chunkindex File A C D B LCI Tutorial

15. Compact storage layout • Data array and metadata stored together in the header Application memory Metadata cache Dataset header Array data …………. Datatype Dataspace …………. Attributes … Array data Data File* * “File” may in fact be a collection of files, memory, or other storage destination. LCI Tutorial

16. Factors affecting I/O performance LCI Tutorial

17. What goes on inside the magic box? • Operations on data inside the magic box • Copying to/from internal buffers • Datatype conversion • Scattering - gathering • Data transformation (filters, compression) • Data structures used • B-trees (groups, dataset chunks) • Hash tables • Local and Global heaps (variable length data: link names, strings, etc.) • Other concepts • HDF5 metadata, metadata cache • Chunking, chunk cache LCI Tutorial

18. Operations on data inside the magic box • Copying to/from internal buffers • Datatype conversion, such as • float  integer • LE  BE • 64-bit integer to 16-bit integer • Scattering - gathering • Data is scattered/gathered from/to application buffers into internal buffers for datatype conversion and partial I/O • Data transformation (filters, compression) • Checksum on raw data and metadata (in 1.8.0) • Algebraic transform • GZIP and SZIP compressions • User-defined filters LCI Tutorial

19. I/O performance depends on • Storage layouts • Dataset storage properties • Chunking strategy • Metadata cache performance • Datatype conversion performance • Other filters, such as compression • Access patterns LCI Tutorial

20. I/O with different storage layouts LCI Tutorial

21. Writing compact dataset Application memory Metadata cache Dataset header …………. Datatype Dataspace …………. Attributes … Array data Data One write to store header and data array File LCI Tutorial

22. Writing contiguous dataset – no conversion Application memory Metadata cache Dataset array data Dataset header …………. Datatype Dataspace …………. Attributes … File LCI Tutorial

23. Writing a contiguous dataset with datatype conversion Dataset array data Metadata cache Dataset header …………. Datatype Dataspace …………. Attribute 1 Conversion buffer 1MB Attribute 2 ………… Application memory File LCI Tutorial

24. Partial I/O with contiguous datasets LCI Tutorial

25. Writing whole dataset – contiguous rows N M One I/O operation Application data in memory M rows File Data is contiguous in a file LCI Tutorial

26. Sub-setting of contiguous datasetSeries of adjacent rows Application data in memory N M One I/O operation M rows Subset – contiguous in file File Entire dataset – contiguous in file LCI Tutorial

27. Sub-setting of contiguous datasetAdjacent, partial rows Application data in memory N Several small I/O operation M N elements … File Data is scattered in a file in M contiguous blocks LCI Tutorial

28. Sub-setting of contiguous datasetExtreme case: writing a column Application data in memory N Several small I/O operation M 1 element … Subset data is scattered in a file in M different locations LCI Tutorial

29. Sub-setting of contiguous datasetData sieve buffer Application data in memory Data is gathered in a sieve buffer in memory 64K memcopy N M 1 element … File Data is scattered in a file in M contiguous blocks LCI Tutorial

30. Performance tuning for contiguous dataset • Datatype conversion • Avoid for better performance • Use H5Pset_buffer function to customize conversion buffer size • Partial I/O • Write/read in big contiguous blocks • Use H5Pset_sieve_buf_size to improve performance for complex subsetting LCI Tutorial

31. I/O with Chunking LCI Tutorial

32. Reminder – chunked storage layout Application memory Dataset array data Metadata cache Dataset header B C D A …………. Datatype Dataspace …………. Chunkindex Attributes … header Chunkindex File A C D B LCI Tutorial

33. Information about chunking • HDF5 library treats each chunk as atomic object • Compression is applied to each chunk • Datatype conversion, other filters applied per chunk • Chunk size greatly affects performance • Chunk overhead adds to file size • Chunk processing involves many steps • Chunk cache • Caches chunks for better performance • Created for each chunked dataset • Size of chunk cache is set for file(default size 1MB) • Each chunked dataset has its own chunk cache • Chunk may be too big to fit into cache • Memory may grow if application keeps opening datasets LCI Tutorial

34. Metadata cache Dataset_1 header ………… ……… Chunk cache Default size is 1MB Dataset_N header Chunking B-tree nodes ………… Application memory Chunk cache LCI Tutorial

35. Writing chunked dataset Chunked dataset Chunk cache A C C B Filter pipeline B A C File ………….. • Compression performed when chunk evicted from the chunk cache • Other filters applied as data goes through filter pipeline LCI Tutorial

36. Partial I/O with Chunking LCI Tutorial

37. Partial I/O for chunked dataset • Example: write the green subset from the dataset , converting the data • Dataset is stored as six chunks in the file. • The subset spans four chunks, numbered 1-4 in the figure. • Hence four chunks must be written to the file. • But first, the four chunks must be read from the file, to preserve those parts of each chunk that are not to be overwritten. 1 2 3 4 LCI Tutorial

38. Partial I/O for chunked dataset • For each of the four chunks: • Read chunk from file into chunk cache, unless it’s already there. • Determine which part of the chunk will be replaced by the selection. • Replace that part of the chunk in the cache with the corresponding elements from the application’s array. • Move those elements to conversion buffer and perform conversion • Move those elements back from conversion buffer to chunk cache. • Apply filters (compression) when chunk is flushed from chunk cache • For each element 3 memcopy performed LCI Tutorial

39. memcopy Partial I/O for chunked dataset Chunk cache Application buffer 3 3 Chunk Elements participating in I/O are gathered into corresponding chunk Application memory LCI Tutorial

40. Partial I/O for chunked dataset Chunk cache Memcopy Conversion buffer 3 Memcopy Application memory Compress and write to file Chunk File LCI Tutorial

41. Variable length data and I/O LCI Tutorial

42. Examples of variable length data • String A[0] “the first string we want to write” ………………………………… A[N-1] “the N-th string we want to write” • Each element is a record of variable-length A[0] (1,1,0,0,0,5,6,7,8,9) [length = 10] A[1] (0,0,110,2005) [length = 4] ……………………….. A[N] (1,2,3,4,5,6,7,8,9,10,11,12,….,M) [length = M] LCI Tutorial

43. Variable length data in HDF5 • Variable length description in HDF5 application typedef struct { size_t length; void *p; }hvl_t; • Base type can be any HDF5 type H5Tvlen_create(base_type) • ~ 20 bytesoverhead for each element • Data cannot be compressed LCI Tutorial

44. How variable length data is stored in HDF5 Actual variable length data Global heap File Pointer intoglobal heap Dataset header Dataset withvariable length elements LCI Tutorial

45. Application buffer Raw data Global heap Variable length datasets and I/O • When writing variable length data, elements inapplication buffer point to global heaps in the metadata cache where actual data is stored. LCI Tutorial

46. There may be more than one global heap Raw data Application buffer Global heap Global heap LCI Tutorial

47. Variable length datasets and I/O Raw data Global heap Global heap File LCI Tutorial

48. VL chunked dataset in a file Chunk B-tree File Dataset header Heaps with VL data Dataset chunks LCI Tutorial

49. Writing chunked VL datasets Application memory Metadata cache B-tree nodes Chunk cache Dataset header ………… Global heap ……… Raw data Chunk cache Conversion buffer Filter pipeline VL chunked dataset with selected region File LCI Tutorial

50. Hints for variable length data I/O • Avoid closing/opening a file while writing VL datasets • Global heap information is lost • Global heaps may have unused space • Avoid alternately writing different VL datasets • Data from different datasets will go into to the same heap • If maximum length of the record is known, consider using fixed-length records and compression LCI Tutorial