Blooming Trees: Space-Efficient Structures for Data Representation

1. Blooming Trees: Space-Efficient Structures for Data Representation Author: Domenico Ficara, Stefano Giordano, Gregorio Procissi, Fabio Vitucci Publisher: ICC 2008 Presenter: Yu-Ping Chiang Date: 2009/05/20

2. Outline • Blooming Tree • Lookup • Insert • Delete • Optimized Blooming Tree • Lookup • Insert • Delete • Simulations

3. item HASH FUNCTION 2 items 3 items 1 item Bit string 0 1 1 0 1 0 0 1 B0 3 bits 1 bit 1 bit 0 1 0 0 1 0 0 1 1 0 1 0 B1 0 1 0 1 0 1 1 1 1 0 0 1 1 1 B2 1 2 1 1 1 1 B3 Blooming Tree index index index

4. Blooming Tree • n items, k0 hash functions, L+2 layers • Layer0 (B0) : • m = nk0/ln2 bits • Layer1~L (B1~BL) : • bits/block ( b=1 in following examples ) • Block numbers is modified • LayerL+1 (BL+1) : • Composed c-bits counters • Hash function • k0 hash functions • log m + L*b bits output • log m bit for layer0 • B bits for layer1~layerL+1

5. 0 1 1 0 1 0 0 1 hash item1 bit string: 01010 1 0 0 1 1 0 1 0 B0 1 1 1 0 0 1 1 1 B1 1 2 1 1 1 1 B2 B3 Blooming Tree - lookup • Algorithm: • Using first log m bits as layer0 index. • Compute a popcount on layer i, that gives index of the couple in layer i+1. • Checking the bit string output by hash function, the bit for layer i. • 0 for first bit. • 1 for second bit. • If processing bit is 0, result NOT FOUND. Otherwise continue search in next layer. • Time complexity: • k0 [ hash + L ( popcount + 2 * check ) ]

6. 0 1 1 0 1 0 0 1 hash bit string: 01010 item1 B0 1 1 1 1 0 0 1 1 1 0 1 B1 1 0 1 0 1 0 1 2 1 1 1 1 B2 B3 Blooming Tree - lookup • Algorithm: • Using first log m bits as layer0 index. • Compute a popcount on layer i, that gives index of the couple in layer i+1. • Checking the bit string output by hash function, the bit for layer i. • 0 for first bit. • 1 for second bit. • If processing bit is 0, result NOT FOUND. Otherwise continue search in next layer. • Time complexity: • k0 [ hash + L ( popcount + 2 * check ) ]

7. 0 1 1 0 1 0 0 1 hash bit string: 01010 item1 1 0 0 1 1 0 1 0 B0 1 B1 0 1 0 1 1 0 1 1 1 1 2 1 1 1 1 B2 Match !! B3 Blooming Tree - lookup • Algorithm: • Using first log m bits as layer0 index. • Compute a popcount on layer i, that gives index of the couple in layer i+1. • Checking the bit string output by hash function, the bit for layer i. • 0 for first bit. • 1 for second bit. • If processing bit is 0, result NOT FOUND. Otherwise continue search in next layer. • Time complexity: • k0 [ hash + L ( popcount + 2 * check ) ]

8. 0 1 1 0 1 0 0 1 hash bit string: 10000 item2 1 0 0 1 1 0 1 0 B0 1 1 1 0 0 1 1 1 B1 NOT FOUND !! 1 2 1 1 1 1 B2 B3 Blooming Tree - lookup • Algorithm: • Using first log m bits as layer0 index. • Compute a popcount on layer i, that gives index of the couple in layer i+1. • Checking the bit string output by hash function, the bit for layer i. • 0 for first bit. • 1 for second bit. • If processing bit is 0, result NOT FOUND. Otherwise continue search in next layer. • Time complexity: • k0 [ hash + L ( popcount + 2 * check ) ]

9. 0 0 1 0 0 0 0 0 0 1 1 0 1 Blooming Tree - insert • Algorithm: • Using first log m bits as layer0 index. • In layer1~layerL+1, using popcount of layer0~layerL and bit for each layer as index. • If bit in layer I already set (means COLLOSION), directly set bit in layer i+1. else, allocate a new block and insert it into original layer i+1 blocks. • Increase count at layer L+1. • Time complexity: • k0 [ hash + L ( popcount + shift + bitset ) ] hash bit string: 01010 item1 B0 allocate a new block (2^b bits) B1 allocate a new block (2^b bits) B2 allocate a new block (2^b bits) B3

10. 0 1 1 0 0 0 0 0 1 0 0 Blooming Tree - insert • Algorithm: • Using first log m bits as layer0 index. • In layer1~layerL+1, using popcount of layer0~layerL and bit for each layer as index. • If bit in layer I already set (means COLLOSION), directly set bit in layer i+1. else, allocate a new block and insert it into original layer i+1 blocks. • Increase count at layer L+1. • Time complexity: • k0 [ hash + L ( popcount + shift + bitset ) ] hash bit string: 00101 item2 B0 allocate a new block (2^b bits) B1 0 1 allocate a new block (2^b bits) 1 1 0 B2 allocate a new block (2^b bits) B3 1 1

11. 0 1 1 0 1 0 0 1 hash bit string: 00101 item3 B0 B1 1 0 0 1 0 1 1 0 B2 Collision occur B3 2 1 Blooming Tree - insert • Algorithm: • Using first log m bits as layer0 index. • In layer1~layerL+1, using popcount of layer0~layerL and bit for each layer as index. • If bit in layer I already set (means COLLOSION), directly set bit in layer i+1. else, allocate a new block and insert it into original layer i+1 blocks. • Increase count at layer L+1. • Time complexity: • k0 [ hash + L ( popcount + shift + bitset ) ]

12. 0 1 1 0 1 0 0 1 1 0 Blooming Tree - delete • Algorithm: • Trace to the last layer, decrease count. • If counter isn’t equal to 0, terminal processing. else, remove the block and checking upper layer if there only this item in the block, if yes, remove that block too. recursive processing upper layers. hash bit string: 00100 item1 B0 B1 0 1 1 0 1 1 0 B2 Remove empty block B3 1 2 1 0

13. 0 1 1 0 1 0 0 1 1 0 Blooming Tree - delete • Algorithm: • Trace to the last layer, decrease count. • If counter isn’t equal to 0, terminal processing. else, remove the block and checking upper layer if there only this item in the block, if yes, remove that block too. recursive processing upper layers. hash bit string: 01001 item2 B0 B1 0 1 0 1 1 0 B2 B3 1 2 1

14. Outline • Blooming Tree • Lookup • Insert • Delete • Optimized Blooming Tree • Lookup • Insert • Delete • Simulations

15. 0 1 1 0 1 0 0 1 1 1 0 1 1 0 0 0 0 1 Hash substrings bitmap 1 0 0 1 01 10 1 1 Optimized Blooming Tree 2 items 3 items 1 item B0 1 0 B1 1 0 1 0 0 0 1 1 1 1 B2 B3 1 2 1 1

16. Optimized Blooming Tree - lookup • Algorithm: • Access B0 • Checking bitmap • If there’s 1 in bitmap, directly compare last L*b bits of hashed bit string, and terminate processing. • Else, lookup method is same as previous defined. • Recursively repeat at each level.

17. 0 1 1 0 1 0 0 1 item1 hash bit string : 10001 0 1 1 0 0 0 Optimized Blooming Tree - lookup • Algorithm: • Access B0 • Checking bitmap • If there’s 1 in bitmap, directly compare last L*b bits of hashed bit string, and terminate processing. • Else, lookup method is same as previous defined. • Recursively repeat at each level. Popcount = 3 B0 Popcount = 2 B1 1 0 1 0 01 0 0 10 Hash substrings bitmap 1 1 1 1 B2 B3 1 2 1 1

18. 1 1 0 1 0 01 01 10 Hash substrings bitmap item1 hash bit string : 01101 0 1 1 0 0 0 Hash substrings bitmap 0 0 01 10 Optimized Blooming Tree - insert • Without collision • Add a zero-block • Set bit string and hash substring B0 1 0 1 1 0 1 0 0 1 B1 1 0 1 0 0 0 1 1 1 1 B2 B3 1 2 1 1

19. 0 1 1 0 1 0 0 1 0 0 1 0 01 1 1 1 0 item2 hash bit string : 01001 0 1 1 0 0 0 Hash substrings bitmap 01 10 0 0 0 0 0 0 1 0 1 1 0 0 1 0 Optimized Blooming Tree - insert • With collision • Set corresponding branches B0 B1 1 0 1 0 0 0 1 1 1 1 B2 B3 1 2 1 1

20. 0 1 1 0 1 0 0 1 0 0 1 0 01 1 0 0 item2 hash bit string : 01001 0 1 0 0 0 0 1 1 0 0 Optimized Blooming Tree - delete B0 B1 1 0 1 0 0 1 1 0 0 0 1 1 1 1 B2 B3 1 2 1 1

21. Outline • Blooming Tree • Lookup • Insert • Delete • Optimized Blooming Tree • Lookup • Insert • Delete • Simulations

22. Simulation • Size comparison

23. Simulation • Build on NP Intel IXP2800