1 / 11

Query Processing

Query Processing. (Two-Pass Algorithms). Priyadarshini.S 006493851 Cs 257_117_ch 15_15.5. Two-Pass Algorithms Using Hashing. General idea: Hash the tuples using an appropriate hash key

roth-gilmore
Télécharger la présentation

Query Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Query Processing (Two-Pass Algorithms) Priyadarshini.S 006493851 Cs 257_117_ch 15_15.5

  2. Two-Pass Algorithms Using Hashing • General idea: • Hash the tuples using an appropriate hash key • For the common operations, there is a way to choose the hash key so that all tuples that need to be considered together has the same hash value • Do the operation working on one bucket at a time

  3. Introduction: Hashing is done if the data is too big to store in main memory buffers. • Hash all the tuples of the argument(s) using an appropriate hash key. • For all the common operations, there is a way to select the hash key so all the tuples that need to be considered together when we perform the operation have the same hash value. • This reduces the size of the operand(s) by a factor equal to the number of buckets.

  4. Partitioning by Hashing initialize M-1 buckets with M-1 empty buffers for each block b of relation R do read block b into the Mth buffer for each tuple t in b do if the buffer for bucket h(t) is full then copy the buffer to disk initialize a new empty block in that buffer copy t to the buffer for bucket h(t) for each bucket do if the buffer for this bucket is not empty then write the buffer to disk

  5. Duplicate Elimination Using Hashing • Hash the relation R to M-1 buckets, R1, R2,…,RM-1 • Note: all copies of the same tuple will hash to the same bucket! • Do duplicate elimination on each bucket Riindependently, using one-pass algorithm • Return the union of the individual bucket results

  6. Analysis of Duplicate Elimination Using Hashing • Number of disk I/O's: 3*B(R) • In order for this to work, we need: • hash function h evenly distributes the tuples among the buckets • each bucket Ri fits in main memory (to allow the one-pass algorithm) • i.e., B(R) ≤ M2

  7. Grouping Using Hashing • Hash all the tuples of relation R to M-1 buckets, using a hash function that depends only on the grouping attributes • Note: all tuples in the same group end up in the same bucket! • Use the one-pass algorithm to process each bucket independently • Uses 3*B(R) disk I/O's, requires B(R) ≤ M2

  8. Union, Intersection and Difference Using Hashing • Use same hash function for both relations! • Hash R to M-1 buckets R1, R2, …, RM-1 • Hash S to M-1 buckets S1, S2, …, SM-1 • Do one-pass {set union, set intersection, bag intersection, set difference, bag difference} algorithm on Ri and Si, for all i • 3*(B(R) + B(S)) disk I/O's; min(B(R),B(S)) ≤ M2

  9. Join Using Hashing • Use same hash function for both relations; hash function should depend only on the join attributes • Hash R to M-1 buckets R1, R2, …, RM-1 • Hash S to M-1 buckets S1, S2, …, SM-1 • Do one-pass join of Riand Si, for all i • 3*(B(R) + B(S)) disk I/O's; min(B(R),B(S)) ≤ M2

  10. Comparison of Sort-Based and Hash-Based • For binary operations, hash-based only limits size of smaller relation, not sum • Sort-based can produce output in sorted order, which can be helpful • Hash-based depends on buckets being of equal size • Sort-based algorithms can experience reduced rotational latency or seek time

  11. Queries?????? Thank you

More Related