Coherency Sensitive Hashing (CSH) Simon Korman and ShaiAvidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference on Computer Vision
Outline • Introduction • Locality Sensitive Hashing for Finding Nearest Neighbors • Coherency Sensitive Hashing • Experiments • Conclusions
Introduction(1/2) Patch : k*k block 1080 query Find the closest patch linear search(search every patch one by one) 1920
Introduction(2/2) • (Streaming) Massive Data Sets => High Dimensional Vectors • E.g. 8*8 patch => v= [ v1, v2, ..., vi , …, vN] , dimension N = 64 • Linear search = find nearest neighbor • For very large databases of high-dimensional items • Time-consuming • Needs to find ApproximateNearest Neighbors (ANN) for each patch in real time. • Curse of dimensionality • Existing ANN methods include trees and hashes. • KD-trees • Locality-Sensitive Hashing(LSH)
Locality Sensitive Hashing for Finding Nearest Neighbors(1/2) Random line • A collision occurs when two points hash to the same value Projection = hash function Look up table Repeat L times =>the closest point will collide most times bucket Hash table
Locality Sensitive Hashing for Finding Nearest Neighbors(2/2) • Hash function: • a is d-dimensional random vector • r is predefine integer • Constant width of each quantization bin • b is random value from range [0, r] • To balance quantizationerror • v is the original vector 1. indexing 2. search
Coherency Sensitive Hashing (CSH) White:1, Black:-1 s=1, 4x4 kernel
4.2.1 Candidate Creation • Patch a, a1,a2of image A; Patch b, b1,b2of image B : Each entry can store k patches from each image=>total k+2*(k+1)+k=4k+2 candidates
4.2.2 Candidate Ranking • Given the candidate set (of size 4k + 2), to find the nearest one. • Main overall time consumer • Approximate the process, have a little impact, greatly reduce time. • Use Walsh Hadamard(WH) projections • Already computed in the indexing stage. • Accumulating the projections of the differences of patches on the WH kernels.
Experiments • We collected 133 pairs of images, taken from 1080p HD
Experiments • We computed the exact nearest neighbor match to serve as a ground truth. • A novel algorithm PatchMatch  to compare : • Not as accurate as LSH or KD-trees. • So fast. The key to this speedup is spatial coherent. Error : not the same match with ground truth  C. Barnes, E. Shechtman, A. Finkelstein, and D. B. Goldman. PatchMatch: A randomized correspondence algorithm for structural image editing. In SIGGRAPH, 28(3), 2009.
Image Reconstruction • Reconstruct image A, use image B • Such reconstructions are very common in many applications. • image editing (e.g. retargetting, inpainting), image denoising and super-resolution… • It simply replaces each pixel with the average of the corresponding pixels. Image B Image A
More results on web : http://www.eng.tau.ac.il/~simonk/CSH/index.html
Conclusions • We proposed an algorithm for computing ANN fields termed Coherency Sensitivity Hashing. • Follows LSH search scheme • But combines image coherency cues • It was shown to be faster and more accurate than PatchMatch.