1 / 33

Spin models for image processing

Lecture. Spin models for image processing. Alexey Abramov abramov _at_ physik3.gwdg.de. Georg-August University, Bernstein Center for Computational Neuroscience, III Physikalisches Institut, Göttingen, Germany. What is raster graphics image (bitmap)?.

bud
Télécharger la présentation

Spin models for image processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture Spin models for image processing Alexey Abramov abramov _at_ physik3.gwdg.de Georg-August University, Bernstein Center for Computational Neuroscience, III Physikalisches Institut, Göttingen, Germany

  2. What is raster graphics image (bitmap)? It’s a data structure representing a generally rectangular grid of pixels, or points of color, viewable via a monitor, paper, or other display medium. width In zoomed view every square represents a pixel. height (R,G,B) = (120,70,90) Pixel is the smallest piece of information in an image. 10-03-11 2/33 Alexey Abramov (BCCN, Göttingen)

  3. Image segmentation Partitioning of an input image into disjoint parts based on some image characteristics. input image segments - homogeneity criteria based methods - clustering - region-based growing - graph cuts - mean shift segmentation 10-03-11 3/33 Alexey Abramov (BCCN, Göttingen)

  4. Superparamagnitec clustering of data The methods of superparamagnetic clustering are nonparametric methods which solve the segmentation problem by finding the equilibrium states of the energy function of a ferromagnetic Potts model (without data term). Before update After update Neighborhood ε2Di Superparamagnetic phase regions of aligned spins correspond to a natural partition of the image data (Blatt et al., 1996). 10-03-11 4/33 Alexey Abramov (BCCN, Göttingen)

  5. Potts model A spin variable σk , which can take on q discrete values v1, v2, …, vq , called spin states, is assigned to each pixel of the image. The energy of the system is described by where the function Jij is a coupling constant, determining the interaction strength, where gi and gj are the respective color vectors of the pixels: where Δ is a normalization constant. 10-03-11 5/33 Alexey Abramov (BCCN, Göttingen)

  6. One iteration of Metropolis algorithm Current configuration S = 1 S = 2 S = 3 S = 4 Assumption for this example: Jij = 1 1. Ecur = -1 2. 4 → 1 Enew= -5 min Emin = -5 3. 4 → 2 Enew= 0 4. 4 → 3 Enew= -2 Proposed configuration Final configuration 5. Enew< Ecurtherefore 4 → 1 (move is accepted) 10-03-11 6/33 Alexey Abramov (BCCN, Göttingen)

  7. Metropolis algorithm (suggested by Metropolis, 1953) Ising model is a statistical model. The probability of each configuration of spins is the Boltzmann distribution with inverse temperature β: To generate configurations using the probability distribution is easiest using the Metropolis algorithm: 1. pick a spin at random and compute its contribution to the energy Ecur 2. flip the value of the spin and calculate the new contribution Enew 3. if Enew < Ecur then the new state is accepted – a good move 4. if Enew > Ecur then the new state is accepted using the probability – a worse move 5. Repeat. Simulated Annealing A certain number of iterations are carried out at each temperature and then the temperature is decreased. This is repeated until the system freezes into a steady state. 10-03-11 7/33 Alexey Abramov (BCCN, Göttingen)

  8. One iteration of parallel Metropolis algorithm 10-03-11 8/33 Alexey Abramov (BCCN, Göttingen)

  9. Implementation of Metropolis on GPU 1. Generate image tiles taking overlapping into account; 2. Construct CUDA data according to memory structure of GPU; 3. Launch kernel function on the GPU; 4. Decompose CUDA data. 10-03-11 9/33 Alexey Abramov (BCCN, Göttingen)

  10. Metropolis algorithm (the cooling schedule) A certain number of iterations are carried out at each temperature and then the temperature is decreased. This is repeated until the system freezes into a steady state. 10-03-11 10/33 Alexey Abramov (BCCN, Göttingen)

  11. Parallel Metropolis algorithm (fast cooling) Domain fragmentation – large uniform areas are being split into sub-segments despite the high attractive forces within them. 10-03-11 11/33 Alexey Abramov (BCCN, Göttingen)

  12. Configurations after short annealing “Cluttered scene” “Filling a cup” “Tsukuba” 10-03-11 12/33 Alexey Abramov (BCCN, Göttingen)

  13. Simulated annealing with short-cut Resolving domain fragmentation 10-03-11 13/33 Alexey Abramov (BCCN, Göttingen)

  14. Resolving Domain Fragmentation For the spin configuration with q = 6 after 10 Metropolis iterations: 10-03-11 14/33 Alexey Abramov (BCCN, Göttingen)

  15. Detected object boundaries “Cluttered scene” “Filling a cup” “Tsukuba” 10-03-11 15/33 Alexey Abramov (BCCN, Göttingen)

  16. Labeling of connected components “Cluttered scene” “Filling a cup” “Tsukuba” 10-03-11 16/33 Alexey Abramov (BCCN, Göttingen)

  17. Fast connected-component labeling Labeling of connected components in a binary image is one of the most fundamental operations in pattern recognition: labeling is required whenever a computer needs to recognize objects (connected components) in a binary image. By use of the labeling operation, a binary image is transformed into a symbolic image in which all pixels belonging to a connected component are assigned a unique label. • Multi-scan algorithms scan an image in the forward and backward raster directions alternately to propagate label equivalences until no label changes. • Two-scan algorithms complete labeling in two scans: during the first scan, they assign provisional labels to object pixels, and record label equivalences. Label equivalences are resolved during or after the first scan. During the second scan, all equivalent labels are replaced by their representative label. • Hybrid algorithms. Algorithms that are between muti-scan algorithms and two-scan algorithms. 10-03-11 17/33 Alexey Abramov (BCCN, Göttingen)

  18. Conventional two-raster-scan algorithms For an N x N binary image, we use b(x,y) to denote the pixel value at (x,y) in the image where 1 ≤ x,y ≤ N, and V0 for the value of object pixels and Vb for that of background pixels. We assume that V0 and Vb are larger than the value of any provisional label, and V0 < Vb. Mask for eight-connected connectivity 10-03-11 18/33 Alexey Abramov (BCCN, Göttingen)

  19. Conventional two-raster-scan algorithms Provisional labeling by the first scan For an N x N binary image, we use b(x,y) to denote the pixel value at (x,y) in the image where 1 ≤ x,y ≤ N, and V0 for the value of object pixels and Vb for that of background pixels. We assume that V0 and Vb are larger than the value of any provisional label, and V0 < Vb. 10-03-11 19/33 Alexey Abramov (BCCN, Göttingen)

  20. Conventional two-raster-scan algorithms With the mask for eight-connected connectivity, for an object pixel b(x,y) there are 16 possible cases in the mask. (2) - (9) and (13) – (16)b(x,y) can take on any existing provisional label in the mask without the need for consideration of label equivalences. 10-03-11 20/33 Alexey Abramov (BCCN, Göttingen)

  21. Conventional two-raster-scan algorithms The order of processing: b(x-1,y-1) → b(x,y-1) → b(x+1,y-1) → b(x-1,y) b(x,y) After b(x,y-1) is processed, the provisional labels for b(x-1,y-1) and b(x,y-1) should have been combined in the same equivalent label set and have the same representative label. Because all provisional labels in the mask have the same representative label, b(x,y) can take on any of the labels without consideration of any label equivalence. 10-03-11 21/33 Alexey Abramov (BCCN, Göttingen)

  22. The Karnaugh map The Karnaugh map (K-map) is a method to simplify boolean algebra expressions. In a Karnaugh map only one boolean variable changes in between adjacent squares. Once the table is generated and the output possibilities are transcribed, the data is arranged into the largest possible groups containing2n cells and the minterm is generated through the axiom laws of boolean algebra. A D C B 10-03-11 23/33 Alexey Abramov (BCCN, Göttingen)

  23. The Karnaugh map for the operation resolve C1 The operation resolve takes much time compared to other operations, it should be avoided as much as possible. C4 C3 C2 10-03-11 24/33 Alexey Abramov (BCCN, Göttingen)

  24. The Karnaugh map for the operation resolve Pseudocode After the first scan, the provisional labels that are assigned to a connected component in the given image will be combined in an equivalent label set and hold the same representative label. Then, by the second scan, each provisional label is replaced by its representative label. Thus, after the second scan, all pixels in a connected component will be assigned a unique label. if (C3  ≠ VB) b(x,y) = C3; else if (C1  ≠ VB) b(x,y) = C1; if (C4  ≠ VB) resolve(C4,C1); else if (C2  ≠ VB) b(x,y) = C2; if (C4  ≠ VB) resolve(C2,C4); else if (C4) b(x,y) = C4; else b(x,y) = m, m = m + 1. 10-03-11 25/33 Alexey Abramov (BCCN, Göttingen)

  25. Possible accelerations on a GPU Representative labels can be assigned to all object pixels simultaneously. Therefore the second scan can be implemented on the GPU in a very efficient manner. 10-03-11 26/33 Alexey Abramov (BCCN, Göttingen)

  26. Execution time on CPU CCL – the contour-tracing connected-component labeling; SAUF – the scan plus array-based union-find algorithm; RTS – the run-based two-scan algorithm. 10-03-11 27/33 Alexey Abramov (BCCN, Göttingen)

  27. Execution time on CPU CCL – the contour-tracing connected-component labeling; SAUF – the scan plus array-based union-find algorithm; RTS – the run-based two-scan algorithm. 10-03-11 28/33 Alexey Abramov (BCCN, Göttingen)

  28. Metropolis procedure with proposed short-cut 10-03-11 29/33 Alexey Abramov (BCCN, Göttingen)

  29. Comparison with graph-based method Original image Proposed method Graph-based 7 7 Felzenszwalb, et al.: Efficient graph- based image segmentation. (2004) 10-03-11 30/33 Alexey Abramov (BCCN, Göttingen)

  30. Timing performance Processing time Total computation time of all steps Proposed method Image size CPU GPU 153.1 ms | 24.4 x 103 ms 512 x 640 px graph-based (CPU) 500.0 ms 10-03-11 31/33 Alexey Abramov (BCCN, Göttingen)

  31. Bibliography • Abramov et al., “Real-time image segmentation on a GPU” (2010) • von Ferber et al., “Cluster update algorithm and recognition” (2000) • Potts, “Some generalized order-disorder transformations” (1952) • Metropolis et al., “Equation of state calculations by fast computing machines” (1953) • Blatt et al., “Superparamagnetic clustering of data” (1996) • Liefeng He et al., “Fast connected-component labeling” (2009) • Felzenszwalb et al., “Efficient graph-based image segmentation” (2004) 10-03-11 32/33 Alexey Abramov (BCCN, Göttingen)

  32. Thank you for your attention ! QUESTIONS ? Göttingen, 10.03.2011

More Related