Segmentation into Planar Patches for Recovery of Unmodeled Objects

1. Segmentation into Planar Patches for Recovery of Unmodeled Objects Kok-Lim Low COMP 290-075 Computer Vision 4/26/2000

2. The Big Picture • Work by Marjan Trobina • “From planar patches to grasps: a 3-D robot vision system handling unmodeled objects.” Ph.D. thesis, ETHZ, 1995. • Overview of whole system acquire range images from 2 or more views segment into planar patches generate object hypotheses compute grasping points robot arm

3. Setup

4. Segmentation into Planar Patches • Objectives • extract planar patches from range images in a robust way • just sufficient info for recovery of unmodeled objects for computing grasping points • Can be viewed as data compression • transforming range image to just a few parameters

5. Related Work • 1) Edge-based segmentation • detect surface discontinuities • find closed edge chains • not robust against noisy data • 2) Split-and-merge paradigm • tessellation of image • using quadtrees or Delaunay triangles

6. Related Work • 3) Clustering • map data to feature space • find clusters • points in feature space have no image-space connectivity • 4) Region growing • grow region until approximation error too large • order dependent, result usually far from optimal • 5) Recover-and-select paradigm

7. input model recovery iterate until remaining models are completely grown model selection output Recover-and-Select Paradigm (RS) • Originally proposed by Leonardis • Very robust against statistical noise and outliers • Consists of 2 intertwined stages:

8. Overview of RS Algorithm • Model recovery • regularly place seeds • grow statistically consistent seeds independently • fit a plane (model) to each region • stop growth of region if planar fit is lousy • stop growth of region if no compatible points can be added • Model selection • select some recovered (plane) models • minimize number of selected models while keeping approximation error low

9. Planar Fitting • Planar patch parameterized by a1, a2, a3 in f(X, Y) = a1X + a2Y + a3 = Z • Distance function from range data point r(m) to planar patch d2(m) = ( r(m) – f(m) )2

10. Planar Fitting • Approximation error for set D of n points is • For each set of points D, minimize  by Linear Least Squares method to obtain plane parameters a1, a2, a3.

11. Model Recovery • Place seeds in regular grid of 7x7 windows • Define model acceptance thresholdT • Grow seed if its  < T (statistically consistent) • Define compatibility constraint C • Add adjacent point m to patch if d2(m) < C (compatible) • Stop when T or no compatible point can be added • Output is a set of overlapping planar patch models

12. Model Selection • Select smallest number of models while keeping approximation error small • Objective function (to be maximized) for model si F(si) = K1ni – K2i– K3Ni where ni = | D | i = approximation error of model si Ni = number of parameters in model si

13. Model Selection • Objective function for M models where p = [ p1...pM ] and pi = 0 or 1 cii = K1ni – K2i– K3Ni cij = ( –K1 | DiDj | + K2ij) / 2 • Use greedy algorithm to find vector p so that F(p) is near to maximum

14. Example

15. Result

16. Result

17. Multiresolution Recover-and-Select (MRS) • In RS, many seeds are grown and then discarded • MRS uses hierarchical approach to reduce waste • Basic idea • build an image pyramid • apply standard RS on coarsest image • selected patches are projected to the next finer level and used as seeds for the new level • start new seeds on the unprojected regions in the next finer level • Speedup of 10 to 20 times

18. Result

19. Viewpoint Invariant Segmentation • Range images from different viewpoints • A planar patch extracted from different views should have same parameters and error measure • Modifications to model recovery stage: • project data points into direct 3-D space prior to the segmentation • minimizing the orthogonal distance to the plane

20. Viewpoint Invariant Segmentation • Planar patch now parameterized by a1, a2, a3, a4 in f(X, Y, Z) = a1X + a2Y + a3Z + a4 = 0 • Distance function from 3-D range data point M to planar patch d2(M) = f(M)2

21. Viewpoint Invariant Segmentation • Approximation error  = 1 where 1is the smallest eigenvalue of the covariance matrix of the n points in the patch • The normal (a1, a2, a3) of the planar patch is the eigenvector with eigenvalue 1

22. Postprocessing • create explicit patch boundary description • post-processing to clean edge • classify patches as “true planes” or “curved patches”, and fit points on curved planes with quadrics • classify adjacency relation as concave or convex • e.t.c.

23. Generating Object Hypotheses • Objects are unmodeled • Group planar patches into Single-View Object Hypotheses (SVOHs) • Combine SVOHs into Global Object Hypotheses (GOHs) • Prefer oversegmentation to undersegmentation — avoid grasping 2 objects at the same time

24. Generating SVOHs • A SVOH is a set of connected patches, such that for any 2 patches, there exists at least one path that does not contain any concave relation

25. Establishing GOHs • A GOH is a set of SVOHs, such that for any SVOHi there is at least one SVOHj (from a different view) such that SVOHi and SVOHj have at least one pair of patches sk (from SVOHi) and sl (from SVOHj) which fulfills the same-surface predicate • Rough idea of “same-surface predicate” • when 2 patches satisfy the same-surface predicate, they are on the same plane or on the same curved surface and they are intersecting each other

26. Result

27. References • Marjan Trobina • “From Planar Patches to Grasps: A 3-D Robot Vision System Handling Unmodeled Objects.” Ph.D. thesis, ETHZ, 1995 • A. Leonardis • “Image Analysis Using Parametric Models: Model-Recovery and Model-Selection Paradigm.” Ph.D. thesis, University of Ljubljana, 1993 • A. Leonardis • “Recover-and-Select on Multiple Resolutions.” Technical report LRV-95, Computer Vision Lab, University of Ljubljana, 1995

28. References • Frank Ade, Martin Rutishauser and Marjan Trobina • “Grasping Unknown Objects.” ETHZ, 1995 • Martin Rutishauser, Markus Stricker and Marjan Trobina • “Merging Range Images of Arbitrarily Shaped Objects.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1994