Seam Carving

1. Seam Carving Kshitij Padalkar 13th Oct 2008

2. Contents: • In one Line… • Need • Solutions and Other Work • Definition and Maths • Energy , Seam • Pixel Removal Criteria • Other Energy Functions? • Discrete Image Resizing • Aspect Ratio Change • Optimal Seam Order • Image Enlarging • Content Amplification • Object Removal • Multi-size Images • Limitations • Links

3. In One Line… Method for Content-Aware Image Resizing

4. Need

5. Different Displays-Different Resolutions-(Different Images??)

6. HTML allows automatic layout of the text, but not the images

7. Solutions and Other Work • Crop and Scale • Saliency • Segmentation

8. Contents: • In one Line… • Need • Solutions and Other Work • Definition and Maths • Energy , Seam • Pixel Removal Criteria • Other Energy Functions? • Discrete Image Resizing • Aspect Ratio Change • Optimal Seam Order • Image Enlarging • Content Amplification • Object Removal • Multi-size Images • Limitations • Links

9. Definition and Maths… • Goal is to remove unnoticeable pixels. • Energy Function: • Remove Pixels with least energy • Sort pixels by Energy, remove smallest pixels (optimum) • Remove Column with smallest total Energy (column) • Remove Seam with smallest total Energy (seam)

10. Seam • Vertical Seam: • an 8-connected path of pixels in the image from top to bottom, containing one, and only one, pixel in each row of the image

11. Seam • Energy of a Seam • Minimum Energy Seam

12. Pixel Removal Criteria • Optimal

13. Pixel Removal Criteria • Pixel

14. Pixel Removal Criteria • Column

15. Pixel Removal Criteria • Seam

16. Pixel Removal Criteria • Comparison

17. Other Energy Functions • Gradient • Entropy • HOG (Histogram of Gradient) • Saliency

18. Contents: • In one Line… • Need • Solutions and Other Work • Definition and Maths • Energy , Seam • Pixel Removal Criteria • Other Energy Functions? • Discrete Image Resizing • Aspect Ratio Change • Optimal Seam Order • Image Enlarging • Content Amplification • Object Removal • Multi-size Images • Limitations • Links

19. Aspect Ratio Change • Change the image from (n X m) to (n X m’) • (assume m’ < m) • Remove m-m’=c seams successively Seam Carving

20. Aspect Ratio Change • Change the image from (n X m) to (n X m’) • (assume m’ < m) • Remove m-m’=c seams successively Scaling

21. Optimal Seam Ordering • Change the image from (n X m) to (n’ X m’) • (assume m’ < m and n’ < n) • What to do? What is the correct order of seam carving? • Remove vertical seams first? • Horizontal seams first? • Or alternate between the two?

22. Optimal Seam Ordering • We optimize • where k = r+c, r = (m−m’), c = (n−n’) and ai is used as a parameter that determine if at step i we remove a horizontal or vertical seam: a = {0,1}

23. Optimal Seam Ordering • Transport map • Matrix of size n X m • Each element T(r,c) holds the minimal cost needed to obtain an image of size n−r Xm−c

24. Discrete Image Resizing • Aspect Ratio Change • Optimal Seam Order • Image Enlarging • Content Amplification • Object Removal

25. Image Enlarging • compute the optimal vertical (horizontal) seam ‘s’ on I and duplicate the pixels of s by averaging them with their left and right neighbors (top and bottom in the horizontal case)

26. Image Enlarging • To reduce the width by ‘k’, compute first ‘k’ vertical seams and duplicate them • All seams duplicated, 50% enlargement. For more do it in steps.

27. Discrete Image Resizing • Aspect Ratio Change • Optimal Seam Order • Image Enlarging • Content Amplification • Object Removal

28. Content Amplification • Scale the image, this will scale everything, content as well as non-content. • Shrink the scaled-image by seam carving, this will carve out the non-content part.

29. Discrete Image Resizing • Aspect Ratio Change • Optimal Seam Order • Image Enlarging • Content Amplification • Object Removal

30. Object Removal • User marks the target object to be removed. • Then seams are removed from the image until all marked pixels are gone. • To regain the original size, seam insertion is employed.

31. Object Removal • Find the missing shoe…

32. Object Removal • Object marking to remove unwanted results

33. Contents: • In one Line… • Need • Solutions and Other Work • Definition and Maths • Energy , Seam • Pixel Removal Criteria • Other Energy Functions? • Discrete Image Resizing • Aspect Ratio Change • Optimal Seam Order • Image Enlarging • Content Amplification • Object Removal • Multi-size Images • Limitations • Links

34. Multi Size Images • Methods mentioned so far are not real-time. • We calculate each seam remove it and calculate the next seam. • For real-time, we calculate- • index map V of size n×m that encodes, for each pixel, the index of the seam that removed it, i.e., V(i, j) = t means that pixel (i, j) was removed by the t-th seam removal.

35. Multi Size Images • Horizontal Index map (H) • Vertical Index map (V) • To get an image of width m’, • we need to remove m-m’ pixels from each row • we gather, in each row, all pixels with seam index greater than or equal to m−m’. • Same for changing the height. • Problem comes when we want to change both!

36. Contents: • In one Line… • Need • Solutions and Other Work • Definition and Maths • Energy , Seam • Pixel Removal Criteria • Other Energy Functions? • Discrete Image Resizing • Aspect Ratio Change • Optimal Seam Order • Image Enlarging • Content Amplification • Object Removal • Multi-size Images • Limitations • Links

37. Limitations • Too much content • No space for seam to avoid content

38. Links • Paper • ftp://ftp1.idc.ac.il/Arik_shamir/SCweb/imret/vidret.pdf • Youtube demo • http://www.youtube.com/watch?v=6NcIJXTlugc • Search google for Seam Carving…. Many many blogs…

39. THANK YOU