Overview of Haze Removal Methods

1. Overview of Haze Removal Methods Matteo Pedone Machine Vision Group, University of Oulu, Finland

2. Overview of Haze Removal Methods Description of the problem Overview of current approaches found in literature Strengths and weaknesses of present methods Description of our method

3. The Atmospheric Scattering Model A particle scatters incident light The nature of scattering depends on material properties, shape and size The exact form and intensity of the scattering pattern varies dramatically with particle size

4. The Atmospheric Scattering Model Haze: constituted of aerosol (small particles suspended in gas) Main sources: volcanic ashes, foliage exudation, combustion products, sea salt… Haze particles are larger than air molecules but smaller than fog droplets. produce a distinctive grayor bluish hue and affects visibility. Extends to altitudes of several Km

5. The Atmospheric Scattering Model Fog: Same origins as haze, associated with an increase in relative humidity of an air Size of water droplets increases Haze can turn into fog (transition state: mist) Reduces visibility more than haze Extends to altitudes of few hundred meters.

6. The Atmospheric Scattering Model

7. The Atmospheric Scattering Model Important physical mechanisms: Attenuation: radiance of a scene-point falls as its distance from the observer increases Airlight: Atmosphere behaving like a source of light. Due to multiple scattering. Increases with distance.

8. The Atmospheric Scattering Model Mathematical Model: I(x) is the observed radiance at x J(x) is the original scene radiance at x A is the airlight t(x), scalar called transmission: describes how the radiance of a point in the scene is attenuated according to its distance d from the observer Note that I, J, A are (R,G,B) triplets

9. The Atmospheric Scattering Model Mathematical Model: In order to remove the effect of haze, one must recover J(x) Quantities A and t are typically unknown I(x) is known

10. The Atmospheric Scattering Model References Narasimhan & Nayar, “Vision and the Atmosphere”, International Journal of Computer Vision, 2001

11. Overview of Haze Removal Methods Description of the problem Overview of current approaches found in literature Strengths and weaknesses of present methods Description of our method

12. Current Methods for Haze Removal Can be grouped into several categories With multiple images With one image + depth-map Single image Subcategories of the ones above are: Requires user interaction Fully automatic We are mostly interested in Single-Image methods

13. Multiple Image Approaches Assume 2+ images of the same scene are taken: Under different weather conditions [1] or With different polarization filters [2] [1] Narasimhan & Nayar, “Vision and the Atmosphere”, 2001 [2] Schechner et al. 2003, “Polarization-based vision through haze”, Applied Optics 42

14. Multiple Image Approaches Narasimhan & Nayar’s method Assumes 2+ bad weather images are given Uses geometric constraints to estimate A The airlight component [1-t(x)] is estimated from corresponding pixels of the two bad weather images

15. (c) Dehazed image, (d) Clear weather images (a),(b) Foggy images Multiple Image Approaches Narasimhan & Nayar’s method, RESULTS

16. One Image + Depth + Texture Kopf et al. Method: Deep Photo project from SIGGRAPH 2008 Assumes a 3D model of the scene is given (e.g.: from Google Maps) Assumes textures of the scene are given (from satellite or aerial photos) Requires user interaction to align the 3D model with the scene Very accurate results

17. Single Image Approaches Do not require information extracted from additional images Do not require depth-information Typically rely upon statistical assumptions, and or the nature of the scene (e.g. part of the sky is visible) Sometimes they require user interaction Most relevant: Fattal’s ”Single-Image Dehazing”, SIGGRAPH 2008 He’s ”Single Image Haze Removal Using Dark Channel Prior”, CVPR 2009

18. Single Image Approaches He’s Method (with Dark Channel Prior) Assumes a portion of the scene is dominated by airlight STATISTICAL ASSUMPTION: ”in most of the non-sky patches, at least one color channel has very low intensity at some pixels. In other words, the minimum intensity in such a patch should have a very low value” The 1st assumption is used to estimate airlight, the 2nd assumption is used to estimate the transmission

19. Single Image Approaches He’s Method (with Dark Channel Prior) Dark Channel: Airlight is estimated by picking up the pixels of the image corresponding to the 0.1% brightest pixels in the dark channel, and then choosing the one with maximum intensity.

20. Single Image Approaches He’s Method (with Dark Channel Prior) Dark Channel: He shows that the transmission can be estimated by calculating:

21. Single Image Approaches He’s Method (with Dark Channel Prior) Dark Channel: He shows that the transmission can be estimated by calculating: Dark channel of the image divided by the airlight color

22. Single Image Approaches He’s Method (with Dark Channel Prior) Airlight and transmission are sufficient to invert the model and retrieve the original radiance of the scene. Dark channel is computed on square neighborhoods  Block artifacts and halos are reduced by using a soft-matting algorithm.

23. Overview of Haze Removal Methods Description of the problem Overview of current approaches found in literature Strengths and weaknesses of present methods Description of our method

24. Summary Multiple image methods require special equipment (polarizers) or same scene under different weather conditions. They don’t necessarily produce better results than single-images approaches

25. Summary One Image + 3D model + textures Accurate and does not require special equipment Requires a considerable amount of special information (3D model, and aerial photos of the scene) Requires user interaction

26. Summary Single-Image methods do not require special equipment, nor extra information They either make assumption on the nature of the scene, or require little interaction by the user

27. Summary Single-Image methods It is known what are the consequences of a bad estimate for the transmission  haze is not completely removed, or it is removed where there is no haze (overboost contrast)

28. Summary None of the aforementioned authors shows what happens when the airlightestimate is inaccurate (motivation of our work)

29. Overview of Haze Removal Methods Description of the problem Overview of current approaches found in literature Strengths and weaknesses of present methods Description of our method

30. Collect statistics of airlight colors from 100+ natural images (daylight and twilight hazy scenes) • Manually select 32x32 pixel patch with “full haze” • Airlight colors are scattered around a 29.8784 degrees line in hue-saturation plane, and most are close to the origin (=> low saturation). Estimation of Airlight

31. Extract patches of 13x13 pixels from hazy image according to the following criteria: • The patch contains pixels with same transmission and hue but with different shades (=> same direction for R but different magnitudes) • The pixels in the patch do not have too low or too high transmission (avoid degenerate cases) Estimation of Airlight

32. Extract patches of 13x13 pixels from hazy image according to the following criteria: • The patch contains pixels with same transmission and hue but with different shades (=> same direction for R but different magnitudes) • The pixels in the patch do not have too low or too high transmission (avoid degenerate cases) • Pixels in the patches do not have too low saturation (hue would not be reliable). • Pixels in the patches are not too dark or too bright in average, and variance should not be too high (noise) or too low (homogeneous areas with no shades). Estimation of Airlight

33. Solve minimization problem: find an airlight color vector that enforces: • Perpendicularity to patches albedos • Closeness to natural airlight hue line • Low saturation Estimation of Airlight • ni : normal to the plane containing the [R,G,B] values of the pixels in the i-th patch extracted • nsky : unit-vector in RGB space having direction corresponding to the statistical hue of airlight in natural images • w: unit vector [1,1,1]/31/2 • c(ni ): scalar associated with the i-th patch (based of residual error, see paper) • : weight parameters (respective default values: 3, 0.02)

34. Results with artifical haze (Middlebury dataset) Estimation of Airlight

35. Results with real images Estimation of Airlight

36. THANK YOU