Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai

1. Practical aspects of HDR capture and acquisition(Deconstructing HDR)Francisco Imai • CREATE 2010 • June 08 2010

2. DisclaimerOnly representing selfViews in this presentations are not of any current or past employer

3. Image as representation Real Ideal

4. Tone editing Ansel Adams

5. What about color HDR? Merced River and El Capitan in Winter, Yosemite Valley [in Nature]

6. Image manipulation

9. Definition Dynamic Range

10. maximum non-saturated signal DR = -------------------------------minimum signal

11. Sensor technologies www.pixim.com

12. HDR sensor technology:1. Multiple gains sensor Advantages: Good low light performance Good linearity Drawbacks: Limited DR extension Additional silicon area Additional power dissipation e.g. Dual column Level amp and ADC [Fowler 09], LOFIC [Adachi 05] from Boyd Fowler [HDRI workshop 2009]

13. HDR capture technology:2. Non-linear pixel response sensor Advantages: Wide DR > 120 dB Instantaneous measurement High fill factor Simple operation Drawbacks: No low-light video Larger FPN Different sensor architecture No solution for color sensors e.g. Logarithmic sensor [Dierickx 95], Dynamic well capacity adjustment [Sayag 91, Decker 98] from Boyd Fowler [HDRI workshop 2009]

14. HDR capture technology:3. Well-capacity recycling sensor Advantages: Good linearity Good color imaging Large DR > 100 dB Best SNR Drawbacks: Large pixel size Poor low light performance High power dissipation Needs high speed read-out e.g. Sigma delta pixel level ADC [Fowler 94], Asynchronous self-reset with multiple capture [Liu 02] from Boyd Fowler [HDRI workshop 2009]

15. HDR capture technology:4. Time to saturation sensor Advantages: Very HDR > 150 dB Drawbacks: High power dissipation Large pixel e.g. Time to saturation [Brajovic 96] from Boyd Fowler [HDRI workshop 2009]

16. HDR capture technology:5. Time varying exposures Advantages: Use conventional sensors Drawbacks: Motion blur e.g. Two sample CID [Nakamura 97], Two sample APS [Yadid-Peckt 97]

17. Blended exposures

18. HDR capture technology:6. Spatially varying exposures Advantages: Excellent linearity Color Imaging DR > 100 dB Drawbacks: High power dissipation Large pixel Poor low light performance Needs high speed read-out e.g. DPS pixel [Yang 99], Fuji Super CCD from Boyd Fowler [HDRI workshop 2009] and www.pixim.com

19. Pixim DPS

20. There are plenty of publication on HDR imaging sensors but most of cameras still do not use them. WHY?

21. Example Spherican 180-360 degrees 26 f-stops, 50 Mpixel 20 s to 1 minute 20 f-stops in HD-video

22. Example FujiFim SuperCCD EXR F200 EXR, F70EXR, F80EXR and S200 EXR

23. What to do with captured HDRI? • Tone map to display in conventional displays • Global tone mapping • Local tone mapping • iCAM • Retinex • Build an HDR display

24. How to adjust colors in HDR display?

25. Off-the-shelf components HDR display SIGGRAPH 2005 - Seetzen et al. 14 bits 1300 cd/m2

26. HDR display modeling

27. HDR display forward model • All possible sextuplets combination is 6256 = 281 trillion • Inaccurate colorimetric measurement in dark region (resulting in green cast in dark areas when model is used) • Problem with cross-terms between LCD and DLP • More robust physical based model is necessary

28. Radiance based Forward model Decompose 12 Component XYZ and perform a priori eigenvector analysis

29. LCD display spectral radiance Red, green and blue ramps

30. LCD additivity Comparison sum of red, green and blue Channels with gray measurements

31. Spectral radiance measurementsPhoto Research PR-650

32. Spectral radiance measurementDLP light reflected on MgO2 Close-up measured area for green ramp Measurement Red, Green, Blue Ramps

33. Additivity DLP channels? White is not active White is active White channel

34. DLP Radiance Transmittance LCD LCD Radiance Transmittance LCD

35. Compare LCD radiance between measurement and estimation usingforward model Blue Green Red

36. Comparison measured and estimated LCD radiance

37. Estimation Final RadianceTraining set

38. Verification 100 “random” samples (20 samples with white on white on DLP) luminance x chroma Hue angle x chroma

39. Inverse HDR model

40. Colorimetric performance100 Random Samples

41. Extra Verification set (280 samples) Neutral ramp (14 levels) 4 octaves of Color Checker 170 objects (Vrhel) Mean CIEDE2000 = 2.4 Maximum CIDE2000 = 7.7 Excluding 7 outliers (out-of-gamut samples)

42. HDR rendering accuracy assessment

43. Veiling glareScene-dependent scatter in optics/camera and in the human eye reduce the HDR Talvala SIGGRPAH 2007

44. Other IQ aspect: SNR from Boyd Fowler [HDRI workshop 2009]

45. Other IQ aspect: Resolution Quick MTF

46. Other IQ aspect: Colour EyeRex – SW by Media Chance

47. Evocative HDR look Nina Aldin Thune Fantoft stavkirke –www.pappafrezzo.com Tunliweb

48. Hype Cycle HDRI

49. Acknowledgments

50. Unanswered questions1. When are we going to have HDR cameras on cell-phone cameras?2. How to deal with white balance in HDR?3. What set of metrics is appropriate for HDR display quality?