Digital Cameras

1. Digital Cameras Engineering Math Physics (EMP) Jennifer Rexford http://www.cs.princeton.edu/~jrex

2. Image Transmission Over Wireless Networks • Image capture and compression • Inner-workings of a digital camera • Manipulating & transforming a matrix of pixels • Implementing a variant of JPEG compression • Wireless networks • Wireless technology • Acoustic waves and electrical signals • Radios • Video over wireless networks • Video compression and quality • Transmitting video over wireless • Controlling a car over a radio link

3. Traditional Photography • A chemical process, little changed from 1826 • Taken in France on a pewter plate • … with 8-hour exposure The world's first photograph

4. Digital Photography • Digital photography is an electronic process • Only widely available in the last ten years • Digital cameras now surpass film cameras in sales

5. Image Formation Digital Camera Film Eye

6. Aperture and Exposure • Aperture • Diameter of the hole allowing light to enter • E.g., the pupil of the eye • Higher aperture leads to more light entering • … though poorer focus across a wider depth of field • Shutter speed • Time for light to enter the camera • Longer times lead to more light • … though blurring of moving subjects • Together, determine the exposure • The amount of light allowed to enter the camera

7. Image Formation in a Pinhole Camera • Light enters a darkened chamber through pinhole opening and forms an image on the further surface

8. Image Formation in a Digital Camera +10V Photon + + + + + +        + CCD sensor • Array of sensors • Light-sensitive diodes that convert photons to electrons • Each cell corresponds to a picture element (pixel) • Sensor technologies • Charge Coupled Device (CCD) • Complementary Metal Oxide Semiconductor (CMOS)

9. Sensor Array: Image Sampling

10. Sensor Array: Reading Out the Pixels • Transfer the charge from one row to the next • Transfer charge in the serial register one cell at a time • Perform digital to analog conversion one cell at a time • Store digital representation Digital-to-analog conversion

11. Sensor Array: Reading Out the Pixels

12. More Pixels Mean More Detail 1280 x 960 1600 x 1400 640 x 480

13. The 2272 x 1704hand The 320 x 240hand

14. Representing Color • Light receptors in the human eye • Rods: sensitive in low light, mostly at periphery of eye • Cones: only at higher light levels, provide color vision • Different types of cones for red, green, and blue • RGB color model • A color is some combination of red, green, and blue • E.g., eight bits for each color • With 28 = 256 values • Corresponding to intensity • Leading to 24 bits per pixel • Red: 255, 0, 0 • Green: 0, 255, 0 • Yellow: 255, 255, 0

15. Number of Bits Per Pixel • Number of bits per pixel • More bits can represent a wider range of colors • 24 bits can capture 224 = 16,777,216 colors • Most humans can distinguish around 10 million colors 8 bits / pixel / color 4 bits / pixel / color

16. Separate Sensors Per Color • Expensive cameras • A prism to split the light into three colors • Three CCD arrays, one per RGB color

17. Practical Color Sensing: Bayer Grid • Place a small color filter over each sensor • Each cell captures intensity of a single color • More green pixels, since human eye is better at resolving green

18. Practical Color Sensing: Interpolating • Challenge: estimating pixels we do not know for certain • For a non-green cell, look at the neighboring green cells • And, interpolate the value • Accuracy of interpolation • Good in low-contrast areas • Poor with sharp edges (e.g., text) Estimate “RGB” at the “G” cells from neighboring values

19. Digital Images Require a Lot of Storage • Three dimensional object • Width (e.g., 640 pixels) • Height (e.g., 480 pixels) • Bits per pixel (e.g., 24-bit color) • Storage is the product • Pixel width * pixel height * bits/pixel • Divided by 8 to convert from bits to bytes • Common sizes • 640 x 480: 1 Megabyte • 800 x 600: 1.5 Megabytes • 1600 x 1200: 6 Megabytes

20. Compression • Benefits of reducing the size • Consume less storage space and network bandwidth • Reduce the time to load, store, and transmit the image • Redundancy in the image • Neighboring pixels often the same, or at least similar • E.g., the blue sky • Human perception factors • Human eye is not sensitive to high frequencies

21. Contrast Sensitivity Curve

22. Lossy vs. Lossless Compression • Lossless • Only exploits redundancy in the data • So, the data can be reconstructed exactly • Necessary for most text documents (e.g., legal documents, computer programs, and books) • Lossy • Exploits both data redundancy and human perception • So, some of the information is lost forever • Acceptable for digital audio, images, and video

23. Examples of Lossless Compression • Huffman encoding • Assign fewer bits to less-popular symbols • E.g., “a” occurs more often than “i” • … so encode “a” as “000” and “i” as “00111” • Efficient when probabilities vary widely • Run-length encoding • Identify repeated occurrences of the same symbol • Capture the symbol and the number of repetitions • E.g., “eeeeeee”  “@e7” • E.g., “eeeeetnnnnnn”  “@e5t@n6”

24. Joint Photographic Experts Group • Lossy compression of images • Starts with an array of pixels in RGB format • With one number per pixel for each of the three colors • Outputs a smaller file with some loss in quality • Exploits both redundancy and human perception • Transforms the data to identify parts that humans notice less • More about transforming the data in Wednesday’s class Uncompressed: 167 KB Good quality: 46 KB Poor quality: 9 KB

25. Conclusion • Digital cameras • Light and a optical lens • Charge and electronic devices • Pixels and a digital computer • Digital images • A two-dimensional array of pixels • Red, green, and blue intensities for each picture • Image compression • Raw images are very large • Compression reduces the image size substantially • By exploiting redundancy and human perception