Colors: Desktop Monitor to the Big Screen (& back)

1. Colors:Desktop Monitor tothe Big Screen (& back) Alan Edelman Dept of Mathematics: MIT MIT Laboratory for Computer Science Frank Wang Arun Rao (Pixar) Graphics Lunch April 25, 2003

2. Other Topics Not Covered Today • Parallel MATLAB • The Bohemian Dome Horizontal Vertical Villarceau A=QQT A=QQ A=QR Hopf Fibration

3. Outline • The Data • The Problem • Step 1: Finding the right three dimensional basis • Step 2: Inverting onto that basis • Step 3: Forming the model

4. Outline • The Data • The Problem • Step 1: Finding the right three dimensional basis • Step 2: Inverting onto that basis • Step 3: Forming the model

5. The data (101 points x 1000 frames) wavelength vs density Greens Blues Reds Grays film density = log(no film / with film)

6. Film Recording and measurements Solid colors sent to film recorder, e.g. reds Negative sent through projector to spectrometer • Log ratio with no film (only bulb) film density = log(no film / with film) Negative is produced: film appears as cyans Energy data at each wavelength Reds

7. Outline • The Data • The Problem • Step 1: Finding the right three dimensional basis • Step 2: Inverting onto that basis • Step 3: Forming the model

8. Movie Making Step I: The artists PixarArtists choose colors on their desktop computer monitors Step II: Color Recording Digital Images Recorded on Film • http://www.pixar.com/companyinfo/press/1999/pr99-02-04a.html • Film Developed Step III: Color Reproduction Film Projected Onto Screen at a movie theatre near you Problem: Colors on the big screen just do not look the same.

9. The Two Stages color recording stage color reconstruction stage

10. The Two Stages color recording stage color reconstruction stage

11. Models, Algorithms, Numerics • physically based models • numerical techniques To invert the color reproduction & recording steps.

12. Outline • The Data • The Problem • Step 1: Finding the right three dimensional basis • Step 2: Inverting onto that basis • Step 3: Forming the model

13. SVD of the data Three significant singular values svd index • Inputs (r,g,b) for 1r,g,b 10 scaled (1000 frames) • Output Space: Densities at 400:3:700 nm’s • Data Structure: 101 x 1000 matrix “A” Compute SVD(A) Project onto best 3 space

14. SVD Basis = no physical meaning Orthogonality Constraint too strong

15. The NNMF Basis = primary colors

16. Non-Negative Matrix Factorization • The NNMF (Lee, Seung 1999) • V  WH Input: Vij>0 Output: Wij>0 Hij>0 (low rank) Algorithm: H  H .* (W’V)./(W’WH) W  W .* (VH’)./(WHH’) • Original Application: Eigenfaces • New Algorithm: Project SVD Into Cone using Convex Hull Algorithm

17. Errors of two NNMF implementations • More than 1Mflop per NNMF iteration. Error flattens after 50000 iteration. • Accuracy of new algorithm improves as samples increase, but not for NNMF. • NNMF can easily generalize to higher dimension.

18. SVD with a geometry tweak

19. Compare bases extracted by the two methods

20. Projection of 1000 spectra onto the basis 101x1000  3x1000  10x10x10

21. Input and output for stage 1 • Find a functional relationship between laser input and output of concentration vectors by either interpolation or a physical model.

22. Outline • The Data • The Problem • Step 1: Finding the right three dimensional basis • Step 2: Inverting onto that basis • Step 3: Forming the model

23. Color matching functions CIERGB CIEXYZ

24. chromaticity diagram (XYZ) spectrum locus purple line

25. CIELAB color space

26. Obtain coefficients from Color • Given a color as (x,y,z) in CIEXYZ coordinates compute c1,c2,c3 such that (x,y,z)=∫λ(x(λ),y(λ),z(λ)) I0(λ) * -(c1b1(λ)+c2b2(λ)+c3b3(λ)) Newton’s Method e dλ

27. The Physical Model Bear’s Law

28. Outline • The Data • The Problem • Step 1: Finding the right three dimensional basis • Step 2: Inverting onto that basis • Step 3: Forming the model

29. The Two Stages color recording stage color reconstruction stage

30. The Two Stages color recording stage color reconstruction stage

31. The tri-pack structure of color film Exposing Color Light blue-sensitive layer yellow filter yellow filter yellow filter green-sensitive layer red-sensitive layer

32. Film development process

33. HD curve of film: density v.s. exposure

34. Physical effects motivates co-linear fit • Inter-layer effects are at play: cross-layer inhibition, cross-layer exposure and cross-layer absorption. • Possible diminishing cross layer exposure effect motivates a bilinear basis in the model. • The model is a least square fit of the data involving only co-linear bases:

36. Summary • Expose 1000 frames of color film to 1000 colors sampled from a RGB color cube. • Collect 1000 spectra by measuring the output color light of the 1000 frames of film. • Invert the second stage: • From the spectra data, extract three bases, i.e. the absorption functions of three dye layers using either NNMF or a geometrical approach involving SVD. • From a given intended color specified in XYZ color coordinates, solve for density vectors using Newton's method. • Invert the first stage: • Compute all 1000 concentration vectors of the 1000 spectra. • Build a functional relationship between the 1000 colors from a RGB cube and the 1000 density vectors using either interpolation or a physical model. • Solve this function for a set of RGB inputs that will give the density vector obtained from previous stage.