Regularized Least-Squares

1. Regularized Least-Squares Regularized Least-Squares

2. Outline • Why regularization? • Truncated Singular Value Decomposition • Damped least-squares • Quadratic constraints Regularized Least-Squares

3. Why regularization? • We have seen that Regularized Least-Squares

4. Why regularization? • We have seen that • But what happens if the system is almost dependent? • The solution becomes very sensitive to the data • Poor conditioning Regularized Least-Squares

5. The 1-dimensional case • The 1-dimensional normal equation Regularized Least-Squares

6. The 1-dimensional case • The 1-dimensional normal equation Regularized Least-Squares

7. The 1-dimensional case • The 1-dimensional normal equation Regularized Least-Squares

8. Why regularization • Contradiction between data and model Regularized Least-Squares

9. A more interesting example:scattered data interpolation Regularized Least-Squares

10. “True” curve Regularized Least-Squares

11. Radial basis functions Regularized Least-Squares

12. Radial basis functions Regularized Least-Squares

13. Rbf are popular • Modeling • J. C. Carr, R. K. Beatson, J. B. Cherrie, T. J. Mitchell,W. R. Fright, B. C. McCallum, and T. R. Evans. Reconstruction and representation of 3d objects with radial basis functions. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, pages 67–76, August 2001. • G. Turk and J. F. O’Brien. Modelling with implicit surfaces that interpolate. ACM Transactions on Graphics, 21(4):855–873, October 2002. • Animation • J. Noh and U. Neumann. Expression cloning. In Proceedings of ACMSIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, pages 277–288, August 2001. • F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. H. Salesin. Synthesizing realistic facial expressions from photographs. In Proceedings of SIGGRAPH 98, Computer Graphics Proceedings, Annual Conference Series, pages 75–84, July 1998. Regularized Least-Squares

14. Radial basis functions • At every point Regularized Least-Squares

15. Radial basis functions • At every point • Solve the least-squares problem Regularized Least-Squares

16. Radial basis functions • At every point • Solve the least-squares problem Regularized Least-Squares

17. Rbf results Regularized Least-Squares

18. pi0 close topi1 Regularized Least-Squares

19. Radial basis functions • At every point • Solve the least-squares problem Regularized Least-Squares

20. Radial basis functions • At every point • Solve the least-squares problem • If pi0 close topi1, A is near singular Regularized Least-Squares

21. pi0 close topi1 Regularized Least-Squares

22. pi0 close topi1 Regularized Least-Squares

23. Rbf results with noise Regularized Least-Squares

24. Rbf results with noise Regularized Least-Squares

25. The Singular Value Decomposition • Every matrix A (nxm) can be decomposed into: • where • U is an nxn orthogonal matrix • V is an mxm orthogonal matrix • D is an nxm diagonal matrix Regularized Least-Squares

26. The Singular Value Decomposition • Every matrix A (nxm) can be decomposed into: • where • U is an nxn orthogonal matrix • V is an mxm orthogonal matrix • D is an nxm diagonal matrix Regularized Least-Squares

27. Geometric interpretation Regularized Least-Squares

28. Solving with the SVD Regularized Least-Squares

29. Solving with the SVD Regularized Least-Squares

30. Solving with the SVD Regularized Least-Squares

31. Solving with the SVD Regularized Least-Squares

32. Solving with the SVD Regularized Least-Squares

33. A is nearly rank defficient Regularized Least-Squares

34. A is nearly rank defficient Regularized Least-Squares

35. A is nearly rank defficient Regularized Least-Squares

36. A is nearly rank defficient • A is nearly rank defficient =>some of the are close to 0 Regularized Least-Squares

37. A is nearly rank defficient • A is nearly rank defficient =>some of the are close to 0 =>some of the are close to Regularized Least-Squares

38. A is nearly rank defficient • A is nearly rank defficient =>some of the are close to 0 =>some of the are close to • Problem with Regularized Least-Squares

39. A is nearly rank defficient • A is nearly rank defficient =>some of the are close to 0 =>some of the are close to • Problem with • Truncate the SVD Regularized Least-Squares

40. pi0 close topi1 Regularized Least-Squares

41. Rbf fit with truncated SVD Regularized Least-Squares

42. Rbf results with noise Regularized Least-Squares

43. Rbf fit with truncated SVD Regularized Least-Squares

44. Choosing cutoff value k • The first k such as Regularized Least-Squares

45. Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph • Skinning Regularized Least-Squares

46. Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph • Skinning ? Regularized Least-Squares

47. Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph • Skinning Regularized Least-Squares

48. Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph • Skinning • Inverse skinning • Let be a set of pairs of geometry and skeleton configurations Regularized Least-Squares

49. Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph • Skinning • Inverse skinning • Let be a set of pairs of geometry and skeleton configurations Regularized Least-Squares

50. Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph • Skinning • Inverse skinning • Let be a set of pairs of geometry and skeleton configurations Regularized Least-Squares