1 / 39

Methods For Nonlinear Least-Square Problems

This overview discusses methods for addressing nonlinear least squares problems, widely applicable in graphics, machine learning, and robotics. The document covers techniques like gradient descent, Newton's method, Gauss-Newton, and Levenberg-Marquardt. Emphasis is placed on the challenges of finding global versus local minima, assumptions for smoothness in cost functions, and stopping criteria for optimization processes. Practical implementations and the role of regularization terms to enhance solution uniqueness are also highlighted, providing a foundation for various applications like inverse kinematics and data fitting.

krystal
Télécharger la présentation

Methods For Nonlinear Least-Square Problems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Methods For Nonlinear Least-Square Problems Jinxiang Chai

  2. Applications • Inverse kinematics • Physically-based animation • Data-driven motion synthesis • Many other problems in graphics, vision, machine learning, robotics, etc.

  3. Where , i=1,…,m are given functions, and m>=n Problem Definition Most optimization problem can be formulated as a nonlinear least squares problem

  4. Data Fitting

  5. Data Fitting

  6. Base Inverse Kinematics Find the joint angles θ that minimizes the distance between the character position and user specified position θ2 θ2 l2 l1 θ1 C=(c1,c2) (0,0)

  7. Global Minimum vs. Local Minimum • Finding the global minimum for nonlinear functions is very hard • Finding the local minimum is much easier

  8. Assumptions • The cost function F is differentiable and so smooth that the following Taylor expansion is valid,

  9. Gradient Descent Objective function: Which direction is optimal?

  10. Gradient Descent Which direction is optimal?

  11. Gradient Descent A first-order optimization algorithm. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient of the function at the current point.

  12. Gradient Descent • Initialize k=0, choose x0 • While k<kmax

  13. Newton’s Method • Quadratic approximation • What’s the minimum solution of the quadratic approximation

  14. Newton’s Method • High dimensional case: • What’s the optimal direction?

  15. Newton’s Method • Initialize k=0, choose x0 • While k<kmax

  16. Newton’s Method • Finding the inverse of the Hessian matrix is often expensive • Approximation methods are often used - conjugate gradient method - quasi-newton method

  17. Comparison • Newton’s method vs. Gradient descent

  18. Gauss-Newton Methods • Often used to solve non-linear least squares problems. Define We have

  19. Gauss-Newton Method • In general, we want to minimize a sum of squared function values

  20. Gauss-Newton Method • In general, we want to minimize a sum of squared function values • Unlike Newton’s method, second derivatives are not required.

  21. Gauss-Newton Method • In general, we want to minimize a sum of squared function values

  22. Gauss-Newton Method • In general, we want to minimize a sum of squared function values Quadratic function

  23. Gauss-Newton Method • In general, we want to minimize a sum of squared function values Quadratic function

  24. Gauss-Newton Method • In general, we want to minimize a sum of squared function values Quadratic function

  25. Gauss-Newton Method • In general, we want to minimize a sum of squared function values Quadratic function

  26. Gauss-Newton Method • Initialize k=0, choose x0 • While k<kmax

  27. Gauss-Newton Method • In general, we want to minimize a sum of squared function values Any Problem? Quadratic function

  28. Gauss-Newton Method • In general, we want to minimize a sum of squared function values Any Problem? Quadratic function

  29. Gauss-Newton Method • In general, we want to minimize a sum of squared function values Any Problem? Quadratic function Solution might not be unique!

  30. Gauss-Newton Method • In general, we want to minimize a sum of squared function values Any Problem? Quadratic function Add regularization term!

  31. Levenberg-Marquardt Method • In general, we want to minimize a sum of squared function values Any Problem?

  32. Levenberg-Marquardt Method • In general, we want to minimize a sum of squared function values Any Problem? Quadratic function Add regularization term!

  33. Levenberg-Marquardt Method • In general, we want to minimize a sum of squared function values Any Problem? Quadratic function Add regularization term!

  34. Levenberg-Marquardt Method • Initialize k=0, choose x0 • While k<kmax

  35. Stopping Criteria • Criterion 1: reach the number of iteration specified by the user K>kmax

  36. Stopping Criteria • Criterion 1: reach the number of iteration specified by the user • Criterion 2: when the current function value is smaller than a user-specified threshold K>kmax F(xk)<σuser

  37. Stopping Criteria • Criterion 1: reach the number of iteration specified by the user • Criterion 2: when the current function value is smaller than a user-specified threshold • Criterion 3: when the change of function value is smaller than a user specified threshold K>kmax F(xk)<σuser ||F(xk)-F(xk-1)||<εuser

  38. Levmar Library • Implementation of the Levenberg-Marquardt algorithm • http://www.ics.forth.gr/~lourakis/levmar/

  39. Constrained Nonlinear Optimization • Finding the minimum value while satisfying some constraints

More Related