100 likes | 197 Vues
Questions. What is “f”? What is “x”? How large is “n”? What is the relation between “f” and “x”?. Convexity. Positive Definite. Iterative Algorithms. Starting at x 0 , generate a sequence of iterates { x k } and terminate when no more progress can be made or solution is obtained.
E N D
Questions • What is “f”? • What is “x”? • How large is “n”? • What is the relation between “f” and “x”?
Iterative Algorithms • Starting at x0, generate a sequence of iterates {xk} and terminate when no more progress can be made or solution is obtained. • From one iterate xk to the next iterate xk+1, the value of the objective must decrease. • To generate the next iterate xk+1, we need information from the current and previous iterates.
Two Types of Algorithms • Line Search (two-steps) • Find a direction to move into • Find an optimal length to travel • Trust Region • Construct a “model function” that approximates the objective function at the current iterate within a “trust region”. • Find the minimizer of the model function. • If the minimizer does not produce enough reduction in the objective function, reduce the approximation region.
Search Directions in Line Search Algorithms– negative gradient
Search Directions in Line Search Algorithms—Quasi-Newton Step
Search Directions in Line Search Algorithms—Conjugate-gradient Step