1. Graphical optimization • Some problems are cheap to simulate or test. • Even if they are not, we may fit a surrogate that is cheap to evaluate. • Relying on optimization software to find the optimum is foolhardy. • It is better to thoroughly explore manually. • In two dimensions, graphical optimization is a good way to go. • For problems with more variables, it may still be good to identify two important variables and proceed graphically. • In higher dimensions, two dimensional cuts can help understand the relationship between local optima.

2. Example • Plot and estimate minimum of • In the range • Can do contour plot or mesh plot

3. Then zoom • Zoomingbyreducingtherangedoesnotchangetheestimate of the position (-1,1.5), butthevalueisnowbelow 0.8.

4. In higher dimensions • Often different solutions of optimization problems are obtained from multiple starting points or even from same starting point. • It is tempting to ascribe this to local optima. • Often it is the result of algorithmic or software failure. • Generate line connecting two solutions

5. Example • Consider function • And consider the two candidate local optima (0,0,0) and (0.94,0.94,0.94). • Line connecting first and second point indicates that they may be local optima.

6. Two-dimensional cut • We can add a third point (not necessarily a minimum) to visualize the function in the plane defined by these three points • Now corresponds to =1,=0, to =0, =1, and to =0, =0

7. Example Now consider the function Using numerical optimization we obtained =[0.25,0.25,-0.25], [0.25,0.75,0.25] We add =[0,0,0] Contour plot confirms that and are likely optima, but is not

8. 10 dimensional constrained example • Aerodynamic design of supersonic transport. • Objective function, take-off gross weight is a cheap and simple function of design variables. • Design variables define geometry of wing and fuselage. • Constraints include range, take-off and landing constraints, and maneuvrability constraints. • The constraints are all uni-modal functions, but in combination, they create a complex, non-convex feasible domain.

9. Two local optima and a third point • Simple constraints create non-convex design space. • Can that happen with linear constraints?

10. Problems • Find graphically the minimum of the function in Slide 5 in the plane x=1. • Find another minimum of the function in Slide 7 that is not along the line connecting the two given optima and plot the function in the triangle connecting the three optima. Limit the figure only to the triangle, by requiring .