1. Multi-Objective Non-linear Optimization via Parameterization and Inverse Function Approximation University of Regina Industrial Systems Engineering M.A.Sc. Thesis Defense May 23, 2003 Mariano Arriaga Marín

2. Thesis Contributions • Novel technique for attaining the global solution of nonlinear optimization problems. • Novel technique for multi-objective nonlinear optimization (MONLO). • Artificial Neural Networks (ANN) Implementation • Methods tested in: • Highly nonlinear optimization problems, • MONLO problems, and • Practical scheduling problem.

3. Current Global Optimization Techniques • Common Techniques • Multistart • Clustering Method • Genetic Algorithms • Simulated Annealing • Tabu Search

4. Multiple Objective Optimization • Current MONLO procedure: • Divide the problem in two parts • Multi-Objective  Single Objective Problem • Solve with a Nonlinear Optimization Technique

5. Multi-Objective  Single Objective • Common Techniques • Weighting Method • E-Constraint • Interactive Surrogate Worth Trade-Off Solution • Lexicographic Ordering • Goal Programming • Problems • Include extra parameters which might be difficult to determine their value. • Determining their value gets more difficult as the number of objective functions increases

6. Proposed Optimization Algorithm • MinF(x) = {f1(x),…,fm(x)} • where xÎ Ân fi(x)Î Â; i = 1,…,m • Optimization of: • Non-Linear functions • Multi-objective • Avoid local minima and inflection points

7. General Idea • Set an initial value for x and calculate f(x) • Decrease the value of the function via a parameter • Calculate corresponding f -1(x) • Note: The algorithm does not necessarily follow the function x0

8. General Idea • When the algorithm reaches a local minima • it looks for a lower value • if this value exists, the algorithm “jumps” to it and continues the process • This process continues until the algorithm reaches the global minimum. x0 xf

9. Inverse Function Approximation • Inverse Function Approximation • Continuous functions • Full Theoretical Justification1 1 Mayorga R.V. and Carrera J., (2002), “A Radial Basis Function Network Approach for the Computation of Inverse Time Variant Functions”, IASTED Int. Conf. on Artificial Intelligence & Soft Computing, Banff, Canada. (To appear in the International Journal of Neural Systems).

10. Global Optimization Example • Consider the function:

11. Initial Model – Part 1 • Initial point in “front” side of curve (1) • Gets out of two local minima (2 & 3) • Converges to the global minimum (4) • v=0 and dZ-1=0

12. Initial Model – Part 2 • Initial point in “back” side of curve • Gets stuck in an inflection point • Does not get to the global minimum • v=0 and dZ-1=0

13. Model with vector v and dZ-1 • Initial point in “back” side of curve • Goes around the curve (null space vector) • Converges to the global minimum

14. Artificial Neural Networks Model • Initial point in “back” side of curve • Calculate J(x) and v with ANNs • Follows almost the same trajectory as previous model • Converges to the global minimum

15. The Griewank Function - Example • Consider the function:

16. Model with dZ-1 & v Initial Model dZ-1=0 & v=0 Model Using ANN Griewank Function Optimization

17. Multi-Objective Nonlinear Example • I-Beam Design Problem2 • Determine best trade-off dimensions • Minimize conflicting objectives • Cross-sectional Area • Static Deflection 2 Osyczka, A., (1984), Multicriterion Optimization in Engineering with FORTRAN programs. Ellis Horwood Limited.

18. What if both objectives are solved separately? • ↓Cross-Sectional Area ↑Static Deflection • ↓ Static Deflection ↑ Cross-Sectional Area

19. I-Beam Results • ―Feasible Solutions • ― Strong Pareto Solutions • ― Weak Pareto Solutions

20. I-Beam Results • Result • Proposed approach achieves very similar results to state-of-the-art Genetic Algorithms (GA) • Gives a diverse set of strong Pareto solutions • The result of the ANN implementations varies by 0.88% • Computational Time3 • If compared to a standard floating point GA4, the computational time decreases in 83% • From 15.2 sec to 2.56 sec 3 Experiments performed in a Sun Ultra 4 Digital Computer. GA: 100 individuals and 50 generations. 4 Passino, K., (1998), Genetic Algorithms Code, September 21st, http://eewww.eng.ohio-state.edu/~passino/ICbook/ic_code.html (accessed February 2003).

21. Multi-Objective Optimization:Just-In-Time Scheduling Problem • Consider 5 products manufactured in 2 production lines • Minimize: • Cost • Line Unbalance • Plant Unbalance • Variables: • Production Rate • Level Loading • Production Time • Production Constraints

22. Scheduling Problem – Optimization Results • Minimize: • Cost

23. Multi-Objective Optimization Example • Minimize: • Cost • Line unbalance • Production Rate variance / line

24. Multi-Objective Optimization Example • Minimize: • Cost • Line unbalance • Production Rate variance / line • Plant unbalance • Distribute production in both production lines

25. Conclusion • Novel global optimization method • It avoids local minima and inflection points • The algorithm leads to convexities via a null space vector v • It can also be used for constraint nonlinear optimization

26. Conclusion (cont.) • Novel MONLO deterministic method • Starts from a single point instead of a population • Computational Time • For the I-Beam example, the computational time is 83% less than Genetic Algorithms • The implementation of ANN reduces the number of calculations to compute the Inverse Function • For the scheduling example, the ANN implementation reduces computational time by 70%

27. Publications • 3rd ANIROB/IEEE-RAS International Symposium of Robotics and Automation, Toluca, Mexico. Sept 1-4, 2002 • Three journal papers already submitted: • Journal of Engineering Applications of Artificial Intelligence • International Journal of Neural Systems • Journal of Intelligent Manufacturing • One paper to be published as a Chapter in a book on Intelligent Systems. Editor: Dr. Alexander M. Meystel