OQGRG: a Scatter Search Approach for Solving Constrained Non-Linear Global Optimization Problems

1. OQGRG: a Scatter Search Approach for Solving Constrained Non-Linear Global Optimization Problems Zsolt Ugray, The University of Texas at Austin, MSIS Dept. Leon Lasdon, The University of Texas at Austin , MSIS Dept. John Plummer, The University of Texas at Austin , MSIS Dept.

2. Heuristic Search • Problem: min f(x) subject to x in X • A heuristic defines a neighborhood N(x) for each x • Example:x=permutation of n jobs • For a pairwise exchange heuristic, N(x) = set of all permutations reachable from x by interchanging 2 components of x,n(n-1)/2 elements • Descent method: • choose x0 in X • make a move: find x1 in N(x0) such that f(x1)<f(x0) • if no such x1 exists, stop • else, replace x0 by x1 and repeat

3. Meta-heuristic Search • Meta-heuristic: a heuristic which guides another heuristic, prevents it from stopping at a local solution • Examples: genetic algorithms, tabu and scatter search, simulated annealing ADVANTAGES • Better chance of finding global optimum • Can handle continuous and discrete variables, nonsmooth functions • Can handle non-algebraic problem forms • routing (traveling salesman) • scheduling DISADVANTAGES • Sometimes no guarantee of success, no bounds • Often slow unless well-designed • Few off-the shelf codes available

4. Simulated Annealing • Choose an initial solution, x, an initial temperature, T, a lower limit on temperature, TLOW, and an inner iteration limit, L. • While (T>TLOW), do { • For k = 1,2,…,L, do{ • Make a random choice of an element, xnew in N(x) . • move_value=f(xnew)-f(x) • If move_value<= 0, (downhill move) set x = xnew • If move_value > 0 (uphill move) set x = with probability exp(-move_value/T). • end inner loop} • Reduce temperature according to an annealing schedule. An example is new T = c*T, where 0<c<1.} • End temperature loop

5. Genetic Algorithms • .Choose an initial population and evaluate the fitness of each individual • .While termination condition not satisfied do{ • 2.1 If crossover condition satisfied then • 2.1.1 {Select parent individuals • 2.1.2 Choose crossover parameters • 2.1.3 Perform crossover} • 2.2 If mutation condition satisfied then • 2.2.1 {choose mutation points • 2.2.2 perform mutation} • 2.3 evaluate fitness of offspring • 2.4 update population}

6. OQGRG-A Multi-start Search Method • Intended for nonlinearly constrained, smooth, non-convex NLP’s and MINLP’s • Combines the OptQuest Callable Library (a scatter search code) of Laguna, Kelly,Glover with a local NLP solver • Currently uses LSGRG2 , a sparse gradient-based GRG code, as the local NLP solver • Written in C

7. OptQuest (OQ) and OQ Callable Library • OQ is the Glover-Laguna-Kelley implementation of scatter search • Option within the Crystal Ball Excel add-in for Monte-Carlo simulation • Handles MINLP’s • OQ callable library: • set up problem • define variables, bounds, constraints, requirements • create initial population • for N iterations: • get trial solution • evaluate objective • put trial solution, objective back to database • regenerate population if necessary

8. Initializing and Updating the Population • Initial Population (size = psize) • include origin,user point,upper and lower bounds • generate stratified random points within bounds • choose a subset whose distance from previous points is maximized • Use population to generate new trial points, evaluate objective at each one • After all evaluated, population is the best psize solutions found so far

9. Scatter Search: trial solution generation Old Solutions New Solutions

12. OptQuest Difficulties • If feasible region is narrow (e.g. equality constraints) then very difficult for OQ to find a feasible solution • Often finds good solution fast, but requires many additional function evaluations to get high accuracy • Gradient-based NLP solvers (GRG,SQP,SLP) are much better at getting feasible and attaining high accuracy, but find “nearest” local optimum.

13. When to Start the LocalSolver • Local solver used is GRG (sparse version) • A GRG call solver is expensive-many function evaluations • Ideally: start once in the basin of attraction of each local solution • Don’t start from a trial point if • too close to a previously found local solution(distance filter) • exact penalty function value is too large(merit filter)

14. Distance Filter • Store max distance traveled to each local optimum • If the distance from a candidate starting point to any local opt is < 0.75*maxdist, don’t start GRG • This assumes regions of attraction are spherical, and maxdist is a good estimate of the radius

15. Merit Filter • Use L1 exact penalty , P(x,w), as merit function • P(x,w)=obj(x)+sum(i,wi*infeasi(x)) • wi > max abs Lagrange multiplier for constraint i over all local solutions • Don’t start from candidate points which have value for P > threshold • Initially threshold=best P value over all candidate points so far • If P > threshold more than 20 consecutive times, threshold <-- threshold +0.2*abs(threshold)

17. Handling Discrete Variables • Current strategy: • GRG treats discrete variables as fixed, at values provided by OQ • OQ varies both discrete and continuous variables • GRG solutions not returned to OQ • Apply distance and merit filters only if discrete variables are the same • Another strategy we will try • as above, but OQ varies only the discrete variables • equivalent to OQ solving the discrete problem, where GRG optimizes over the continuous ones

18. GAMS Interface • Motivation: Large Set of Test Problems Coded in GAMS by Floudas et.al., best known sol available • downloadable at http://titan.princeton.edu/TestProblems/ • Most from Chemical process design or operation • 144 problems • See book: Handbook of Test Problems in Local and Global Optimization, by Floudas et. al., Kluwer Academic Publishers, ISBN 0-7923-5801-5 • Uses GAMS C Language Library Routines • most small but some with over 100 variables, a few with over 1000. • Most arise from chemical engineering

20. Parameters of base case • Sort by max abs x value of best known solution • Solve all probs with max x between 10**k and 10**(k+1) in a group. Set bounds of + and - 10**(k+1) on all unbounded variables • Use 200 initial OQ iterations and 1000 total, but try again with (1000,5000) if fall short of best known sol. • OQ knows linear constraints, generates trial points which satisfy them

21. Floudas problems-results • Best known solution found or improved on in 118 of 121 problems • Best OQGRG solution found on first GRG call (itn 201) in 65 problems • Average GRG calls to find best OQGRG solution: bnd GRGcalls vars • 1 2.22 7.5 • 10 2.13 9.2 • 100 3.83 11.5 • 10,000 1.4 52.6 • 1000 7.8 69.1

22. Floudas results-group averages • Bnd total GRG fcn to best vars • 1 16.4 414 7.5 • 10 11.9 592 9.2 • 100 19.9 985 11.5 • 10,000 15.3 1356 52.6 • 1000 23.7 2538 69.1

23. Advantages and Disadvantages • Advantages • Finds good or optimal solutions rapidly • Can handle problems of general form with hundreds or thousands of continuous variables and/or constraints • Can handle MINLP’s with many discrete variables • strong ability to find a feasible solution (due to GRG) • works very differently from DICOPT or branch and bound • Disadvantages • No guarantee of finding a global solution • No way to tell if you have found a global solution

24. Future Work • Determine effects of varying important OQGRG options and parameters • basin and merit filter parameters • OQ treats continuous vars as discrete • OQ strategies to intensify or diversify • fractional change criterion for overall method • Comparison with LGO,MLSL of Fylstra, Random starts, DICOPT • Improvements to OQGRG algorithm motivated by test results