New Optimization Methods in Geometric Computing
New Optimization Methods in Geometric Computing. By Paul Zimmons CS 258 Fall 2000. Preface. Optimization techniques are optimization techniques whether applies to geometry or otherwise The topics I will cover have applications in both geometric and non-geometric computation.
New Optimization Methods in Geometric Computing
E N D
Presentation Transcript
New Optimization Methods in Geometric Computing By Paul Zimmons CS 258 Fall 2000
Preface • Optimization techniques are optimization techniques whether applies to geometry or otherwise • The topics I will cover have applications in both geometric and non-geometric computation
Basis for this Talk • The basis for this talk was the book New Ideas in Optimization by David Corne, Marco Dorigo, and Fred Glover (UK book)
Overview • Current Optimization • Ant Colony Methods • Differential Evolution • Immune System Methods • Swarm Method
What is Optimization? • “doing better”– NIIO • Different possible solutions and some measure of the quality of those solutions • Real problems are ‘hard’ • Can’t guarantee to find the best solution in acceptable amounts of time
Types of Optimization • Problem specific methods and very general • The methods discussed here are based on the idea of stochastic iterative search • General method
General Method • 1. Generate and evaluate an initial collection of candidate solutions S • 2. Produce and evaluate a collection of new candidate solution S’ by making random changes to selected members of S • 3. Replace some members of S with some of the members of S’ and then back to 2
Why Do This? • Simple to Implement • Very General • No function assumptions • Very Successful • Relatively little effort • Easily tailored to use problem specific knowledge
Ant Colony Optimization • Inspired by ant colonies • Equates the notion of a candidate solution with the route taken by an ant between two places • Ants leave a trail in which better solutions leave a stronger trail • The trail affects future ants
Ant Algorithm • ‘Multi-agent system’ in which each member acts like an ant or a super-ant • Good for routing, traveling salesman problem, etc. • Path Planning
History • Goss et al (1989) studied ants in a laboratory • Ant colony was given access to food linked to the nest by two bridges of different lengths • Ants must choose one way or the other • Ants converged on the smaller path
Experimental Setup Nest Food
More History • Autocatalytic effect • Indirect communication and differential path length • Ants deposit pheromones on the ground • When they reach a decision, they are biased by the amount of pheromone
Simple Ant Colony Opt. • G = (N,A) is a graph with n = |N| nodes • Source node f and destination node d and path length is given by # of hops • Each arc (i,j) has a variable ij which is called the artificial pheromone trail • Pheromone trails are read and written by ants • Intensity is proportional to utility
More S-ACO • At each node, information on the node or the outgoing arcs are used stochastically to decide the next move • For ant k at node i uses pheromone trails ij to compute the probability of moving to node jNi as the next move
More S-ACO • Ants deposit pheromone on the arcs they use • ij(t) ij(t) + • Autocatalysis and differential path length will again favor the smaller path • BUT this can converge inappropriately to a suboptimal path
Better Pheromone Model • Similar to real pheromones, artificial pheromone trails can evaporate • Decrease the pheromone trails exponentially • (1-) , (0,1] • Multiply for each iteration of the algorithm • Pheromone trails encode a long-term memory about the whole search process
S-ACO Problems • Increasing the complexity by introducing multiple shortest paths makes the algorithm less stable and effective • Parameter choices are more important • More sophisticated approaches are necessary • Ex. Increase the pheromone by the ‘goodness’ of the solution or give ants memories • Good for discrete optimization
S-ACO Notes • Good quality solutions can usually only emerge from collective interaction among the ants • Each ant makes use only of private information and local information about the node it is visiting • Ants do not adapt, they adaptively modify the way the problem is represented and perceived by other ants
When to use Ants • NP-hard problems where the search space is exponential in the dimension of the problem representation • Cases where the properties of the graph change over time concurrently with the optimization process • Problems where the computational architecture is spatially distributed (networks) because ants are inherently distributed, asynchronous, and multi-agent
Performance • Better than general heuristics like evolutionary computation or simulated annealing • Adding a local search makes the algorithm competitive with more sophisticated approaches • Does not beat the best TSP out there (Iterated Lin-Kernighan, 1997) • Better than most for network problems (cause of the changes)
Token Geometry App • Path planning with moving objects (??) • Sophisticated game engine where CPU/Graphics/Memory resources must be dynamically allocated • Camera path and choosing best reference images • Nodes are images and edges are possible choices
Differential Evolution • “Differential Evolution (DE) is a simple yet powerful population-based, direct-search algorithm for globally optimizing functions defined on totally ordered spaces, including especially functions with real-valued parameters.”
When to Use DE • Problem specific methods will usually do better when compared to general methods • But no deterministic algorithm does well with functional pathologies
What can go wrong? • Non-linearity • High dimensionality • Multi-modality (multiple local optima) • Parameter interaction • Constraints • Flatness • Non-Differentiability
Generic Evol. Alg. • Initialize population • Evaluate fitness • Do(until we win or are tired) • Mutate the population • Recombine the mutants to keep the population size down • Evaluate fitness • Select successors
DE Overview • Overview of algorithm • Encoding • Population and parameters • Mutation • Selection • Recombination
DE Algorithm • R1, r2, r3 {1,2,…,NP} • (randomly selection : r1r2r3I) • Jrand = int(randi[0,1)*D)+1 • For(j=1, j D; j=j+1) • If(randi[0,1] < CR j=jrand) • Uj,I,G+1=vj,I,G+1=xj,r3,G+F*(xj,r1,G-xj,r2,G) • Else uj,i,G+1 = xj,i,G
Evaluating a Vector Vi,G+1=Xr3,G+F*(Xr1,G- Xr2,G) = potential child F*(Xr1,G- Xr2,G) Ui,G+1 Ui,G+1 Xr3,G Ui,G+1 Xi,G Vi,G+1 Xr1,G- Xr2,G Xr1,G Xr2,G
Global Mutations • Say two ellipsoids • Usually a search will converge on one of them and ignore the other • In DE since all the object vectors are mutated by the same distribution, if object vectors occupy both ellipsoids then the distribution will contain a mixture • Inclusion of large scale mutations from vectors form both ellipses give some form of globally correlated mutation
DE Results • Telescope design with DE and ray tracer • DE chooses curvature, number of lenses, type of lenses • Mechanical problems • Discrete and continuous solutions used the same parameter choices • Function approximation, curve constraints, surface constraints, inverse problems
Immune System Method • I’m not a doctor • Immune system is a distributed system with several functional components in strategic locations
Artificial Immune Sys. • Emulate some facet of the immune system • Learning classifier • Recruitment mechanism • Similar to biological evolution • Feedback • Discover and maintain coverage of diverse patterns • Solutions are drawn from a library of separate gene segements (put the pieces together)
Applications • Virus detection • Scheduling • Antibodies evolve that recognize common patterns of job sequences • Those patterns are used to produce schedules • Optimizing structures (truss)
Graphics Engine as Immune System • Objects are the antibodies • Trying to dispose of them as quickly as possible • Keep a library of drawing routines • Make populations of drawing different parts or the whole object by these different routines • glVertex vs. Display List vs. glArrays vs glStream
GE as IS • When a routine is found to draw the object quickly, remember the kind of object • # vertices, vertex colors, textures • This is kind of like molecules designed for one type of antibody or generalizing to many types • Keep a population of these good solutions (to prevent explosion of drawing programs, dictates type) • Kind of like learned pattern recognition of geometric objects
Particle Swarm (1995) • Social interaction is crucial to human cognition • PS models the exploration of problem space by a population • Individual’s successes influence the searches of their peers
Moving in Hyperspace • For each iteration, an individual’s position is changed by adding a velocity v to its coordinates • Xi(t) = Xi(t-1) + Vi(t) • Vi(t)=Vi(t-1)+(Pi-Xi(t-1)) • If f(xi) < pbesti then pi = xi • is random
On To Groups • A neighborhood is defined for an individual • Can use whole population (gbest) or a subset (lbest) • An individual looks at the best solution so far pg • (pg-xi) is the distance from individual to best (vector of social influence)
Revised Movement • Vi(t)=Vi(t-1)+1(Pi-Xi(t-1))+2(PG-Xi(t-1)) • Xi(t) = Xi(t-1) + Vi(t) • vMax is max velocity else the system might explode • 1 , 2 usually less than 2 1(Pi-Xi(t-1)) 2(PG-Xi(t-1)) Vi(t-1)
Parameters • Dimensionality • Works better on high than low • Number of Individuals • Inverse relationship between the size of the population and the number of iterations • i should be < 2 • VMAX– might be able to eliminate
More Parameters • Using global best is simpler • Local neighborhood is slower but less susceptible to local minima • Similar ‘cultures’ within the population can emerge • Settle on a local optima that is ‘good enough’ • Inertia Weight • Controls the impact of previous velocities • Can balance global and local • Can decrease over time to focus the search
Applications • Generally used for optimizing high dimensional functions • Very good at making neural nets • Same type of geometric applications (function optimization, complex bounding surfaces…)
Conclusion • We have seen 4 different approaches to the same task of optimization using stochastic methods • Randomness can be smart • The theory behind these methods is still immature • The application to graphics is still open to debate
