Créer une présentation
Télécharger la présentation

Télécharger la présentation
## Dynamic Optimisation

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Dynamic Optimisation**• So far have looked at the use of EA’s as function optimisers for stationary functions, i.e. where the fitness landscape remains static w.r.t. time. • Many interesting applications are non-stationary: given a potential solution x and a fitness function f, then f(x, t1) and f(x,t2) may differ.**Types of non-stationary function**• Cobb (1990) defines two ways of classifying non-stationary functions • Switching vs. Continuous • based on time-scale of change w.r.t. rate of evaluation • Markovian vs. State Dependant • f(x,t) comes fromasymptotic distribution or is strictly a function of time.**Strategies Used**• Cobb(1990) identifies two different classes of strategies for NS environments 1: Expand the memory of the EA. • e.g. Goldberg & Smith (87), Smith (88) , Dasgupta & MacGregor(1992). • good solution if there are a fixed number of states in MSE 2: (Adaptively) expand variation in the population**Experiments with Std GA**• Cobb (1990) observed the behaviour of a standard GA on a parabolic function with a the optima moving sinusoidally in space. • Offline performance decreases as rate of change increases for all p_m • As rate of change increases the value of p_m which gives optimal offline performance increases • As problem difficulty increased, rate of change that GA could track decreased**Triggered Hyper Mutation(HM)**• Cobb proposed adaptive mutation mechanism with two mutation rates 0.5/0.001 • Higher rate triggered by drop in fitness of best in generation • Tracked optima far better than SGA, especially when the time-optima were spatially close**Pitfalls with HM**• Grefenstette (1992) noted that under certain conditions HM might never get triggered. • Function composed of 14 sinusoidal hills • initial optima 5.0, subsequent optima with height 10.0, appears after 20 generations, and moves every 20 generations • SGA goes to 5.0, rises to 10.0 when optima close but does not track • HM, goes to 5, only tracks 10.0 after optima moves close to original peak, then triggering works**Random Immigrants (RI)**• RI strategy proposed, replace proportion of population with randomly created members at every generation • 30% replacement gives best off-line tracking • too high and it can’t converge between changes • however off-line performance decreases with proportion replaced**Comparison of techniques**• Cobb & Grefenstette (1993) compared HM vs. RI vs SGA (with high mutation rate) • Noted difference in nature of mutation • SGA uniform in pop and time • HM uniform in pop, not in time • RI uniform in time, not in pop. • NB this (later) version of HM triggered by decrease in running mean over 5 generations of best in pop.**Comparison Experiments**• 2 landscapes, A as per picture, B composed of various shaped hills. 32 bits each • 3 types of change: • linear motion in A, moving 1 step along an axis every 2/5 generations • randomly shifting the optima in A every 20 generations • swap between A and B every 2/20 generations**Comparison Conclusions**• ran lots of experiments to find good settings for each algorithm with each problem • SGA: • p_m 0.1 was reasonably good at the translation tasks, but gave v. poor online performance. • couldn’t track moving optimum or oscillation • p_m needs to be matched to degree of change**Comparison Conclusions cont..**• HM • High variance in performance noted • HM rate needed tuning a lot to problem • Much better at tracking sudden changes than SGA, and gave better online performance than SGA or RI`when rate of change was slow enough to allow lower rate of mutation**Comparison Conclusions cont.**• RI • not very good at tracking linear movement • best at the oscillating task • they hypothesise ( but don’t show) that this is because it allows the preservation of niches • poor performance on stationary problems and slow movements**Vavak 1993->**• Concentrated on on-line performance, for control of industrial systems • Compared generational vs. steady state GA’s on a 1 dimensional moving problem • SSGA better from random or converged population. • entropy reacts faster and reaches lower value for SSGA**Design choices for SSEAs**• Smith & Vavak showed choice of replacement strategy was critical • Best algorithms • Use elitism, but • Systematically re-evaluate all points in population • Overall a “conservative” delete-oldest policy was found to work best,**Self-Adaptive Approaches**• Back et al showed that it is not necessary to use a trigger if you have a self-adaptive system. • ES was able to track a moving optima reliably**Co-evolution**• Sometimes the fitness of a solution may not be externally defined, e.g.: • Individuals compete against others from same population • E.g. evolving game playing strategies • Individuals must be evaluated in context of solutions from other populations representing • Other parts of decomposed problem • Competing species • Obviously in biology the fitness in our “adaptive landscape” metaphor is a function of the other competing species rather than being fixed.**Some terminology**• Mutualism or Symbiosis: • The beneficial co-adaptation of different species, may become co-located (e.g. gut flora) • In EC known as Co-operative Coevolution: e.g. lots of work by Bull, Potter and DeJong on function decomposition • Predation or Parasitism: • Antagonistic co-adaption e.g. fox/rabbit • Also known as competitive coevolution: e.g. lots of work on test-soluton problems, game-playing etc. • Note that both types of coevolution can occur in single population or multi-population forms**How to evaluate fitness?**• Single population: • Test each solution against every other (or a randomly selected sample if pop. is large) • Multiple populations • Need to decide a pairing strategy e.g.: • Evolve populations in turn, use best of other pops. • Pair with randomly chosen members of other pops • Lots of work by Bull, Husbands verdict still out • Neat solution by Husbands uses cellular model with one of each pop on each grid point • Paredis uses “Life-Time-Fitness-Evalaution” – mean of last twenty encounters with another species**Some examples**• Hillis 1993: • Co-evolved sorting networks with “difficult” test sequences - found network of smallest size known at the time • Axelrod: • Organised competitions for Iterated Prisoners Dilemma • Showed he could evolve winning strategies • Paredis: • Coevolved solutions and their representations to remove operator bias • Also used coevolution for constraint satisfaction problems**This sounds great, why don’t we all do it ….**• Desirable behaviour in competitive evolution is known as “arms races” • Red queen effect: run faster and faster just to keep up • But problems arise: • Cycling • Mediocre stable states • disengagement**What does the future hold**• Several groups actively researching solutions to these problems • LOTS of potential applications: • Agent based systems for e-commerce, trading, game-playing etc. • Investigating difficult problems in abstract fields eg maths • Evolving team behaviour • Optimisation with (automatic? ) problem decomposition