Evolutionary Computation and beyond

1. Evolutionary Computationand beyond Elhanan Borenstein May. 2004 www.cs.tau.ac.il/~borens

2. Evolution vs. Learning • Process: Random (?)Directional (?) • Level: PopulationIndividual • Entity: GenotypePhenotype • Time Scale: GenerationsLifetime • Operators: Mutation, CrossoverGradient Descent • Search: GlobalLocal • Problems: Slow / RandomInitial conditions Evolution Learning

3. Talk Outline • Evolution – An Inevitable Process • Genetic Algorithms • Motivation • Techniques & Pseudo Code • Evolution as a Stochastic Search Process • Fitness Landscapes • Enhanced Genetic Algorithms • Artificial Life and Evolutionary Autonomous Agents • Life as it could be… • Definition & Motivation • Putting is All Together • Experiments Demo

4. Evolution – An inevitable Process

5. Evolution – An inevitable Process • Any system, that embeds the following three processes, will inevitably undergo evolution: • Reproduction (inheritance) • Variation • Selection • A process of selective reproduction and substitution in a population of individuals. Initial Population NextGeneration Selection Reproduction Variation

6. Domestic Animals Breeding • Life on Earth • Strategies (Financial…) • Evolutionary Game Theory • Ideas, Beliefs, knowledge • culture • memes Evolution – An inevitable Process Just a few examples may include:

7. Evolutionary Computation • If it works so well in nature… why not using it to solve computational problems… • Many problems in computer science (and mathematics) have the following characteristics: • We are searching for the optimal solution… • The number of possible solutions (search space) is huge. • We donot have an analytical method to reach the optimal solution… • but… given a solution, we can evaluateits quality. • Evolution is a search process !!!

8. For Example …

9. Genetic Algorithms • Each possible solution will be represented (coded) as a string of characters (genome) For example: LPMRSBN (London Paris  Madrid  Rome  Lisbon  Berlin  Monaco) • Generate a random collection (population) of solutions: LPMRSBN, MSBRNLP, NBSMPLR, MRLNSBP, … • Evaluate the quality (fitness) of each solution • Generate the next generation of solutions: • Delete (kill) all bad solutions. • Duplicate (reproduce) a number of copies of each remaining (surviving) solution. • Insert random modifications (mutations) to the newly create solutions. • Repeat (life goes on) until the optimal solution was found.

10. Genetic Algorithm – Pseudo Code Initialize population: P  Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P  Ps Evaluated: For each h in P, compute Fitness(h) return

11. Genetic Algorithm – Pseudo Code Initialize population: P  Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P  Ps Evaluated: For each h in P, compute Fitness(h) return Population • Genotype vs. Phenotype • Representation • Coding • Direct • Developmental • Stochastic 2.31 1.02 -0.28 -7.93 6.32 1.87 -1.72 2.46 0.45 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 0 1 0 0 1 1 1 1 1 1 1 0

12. Genetic Algorithm – Pseudo Code Initialize population: P  Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P  Ps Evaluated: For each h in P, compute Fitness(h) return Evaluation – The fitness function • Reliable assessment • Average fitness • Scaling (rank-based, linear, quadratic, Baltzmann) • Tournament

13. Genetic Algorithm – Pseudo Code Initialize population: P  Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P  Ps Evaluated: For each h in P, compute Fitness(h) return Termination Criteria • Maximal Fitness • Convergence of the population • No improvement • Maximal # of generations

14. Genetic Algorithm – Pseudo Code Initialize population: P  Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P  Ps Evaluated: For each h in P, compute Fitness(h) return Selection (survival of the fittest) • A stochastic process !!! • Roulette wheel selection • Tournament • Competition • Elitism

15. Genetic Algorithm – Pseudo Code Initialize population: P  Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P  Ps Evaluated: For each h in P, compute Fitness(h) return Reproduction • Sexual vs. Asexual • Haploid vs. Diploid

16. Genetic Algorithm – Pseudo Code Initialize population: P  Generate p hypotheses at random Evaluate: For each h in P, compute Fitness(h) While [max Fitness(h) < threshold] do Select: Probabilistically select p/2 pairs of hypotheses from P. Pr(hi) = Fitness(hi) / Σ Fitness(hi) Reproduction: Produce two offspring for each pair. Add to Ps With probability pc apply crossover operator mutate: For each gene, with probability pm, apply mutation operator Update: P  Ps Evaluated: For each h in P, compute Fitness(h) return Genetic Operators - Variation 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 0 1 0 0 1 1 1 1 1 1 1 0 • Mutation: • Crossover (one point/two points): 0 1 1 1 0 1 1 0 1 1 1 1 0 0 1 0 1 0 0 1 1 1 1 1 1 1 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 0 1 0 0 1 1 1 1 1 1 1 0 1 1 0 1 0 0 1 1 0 0 0 0 1 0 0 1 0 1 1 0 0 1 1 1 0 0 1 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 0 1 0 1 0 0 1 1 1 0 0 1 1 1 0 1 0 0 1 1 0 0 0 0 1 0 0 1 0 1 0 1 1 1 1 1 1 1 0

17. Genetic Algorithm – Pseudo Code Determining which of the numerous extensions and variations of Genetic Algorithms matches a specific task is still very much a matter of art and experience.

18. Genotype/Phenotype Coding • A few notes to bear in mind about genetics and evolutionary computation: • Most evolutionary computation applications, implement some sort of distinction between genotypes and phenotypes, in an analogous manner to biological evolution. • While the genotype is the one that transferred across generations, it is the phenotype that goes trough the selection process. • We work mostly in a strict Darwinian framework, where lifetime adaptations are not inherited. • The Genotype to Phenotype coding may have a dramatic effect on the evolutionary search !!!

19. Evolution – A Stochastic Search The Fitness Landscape Optimal Solution Fitness Value Gene A Configuration Gene B Configuration

20. Evolution – A Stochastic Search Illustration – Initial Population Fitness Value Gene A Configuration Gene B Configuration

21. Evolution – A Stochastic Search Illustration – Natural Selection Fitness Value Gene A Configuration Gene B Configuration

22. Evolution – A Stochastic Search Illustration – Reproduction & Variation Fitness Value Gene A Configuration Gene B Configuration

23. Evolution – A Stochastic Search Illustration – The Next Generation Fitness Value Gene A Configuration Gene B Configuration

24. Incremental • Ruggedness • Local optima • Size of basins of attraction B C D A Fitness Fitness Fitness Fitness Genetic Configuration Genetic Configuration Genetic Configuration Genetic Configuration Evolution – A Stochastic Search • Which properties of the fitness landscape affect the evolutionary search? • Size!!!

25. Fitness Genetic Configuration Evolution – A Stochastic Search • Can we control these properties? • Yes (to some degree) !!! • Genetic Representation • Dimensionality • Neighborhood function • Building blocks • Stochastic encoding • Fitness Function • Smooth • Monotone / incremental • Resolution • Much is still dependent on the Task !!!

26. Enhancing Genetic Algorithms • How can we push the evolutionary process forward? • Taking another lesson from nature • Co-evolution: • host-parasite / predator-pray • Red queen effect (“You have to run faster and faster just to stay in the same place!”). • Incremental evolution • Lifetime learning: • Darwinian framework vs Lamarckian framework. • Social Learning • Cultural evolution

27. Life As It Could BeA Short Introduction to Artificial LifeandEvolutionaryAutonomous Agents

28. A Multidisciplinary Research Biology Computer Science ArtificialLife Neuroscience Mathematics Psychology Animal Behavior Physics Medical Research Robotics

29. Artificial Life (ALife) - Definition “Artificial Life (ALife) is the study of man-made systems that exhibit behaviors characteristic of natural systems. It complements the traditional biological science concerned with the analysis of living organisms by attempting to synthesize lifelike behaviors within computers and other artificial media. …by extending the empirical foundation upon which biology is based beyond the carbon-chain life that has evolved on Earth… ALife can contribute to theoretical biology by locating life-as-we-know-it within the large picture of life-as-it-could-be.” Chris Langton

30. Artificial Life (ALife) - Motivation ALife motivation is twofold: ComputationalCreate better artificial systems that are inspired by biological systems BiologicalGain a better understanding of biological systems

31. Modeling Complex Systems • What advantages are there in artificial models? • Full access to all data (all vs. tens, all vs. fossil record) • Time resolution (milliseconds vs. years) • Intervention (all and any vs. few) • But…will our findings in such models (systems) apply to real natural systems? • How accurate should our model be? • Which parameters should be included? על הדיוק במדע: ... באימפריה זו הגיעה אומנות כתיבת-המפות למידת שלמות כזו, שמפתו של מחוז אחד השתרעה על פני עיר שלמה ואילו מפתה של האימפריה כולה – על פני המחוז כולו. ברבות הימים לא סיפקו עוד מפות-ענק אלה את התושבים וועדות כותבי המפות החלו להכין את מפת האימפריה שגודלה כגודל האימפריה עצמה, והיא זהה עמה בכל נקודה ונקודה... חורחה לואיס בורחס

32. Output Input Artificial Neural Networks (ANN) • Artificial Neural Network: • Such neural network can be thought of as a “black-box” that compute some function of the input. Different assignment of the synaptic weights will result in a different function. w6 w1 w21 w2 w3 w8

33. w6 w1 w21 w2 w3 w8 Autonomous Agents • Such artificial creatures may be either real autonomous robots or simulated autonomous agents.

34. Developing the Agent’s Brain • So…we have manage to construct a good model for a neural-network brain, controlling an artificial agent… • But…we are back to square one!!! How will we develop the agent’s brain and get “working” and interesting brain models? • There are two approaches we can take (both inspired again by biological process). • Lifetime Learning • “Genetic” Evolution

35. Evolutionary Autonomous Agents 100 brains (random genome) Create a first random generation Evaluate each “brain” in the (virtual) world Insert the “brain” into an agent and let it operate in the world fitness value Allow the successful agent to reproduce mutations,crossovers A new generation is created

36. Evolutionary Autonomous Agents Agents developed by evolution and imitation>>

37. Evolutionary Autonomous Agents Agent in a fire-fighting contest

38. Evolutionary Autonomous Agents • One of the main objectives of our study is to develop methodologies for neuroscience research. • We thus need to develop more complex / interesting brains (neural networks) !!! Complex Task

39. Evolutionary Autonomous Agents

40. Evolutionary Autonomous Agents • How would you solve this task?

41. Evolutionary Autonomous Agents S10: a forager-explorer agent Generation 200>> S10: a forager-explorer agent Generation 900>> Emergent Phenomena !!!

42. Evolutionary Autonomous Agents • Analyzing the emergent neural network: • Much work is done to put forward analytical methods that could be applied to biological neuroscience research !! S10: a forager-explorer agent Command neuron clamp to 0>>

43. Summary Questions? • Evolutionary computation is a powerful tool that can be used to “solve” a wide range of tasks. • It is still hard to provide rules of thumb to match the numerous extensions and variations of Genetic Algorithms to a specific task. • Artificial Life and biological system modeling (in-silico biology) is a growing and exiting field that can shed new light on natural biological systems.