1 / 22

CAP6938 Neuroevolution and Developmental Encoding NeuroEvolution of Augmenting Topologies (NEAT)

CAP6938 Neuroevolution and Developmental Encoding NeuroEvolution of Augmenting Topologies (NEAT). Dr. Kenneth Stanley September 25, 2006. TWEANN Problems Reminder. Competing conventions problem Topology matching problem Initial population topology randomization Defective starter genomes

denzel
Télécharger la présentation

CAP6938 Neuroevolution and Developmental Encoding NeuroEvolution of Augmenting Topologies (NEAT)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CAP6938Neuroevolution and Developmental EncodingNeuroEvolution of Augmenting Topologies(NEAT) Dr. Kenneth Stanley September 25, 2006

  2. TWEANN Problems Reminder • Competing conventions problem • Topology matching problem • Initial population topology randomization • Defective starter genomes • Unnecessarily high-dimensional search space • Loss of innovative structures • More complex can’t compete in the short run • Need to protect innovation • NEAT directly addresses these challenges

  3. Solutions: NEAT • Historical markings match up different structures • Speciation • Keeps incompatible networks apart • Protects innovation • Incremental growth from minimal structure, i.e. complexification • Avoids searching in unnecessarily high-d space • Makes finding high-d solutions possible

  4. Genetic Encoding in NEAT

  5. Topological Innovation

  6. Link Weight Mutation • A random number is added or subtracted from the current weight/parameter • The number can be chosen from uniform, Gaussian (normal) or other distributions • Continuous parameters work best if capped • The probability of mutating a particular gene may be low or high, and is separate from the magnitude added • Probabilities and mutation magnitudes have a significant effect

  7. Link Weight Mutation in NEAT C++ randnum=randposneg()*randfloat()*power; if (mut_type==GAUSSIAN) { randchoice=randfloat(); if (randchoice>gausspoint) ((*curgene)->lnk)->weight+=randnum; else if (randchoice>coldgausspoint) ((*curgene)->lnk)->weight=randnum; } else if (mut_type==COLDGAUSSIAN) ((*curgene)->lnk)->weight=randnum; //Cap the weights at 3.0 if (((*curgene)->lnk)->weight > 3.0) ((*curgene)->lnk)->weight = 3.0; else if (((*curgene)->lnk)->weight < -3.0) ((*curgene)->lnk)->weight = -3.0;

  8. Topology Matching Problem • Problem arises from adding new genes • Same gene may be in different positions • Different genes may be in same positions

  9. Biological Motivation • New genes appeared over biological evolution as well • Nature has a solution to still know which is which • Process of aligning and matching genes is called synapsis • Uses homology to align genes: “. . .Crossing over thus generates homologous recombination; that is, it occurs between 2 regions of DNA containing identical or nearly identical sequences.” (Watson et al. 1987)

  10. Artificial Synapsys: Tracking Genes through Historical Markings The numbers tell exactly when in history particular topological features appeared, so now they can be matched up any time in the future. In other words, they reveal gene homology.

  11. Matching up Genes

  12. Second Component: Speciation Protects Innovation • Originally used for multimodal function optimization (Mahfood 1995) • Organisms grouped by similarity (compatibility) • Fitness sharing (Goldberg 1987, Spears 1995): Organisms in a species share the reward of their fitness peak • To facilitate this, NEAT needs • A compatibility measure • Clustering based on compatibility, for fitness sharing

  13. Measuring Compatibility • Possible in NEAT through historical markings • 3 factors affect compatibility via historical markings on connection genes: • Excess • Disjoint • Average Weight Distance W • Compatibility distance

  14. Clustering Into Species

  15. Dynamic Compatibility Thresholding

  16. Fitness Sharing: Assigning Offspring to Species

  17. Third Component: Complexification from Minimal Structure • Addresses initialization problem • Search begins in minimal-topology space • Lower-dimensional structures easily optimized • Useful innovations eventually survive • So search transitions into good part of higher-dim. space • The ticket to high-dimensional space

  18. NEAT Performed Well on Double Pole Balancing Without Velocity Inputs

  19. DPNV Solutions Are Compact

  20. Harder DPNV (0.3m short pole) solution

  21. Visualizing Speciation

  22. Next Class: More NEAT • Implementation issues • Where NEAT can be changed • Areas for advancement • Issues in applying NEAT (e.g. sensors and outputs) Evolving a Roving Eye for Go by Kenneth O. Stanley and Risto Miikkulainen (2004) Neuroevolution of an Automobile Crash Warning System by Kenneth O. Stanley and Risto Miikkulainen (2005)

More Related