Evolving Winning Controllers for Virtual Race Cars

1. Evolving Winning Controllersfor Virtual Race Cars Yonatan Shichel & Moshe Sipper

2. Outline • Introduction • Artificial Intelligence • AI in games • Robocode: Java-based tank-battle simulator • RARS: Robot Auto Racing Simulator • Evolutionary Computation • Key concepts in evolution • Genetic Algorithms (GA) • Genetic Programming (GP) • GP-RARS: evolution of winning controllers for virtual race cars • Game description • Previous work • Evolutionary environment setup & calibration • Experiments and Results • Discussion • Result Analysis • Concluding Remarks

3. Introduction

4. Artificial Intelligence (AI) Definition (Russell & Norvig, 2003): “systems that [act/think] [like humans/rationally]”

5. Artificial Intelligence (AI) Definition (Russell & Norvig, 2003): “systems that [act/think] [like humans/rationally]”

6. Artificial Intelligence (AI) Definition (Russell & Norvig, 2003): “systems that [act/think] [like humans/rationally]”

7. Artificial Intelligence (AI) Definition (Russell & Norvig, 2003): “systems that [act/think] [like humans/rationally]”

8. Artificial Intelligence (AI) Definition (Russell & Norvig, 2003): “systems that [act/think] [like humans/rationally]”

9. AI in Games • games are natural candidates for AI • games provide a variety of challenges • games allow exploration of real-world realms • games allow comparison to human behavior • games can be rewarding to master • games are fun!

10. Robocode

11. Robocode • tank-battle simulation • Java-based, open-source programming game • simplistic physical model • active gamer community • extensive online robot library • ongoing tournaments

12. RARS: Robot Auto Racing Simulator

13. RARS: Robot Auto Racing Simulator

14. RARS: Robot Auto Racing Simulator

15. RARS: Robot Auto Racing Simulator

16. RARS: Robot Auto Racing Simulator • car-race simulation • C++-based, open-source programming game • sophisticated physical model • inactive gamer community • limited online robot library • tournaments held between 1995 and 2003

17. Evolutionary Computation “a family of algorithmic approachesaimed at finding optimal solutions tosearch problems of high complexity”

18. Key concepts in Evolution The Origin of Species (Darwin, 1859): • a population is composed of many individuals • individuals differ in characteristics, which are inheritable by means of sexual reproduction • environment consists of limited resources, leading to a struggle for survival

19. Key concepts in Evolution The Origin of Species (Darwin, 1859): • fitter individuals are more likely to survive and reproduce, passing their characteristics to their offspring • as time passes, populations slowly adapt to their surrounding environment

20. Genetic Algorithms (GA) Inspired by Darwin’s evolutionary principles: • a fixed-size population is composed of many solution instances for the problem at hand • solutions are encoded in genomes • a fitness function determines how fit each individual is • population is re-populated on each generation • fitter individuals have higher probabilities to be selected to next generation

21. Genetic Algorithms (GA) • genetic operators – crossover and mutation – are applied on selected individuals for the creation of new individuals • process is repeated for many generations

22. Genetic Algorithms (GA) A schematic flow of a basic GA: g=0 initialize population P0 evaluate P0 //assign fitness values to individuals while (termination condition not met) do g=g+1; select Pg from Pg-1 crossover Pg mutate Pg evaluate Pg end while

23. Genetic Algorithms (GA) GA customization: • genome representation • fitness measure • selection method • crossover method • mutation method • termination condition • initial population creation

24. Genetic Programming (GP) “an evolutionary computation approachaimed at the creation of computer programsrather than static solutions”

25. Genetic Programming (GP) • individual’s genome is composed of LISP expressions

26. Genetic Programming (GP) example of LISP expression: (+ (* x x) 1) ==> x2+1 + * 1 x x

27. Genetic Programming (GP) • individual’s genome is composed of LISP expressions • LISP expressions are composed of functions and terminals

28. Genetic Programming (GP) functions: {+, *} terminals: {1, x} + * 1 x x

29. Genetic Programming (GP) functions: {+, *} terminals: {1, x} + * 1 x x

30. Genetic Programming (GP) functions: {+, *} terminals: {1, x} + * 1 x x

31. Genetic Programming (GP) functions: {+, *} terminals: {1, x} + 1 * x x

32. Genetic Programming (GP) • individual’s genome is composed of LISP expressions • LISP expressions are composed of functions and terminals • LISP expressions evaluate to numeric values, hence representing functions

33. Genetic Programming (GP) evaluation of LISP expression:

34. Genetic Programming (GP) • individual’s genome is composed of LISP expressions • LISP expressions are composed of functions and terminals • LISP expressions evaluate to numeric values, hence representing functions • genetic operators are defined to operate on (and return) LISP expressions

35. Genetic Programming (GP) subtree substitution crossover: - + 1 * * 1 1 x x + x 1 (+ (* x x) 1) x2+1 (- 1 (* 1 (+ x 1))) -x

36. Genetic Programming (GP) subtree substitution crossover: - + 1 * * 1 1 x x + x 1 (+ (* x x) 1) x2+1 (- 1 (* 1 (+ x 1))) -x

37. Genetic Programming (GP) subtree substitution crossover: - + 1 * * 1 1 x x + x 1 (+ (* x x) 1) x2+1 (- 1 (* 1 (+ x 1))) -x

38. Genetic Programming (GP) subtree substitution crossover: - + 1 * 1 1 + x 1 (- 1 (* 1 (+ x 1))) -x

39. Genetic Programming (GP) subtree substitution crossover: - + 1 1

40. Genetic Programming (GP) subtree substitution crossover: - + * 1 1 1 + x 1 (+ (* 1 (+ x 1)) 1) x+2

41. Genetic Programming (GP) subtree substitution crossover: - + * 1 * 1 1 x x + x 1 (+ (* 1 (+ x 1)) 1) x+2 (- 1 (* x x)) 1-x2

42. Genetic Programming (GP) random subtree growth mutation: + * 1 x x (+ (* x x) 1) x2+1

43. Genetic Programming (GP) random subtree growth mutation: + * 1 x x (+ (* x x) 1) x2+1

44. Genetic Programming (GP) random subtree growth mutation: + * 1 x

45. Genetic Programming (GP) random subtree growth mutation: - + 1 1 * 1 x (+ (* x (- 1 1)) 1) 1

46. Genetic Programming (GP) A schematic flow of a basic GP: g=0 initialize population P0 evaluate P0 //assign fitness values to individuals while (termination condition not met) do g=g+1; while (Pg is not full) do OP = choose a genetic operator select individual or individuals from Pg-1 according to OP's inputs apply OP on selected individuals add the resulting individuals to Pg end while evaluate Pg end while

47. GP-RARS evolution of winning controllers for virtual race cars

48. Basic Rules • one or more cars drive on a track for given number of laps • cars are damaged when colliding or driving off track • car may be disabled and disqualified if its damage exceeds a certain level • the winner is the driver that finishes first

49. Game Variants • number of cars: one, two, multiple • number of tracks: one, multiple • race length: short, long • controller program: generic, specialized • driver class: reactive (c2), optimal-path (c1)

50. Game Variants • number of cars: one, two, multiple • number of tracks: one, multiple • race length: short, long • controller program: generic, specialized • driver class: reactive (c2), optimal-path (c1)