How Letting Go of Goals Helps Creativity and Discovery

1. How Letting Go of Goals Helps Creativity and Discovery Kenneth O. Stanley Evolutionary Complexity Research Group University of Central Florida School of EECS <kstanley>@eecs.ucf.edu In Collaboration with Joel Lehman, Sebastian Risi, Jimmy Secretan, Nick Beato, Adam Campbell, David D'Ambrosio, Adelein Rodriguez, Jeremiah T. Folsom-Kovarik, Brian Woolley

2. A Sobering Message… If you put your mind to it, you can accomplish anything (Marty McFly in Back to the Future, 1985)

3. A Sobering Message… If you put your mind to it, you can accomplish anything (Marty McFly in Back to the Future, 1985)

4. …with a Paradoxical Silver Lining If you do not put your mind to it, you can accomplish anything

5. Seeking AI: Trying to Understand How Complexity Evolves • 100 trillion connections in the human brain • The most complex structure known to exist • How can Darwinian evolution produce such astronomical complexity? • We can investigate this question through evolutionary computation (artificial evolution) • In the process, other fundamental principles of innovation and discovery are uncovered • Evolution is a kind of search

6. Innovation, Creativity, and Discovery are Forms of Search • We search the space of possible artifacts • All possible images • All possible three-dimensional morphologies • All possible combinations of words • All possible minds • Most of the search space is desolate • The gems are needles in a haystack • The mind is a powerful search operator

7. Argument Preview • Highly ambitious objectives ultimately block their own achievement • The greatest human achievements are not the result of objective optimization • All of search is cloaked in futility • Yet in this realization there is genuine liberation • New opportunities will open for discoveries that work in different ways

8. Picbreeder:A Microcosm for Innovation • Website: http://picbreeder.org • Crowd-sourced picture-breeding online service • Three years of operation • 7,500 evolved images • 500 users • Like Dawkins’ BioMorphs (from The Blind Watchmaker, 1986) on steroids • The question: How do users discover the best images in a vast and desolate space?

12. How Users Breed Images • First must decide where to start • From “scratch” : Random initial population • By “branching” : Offspring of existing image Typical Start from Scratch Starting from a Butterfly

13. How Users Breed Images • First must decide where to start • From “scratch” : Random initial population • By “branching” : Offspring of existing image Typical Start from Scratch Starting from a Face

14. How Users Breed Images • Next: Select parent(s) – Which do you like?

15. How Users Breed Images • Next: Select parent(s) – Which do you like? Press “Evolve” after selecting parents

16. How Users Breed Images • Next: The offspring (next generation) appear Parent

17. How Users Breed Images • Next: Repeat until satisfied and then Publish Parent

18. Important: Everything Ultimately Derives from Scratch • The beginning of evolution • However, the images become more complex over generations

19. Breeding from Scratch

20. Breeding from Scratch

21. Breeding from Scratch Parent

22. Breeding from Scratch Parent

23. Breeding from Scratch And so on… Parent

24. The Result:Large, Growing Phylogenies • Users build upon each other’s discoveries (30 users built this one)

25. Images Evolved by Picbreeder Users (All are 100% evolved: no retouching)

26. Picbreeder Image Representation • Images are represented by compositional pattern-producing networks (CPPNs) • A composition of simple functions • A new node is sometimes introduced through mutation • What kind of search space does this representation induce? y f(x,y,d)= HSB Gaussian x H S B Sigmoid Sig Sin Gaus Sine Sig Sin f Gaus Gaus Linear x y d

27. The Search Space is Desolate • Almost every image looks like these • They have random weights and topologies:

28. Yet Picbreeder Users Found These… (All are 100% evolved: no retouching)

29. Yet Picbreeder Users Found These… (All are 100% evolved: no retouching)

30. …and entire “species”…

31. …and these… (All are 100% evolved: no retouching)

32. …and these…How? (All are 100% evolved: no retouching)

33. Paradox: Images Cannot Be Re-evolved! • Pick an image • Make it an objective (i.e. goal) for evolution on computer (NEAT) • Run automated evolution • Output is terrible • This result is universal • Why? (74 cumulative generations) (best results from over 30,000 generations each)

34. Why Is It Impossible to Re-discover Former Discoveries? • You cannot find something on Picbreeder simply by looking for it • Serendipity plays a powerful role • Yet if it is serendipity, then how can it happen so often? (best results from over 30,000 generations each)

35. The Story of the Car • Would you expect to find a car in this space? • I didn’t • But then I found one

36. How Did I Find the Car? • I was not looking for a car • Rather, I chose to evolve the alien (ET) face to get more alien faces:

37. How Did I Find the Car? • But then, the alien’s eyes descended and turned into wheels

38. How Did I Find the Car? • The only way to find the car was by not looking for it • Otherwise, I would never have selected the alien • It does not look like a car

39. How Did I Find the Car? • But I would not have evolved the alien either! • Someone else had to evolve it for me to make my discovery

40. Most Top Images Have the Same Story The stepping stones almost never resemble the final product

41. Moral: It Works Because There Is No Unified Goal • The only way to find the needles in the haystack • …is by not collectively looking for them • Users have conflicting goals and interests • Some explore with no goal • Some may have their own goals • Yet the system as a whole has no unified objective • Every discovery is a potential stepping stone for someone else

42. Stepping Stones • Steps in search space that lead to objective • May be hard or impossible to identify a priori • Especially for ambitious problems • May not induce positive change in objective function

43. Objectives Can Be Deceptive • The Chinese Finger Trap • What are the stepping stones?

44. Thought Experiment • You have a Petri dish the size of the world • …and organisms equivalent to the very first cells on Earth • Your objective: Evolve human-level intelligence • You have 4 billion years • What is your selection strategy?

45. Thought Experiment • You have a Petri dish the size of the world • …and organisms equivalent to the very first cells on Earth • Your objective: Evolve human-level intelligence • You have 4 billion years • What is your selection strategy? • Administer intelligence tests to single-celled organisms!

46. Thought Experiment • You have a Petri dish the size of the world • …and organisms equivalent to the very first cells on Earth • Your objective: Evolve human-level intelligence • You have 4 billion years • What is your selection strategy? • Administer intelligence tests to single-celled organisms!

47. Intelligence Does Not Resemble Its Stepping Stones • The stepping stones: • Multicellularity • Bilateral Symmetry • Yet they are essential to its discovery • Humans were only found because we are not the objective • A proliferation of agnostic stepping-stones was the necessary prerequisite • Natural evolution has no final objective

48. The Limits of Human Innovation • Thought experiment: Good idea or not? • 5,000 years ago sequester all the greatest minds to build a computer

49. The Limits of Human Innovation • Thought experiment: Good idea or not? • 5,000 years ago sequester all the greatest minds to build a computer • Bad idea! • Vacuum tubes were not invented with computation in mind • Electricity was not discovered with computation in mind • Almost no prerequisite to any major invention was invented with that invention in mind!

50. The Limits of Human Innovation • This realization touches many of our greatest enterprises • Artificial intelligence • Including personal goals (e.g. make $1M)