Three Kinds of Learning

1. Three Kinds of Learning Chrisantha Fernando Marie Curie Fellow, Collegium Budapest

2. Aim • I present some examples of learning • Associative learning • Causal inference • Insight based problem solving • My aim is to understand the mechanisms underlying these learning behaviours.

3. Classical Conditioning Sound of Metronome (CS) Smell of Food (US) Salivation (Response) Ivan P. Pavlov (1927)

4. Light touch (CS) Withdrawal (Response) Electric Shock (US) Hawkins et al, 1989

5. Coincidence detectors Pre-Synaptic (Eccles) Post-Synaptic (Hebb)

6. Paramecia

7. What could they learn? • Temperature change precedes O2 change in marine ecosystems by 20 minutes. • Photon flux may precede temperature changes. • Aerobic to anaerobic respiration from mouth to gut (signaled by increasing temperature).

8. David S Goodsell, 1998 “The Machinery of Life”

9. Previous Work na = [A] + [AB] Gandhi et al, 2007 nb = [B] + [AB]

10. A Simple Learning Circuit • In collaboration with molecular biologists, I have designed Hebbian learning circuits for plasmids carried by E. coli. v = w.u dwi/dt = uiv

11. A Gene Regulatory Network

13. Output P No pairing Pairing

14. Artificial Evolution in Silico SBMLEvolver Synthetic Biology Toolbox

16. Evolution in vivo What will evolve?

17. Fitness function

18. Presynaptic Subtractive Norm Oja (L2 Norm) BCM Weight consolidation

19. MAPK Implementation

20. Why?

21. Later….

22. A R A R C C B B Susan Jeffery Learns patient specific (contingent) associations

23. Tagkopoulos et al, 2008 (in press) Similar Recent Work… • Prediction, but no associative learning.

24. Limitations of Associative Learning • Can only learn associations between pre-specified stimuli • Learns only associations, not cause and effect relationships.

25. “A Concept of Cause and Effect” • “My argument is that causal understanding gave rise to tool-making; that was the evolutionary advantage. It's tool-making that's really driven human evolution. This is not widely accepted, I'm afraid, but there's no question about it. It's tools that really made us human. They may even have given rise to language.” Lewis Wolpert, 2007

26. What is causal Inference? • Does dropping a coin into a tin of coins cause the number of coins in the tin go up? • Can moving a piece on a chess-board cause the opponent's queen to be pinned? • Can ignorance cause poverty? • Can poverty cause crime? • Can ignorance cause a TV set to be moved through a broken window? • Can inserting a certain sort of twig in a certain way into a particular partly built nest cause the nest to become more rigid? • Analysing the concept of causation is probably the hardest unsolved philosophical problem. It's at the root of problems about relations between mind and body (or relations between virtual and physical machines). Sloman, 2008 (pc)

27. Causal Inference • What is the difference between causal inference and associative learning? • Weak: To utilize more than pair-wise correlations (perhaps unconsciously). • Strong: Combining observation of conditional probability P(X|Y) with novel appropriate interventions • i.e. why don’t Pavlov’s dogs spontaneously ring the bell when they are hungry? (without reinforcement). • Humans do generate hypotheses based on CP and produce interventions to test causal models. • Parties -> Wine -> Insomnia • Wine <- Parties -> Insomnia

28. Structuring Interventions • A --> B --> C --> D • Intervene at C: A-> B C-->D • A <-- B <-- C <-- D • Intervene at C: A <-- B <-- C D

29. Algorithms exist to discover causal networks. • Bayesian learning • Know prior probability of causal graphs • Know probability of observations given each graph • Use Bayes theorum to calculate probability of graph given observations and priors • Fined the best graph • Constraint-based learning • For each pair of variables a and b in V, search for a set Sabsuch that (a ||_ b |Sab) holds in P, i.e. a and b should be independent in P, conditioned on Sab. Construct an undirected graph G such that vertices aand bare connected with an edge iff no set Sabcan be found. Connect dependent nodes • For each pair of non-adjacent variables a and b with a common neighbor c check if c is an element of Sab. If it is continue, if it is not then add arrow heads pointing at c i.e. a--> c <-- b. • In the partially directed graph that results, orient as many of the undirected edges as possible subject to two conditions: • i. the orientation should not create a new v-structure, • ii.the orientation should not create a directed cycle.

30. Crows • What is the evidence for causal understanding in crows?

32. I’m not convinced… • Tool use is innate in New Calodonian crows; no social learning is required. • Crows can make the right length and thickness of tool for the right hole, on the first trial. This could still be associative learning. • Must exclude simple strategies, e.g. random search, win-stay loose shift, reinforcement learning, operant conditioning, etc… • “Rather than giving subjects a defined set of choices, they are placed in a situation where they have a low probability of solving a task by chance alone (for example, in a hook making task an animal may be given a piece of pliable material that can be changed into an infinite number of shapes, but only a small subset of these shapes would be functional)” • How does one define the null-hypothesis, e.g. what is the probability of manufacturing a hook-shaped object by chance alone?

33. Both Betty and Bob use trial and error search.

34. Causal Inference in Rats • What is the evidence for causal inference in rats?

35. Causal Inference in Children • What is the evidence for causal inference in humans? • Understanding interventions (monkey sneezing + blickets, etc….) • A+ --> B- --> AB+ --> AB+ (Children choose A) • A+ --> A+ --> A+ --> B- --> B+ --> B+ (Choose randomly) • Retrospective disambiguation (by children) • e.g. AB+ --> A-, AB+ --> A+

36. Gopnik & Schultz, 2004

37. Gopnik & Schultz, 2004

38. Gopnik & Schultz, 2004

39. Our approach • To study intra-brain causal inference.

40. Insight in Humans

41. How to Solve it? • What is an ‘insight problem’? A problem that requires restructuring of the initial problem representation, e.g. goal states, operators, constraints. • What kinds of algorithm are used to solve these and related problems? What determines the set of goal, operators, constraints?

42. Missionaries and Cannibals • 3 missionaries and 3 cannibals must cross a river using a boat which can carry at most two people. • For both banks, if there are missionaries present on the bank, they cannot be outnumbered by cannibals. • The boat cannot cross the river by itself with no people on board.

43. Intermediate goals may be used • Early moves balance number of M & C on each side of river. • Intermediate moves maximize progress from one side to other. • Later moves avoid revisiting previous states.

44. There is some evidence… • Non-maximal moves that allow a subsequent move to make more progress are retained as promising states for future trials. • Goal criteria are relaxed and changed based on the quality (immediate benefit) of generated solutions. • Sometimes hill-climbing to a ‘wrong’ goal criteria can get stuck in local minima.

45. 3 moves • 7426 legal 3-move sequences • 2 reach ring solution • 176 reach 2 group solution • 7426 sequences are not eqiprobable under random selection assumption. • < 1/3 participants solve problem within 10 minutes. • Choice of the correct first move based on the improved goal scores available from the second move was crucial. • Few subjects even conceived of a two group solution when asked to “produce a shape where each coin only touches two others”. CHRONICLE, MACGREGOR, AND ORMEROD,2004

46. Brain damage helps some problem solving! II = III + I Type A IV = III - I Type B VI = VI + VI Type C Solutions here Reverberi et al 2005

47. Our approach • To study mechanisms for restructuring of problem representations. Poelwijk et al 2007

48. Conclusions • What neural mechanisms underlie causal inference, and solving insight problems? • What changes allow humans to have these capacities but precludes other apes from having them? • What algorithms can best predict human performance in such problems?

49. Thanks to • Eors Szathmary • Lewis Bingle • Anthony Liekens • Aaron Sloman • Jon Rowe • Dov Stekel • Christian Beck & Thorsten Lenser