Karl Popper (1902-1994) Demarcation between science and non-science

1. Karl Popper (1902-1994) Demarcation between science and non-science Zoltán Dienes, Philosophy of psychology

2. What is science? What is the difference between science and pseudo-science? What is the difference between good science and bad science? On what grounds should some papers submitted to scientific journals be rejected or accepted? Are Christian Science, Creation Science, Scientology, astrology sciences? If not, why not and why does it matter? Is psychology a science? Good science or bad science? How does knowledge grow?

3. What is science? “Systematic and formulated knowledge” (Pocket Oxford) based on empirical observation. So astrology is a science? Why has there been such an explosive growth in knowledge over the last few hundred years? More generally, what is it that facilitates the growth of knowledge? Need a better demarcation criterion that answers this question. How can we arrive at knowledge of general rules from empirical observation?

4. Deduction: Going from certain truths to further certain truths. All swans are white Sam is a swan ____________________ Sam is white Induction: Going from particular observations to universal rules. Sam the swan is white; Georgina the swan is white; Fred the swan is white; … Emma the swan is white _____________________ All swans are white (?)

5. Inductivists believe that science proceeds by induction; Science is objective because it is based on actual observations rather than just speculation Science goes from particular observation statements inductively to general rules. David Hume (1711-1776): But how is this possible? How can we ever justify going from particular observations to universal rules? (We are very confident that the sun will continue to rise each morning – but one morning it WILL NOT!) The problem of induction If science is inductive reasoning, and if it is always questionable to go from particulars to universals, how is science possible?

6. Problem of induction solved: Induction does not exist. Science consists of freely, creatively inventing theories then testing them. Theories are never shown to be true, but can be falsified. Testing is deductive: Accepting certain singular statements means by deduction that the theory is false. Science can only work in this way if a theory is falsifiable: the theory says certain things cannot happen.

7. Problem of induction solved: Induction does not exist. Science consists of freely, creatively inventing theories then testing them. Theories are never shown to be true, but can be falsified. Testing is deductive: Accepting certain singular statements means by deduction that the theory is false. Science can only work in this way if a theory is falsifiable: the theory says certain things cannot happen. Science consists in proposing falsifiable theories then rigorously attempting to falsify them: It is only when theories are falsified that we get feedback from Nature and a chance to improve our knowledge. Theories that survive rigorous attempts at falsification are NOT proved; they are “corroborated” but can only be held tentatively

8. Popper (1934) Distinguish • the process of inventing a theory (“the context of discovery”, which is not logical but creative) • the process of testing the theory (“the context of justification”) There is no particular method of inventing theories.

9. Accept Hume: Cannot go from singular statements (apply to a specific event or individual) to universals (a general assertion to be applied to an unlimited number of individuals) There is no method of showing a theory is true! BUT one can go from singular statements to concluding a universal statement is false! Peter the Swan is black _____________________ Not all swans are white So “All swans are white” cannot be verified by any number of singular statements BUT it can be falsified

10. Popper was impressed by two opposing types of experiences in 1919: On the one hand with Marxism and psychoanalysis; on the other hand, with Einstein. “Admirers of Marx, Freud and Adler were impressed by the ability of the theories to explain everything that happened within their domain. They saw confirming instances everywhere; whatever happened always confirmed it. Its truth appeared manifest; people who did not see the truth refused to because of their class interest or because of their repressions which were crying out for treatment. A Marxist could not open a newspaper without finding on every page confirming evidence for his interpretation of history“ (1963, p. 45)

11. Popper briefly worked for Alfred Adler (a student of Freud’s). “Once in 1919 I reported to [Adler] a case which did not seem particularly Adlerian, but which he found no difficulty in analysing in terms of his theory of inferiority feelings, although he had not even seen the child. Slightly shocked, I asked him how could he be so sure. ‘Because of my thousand-fold experience’ he replied; whereupon I could not help saying, ‘And with this new case, I suppose, your experience has become one thousand-and-one-fold’”. What do these confirmations mean if every conceivable case could be interpreted in the light of Adler’s (or Freud’s) theory? How would Adler (Freud, Marx) ever get any indication that he was wrong?

12. Contrast • a man who pushes a child into water to drown it • a man who sacrifices his life attempting to save the child. According to Freud: • the first man suffered from repression (say of some component of the Oedipus complex) • the second had achieved sublimation. According to Adler: • the first man suffered from feelings of inferiority (producing the need to prove himself that he dared to commit such a crime) • and so did the second man (whose need was to prove himself that he dared to rescue the child).

13. If a patient accepts the interpretation: It was the right one If a patient rejects the interpretation, particularly with some vigour: It hit pretty close to home. What would give you the slightest inkling that you, the analyst, were wrong? If we cannot learn from our mistakes, how can we improve our theories?

14. In 1919 Popper went to a lecture by Einstein which impressed him greatly. Einstein said if in a particular set-up light were not observably bent his general theory of relativity would be untenable. (The prediction was tested in May 1919 by Eddington and the effect was found.) If someone proposes a scientific theory they should answer, as Einstein did “Under what conditions would I admit that my theory is untenable?”

15. In 1919 Popper went to a lecture by Einstein which impressed him greatly. Einstein said if in a particular set-up light were not observably bent his general theory of relativity would be untenable. (The prediction was tested in May 1919 by Eddington and the effect was found.) If someone proposes a scientific theory they should answer, as Einstein did “Under what conditions would I admit that my theory is untenable?” Lakatos: “I used to put this question to Marxists and Freudians: ‘Tell me what specific social and historical events would have to occur in order for you to give up your Marxism?’. I remember this was accompanied by either stunned silence or confusion. But I was very pleased with the effect.” (1999, p. 26)

16. Falsifiability distinguishes science from non-science (metaphysics) NOT meaningful from nonsense etc Non-falsifiability of Freudian and Alderian views etc does not mean they were not often seeing things correctly “I personally do not doubt that much of what they say is of considerable importance and may well play its part one day in a psychological science which is testable” (Popper, 1963)

17. Falsifiability distinguishes science from non-science (metaphysics) NOT meaningful from nonsense etc Non-falsifiability of Freudian and Alderian views etc does not mean they were not often seeing things correctly “I personally do not doubt that much of what they say is of considerable importance and may well play its part one day in a psychological science which is testable” (Popper, 1963) They are like “myths”. “Historically speaking almost all scientific theories originate from myths. If a theory is non-scientific or metaphysical it is not thereby found to be unimportant or nonsensical” But to get feedback from Nature, the metaphysical frameworks must ultimately provide scientific – falsifiable - theories

18. Is it actually possible to distinguish falsifiable and non-falsifiable systems? According to Popper, observations are always “theory impregnated”. Theories are needed to determine what an observation is. Is it an accurate clock or a rigid rod? Can only refer to our theories to answer this. (Consider a measurement of : how extroverted a participant is Working memory span) A theoretical system can always escape falsification by e.g. doubting the observations (“The extroversion scale has limited validity”), or changing a definition (“A non-white swan is not a swan”).

19. Popper 1934: Observation statements are never given directly from experience. Every statement uses universal names, every statement has the character of a theory or hypothesis. “Here is a glass of water” cannot be verified or justified by any observational experience. Experiences however can motivate observation statements.

20. Popper 1934: Observation statements are never given directly from experience. Every statement uses universal names, every statement has the character of a theory or hypothesis. “Here is a glass of water” cannot be verified or justified by any observational experience. Experiences however can motivate observation statements. Observation statements are accepted by decision or agreement. Theory dominates experimental work from its initial planning up its finishing touches in the laboratory. We must decide which observation statements we will accept. (The decision is fallible and amounts to tentatively accepting a low level empirical hypothesis which describes the effect: “Peter is an extrovert”, “This extrovert was asleep at 7 am” etc) 

21. Popper (1934): The question of whether a theoretical system as such is falsifiable or not is misconceived. “It is only with regards to the methods applied to the system can we ask whether it is a falsifiable theory or not.” Consider: Astrology makes falsifiable predictions but these are scarcely used to test and modify the theory.

22. Popper (1934): The question of whether a theoretical system as such is falsifiable or not is misconceived. “It is only with regards to the methods applied to the system can we ask whether it is a falsifiable theory or not.” Consider: Astrology makes falsifiable predictions but these are scarcely used to test and modify the theory. How do scientists as a community treat the theory – are they seeking to test and falsify it? (Astrologists in general don’t).

23. Popper (1934): The question of whether a theoretical system as such is falsifiable or not is misconceived. “It is only with regards to the methods applied to the system can we ask whether it is a falsifiable theory or not.” Consider: Astrology makes falsifiable predictions but these are scarcely used to test and modify the theory. How do scientists as a community treat the theory – are they seeking to test and falsify it? (Astrologists in general don’t). Decide in advance the sort of observation statements you will accept, the conditions that will falsify your theory. The decision is fallible, so the feedback from nature is noisy, but at least we are exposing ourselves to feedback! We are giving ourselves a chance to learn from our mistakes!

24. Popper 1934: To be honest, scientists we must decide to avoid excuses for saving a theory The aim of the empirical method is not to save the lives of untenable systems but, on the contrary, to select the one which is by comparison the fittest, by exposing them all to the fiercest struggle for survival.

25. Rule: Only revise theories, introduce new theories, or introduce those auxiliary hypotheses that do not diminish the degree of falsifiability of the system Contrast: “All swans are white except for Peter” Ad hoc: The amendment to the theory decreased its falsifiability

26. Rule: Only revise theories, introduce new theories, or introduce those auxiliary hypotheses that do not diminish the degree of falsifiability of the system Contrast: “All swans are white except for Peter” Ad hoc: The amendment to the theory decreased its falsifiability Theory: “God created the Earth, with all current species in it, 4000 years ago” Falsifying evidence: Fossil record Ad hoc amendment: God put the fossil record there to test our faith.

27. Rule: Only revise theories, introduce new theories, or introduce those auxiliary hypotheses that do not diminish the degree of falsifiability of the system Contrast: “All swans are white except for Peter” Ad hoc: The amendment to the theory decreased its falsifiability Theory: “God created the Earth, with all current species in it, 4000 years ago” Falsifying evidence: Fossil record Ad hoc amendment: God put the fossil record there to test our faith. In psychology, attempts to save theories in ways that don’t suggest new tests are often called “post hoc” The more falsifiable the theory, the more preferable it is as a scientific theory. How can we measure degree of falsifiability?

28. A potential falsifier is a potential observation statement that would contradict the theory (“Peter the swan is black”) How can one theory be more falsifiable than another? If the class of potential falsifiers is larger.

29. A potential falsifier is a potential observation statement that would contradict the theory (“Peter the swan is black”) How can one theory be more falsifiable than another? If the class of potential falsifiers is larger. E.g. One needs fewer data points to rule out a straight line relationship than a quadratic

30. A potential falsifier is a potential observation statement that would contradict the theory (“Peter the swan is black”) How can one theory be more falsifiable than another? If the class of potential falsifiers is larger. E.g. One needs fewer data points to rule out a straight line relationship than a quadratic (Scientists prefer simple theories. But what is simplicity? Simple theories are better testable. Straight lines are simpler than curves.)

31. “A is positively correlated with B” allows all the positive regression lines: A B And just rules out all the negative ones; i.e. 50% of possible lines are excluded.

32. “A is correlated with B” rules out practically nothing. All positive and negative regression lines are permitted:

33. “A is correlated with B” rules out practically nothing. All positive and negative regression lines are permitted: “group A will score differently from group B” also rules out virtually nothing; it’s a very weak theory. “Group A will score higher than B” is better: It rules out 50% of the possible difference scores. Group A will perform 30% better than group B rules out a lot! It would be a prediction of a very falsifiable theory

34. Popper: New hypotheses replacing old rejected ones should have more falsifiability But: Unrealistic to always expect more falsifiability? If data rule out a straight line, surely one can accept a curve? Nature may be simple, but she is only so simple! Sometimes the more complex answer is right. But of course one always strives for the simplest one can get away with.

35. Computational model: A computer simulation of subjects, exposed to the stimuli subjects received and gives actual responses. E.g. neural networks - collections of artificial neurons connected together in a certain way; can learn according to some learning rule. A computational model has a number of “free parameters” – numbers that have to be fixed, like the number of artificial neurons used, the learning rate between the connections etc. Boucher and Dienes 2003: Contrasted two models, trained on the same stimuli as subjects were.

36. The axes are dependent variables. Predictions of each model were determined for the full range of allowable parameter values. The graph is a space of possible outcomes of human data. Notice the model in grey occupies less area than the model in black. The grey model is easier to falsify.

37. (In fact, the human data occupied the same region of space as the grey model: The grey model was easy to falsify, but it survived this attempt a falsification. It was not proven true but it was corroborated: It proved its mettle.)

38. Hypothetical case: Model B is more falsifiable than A. As the human data is consistent with both models, model B is to be preferred. (The point is obvious but often disregarded: Often modellers just try to find any model – set of parameter values - that fits the data. )

39. Of course, if the data were inconsistent with B, B would have to be rejected. A could be tentatively accepted until a better (more simple, more falsifiable) theory could be discovered. Notice A does not rule out very much; it does not really explain why people behaved as they do.

40. One should favour theories with: • the highest level of universality • the highest degree of precision 1) the orbits of all planets are circles 2) the orbit of mars is a circle 3) the orbits of all planets are ellipses

41. One should favour theories with: • the highest level of universality • the highest degree of precision 1) the orbits of all planets are circles 2) the orbit of mars is a circle 3) the orbits of all planets are ellipses 1) is more universal than 2), hence easier to falsify: 1) says more 1) is more precise than 3), hence easier to falsify (more position-time coordinates are needed to falsify 3 than 1).

42. Popper 1934: The empirical content of a theory increases with its degree of falsifiability. The more a statement forbids, the more it says about the world. A theory that forbids many outcomes is easy to falsify – it asserts a lot about he world. A theory that allows everything explains nothing.

43. Popper 1934: The empirical content of a theory increases with its degree of falsifiability. The more a statement forbids, the more it says about the world. A theory that forbids many outcomes is easy to falsify – it asserts a lot about he world. A theory that allows everything explains nothing. Theoretical science aims at obtaining theories that are easily falsifiable, that restricts the range of permitted events to a minimum (and any further restriction would lead to the actual falsification of the theory). The theory would describe our particular world as precisely as a theory can. One should favour the theory with the highest possible empirical content.

44. Popper 1934: The search for truth is the strongest motive for scientific discovery. But we cannot know, we can only guess. Our guesses are guided by the unscientific, the metaphysical (though biologically explicable) faith in laws that we can discover.

45. Popper 1934: The search for truth is the strongest motive for scientific discovery. But we cannot know, we can only guess. Our guesses are guided by the unscientific, the metaphysical (though biologically explicable) faith in laws that we can discover. Bold ideas, unjustified anticipations, and speculative thought are our only means for interpreting nature. Those among us who are unwilling to expose their ideas to the hazard of refutation do not take part in the game of science. Weyl “Nature knows so well how to meet our theories with a decisive no – or with an inaudible yes”.

46. “But as for certain truth, no man has known it Nor will he know it; neither of the gods, Nor yet of all the things of which I speak. And even if by chance he were to utter The perfect truth, he would himself not know it; For all is but a woven web of guesses “ Xenophanes (570 – 465 BCE)

47. Popper’s ideas were often endorsed by great scientists. E.g. Feynman: “The scientist does not try to avoid showing that the rules are wrong; there is progress and excitement in the exact opposite. He tries to prove himself wrong as quickly as possible. . . In science we are not interested in where an idea comes from. There is no authority that decides what is a good idea. … there is no interest in the background of the author of an idea or his motive in expounding it. You listen and if it sounds like a thing worth trying you get excited…” Feynman emphasized the fallibility of all scientific knowledge and the importance of doubt not certainty: “If we did not doubt we would not get any new ideas”. (1998/1963)