History of Artificial Intelligence

1. History of Artificial Intelligence Dana Nejedlová Department of Informatics Faculty of Economics Technical University of Liberec

2. What is Intelligence? • Common definition of artificial intelligence: • AI is a field which attempts to build intelligent machines and tries to understand intelligent entities. • But what is intelligence? • Learning, manipulating with facts, but also creativity, consciousness, emotion and intuition. • Can machines be intelligent? • Up to the present day it is not sure whether it is possible to build a machine that has all aspects of intelligence. • This kind of research is central in the field of AI.

3. What Is Artificial Intelligence? • Building machines that are able of symbolic processing, recognition, learning, and other forms of inference • Solving problems that must use heuristic search instead of analytic approach • Using inexact, missing, or poorly defined information • Finding representational formalisms to compensate this • Reasoning about significant qualitative features of a situation • Working with syntax and semantics • Finding answers that are neither exact nor optimal but in some sense „sufficient“ • The use of large amounts of domain-specific knowledge • The use of meta-level knowledge (knowledge about knowledge) to effect more sophisticated control of problem solving strategies

4. Before the Creation of Electronic Computers • Ancient and medieval myths • Talos, Pandora, Golem • artificial men, robots • Research in the antiquity till the 17th century • Aristotle, Gottfried Wilhelm Leibniz • automation of reasoning • Thomas Hobbes, René Descartes • mechanistic understanding of living beings • 20th century, 1948 • Norbert Wiener – Cybernetics: Or the Control and Communication in the Animal and the Machine. • Intelligent behavior is the result of the feedback mechanism.

5. The Beginnings of Electronic Computers • John Louis von Neumann (1903 – 1957) • Von Neumann’s architecture of a computer • Consultations on the EDVAC Project (1945) • Game Theory (1944) • It can be applied to the interacting intelligent agents. • Cellular automata (1966) • They have computational capacity. • Alan Mathison Turing (1912 – 1954) • Turing Machine (1936) • formalization of algorithm, abstraction of computer • Turing Test (1950) • proposal how to test the ability of a machine to demonstrate thinking • Programming of “Manchester Mark I” computer (1949)

6. The birth of “Artificial Intelligence” • John McCarthy used the term “Artificial Intelligence” for the first time as the topic of the Dartmouth conference in 1956. • Venue: • Dartmouth College, Hanover, state New Hamphshire, USA • Organizers: • John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon • Participants: • Ray Solomonoff, Oliver Selfridge, Trenchard More, Arthur Samuel, Herbert Simon, and Allen Newell • Proposal: • To prove that every aspect of learning or any other feature of intelligence can be so precisely described that a machine can be made to simulate it.

7. Approaches to Artificial Intelligence • Good Old-fashioned Artificial Intelligence (GOFAI) or symbolic artificial intelligence (John Haugeland, 1985) • Program (e.g. classifier) in the GOFAI style is composed of parts (e.g. rules), that have clear relation to the real world. • New-fangled Artificial Intelligence • The most important branch was connectionism – artificial neural networks (McCulloch – Pitts, 1943). • Genetic algorithms (Holland, 1975) and other kinds of biologically inspired information processing • Strong AI (John Searle, 1980) • Artificial intelligence is real intelligence. • Solution of complex problems, e.g. robotics. • Weak AI • Artificial intelligence is a mere imitation of human real intelligence. • Solution of a specific problems that do not cover the whole scale of human capabilities, e.g. OCR or chess.

8. Motivations for Biologically Inspired Information Processing • Danny Hillis: The Connection Machine (1985) • Machines programmed in a GOFAI style tend to slow down as they acquire more knowledge. • They must search their knowledge base. • Humans have the opposite property. • They have massively parallel brain architecture. • Humans were not produced by an engineering process. • They are the result of evolution. • Marvin Minsky: The Society of Mind (1986) • Model of human intelligence which is built from the interactions of simple parts called agents which are themselves mindless. • It would be difficult to imagine how evolution could shape a single system as complex as mind. • Evolution could, however, shape individual specialized cognitive units and form the mechanisms that enable the modules to interact. • Marvin Minsky: The Emotion Machine (2006) • Emotions are different ways to think that our mind uses to increase our intelligence.

9. Artificial Intelligence Philosophy • What is intelligence and thinking? • Turing test (1950) • According to GOFAI thinking is symbol manipulation, that is why program in the GOFAI style is thinking. • Chinese Room Problem (John Searle, 1980) • Thinking of humans and computers is different. • Is human intelligence inseparable from mind and emotions? • In what sense can we say that a computer can understand natural language? • Who is responsible for the decisions made by AI? • What should be the ethics of people of dealing with the creations of artificial intelligence?

10. Hard Versus Soft Computing • Good Old-fashioned Artificial Intelligence • IF – THEN Rules • Heuristics • New-fangled Artificial Intelligence • Neural networks • Fuzzy logic • Probabilistic reasoning • belief networks (Bayes networks) • genetic algorithms • chaos theory • parts of learning theory (machine learning)

11. Heuristics • Problem-solving method that is usually successful, but can fail i some situations • Unclearly defined problems with missing or ambiguous data • Medical diagnosis • Vision, speech recognition • Helps to decide among infinite number of possible interpretations. • A problem may have an exact solution, but the computational cost of finding it may be prohibitive. • Chess, tic-tac-toe, 15 or 8-puzzle, scheduling, path-finding… • Heuristic evaluation function • Evaluates each stage of solution. • Number of conflicts in a number of possible schedules • Helps to decide about the next step leading to the goal. • Selecting the schedule with minimum number of conflicts for the next small changes attempting to find some correct schedule

12. Expectations from Artificial Intelligence • Predictions of Herbert Simon and Allen Newell (Heuristic Problem Solving, 1958), that within ten years • a digital computer will be the world's chess champion, • a digital computer will discover and prove an important new mathematical theorem, • a digital computer will compose critically acclaimed music, • most theories in psychology will take the form of computer programs.

13. Typical AI Problem • 8 Queens Puzzle • Is there a way of placing 8 queens on the chessboard so that no two queens would be able to attack each other?

14. Hard Problem for AI • Truncated Chessboard Problem • Is there a way of placing dominos on the board so that each square is covered and each domino covers exactly two squares?

15. Limitations of Artificial Intelligence • David Hilbert (1862 – 1943) andKurt Gödel (1906 – 1978) • Gödel‘s Incompleteness Theorem (1931) • Consistency of a formal system cannot be proved within the system, because it can contain statements with self-reference – logical paradoxes of the type: • This statement is false. • Some tasks have no algorithms. • The halting problem • It is not decidable whether the algorithm will halt or not. • The algorithms in question contain again self-reference. • Complexity Theory (NP-completeness, 1971) • Some tasks have algorithms, but the computation cannot be completed in practice (on a real computer), because it would take too much time.

16. Gödel‘s Incompleteness Theorem • There are unprovable statements in every axiomatic mathematical system expressive enough to define the set of natural numbers. • Example theorem 1 = 2 • Proof of the theorem: • If a = b, a ≠ 0, b ≠ 0, • then the two following equalities are also true: a2 – b2 = (a – b) ∙ (a + b), a2 – b2 = a2 – ab. • And the following statements can be derived from them: a2 – ab = (a – b) ∙ (a + b) a ∙ (a – b) = (a – b) ∙ (a + b) a = a + b a = a + a a = 2a 1 = 2 • Truth can be verified only when knowledge beyond the natural finite numbers arithmetic is used.

17. The Logic Theorist – The First Artificial Intelligence Program • Allen Newell, J.C. Shaw and Herbert Simon at Carnegie Institute of Technology, now Carnegie Mellon University, in 1955 • It did logic proofs from the book “Principia Mathematica” (Bertrand Russell and Alfred North Whitehead, 1910). • It used mental processes of human experts. • cognitive science • To implement Logic Theorist on a computer, the three researchers developed a programming language, IPL, a predecessor of Lisp.

18. Programming Languages • Tasks like natural language processing, knowledge representation, or theorem proving needed a special language allowing processing of symbolic data. • Lisp (John McCarthy, USA, 1958) • functional paradigm / list processing • Program consists of functions of nested functions. • Data and programs are represented the same way: a list. • (+ 1 2 3) is a both a list of 4 atoms and a function returning value 6. • Program can serve as data for another program! • Powerful feature allowing flexible and productive coding. • Prolog (Alain Colmerauer, Europe, 1972) • declarative paradigm / logic programming • Program consists of facts and rules. • Programmer describes (i.e. declares) a problem. • Compiler deduces new facts from them. • Programmer does not write the algorithm for the solution.

19. Programs with Symbolic Artificial Intelligence • The General Problem Solver (1957) • It was solving formalized symbolic problems, e.g. mathematical proofs and chess. • The Geometry Theorem Prover (1958) • It was proving theorems with the help of explicitly represented axioms. • SAINT (Symbolic Automatic INTegrator) • Integral calculus (1961) • ANALOGY (1963) • The picture A is to picture B like picture C to picture D. • IQ tests are used for measuring the intelligence of people. • Computers can be programmed to excel in IQ tests. • But those programs would be stupid in real-world situations.

20. Natural Language Processing • STUDENT (1964, 1967) • It was solving word problems in algebra. • SIR (Semantic Information Retrieval, 1968) • It was reading simple sentences and answered questions. • ELIZA (1965) • It was simulating psychologist. • TLC (Teachable Language Comprehender) (1969) • It was reading text and making semantic network. • SUR (Speech Understanding Research) (1971) • 5-year plan of the ARPA (today DARPA) agency of a research in continuous speech recognition

21. Expert Systems • They belong to the symbolic AI. • They use a set of rules and heuristics. • MACSYMA (MIT, 1968 -1982) • It was doing symbolic math calculations. • DENDRAL (SRI, 1965) • It is identifying chemicals. • MYCIN (SRI, Edward Shortliffe, 1974) • It diagnosed infectious blood diseases. • The following systems: EMYCIN, PUFF, INTERNIST - CADUCEUS

22. Commercial Expert Systems • PROSPECTOR (SRI, 1974 – 1983) • It is analyzing geological data and searching for deposits of minerals. • XCON – eXpert CONfigurer (CMU, 1978) • It was configuring DEC’s VAX computers. • TEIRESIAS (SRI, Randall Davis, 1976) • Knowledge Acquisition System (KAS) • It is acquiring knowledge from human experts. • It is building knowledge bases for expert systems.

23. Robotics • Marvin Lee Minsky (* 1927) • Freddy (University of Edinburgh,1973) • SHAKEY (SRI, 1969) • SHRDLU (MIT, Terry Winograd, 1970) • blocks worlds (MIT, 1970) • Robot has to manipulate building blocks according to instructions. • computer vision • natural language understanding • planning

24. The First Artificial Neural Networks • Warren McCulloch and Walter Pitts • Model of artificial neuron (1943) • Neuron represents functions. • Donald Olding Hebb • Rule for neural network training (1949) • Marvin Minsky and Dean Edmonds have built the first computer with neural network. • SNARC (1951)

25. Other Artificial Neural Networks • Frank Rosenblatt • Perceptron (1957) • a single-layer network and its learning rule capable of learning linearly separable functions • Bernard Widrow and Marcian Ted Hoff • Minimization of network’s root square error • Delta rule (learning rule of a neural network) • ADAptive LINEar Systems or neurons or ADALINEs (1960) • MADALINEs (1962) • multi-layer versions of ADALINEs

26. Neural Networks Critique • Book „Perceptrons“ (Marvin Minsky and Seymour Papert, 1969) • When single-layer neural networks of a Perceptron type cannot learn XOR function (it is linearly inseparable), also multi-layer networks cannot learn it. • Hence funding of neural network research was stopped until the beginning of the 20th century 80’s. • But multi-layer neural networks can learn the XOR function. • All that is needed for this is to find the right algorithm for their training.

27. Neural Networks Resurrection • Hopfield net (John Hopfield, 1982) • It can learn a couple of pictures (patterns). • Self-Organizing Map (SOM) (Teuvo Kohonen, 1982) • It can do unsupervised learning. • Backpropagation (Arthur Bryson and Yu-Chi Ho, 1969) • algorithm for training of a multilayer neural network • It needs network’s neurons not to have a sharp threshold. • Because it was not noticed, it was then rediscovered several times in the 70’s and the 80’s of the 20th century and popularized in 1986. • NETtalk (Terry Sejnowski and Charles Rosenberg, 1986) • Multi-layer neural network, that learned English pronunciation and could generalize. • It used the backpropagation algorithm.

28. The Most Important AI Laboratories • MIT (Massachusetts Institute of Technology) • 1959 - John McCarthy and Marvin Minsky founded Artificial Intelligence Laboratory. • SRI (Stanford Research Institute) • 1963 - John McCarthy founded AI Laboratory. • CMU (Carnegie Mellon University) • 1980 - Raj Reddy founded The Robotics Institute. • IBM • AT&T Bell Labs • University of Edinburgh

29. Present • Robotic toys, space probes • Robotics in machinery • Home appliances (washers, vacuum cleaners) • Data Mining, fraud detection, spam filtering • Searching on the Internet (web agents) • Modeling of interactive processes (agents) • E-business – e-shops personalization • Intelligent tutoring systems and SW interfaces • Role-playing games, chess programs • Speech and video recognition • Machine translation