John McCarthy

1. John McCarthy • Pioneer in AI • Formalize common-sense reasoning • Also • Proposed timesharing • Mathematical theory • …. • Lisp stems from interest in symbolic computation (math, logic)

2. Language speeds www.bagley.org/~doug/shoutout: Completely Random and Arbitrary Point System

3. Example of formula (defun roots (a b c) (list (/ (+ (- b) (sqrt (- (expt b 2) (* 4 a c)) )) (* 2 a)) (/ (+ (- b) (sqrt (- (expt b 2) (* 4 a c)) )) (* 2 a))))

4. eval and quote > (eval (cdr '(a + 2 3))) 5 > (setq a 'b) b > a b > b error: unbound variable - b if continued: try evaluating symbol again 1> [ back to top level ] > (set 'a 'b) b > (eval (eval ''a)) b > 'a a

5. eval and quote > (eval (eval '(quote a))) b > 'a a > (eval '(list '* 9 6)) (* 9 6) > (eval (eval '(list * 9 6))) error: bad function - (* 9 6) 1> [ back to top level ] > (eval (eval '(list '* 9 6))) 54

6. Examples of tail recursion • If last operation in function is recursive call, overwrite actuals and go to beginning of code: (defun last (lis) ; finds the last element of the list (if (null? (cdr lis) (car lis)) (last (crd lis)))) ; can be done with loop (defun length (lis) ; calculates the length of the list (if (null? lis) 0) (+ 1 (length (cdr lis)))) ; not tail recursive!

7. Example of Tree Recursion: Fibonacci • Writing a function to compute the nth Fibonacci number • Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, … • fib(0) = 0 • fib(1) = 1 • fib(n) = fib(n-2) + fib(n-1)

8. Short Version of tree recursion (defun fib (n) (cond ((eql n 0) 0) ; base case ((eql n 1) 1) ; base case (t (+ (fib (- n 1)) ; recursively compute fib(n) (fib (- n 2))))))

9. Complete Version with Error Checking and Comments (defun fib (n) "Computes the nth Fibonacci number." (cond ((or (not (integerp n)) (< n 0)) ; error case (error "~s must be an integer >= 0.~&" n)) ((eql n 0) 0) ; base case ((eql n 1) 1) ; base case (t (+ (fib (- n 1)) ; recursively compute fib(n) (fib (- n 2))))))

10. Problems: 1. Write a function (power 3 2) = 3^2 = 9 2. Write a function that counts the number of atoms in an expression. (count-atoms '(a (b) c)) --> 3 3. (count-anywhere 'a '(a ((a) b) a)) --> 3 4. (dot-product '(10 20) '(3 4)) --> 10x3 + 20x4 = 110 5. Write a function (flatten '(a (b) () ((c)))) --> (a b c) which removes all levels of parenthesis and returns a flat list of atoms. 6. Write a function (remove-dups '(a 1 1 a b 2 b)) --> (a 1 b 2) which removes all duplicate atoms from a flat list. (Note: there is a built-in remove-duplicates in Common Lisp, do not use it).

11. Solutions 1-3 (defun power (a b) "compute a^b - (power 3 2) ==> 9" (if (= b 0) 1 (* a (power a (- b 1))))) (defun count-atoms (exp) "count atoms in expresion - (count-atoms '(a (b) c)) ==> 3" (cond ((null exp) 0) ((atom exp) 1) (t (+ (count-atoms (first exp)) (count-atoms (rest exp)))))) (defun count-anywhere (a exp) "count performances of a in expresion - (count-anywhere 'a '(a ((a) b) (a))) ==> 3" (cond ((null exp) 0) ((atom exp) (if (eq a exp) 1 0)) (t (+ (count-anywhere a (first exp)) (count-anywhere a (rest exp))))))

12. Solutions (defun flatten (exp) "removes all levels of paranthesis and returns flat list of atomsi (flatten '(a (b) () ((c)))) ==> (a b c)" (cond ((null exp) nil) ((atom exp) (list exp)) (t (append (flatten (first exp)) (flatten (rest exp))))))

13. Iteration – adding all elements from a list • Iteration is done by recursion • Analogous to while-loop (defun plus-red (a) (if (null a) 0 (plus (car a) (plus-red (cdr a)) )) )

14. Nested Loops • Example : Cartesian product (defun all-pairs (M N) (if (null M) nil (append (distl (car M) N) (all-pairs (cdr M ) N )) )) (defun distl (x N) (if (null N) nil (cons (list x (car N)) (distl x (cdr N)) )) )

15. Hierarchical structures • Are difficult to handle iteratively example: equal function • eq only handles atoms • initial states • If x and y are both atoms (equal x y) = (eq x y) • If exactly one of x and y is atom (equal x y) = nil (and (atom x) (atom y) (eq x y)) • use car and cdr to write equal recursively

16. Equivalency of recursion and iteration • it may be seemed that recursion is more powerful than iteration • in theory these are equivalent • As we said iteration can be done by recursion • by maintaining a stack of activation records we can convert a recursive program to an iterative one.

17. Functional arguments and abstraction • Suppress details of loop control and recursion example: applying a function to all elements of list (defunmapcar (f x) (if (null x) nil (cons (f (car x)) (mapcar f (cdr x)) )) ) (defun reduce (f a x) (if (null x) a (f (car x) (reduce f a (cdr x) )) ) )

18. Few more functions let* • let*is similar tolet, but thebindingsof variables are performed sequentially rather than in parallel. • The expression for theinit-formof avarcan refer tovarspreviously bound in thelet*. • The form (let* ((var1 init-form-1) (var2 init-form-2) ... (varminit-form-m)) declaration1 declaration2 ... declarationp form1 form2 ... formn) • first evaluates the expressioninit-form-1, • then binds the variablevar1to that value; • then it evaluatesinit-form-2and bindsvar2, and so on. • The expressionsformjare then evaluated in order; the values of all but the last are discarded (that is, the body oflet*is an implicitprogn).

19. Few more functions reduce reduceuses a binary operation,function, to combine theelementsofsequenceboundedbystartandend. Thefunctionmust accept asargumentstwoelementsofsequenceor the results from combining thoseelements. Thefunctionmust also be able to accept no arguments. Ifkeyis supplied, it is used is used to extract the values to reduce. Thekeyfunction is applied exactly once to each element ofsequencein the order implied by the reduction order but not to the value ofinitial-value, if supplied. Thekeyfunction typically returns part of theelementofsequence. Ifkeyis not supplied or isnil, thesequenceelementitself is used. The reduction is left-associative, unlessfrom-endistruein which case it is right-associative. Ifinitial-valueis supplied, it is logically placed before the subsequence (or after it iffrom-endistrue) and included in the reduction operation. In the normal case, the result ofreduceis the combined result offunction's being applied to successive pairs ofelementsofsequence. If the subsequence contains exactly oneelementand noinitial-valueis given, then thatelementis returned andfunctionis not called. If the subsequence is empty and aninitial-valueis given, then theinitial-valueis returned andfunctionis not called. If the subsequence is empty and noinitial-valueis given, then thefunctionis called with zero arguments, andreducereturns whateverfunctiondoes. This is the only case where thefunctionis called with other than two arguments.

20. Few more functions reduce Examples: (reduce #'* '(1 2 3 4 5)) => 120 (reduce #'append '((1) (2)) :initial-value '(i n i t)) => (I N I T 1 2) (reduce #'append '((1) (2)) :from-end t :initial-value '(i n i t)) => (1 2 I N I T) (reduce #'- '(1 2 3 4)) == (- (- (- 1 2) 3) 4) => -8 (reduce #'- '(1 2 3 4) :from-end t) ;Alternating sum. == (- 1 (- 2 (- 3 4))) => -2 (reduce #'+ '()) => 0 (reduce #'+ '(3)) => 3 (reduce #'+ '(foo)) => FOO (reduce #'list '(1 2 3 4)) => (((1 2) 3) 4) (reduce #'list '(1 2 3 4) :from-end t) => (1 (2 (3 4))) (reduce #'list '(1 2 3 4) :initial-value 'foo) => ((((foo 1) 2) 3) 4) (reduce #'list '(1 2 3 4) :from-end t :initial-value 'foo) => (1 (2 (3 (4 foo))))

21. Few more functions reduce • SOME function searches the sequences for values for which predicate returns true. • It there is such list of values that occupy same index in each sequence, return value is true, otherwise false. (some #'alphanumericp "") => NIL (some #'alphanumericp "...") => NIL (some #'alphanumericp "ab...") => T (some #'alphanumericp "abc") => T (some #'< '(1 2 3 4) '(2 3 4 5)) => T (some #'< '(1 2 3 4) '(1 3 4 5)) => T (some #'< '(1 2 3 4) '(1 2 3 4)) => NIL

22. Building Problem Solvers in LISP • http://www.qrg.northwestern.edu/bps/directory.html

23. Genetic algorithm Common Lisp code

24. Genetic algorithm Common Lisp code from Dean: Takes population as argument (defun reproduce (population)(let ((offspring nil)(d(distribution population) ))(dotimes (i (/ (length population) 2) )(let ((x (selectone d) )(y (selectone d)) ) (crossover x y)(setq offspring(nconc (list x y) offspring) )))offspring)) Initializes offspring to empty list Distributes initial population Distributed population Repeats for the length of half population Selects parent x, one from population Selects parent y, one from population Does crossover of parents x and y End of adding new children to list offspring Creates new list offspring by adding new children x and y to old list offspring Returns new list offspring

25. Genetic algorithm Common Lisp code (importance of drawing trees for larger functions) (defun reproduce (population)(let ((offspring nil)(d (distribution population) ))(dotimes (i (/ (length population) 2) )(let ((x (selectone d) )(y (selectone d)) ) (crossover x y)(setq offspring (nconc (list x y) offspring) ))) offspring)) reproduce offspring distribution dotimes let length Selects parent x, one from population d selectone setq Selects parent y, one from population d crossover Does crossover of parents x and y nconc Creates new list offspring I found it very useful to create for myself such trees to know who is calling whom Returns list offspring

26. User defined function distribution Takes the initial population and distributes it according to fitness function Distributes initial population (defundistribution (population) (let* ((genotypes(remove-duplicates population :test #'equal))(sum(apply #'+ (mapcar#'fitness genotypes)))) (mapcar #'(lambda (x) (cons (/ (fitness x) sum) x))genotypes))) Creates genotypes by removing the duplicates from population Uses function fitness Creates sum of fitness values Calculates fitness of all elements from list genotypes Creates a pair of normalized fitness and a genotype x Creates list of pairs for all elements of list genotypes

27. User defined function selectone This function was used in function reproduce two pages earlier (defunselectone (distribution)(let ((random (random 1.0))(prob 0)genotype)(some #'(lambda (pair)(incfprob (first pair))(if (> random prob) nil ;;else(setqgenotype (rest pair)))) distribution)(mutate genotype))) Selects one parent from population Initializes random Initializes prob Initializes genotype Compares elements of distribution. Selects those with higher prob Apply to the original distribution Calls function mutate to mutate the genotype

28. User defined functions for crossover and mutation x and y are chromosomes (defuncrossover (x y) (if (> (random 1.0) 0.6) (list x y);; in this case do nothing, return x and y as in input;;else (let* ((site(random (length x)))(swap(rest (nthcdr site x))))(setf(rest (nthcdrsite x))(rest (nthcdrsite y)))(setf (rest (nthcdrsite y)) swap)))) (defunmutate (genotype)(mapcar #'(lambda (x)(if (> (random 1.0) 0.03) x;; else(if (= x 1) 0 ;; else 1)))genotype)) Site is a place of cut Swap is temporary location Creates child 1 Execute crossover Creates child 2 Does mutation of a single genotype by flipping bits Can do several mutations at once

29. Calculate fitness function Calculates fitness of a genotype x (defun fitness (x)(let ((xarg(/ (string2num x) 1073741823.0))(v '(0.5 0.25 1.0 0.25))(c '(0.125 0.375 0.625 0.875))(w 0.003)) (reduce #'+ (mapcar #'(lambda (vici)(let ((xc (- xargci)))(* vi (exp(* -1 (/ (* 2 w)) xc xc)))))v c)))) (defunstring2num (s) (loop for xi in (reverse s) for p = 1 then (* p 2) sum (* xi p))) Set parameters Converts string to a number