Heuristic Programming

1. Heuristic Programming CIS 487/587 Bruce R. Maxim UM-Dearborn

2. State Space Search • Many AI problems can be conceptualized as searching a well defined set of related problem states • Puzzle solving • Game playing • Generate and test • Genetic algorithms • Scheduling algorithms

3. Water Jug Problem • Problem • You have a four gallon jug and a three gallon jug and the goal is to come up with exactly two gallons of water. • Operations • Dump bucket contents on the ground • Dump bucket contents in other bucket • Fill bucket to top with water

4. Tree Representation (0,0) (4,0) (0,3)   (0,3) (0,0) (4,3) (1,3) (3,0) (4,3) (0,0)

5. Digraph Representation (0,0) (4,0) (0,3) (1,3) (4,3) (3,0)

6. Adjacency List • (0 0)  ((4 0) (0 3)) • (4 0)  (( 4 3) (0 0) (1 3) (0 3)) • (0 3)  ((4 3) (3 0) (0 0)) • (1 3)  ((1 0) (4 3) (4 0) (0 3))

7. Good Knowledge Representations • Important things made clear /explicit • Expose natural constraints • Must be complete • Are concise • Transparent (easily understood) • Information can be retrieved & stored quickly • Detail suppressed (can be found as needed) • Computable using existing procedures

8. River Puzzle • Problem • There are four items a farmer, wolf, goose, and corn. The farmer can only take one item across the river at a time. • Constraints • Wolf will eat the goose if left alone with it • Goose will eat the corn if left alone with it

9. Problem Analysis • Each of the 4 problem elements can be considered a Boolean variable based on its bank location • There are 24 or 16 possible states • 6 of these states fail the “no eat” test • That leaves 10 states to consider which could be linked in C(10,2) = 45 ways • However only 10 of these links can really be used to connect states

10. F=Farmer W=Wolf G=Goose C=Corn ~=River W F ~ W F G F W F G ~ G F ~ W C W C C ~ G F G ~ C F ~ W C F ~ C F W W G ~ G G ~ G C C C F C W ~ G W

11. Solution • Once graph is constructed finding solution is easy (simply find a path) • AI programs would rarely construct the entire graph explicitly before searching • AI programs would generate nodes as needed and restrict the path search to the nodes generated • May use forward reasoning (initial to goal) or backward reasoning (goal to initial)

12. Rules • Many time these state transitions are written as rules: If ((Wolf Corn) (Farmer Goose)) Then ((Wolf Corn Farmer) (Goose)) • Depending on which direction you are reasoning you match either the left (if) or right (then) part

13. Potential Problems • If you have more than one state to move to then you need a procedure to choose one • Exact matches often do not occur in the real world so you may need to measure how “close” you are to an exact match

14. Control Strategies • A good control strategy causes motion (ideally toward the goal state) • The control strategy should be systematic and not allow repeated use of the same rule sequences (to avoid infinite loops)

15. Heuristics • AI programming often relies on the use of heuristics • Heuristics are techniques that improves efficiency by trading “speed” for “completeness”

16. Search Considerations • Can the search algorithm work for the problem space? • Is the search algorithm efficient? • Is it easy to implement? • Is search the best approach or is more knowledge better?

17. Example Graph A F S B C

18. Adjacency List (setf (get 's 'children) '(a b)) (setf (get 'a 'children) '(s b f)) (setf (get 'b 'children) '(s a c)) (setf (get 'c 'children) '(b f)) (setf (get 'f 'children) '(a c))

19. “Any Path” Search Algorithms • Depth First Search • Hill Climbing • Best First Search • Breadth First Search • Beam Search

20. Depth First Search Add root to queue of partial paths Until queue is empty or goal is attained If first queue element equals the goal then do nothing Else remove the first queue element add its children to the front of the queue of the partial paths If goal is attained then announce success

21. Depth First Output > (depth 's 'f) ((s)) ((a s) (b s)) ((b a s) (f a s) (b s)) ((c b a s) (f a s) (b s)) ((f c b a s) (f a s) (b s)) (s a b c f)

22. Depth First Weakness • Depth first is not good for “tall” trees when it is possible to over commit to a “bad” path early • In our example depth first missed a complete solution because it focused on checking the first partial path and did not test the others in the queue

23. Improving Search • We will need to add knowledge to our algorithms to make them perform better • We will need to add some distance estimates, even using “best guesses” would help • We will need to begin sorting part of the the queue of partial paths

24. Example Graph 1 A 0 2 F S B C 2 1

25. British Museum Search • If enough monkeys had enough type writers and enough time then they could recreate all the knowledge housed in the British Museum • So we could compute every path by trial and error then pick the shortest

26. Branch and Bound • An optimal depth first search • Requires the ability to measure the distance traveled “so far” using an ordinal scale (does not require exact distances) • Entire queue of partial path is sorted after it is modified

27. Branch and Bound Add root to queue of partial paths Until queue is empty or goal is attained If first queue element equals the goal then do nothing Else remove the first queue element add its children to the front of the queue of the partial paths sort the queue of partial paths by distance traveled If goal is attained then announce success

28. Branch and Bound Output > (branch 's 'f) ((s)) ((a s) (b s)) ((b s) (b a s) (f a s)) ((a b s) (c b s) (b a s) (f a s)) ((c b s) (b a s) (f a s) (f a b s)) ((b a s) (f a s) (f c b s) (f a b s)) ((f a s) (c b a s) (f c b s) (f a b s)) (s a f)

29. A* • Makes use of both a cost measure and remaining distance underestimator • Removes redundant paths from the queue of partial paths

30. A* Add root to queue of partial paths Until queue is empty or goal is attained If first queue element equals the goal then do nothing Else remove the first queue element add its children to the front of the queue of the partial paths sort the queue of partial paths by distance traveled plus the estimate of distance to goal remove redundant paths If goal is attained then announce success

31. A* > (a* 's 'f) ((s)) ((a s) (b s)) ((f a s) (b s)) (s a f)

32. A* Weaknesses • Still requires the use of a “natural” distance measure • Underestimator is a guess as best • Sorting takes time • Removing paths takes time

33. Generate and Test • Search can be viewed generate and test procedures • Testing for a complete path is performed after varying amount of work has been done by the generator • At one extreme the generator generates a complete path which is evaluated • At the other extreme each move is tested by the evaluator as it is proposed by the generator

34. Improving Search-Based Problem Solving Two options • Improve “generator” to only generate good moves or paths • Improve “tester” so that good moves recognized early and explored first

35. Using Generate and Test • Can be used to solve identification problems in small search spaces • Can be thought of as being a depth-first search process with backtracking allowed • Dendral – expert system for identifying chemical compounds from NMR spectra

36. Dangers • Consider a safe cracker trying to use generate a test to crack a safe with a 3 number combination (00-00-00) • There are 1003 possible combinations • At 3 attempts/minute it would take 16 weeks of 24/7 work to try each combination in a systematic manner

37. Generator Properties • Complete • capable of producing all possible solutions • Non-redundant • don’t propose same solution twice • Informed • make use of constraints to limit solutions being proposed

38. Dealing with Adversaries • Games have fascinated computer scientists for many years • Babbage • playing chess on Analytic Engine • designed Tic-Tac-Toe machine • Shanon (1950) and Turing (1953) • described chess playing algorithms • Samuels (1960) • Built first significant game playing program (checkers)

39. Why games attracted interest of computer scientists? • Seemed to be a good domain for work on machine intelligence, because they were thought to: • provide a source of a good structured task in which success or failure is easy to measure • not require much knowledge (this was later found to be untrue)

40. Chess • Average branching factor for each position is 35 • Each player makes 50 moves in an average game • A complete game has 35100 potential positions to consider • Straight forward search of this space would not terminate during either players lifetime

41. Games • Can’t simply use search like in “puzzle” solving since you have an opponent • Need to have both a good generator and an effective tester • Heuristic knowledge will also be helpful to both the generator and tester

42. Ply • Some writers use the term “ply” to mean a single move by either player • Some insists “ply” is made up of a move and a response • I will use the first definition, so “ply” is the same as the “depth - 1” of the decision tree rooted at the current game state

43. Static Evaluation Function • Used by the “tester” • Similar to “closerp” from our heuristic search work in A* type algorithms • In general it will only be applied to the “leaf” node of the game tree

44. Static Evaluation Functions • Turing (Chess) sum of white values / sum of black values • Samuels (Checkers) linear combination with interaction terms • piece advantage • capability for advancement • control of center • threat of fork • mobility

45. Role of Learning • Initially Samuels did not know how to assign the weights to each term of his static evaluation function • Through self-play the weights were adjusted to match the winner’s values c1 * piece advan + c2 * advanc + …

46. Tic Tac Toe

47. Tic Tac Toe 100A + 10B + C – (100D + 10E + F) A = number of lines with 3x’s B = number of lines with 2 x’s C = number of lines with single x D = number of lines with 3 O’s E = number of lines with 2 O’s F = number of lines with a single O

48. X X O O O X A = 0 B = 0 C = 1 D = 0 E = 1 F = 1 100 (0) + 10(0) + 1 – (100 (0) + 10(1) + 1) = 1 – 11 = -10 Example

49. Weakness • All static evaluation functions suffer from two weaknesses • information loss as complete state information mapped to a single number • Minsky’s Credit Assignment problem • it is extremely difficult to determine which move in a particular sequence of moves caused a player to win or loss a game (or how much credit to assign to each for end result)

50. What do we need for games? • Plausible move generator • Good static evaluation functions • Some type of search that takes opponent behavior into account for nontrivial games