Problem Solving by Search by Jin Hyung Kim Computer Science Department KAIST

1. Problem Solving by Searchby Jin Hyung KimComputer Science DepartmentKAIST

2. Example of Representation • Euler Path

3. Graph Theory • Graph consists of • A set of nodes : may be infinite • A set of arcs(links) • Directed graph, underlying graph, tree • Notations • node, start node(root), leaf (tip node), root, path, ancestor, descendant, child(children, son), parent(father), cycle, DAG, connected, locally finite graph, node expansion

4. State Space Representation • Basic Components • set of states {s} • set of operators { o : s -> s } • control strategy { c: sn -> o } • State space graph • State -> node • operator -> arc • Four tuple representation • [N, A, S, GD], solution path

5. Examples of SSR • TIC_TAC_TOE • n2-1 Puzzle • Traveling salesperson problem (TSP)

6. Search Strategies • A strategy is defined by picking the order of node expansion • Search Direction s • Forward searching (from start to goal) • Backward searching (from goal to start) • Bidirectional • Irrevocable vs. revocable • Irrevocable strategy : Hill-Climbing • Most popular in Human problem solving • No shift of attention to suspended alternatives • End up with local-maxima • Commutative assumption Applying an inappropriate operators may delay, but never prevent the eventual discovery of solutions. • Revocable strategy : Tentative control • An alternative chosen, others reserve

7. Evaluation of Strategies • Completeness • Does it always find a solution if one exists ? • Time Complexity • Number of nodes generated/expanded • Space complexity • Maximum number of nodes in memory • Optimality • Does it always find a least-cost solution ? • Time and Space complexity measured by • b – maximum branching factors of the search tree • d – depth of least-cost solution • m - maximum depth of the state space (may be

8. Implementing Search Strategies • Uninformed search • Search does not depend on the nature of solution • Systematic Search Method • Breadth-First Search • Depth-First Search (backtracking) • Depth-limited Search • Uniform Cost Search • Iterative deepening Search • Informed or Heuristic Search • Best-first Search • Greedy search (h only) • A* search (g + h) • Iterative A* search

9. start put s in OPEN Fail OPEN empty ? Select & Remove the a node of OPEN and put it in CLOSE (call it n) Success any succesor = goal ? X-First Search Algorithm yes Expand n. Put successors at the end of OPEN pointers back to n yes

10. Comparison of BFS and DFS • BFS always terminate if goal exist cf. DFS on locally finite infinite tree • Gurantee shortest path to goal - BFS • Space requirement • BFS - Exponential • DFS - Linear, keep children of a single node • Which is better ? BFS or DFS ?

11. Uniform Cost Search • A Genaralized version of Breadth-First Search • C(ni, nj) = cost of going from ni to nj • g(n) =(tentative minimal) cost of a path from s to n. • Guarantee to find the minimum cost path • Dijkstra Algorithm

12. start put s in OPEN, set g(s) = 0 Fail OPEN empty ? Remove the node of OPEN whose g(.) value is smallest and put it in CLOSE (call it n) Success Expand n. calculate g(.) of successor Put successors to OPEN pointers back to n Uniform Cost Search Algorithm yes yes n = goal ?

13. Iterative Deepening Search • Compromise of BFS and DFS • Save on Storage, guarantee shortest path • Additional node expansion is negligible • proc Iterative_Deeping_Search(Root) • begin • Success := 0; • for (depth_bound := 1; depth_bound++; Success == 1) • { depth_first_search(Root, depth_bound); • if goal found, Success := 1; • } • end

14. Iterative Deeping (l=0)

15. Iterative Deeping (l=1)

16. Iterative Deeping (l=2)

17. Iterative Deeping (l=3)

18. Properties of IDS • Complete ?? • Time ?? (d+1)b0 + db1 + (d-1)b2 + … + bd = O(bd) • Space ?? : O(bd) • Optimal ?? Yes, if step cost = 1 • Can be modified to explore uniform cost tree ? • Numerical comparison b=10 and d=5, solution at far right N(IDS) = 50 + 400 + 3,000 + 20,000 + 100,000 = 123,450 N(BFS) = 10 + 100 + 1000 + 10000 + 100000 + 999,990 = 1,111,100

19. Informed Search

20. x x x o x o o x o x x x x o o x o x o Use of Heuristics to select the Best • Tic-tac-toe

21. Tic-tac-toe • Most-Win Heuristics x x x 2 win 3 win 4 win

22. 8-Puzzel Heuristics 3 1 2 3 2 1 8 8 4 4 6 5 7 6 5 7 • # of Misplaced tiles • Sum of Manhattan distance 2 3 8 3 2 2 3 1 1 4 8 4 8 1 4 7 7 6 5 5 7 6 5 6 a c b

23. Heuristics for Road Map Problem

24. start put s in OPEN, compute f(s) Fail OPEN empty ? Remove the node of OPEN whose f(.) value is smallest and put it in CLOSE (call it n) Success Expand n. calculate f(.) of successor Put successors to OPEN pointers back to n Best First Search Algorithm( for tree search) yes yes n = goal ?

25. Algorithm A • Best-First Algorithm with f(n) = g(n) + h(n) where g(n) : cost of n from start to node n h(n) : heuristic estimate of the cost from n to a goal • Algorithm is admissible if it terminate with optimal solution • What if f(n) = f*(n) where f*(n) = g*(n) + h*(n) where g*(n) = shortest cost to n h*(n) = shortest actual cost from n to goal

26. Algorithm A* (Branch and Bound method) • Algorithm A becomes A* if h(n) h*(n) • Algorithm A* is admissible • can you prove it ? • If h(n) 0, A* algorithm becomes uniform cost algorithm • Uniform cost algorithm is admissible • If n* is on optimal path, f*(n*) = C* • f*(n) > C* implies that n is not on optimal path • A* terminate in finite graph

27. Examples of Admissible Heuristics • 8 puzzle heuristic • N queen problem, Tic-tac-toe • Air distance heuristic • Traveling Salesperson Problem • Minimum spanning tree heuristics • ….

28. Iterative Deeping A* • Modification of A* • use threshold as depth bound • To find solution under the threshold of f(.) • increase threshold as minimum of f(.) of previous cycle • Still admissible • same order of node expansion • Storage Efficient – practical • but suffers for the real-valued f(.) • large number of iterations

29. start put s in OPEN, compute f(s) OPEN empty ? Remove the node of OPEN whose f(.) value is smallest and put it in CLOSE (call it n) Success Iterative Deepening A* Search Algorithm ( for tree search) set threshold as h(s) yes threshold = min( f(.) , threshold ) yes n = goal ? • Expand n. calculate f(.) of successor • if f(suc) < threshold then • Put successors to OPEN if • pointers back to n

30. Memory-bounded heuristic Search • Recursive best-first search • A variation of Depth-first search • Keep track of f-value of the best alternative path • Unwind if f-value of all children exceed its best alternative • When unwind, store f-value of best child as its f-value • When needed, the parent regenerate its children again. • Memory-bounded A* • When OPEN is full, delete worst node from OPEN storing f-value to its parent. • The deleted node is regenerated when all other candidates look worse than the node.

31. Monotonicity (consistency) • A heuristic function is monotone if for all states ni and nj = suc(ni) h(ni) - h(nj) cost(ni,nj) and h(goal) = 0 • Monotone heuristic is admissible

32. More Informedness (Dominate) • For two admissible heuristic h1 and h2, h2 is more informed than h1 if h1(n) h2(n) for all n • for 8-tile problem • h1 : # of misplaced tile • h2 : sum of Manhattan distance h1(n) h2(n) h*(n) 0

33. Generation of Heuristics • Relaxed problem solution is an admissible heuristics • Manhattan distance heuristic • Solution of subproblems • combining several admissible heuristics h(n) = max{ h1(n), …, hn(n)} • Use of Pattern databases • Max of heuristics of sub-problem pattern database • 1/ 1000 in 15 puzzle compared with Manhattan • Addition of heuristics of disjoint sub-problem pattern database • 1/ 10,000 in 24 puzzle compared with Manhattan • disjoint subdivision is not possible for Rubic’s cube

34. Semi-addmissible heuristicsDynamic Weighting, Risky heuristics • If { h(n)h*(n) + e }, then C(n) (1+e) C*(n) • f(n) = g(n) + h(n) + e[1-d(n)/N] h(n) • At shallow level : depth first excursion • At deep level : assumes admissibility • Use of non-admissible heuristics with risk • Utilize heuristic functions which are admissible in the most of cases • Statistically obtained heuristics PEARL, J., AND KIM, J. H. Studies in semi-admissible heuristics. IEEE Trans. PAMI-4, 4 (1982), 392-399

35. Performance Measure • Penetrance • how search algorithm focus on goal rather than wander off in irrelevant directions • P = L / T • Effective Branching Factor (B) • B + B2 + B3 + ..... + BL = T • less dependent on L

36. Planning : Monkey and Banana • Monkey is on floor at (x1, y1), banana is hanging at (x2, y2), and box is at (x3, y3). Monkey can grab banana if he push box under banana and climb on it. Develop a state-space search representation for this situation and show how monkey can grab banana.

37. Local Search and Optimization • Local search • less memory required • Reasonable solutions in large (continuous) space problems • Can be formulated as Searching for extreme value of Objective function find i = ARGMAX { Obj(pi) } where pi is parameter

38. Search for Optimal Parameter • Deterministic Methods • Step-by-step procedure • Hill-Climbing search, gradient search • ex: error back propagation algorithm • Finding Optimal Weight matrix in Neural Network training • Stochastic Methods • Iteratively Improve parameters • Pseudo-random change and retain it if it improves • Metroplis algorithm • Simulated Annealing algorithm • Genetic Algorithm

39. Hill Climbing Search 1. Set n to be the initial node 2. If obj(n) > max { obj(childi(n)) } then exit 3. Set n to be the highest-value child of n 4. Return to step 2 • No previous state information • No backtracking • No jumping • Gradient Search • Hill climbing with continuous, differentiable functions • step width ? • Slow in near optimal

40. State space landscape Real World

41. Hill-climbing :Drawbacks • Local maxima • At Ridge • Stray in Plateau • Slow in Plateau • Determination of proper Step size • Cure • Random restart • Good for Only few local maxima Global Maximum

42. Local Beam Search • Keep track of best k states instead of 1 in hill-climbing • Full utilization of given memory • Variation: Stochastic beam search • Select k successors randomly

43. Iterative Improvement Algorithm • Basic Idea • Start with initial setting • Generate a random solution • Iteratively improve the quality • Good For hard, practical problems • Because it Keeps current state only and No look-ahead beyond neighbors • Implementation • Metropolis algorithm • Simulated Annealing algorithm • Genetic algorithm

44. Metropolis algorithm • Modified Monte Carlo method • Suppose our objective is to reach the state minimizing energy function 1. Randomly generate a new state, Y, from state X 2. If E(energy difference between Y and X) < 0 then move to Y (set Y to X) and goto 1 3. Else 3.1 select a random number,  3.2 if  < exp(- E / T) then move to Y (set Y to X) and goto 1 3.3 else goto 1

45. From Statistical Mechanics • In thermal equilibrium, probability of state i • energy of state i • absolute temperature • Boltzman constant • In NN • define

46. p 1 eDE/T 0 DE Simulated AnnealingProbability distribution

47. Simulated Annealing algorithm • What is annealing? • Process of slowly cooling down a compound or a substance • Slow cooling let the substance flow around  thermodynamic equilibrium • Molecules get optimum conformation contraction : cause stress

48. Simulated Annealing • Simulates slow cooling of annealing process • Solves combinatorial optimization • variant of Metropolis algorithm • by S. Kirkpatric (83) • finding minimum-energy solution of a neural network = finding low temperature state of physical system • To overcome local minimum problem • Instead always going downhill, try to go downhill ‘most of the time’

49. Iterative algorithm comparison • Simple Iterative Algorithm 1. find a solution s 2. make s’, a variation of s 3. if s’ is better than s, keep s’ as s 4. goto 2 • Metropolis Algorithm • 3’ : if (s’ is better than s) or ( within Prob), then keep s’ as s • With fixed T • Simulated Annealing • T is reduced to 0 by schedule as time passes

50. Simulated Annealing algorithm functionSimulated-Annealing(problem, schedule) returns a solution state inputs: problem, a problem local variables: current, a node next, a node T, a “temperature” controlling the probability of downward steps current Make-Node(Initial-State[problem]) for t1 to infinity do T  schedule[t] if T=0 thenreturncurrent next  a randomly selected successor of current DE  Value[next] – Value[current] ifDE>0 thencurrentnext elsecurrentnext only with probability eDE/T