Explorations in Artificial Intelligence

1. Explorations in Artificial Intelligence Prof. Carla P. Gomes gomes@cs.cornell.edu Module 4-1 Constraint Programming

2. Constraint Programming • Constraint programming is a research area for declarative description • and effective solving of large, particularly combinatorial; • Combines different fields: • Artificial Intelligence; • Programming Languages; • Symbolic Computing; • Computational Logic; • Operations Research; Very competitive and even outperforming other approaches for solving hard combinatorial problems Recent years there have been several successful applications – it is now part of standard OR books.

3. Began in 1980s from AI world • Prolog III (Marseilles, France) • CLP(R) • CHIP (ECRC, Germany) • Application areas • Image processing, Scheduling, sequencing, resource and personnel allocation, etc. etc. • Active research area • Specialized conferences (CP, CP/AI-OR, …) • Journal (Constraints) • Companies (Ilog and others)

4. Constraint Programming • Two main contributions • A new approach to combinatorial optimization • Orthogonal and complementary to standard OR methods • Combinatorial versus numerical • A new language for combinatorial optimization • Rich language for constraints • Language for search procedures

5. Modeling in Constraint Programming • A rich constraint language • Arithmetic, higher-order, logical constraints • Global constraints for natural substructures • Specification of a search procedure • Definition of search tree to explore • Specification of search strategy Satisfiability is a particular case of a constraint programming language.

6. Constraint Programming • 1- Problem Formulation: • A problem is a finite set of constraints involving a finite set of variables. • Constraint Satisfaction Problems (CSP)  feasibility problem only and --- SAT is a particular case of CSP; • Constraint Optimization Problems (COP) if in addition the solution is required to maximize an objective function • 2- Problem Solution: • Domain specific methods • General Solution Methods

7. Constraint programming is the study of computational systems based on constraints. • A constraint  • Logical relation among several unknowns (or variables), each taking a value in a given domain. A constraint thus restricts the possible values that variables can take, it represents some partial information about the variables of interest. • Very general framework. Very expressive;

8. Constraint Satisfaction Problems • A Constraint Satisfaction Problem (P) consists of: • A set of variables, X1,…,Xn ; • For each variable Xi, a domain of possible values, Dom(Xi). • A set of constraintsC1,…,Cm . • A constraint Ci on k variables Xci1,…,Xcik is a relation R(Xci1,…,Xcik)Dci1x…xDcik that specifies the valid combinations of values for the variables involved. • A (feasible) solution to a CSP is an assignment of values to all the variables, one value per variable, satisfying all the constraints. Why is Satisfiability a particular case of CSP?

9. A variable V can be assigned a value v (V  v) if and only if v Є Dom(V). • A consistent set of assignments is a set of assignments {V1  v1, …, Vk  vk}, such that all the constraints are satisfied, and each of the Vi is unique; • A (feasible) solution is a complete consistent set of assignments, i.e., one consistent set of assignments in which the number of variables in the set is equal to the number of variables in the problem (n).

10. Any subset of a (feasible) solution is consistent; • We might be interested in finding: • Any (feasible) solution • Enumerating or counting all the (feasible) solutions or all optimal solutions • Finding the optimal solution --- i.e., (feasible) solution(s) that optimize(s) an objective function

11. Constraint Programming At a Glance: Modeling Graph Coloring Problem: Can we color the regions of the map of Australia, using three colors – red, green, and blue - such that adjacent regions have different colors? Source: AIMA – Russell and Norvig

12. Constraint Programming At a Glance: Modeling 1 – Modeling Rich language Different Models (Binary, N-ary, Global Constraints,etc) Source: AIMA – Russell and Norvig

14. The Constraint Graph

15. Planar Graphs • Special type of graphs  planar graphs. • Planar Graph – is a graph that can be drawn (mathematicians say "can be embedded • in the plane") so that no edges intersect. A nonplanar graph cannot be drawn without • edge intersections. The graph representation of a map (each region corresponds to a node; edges link neighbor regions) is a planar graph. B C G B G D A C D F F A E E

16. Yes Planar graph? Planar vs. NonPlanar Graphs Planar graph K5 K3,3 NonPlanar graph NonPlanar graph

17. A subdivision of a graph results from inserting vertices into edges (for example, changing an edge •——• to •—•—•) and repeating this zero or more times. Example of a subdivision of K3,3. Planar Graph - A finite graph is planar iff it does not contain a subgraph that is a subdivision of K5 (the complete graph on five vertices) or K3,3 (complete bipartite graph on six vertices, three of which connect to each of the other three).

18. Four Color Theorem • The chromatic number of a graph is the least number of colors • that are required to color a graph. • The Four Color Theorem – the chromatic number of a planar graph • is no greater than four. • Proof: Appel and Haken 1976; careful case analysis performed by computer; proof • reduced the infinitude of possible maps to 1,936 reducible configurations (later • reduced to 1,476) which had to be checked one by one by computer. This • reducibility part of the work was independently double checked with different • programs and computers. The computer program ran for hundreds of hours. Four color map.

19. Graph Coloring for Arbitrary Graphs • Four Color Theorem – only applies to planar graphs; • For an arbitrary graph – • finding the chromatic number of an arbitrary graph takes exponential time (in the worst-case); • even finding an approximation to the chromatic number up to a factor of 2 (i.e., a bound which is no more than double the chromatic number) is hard – if we found a polynomial time approximation algorithm with less than a factor of 2, then we would also find a polynomial time algorithm for finding the excat chromatic number of the graph.

20. 1 1 7 7 2 2 6 6 3 3 5 5 4 4 Application of Graph Coloring • Lots of applications involving scheduling and assignments. • Scheduling of final exams – nodes represent finals, edges between finals • denote that both finals have common students (and therefore they have to • have different colors, or different periods). Time Period  courses I (red)  1,6 II (blue)2 III (green)3,5 IV (black) Graph of finals for 7 courses

21. Constraint Programming At a Glance: Search • 2 – Search • Different variants of backtrack search (Depth First Search; Breadth-First Search; Iterative Deepening; branch and bound, etc) •  core of AI technology • Deterministic vs. Randomized backtrack search

22. Standard search formulation (incremental) • Let's start with the straightforward approach, then fix it • States are defined by the values assigned so far • Initial state: the empty assignment { } • Successor function: assign a value to an unassigned variable that does not conflict with current assignment  fail if no legal assignments • Goal test: the current assignment is complete • This is the same for all CSPs • Every solution appears at depth n (or less than n) with n variables use depth-first search

23. Backtracking search • Variable assignments are commutative}, i.e., • [ WA = red then NT = green ] same as [ NT = green then WA = red ] • Only need to consider assignments to a single variable at each node  b = d and there are $d^n$ leaves • Depth-first search for CSPs with single-variable assignments is called backtracking search • Backtracking search is the basic uninformed algorithm for CSPs

24. Backtracking search

25. Backtracking example

26. Backtracking example

27. Backtracking example

28. Backtracking example

29. Improving backtracking efficiency • General-purpose methods can give huge gains in speed: • Which variable should be assigned next? • In what order should its values be tried? • Can we detect inevitable failure early?

30. Most constrained variable • Most constrained variable: choose the variable with the fewest legal values • a.k.a. minimum remaining values (MRV) heuristic

31. Most constraining variable • Tie-breaker among most constrained variables • Most constraining variable: • choose the variable with the most constraints on remaining variables

32. Least constraining value • Given a variable, choose the least constraining value: • the one that rules out the fewest values in the remaining variables • Combining these heuristics makes 1000 queens feasible

33. More advanced… Constraint Programming At a Glance: Consistency • 3 – Consistency • Key concept in Constraint Programming idea: remove inconsistent values from variables’ domains; • Different notions of consistency: • Node consistency (1 variable at a time) • Arc consistency (2 variables at a time) • Forward checking (simple version of arc consistency; • 2 variables at a time) • Hyper-arc consistency (k>2 variables at a time) • Global Constraints

34. Node Consistency • The simplest form of consistency technique; Let’s G=(V,C) denote the constraint • graph of a binary CSP: • Node Consistency • The node representing a variable V in the constraint graph is node consistent if for every value x in the current domain of V, each unary constraint on V is satisfied. • Node consistency achieved by simply removing values from the domain D of each variable V that do not satisfy unary constraints on V.

35. Forward checking • Idea: • Keep track of remaining legal values for unassigned variables • Terminate search when any variable has no legal values

36. Forward checking • Idea: • Keep track of remaining legal values for unassigned variables • Terminate search when any variable has no legal values

37. Forward checking • Idea: • Keep track of remaining legal values for unassigned variables • Terminate search when any variable has no legal values

38. Forward checking • Idea: • Keep track of remaining legal values for unassigned variables • Terminate search when any variable has no legal values

39. Constraint propagation • Forward checking propagates information from assigned to unassigned variables, but doesn't provide early detection for all failures: • NT and SA cannot both be blue! • Constraint propagation repeatedly enforces constraints locally

40. T Arc consistency • Simplest form of propagation makes each arc consistent • X Y is consistent iff for every value x of X there is some allowed y

41. T Arc consistency • Simplest form of propagation makes each arc consistent • X Y is consistent iff for every value x of X there is some allowed y

42. T Arc consistency • Simplest form of propagation makes each arc consistent • X Y is consistent iff for every value x of X there is some allowed y • If X loses a value, neighbors of X need to be rechecked

43. T Arc consistency • Simplest form of propagation makes each arc consistent • X Y is consistent iff for every value x of X there is some allowed y • If X loses a value, neighbors of X need to be rechecked • Arc consistency detects failure earlier than forward checking • Can be run as a preprocessor or after each assignment

44. Arc consistency algorithm AC-3 • Time complexity: O(n2d3)

45. Constraint Programming At a Glance: Arc Consistency If X’s domain is reduced, neighbors of X have to be rechecked.

46. look back look ahead Labelling order Constraint Programming At a Glance: Integration of Search + Consistency • 4 – Search + Consistency Constraint Propagation • Combination of search (backtracking) with consistency techniques methods: • look ahead (preventing conflicts) • forward checking, full look-ahead, etc • look back (learning from conflicts) More advanced R. Bartak 99

47. Examples of Applications of Graph Coloring

48. Scheduling of Final Exams • How can the final exams at Cornell be scheduled so that no student nhas • two exams at the same time? • A vertex correspond to a course; • An edge between two vertices denotes that there is at least one common • Student in the courses they represent; • Each time slot for a final exam is represented by a different color. • A coloring of the graph corresponds to a valid schedule of the exams.

49. 1 Time Period I II III IV Courses 1,6 2 3,5 4,7 2 7 6 3 5 4 Scheduling of Final Exams 1 2 7 6 3 5 4 What are the constraints between courses? Find a valid coloring

50. Frequency Assignments • T.V. channels 2 through 13 are assigned to stations in North • America so that no no two stations within 150 miles can operate on • the same channel. How can the assignment of channels be • modeled as a graph coloriong? • A vertex corresponds to one station; • There is a edge between two vertices if they are located within 150 miles • of each other • Coloring of graph --- corresponds to a valid assignment of channels; • each color represents a different channel.