CS 4700: Foundations of Artificial Intelligence

CS 4700:Foundations of Artificial Intelligence Carla P. Gomes gomes@cs.cornell.edu Module: CSP1 (Reading R&N: Chapter 5)

Outline • Constraint Satisfaction Problems (CSP) • Backtracking search for CSPs

Motivational Example:8-Queens Goal: place 8 non-attacking Queens.

Intro Example: 8-Queens • Purely generate-and-test • The “search” tree is only used to enumerate all possible 648 combinations

Intro Example: 8-Queens Another form of generate-and-test, with no redundancies  “only” 88 combinations.

Intro Example: 8-Queens After placing the first queen, what would you do for the 2nd ?

General Search vs.Constraint satisfaction problems (CSPs) • Standard search problem: • state is a "black box“ – can be accessed only by the problem-specific routines: •successor function; •heuristic function; and •goal test. What is needed: Not just a successor function and goal test. Also a means of propagating the constraints (e.g. imposed by one queen on the others and an early failure test)  Explicit representation of constraints and constraint manipulation algorithms  Constraint Satisfaction Problems (CSP)

General Search vs.Constraint satisfaction problems (CSPs) CSP: States and goal test have astandard, structured, and simple representation. • state is defined by variablesXi with values from domainDi • goal test is a set of constraints specifying allowable combinations of values for subsets of variables Interesting tradeoff!!! Example of a (restricted) formal representation language. Allows useful general-purpose algorithms more powerful than standard search algorithms that have to resort to problem specific heuristics to enable solution of large problems.

Constraint Satisfaction Problem • Set of variables {X1, X2, …, Xn} • Each variable Xi has a domain Di of possible values • Usually Di is discrete and finite • Set of constraints{C1, C2, …, Cp} • Each constraint Ck involves a subset of variables and specifies the allowable combinations of values of these variables Goal: Assign a value to every variable such that all constraints are satisfied

Motivational Example:8-Queens How do we represent 8-Queens as a CSP: Variables? Constraints?

Example: 8-Queens Problem • 64 variables Xij, i = 1 to 8, j = 1 to 8 • Domain for each variable {0,1} • Constraints are of the form: • Row and columns • Xij = 1  Xik = 0 for all k = 1 to 8, kj • Xij = 1  Xkj = 0 for all k = 1 to 8, kI • Diagonals • Xij = 1  Xi+l,k+l = 0 l = 1 to 8, i+l 8; k+l8 (right and up) • Xij = 1  Xi-l,k+l = 0 l = 1 to 8, i-l 1; k+l8 (right and down) • Xij = 1  Xi-l,k-l = 0 l = 1 to 8, i-l 1; k-l1 (left and down) • Xij = 1  Xi+l,k-l = 0 l = 1 to 8, i+l 8 ; k-l1 (left and up)

Example: 8-Queens Problem • 8 variables Xi, i = 1 to 8 • Domain for each variable {1,2,…,8} • Constraints are of the forms: • Xi Xj when ji • Queens in same diagonal • Xi – Xj  i – j and • Xi – Xj  j – i

Crypto-arithmetic Puzzle • SEND 9567 • +MORE + 1085 • ----------- ---------- • MONEY 10652 Variables: S E N D M O R Y Domains: [1..9] for S and M [0..9] for E N D O R Y

Crypto-arithmetic Puzzle Constraints 1 • 1 single constraint 1000 S + 100 E + 10 N + D + 1000 M + 100 O + 10 R + E = 10000 M + 1000 O + 100 N + 10 E + y Or 5 equality constraints, using “carry” variables C1, …, C4 Є [0..9] SEND +MORE ----------- MONEY D + E = 10 C1 + Y; C1 + N + R = 10 C2 + E; C2 + E + O = 10 C3 + N; C3 + S + M = 10 C4 + O; C4 = M

Crypto-arithmetic Puzzle Constraints 2 • 28 not-equal constraints • X≠ Y, X,Y Є {S E N D M O R Y} • Or • A single constraint • Alldifferent(S, E, N, D, M, O, R, Y) Alldifferent(X1, … , Xn)  it states in a compact way that the variables X1, … , Xn have all different values assigned to them  Global constraint (it involves n-ary constraint)  Special procedures to handle this constraint

Example: Map-Coloring • VariablesWA, NT, Q, NSW, V, SA, T • DomainsDi = {red,green,blue} • Constraints: adjacent regions must have different colors • e.g., WA ≠ NT, or (WA,NT) in {(red,green),(red,blue),(green,red), (green,blue),(blue,red),(blue,green)}

Example: Map-Coloring • Solutions are complete and consistent assignments, e.g., WA = red, NT = green,Q = red,NSW = green,V = red,SA = blue,T = green

Constraint graph • Binary CSP: each constraint relates two variables • Constraint graph: nodes are variables, arcs are constraints Two variables are adjacent or neighbors if they are connected by an edge or an arc

T1 T2 T4 T3 Example: Task Scheduling • T1 must be done during T3 • T2 must be achieved before T1 starts • T2 must overlap with T3 • T4 must start after T1 is complete • Are the constraints compatible? • Find the temporal relation between every two tasks

our focus Varieties of CSPs • Discrete variables • finite domains: • n variables, domain size d  O(dn) complete assignments • infinite domains: • integers, strings, etc. • e.g., job scheduling, variables are start/end days for each job • need a constraint language, e.g., StartJob1 + 5 ≤ StartJob3 • Continuous variables • e.g., start/end times for Hubble Space Telescope observations • linear constraints solvable in polynomial time by linear programming • (including Boolean satisfiability 1st problem to be shown NP-complete!)

Varieties of constraints • Unary constraints involve a single variable, • e.g., SA ≠ green • Binary constraints involve pairs of variables, • e.g., SA ≠ WA • Higher-order constraints involve 3 or more variables, • e.g., cryptarithmetic column constraints

Example: Cryptarithmetic • Variables: F T U W R O X1 X2 X3 • Domains: {0,1,2,3,4,5,6,7,8,9} • Constraints: Alldiff (F,T,U,W,R,O) • O + O = R + 10 ·X1 • X1 + W + W = U + 10 · X2 • X2 + T + T = O + 10 · X3 • X3 = F, T ≠ 0, F≠ 0

Analysis of Polyhedral Scenes • Origins of constraint satisfaction problems  • researchers in computer vision and graphics in the 60s-70s were interested in developing a procedure to assign 3- dimensional interpretations to scenes; • They identified • Four types of junctions • Three types of edges

Edge Types • Hidden – if one of its planes cannot be seen •  represented with arrows: •  or  • Convex – from the viewer’s perspective •  represented with • + • Concave – from the viewer’s perspective •  represented with • -

Types of Junctions Type of junction: L Fork T Arrow

CSP Model • Variables  Edges; • Domains  {+,-,,} • Constraints: • 1- The different type junctions define constraints: • L, Fork, T, Arrow; • L = {(, ) , ( , ), (+, ), (,+), (-, ), (,-)} • Fork = { (+,+,+), (-,-,-), (,,-), (,-,),(-,,)} L(A,B)  the pair of values assigned to variables A,B has to belong in the set L; Fork(A,B,C)  the trio of values assigned to variables A,B,C has to belong in the set Fork;

CSP Model • T = {(, , ) , ( ,,), (,,+), (,-)} • Arrow = { (,,+), (+,+,-), (-,-,+)} T(A,B,C)  the trio of values assigned to variables A,B,C has to belong in the set T; Arrow(A,B,C)  the trio of values assigned to variables A,B,C has to belong in the set Arrow; 2- For each edge XY its reverse YX has a compatible value Edge = { +,+), (-,-), (,),(,)} Edge(A,B)  the pair of values assigned to variables A,B has to belong in the set Edge;

E F A B G C D CSP Model - Cube How to label the cube?

E F A B G C D CSP Model • Variables: Edges: AB, BA,AC,CA,AE,EA,CD, • DC,BD,DB,DG,GD,GF,FG,EF,FE,AE,EA; • Domains  {+,-,,} • Constraints: • L(AC,CD); L(AE,EF); L(DG,GF); • Arrow(AC,AE,AB); Arrow(EF,FG,BF); Arrow(CD,DG,DB); • Fork(AB,BF,BD); • Edge(AB,BA); Edge(AC,CA); Edge(AE,EA); • Edge(EF,FE); Edge(BF,FB); Edge(FG,GF); • Edge(CD,DC); Edge(BD,DB); Edge(DG,GD);

E F A B G C D CSP Model • Variables: Edges: AB, BA,AC,CA,AE,EA,CD, • DC,BD,DB,DG,GD,GF,FG,EF,FE,AE,EA; • Domains  {+,-,,} • Constraints: • L(AC,CD); L(AE,EF); L(DG,GF); • Arrow(AC,AE,AB); Arrow(EF,FG,BF); Arrow(CD,DG,DB); • Fork(AB,BF,BD); • Edge(AB,BA); Edge(AC,CA); Edge(AE,EA); • Edge(EF,FE); Edge(BF,FB); Edge(FG,GF); • Edge(CD,DC); Edge(BD,DB); Edge(DG,GD); + + +

E F A B G C D CSP Model + + + One (out of four) possible labelings

Escher:Waterfall, 1961

Escher:Belvedere, May 1958

Escher:Ascending and Descending, 1960

The Impossible Objects is Escher’s Worlds Penrose & Penrose Stairs Penrose Triangle

Impossible Objects:No labeling!

Real-World Applications • Hardware and Software Configuration • Hardware and Software Verification • Timetabling • Sport Scheduling • Floor-Planning • Car Sequencing • Transportation scheduling • … • Many other applications

Search

CSP as a Search Problem • Initial state: empty assignment • Successor function: a value is assigned to any unassigned variable, which does not conflict with the currently assigned variables • Goal test: the assignment is complete • Path cost: irrelevant

CSP as a Search Problem • Initial state: empty assignment • Successor function: a value is assigned to any unassigned variable, which does not conflict with the currently assigned variables • Goal test: the assignment is complete • Path cost: irrelevant • n variables of domain size d O(dn) distinct complete assignments

Remark • Finite CSP include 3SAT as a special case (see class on logic) • 3SAT is known to be NP-complete • So, in the worst-case, we cannot expect to solve a finite CSP in less than exponential time

Solving CSP by search : Backtracking Search • BFS vs. DFS • BFS  terrible! • A tree with n!dn leaves : (nd)*((n-1)d)*((n-2)d)*…*(1d) = n!dn • Reduction by commutativity of CSP • A solution is not in the permutations but in combinations. • A tree with dn leaves • DFS • Used popularly • Every solution must be a complete assignment and therefore appears at depth n if there are n variables • The search tree extends only to depth n. • A variant of DFS : Backtracking search • Chooses values for one variable at a time • Backtracks when failed even before reaching a leaf. • Better than BFS due to backtracking, but still inefficient!!

Backtracking search • Variable assignments are commutative}, i.e., • [ WA = red then NT = green ] same as [ NT = green then WA = red ] • Only need to consider assignments to a single variable at each node  b = d and there are dn leaves • Depth-first search for CSPs with single-variable assignments is called backtracking search • Backtracking search is the basic uninformed algorithm for CSPs • Can solve n-queens for n≈ 25

function BACKTRACKING-SEARCH (csp) returns a solution, or failure return RECURSIVE-BACKTRACKING({}, csp) function RECURSIVE-BACKTRACKING(assignment, csp) returns a solution, or failure ifassignment is complete then returnassignment var SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp], assignment, csp) for eachvalue in ORDER-DOMAIN-VALUES(var, assignment, csp) do ifvalue is consistent with assignment according to CONSTRAINTS[csp] then add {var=value} to assignment result RECURSIVE-BACKTRACKING(assignment, csp) ifresult != failurethen returnresult remove {var = value} from assignment return failure ☜ BACKTRACKING OCCURS HERE!! Solving CSP by search : Backtracking Search

empty assignment 1st variable 2nd variable 3rd variable  Backtracking Search Assignment = {(var1=v11)}

empty assignment 1st variable 2nd variable 3rd variable  Backtracking Search Assignment = {(var1=v11),(var2=v21)}

empty assignment 1st variable 2nd variable 3rd variable  Backtracking Search Assignment = {(var1=v11),(var2=v21),(var3=v31)}

empty assignment 1st variable 2nd variable 3rd variable  Backtracking Search Assignment = {(var1=v11),(var2=v22)}

CS 4700: Foundations of Artificial Intelligence