Modelling & Solving with Constraints

1. Modelling & Solving with Constraints Prof. Toby Walsh University College Cork/Uppsala University

2. Overview • Introduction to constraint programming • Constraint propagation • Backtracking search • Modelling case studies • Simple recipe • Solving with constraints • Global constraints • Set variables • Branching heuristics

3. Resources • Course links • www.cs.york.ac.uk/~tw/Links/csps/ • Benchmark problems • www.csplib.org • Constraints solvers • LP based like ECLIPSE, Java based solvers like JCL, …

4. Constraint programming • “Dream” of declarative programming • State the constraints • Solver finds a solution • Method of choice for many hard combinatorial problems • Scheduling, assignment, routing, …

5. Constraints are everywhere! • No meetings before 10am • Network traffic < 100 Gbytes/sec • PCB width < 21cm • Salary > 45k Euros …

6. Constraint satisfaction • Constraint satisfaction problem (CSP) is a triple <V,D,C> where: • V is set of variables • Each X in V has set of values, D_X • Usually assume finite domain • {true,false}, {red,blue,green}, [0,10], … • C is set of constraints Goal: find assignment of values to variables to satisfy all the constraints

7. Example CSP • Course timetabling • Variable for each course CS101, OS102 .. • Domain are possible times wed9am, fri10am, .. • Constraints: CS101 \= wed9am Capacity constraints: atmost(3,[OS102,DB103..],wed9am) Lecturer constraints: alldifferent([CS101,DB103,…])

8. Constraint optimization • CSP + objective function • E.g. objective is Profit = Income - Costs • Find assignment of vals to vars that: • Satisfies constraints • Maximizes (minimizes) objective • Often solved as sequence of satisfaction problems Profit > 0, Profit > Ans1, Profit > Ans2, …

9. Constraint programming v. Constraint logic programming • Constraints declaratively specify problem • Logic programming natural approach Assert constraints, call “labelling” strategy (backtracking search predicate) • Imperative & functional toolkits • C++, Java, CAML, …

10. Constraints • Constraints are tuples <S,R> where • S is the scope, [X1,X2, … Xm] • list of variables to which constraint applies • R is relation specifying allowed values (goods) • Subset of D_X1 x D_X2 x … x D_Xm • May be specified intensionally or extensionally

11. Constraints • Extensional specification • List of goods (or for tight constraints, nogoods) • Intensional specification • X1 =/= X2 • 5*X1 + 6*X2 < X3 • alldifferent([X1,X2,X3,X4]), …

12. Binary v non-binary • Binary constraint • Scope covers 2 variables • E.g. not-equals constraint: X1 =/= X2. • E.g. ordering constraint: X1 < X2 • Non-binary constraint • Scope covers 3 or more variables • E.g. alldifferent(X1,X2,X3). • E.g. tour(X1,X2,X3,X4). “Non-binary constraints” usually do not include unary constraints!

13. Constraint graph • Nodes = variables • Edge between 2 nodes iff constraint between 2 associated variables • Few constraints, sparse constraint graph • Lots of constraints, dense constraint graph

14. Some non-binary examples • Timetabling • Variables: Lecture1, Lecture2, … • Values: time1, time2, … • Constraint that lectures taught by same lecturer do not conflict: alldifferent(Lecture1,Lecture5,…).

15. Some non-binary examples • Scheduling • Variables: Job1. Job2, … • Values: machine1, machine2, … • Constraint on number of jobs on each machine: atmost(2,[Job1,Job2,…],machine1), atmost(1,[Job1,Job2,…],machine2).

16. Why use non-binary constraints? • Binary constraints are NP-complete • Any non-binary constraint can be represented using binary constraints • E.g. alldifferent(X1,X2,X3) is “equivalent” to X1 =/= X2, X1 =/= X3, X2 =/= X3 • In theory therefore they’re not needed • But in practice, they are!

17. Modelling with non-binary constraints • Benefits include: • Compact, declarative specifications (discussed next) • Efficient constraint propagation (discussed second)

18. Consider writing your own alldifferent constraint: alldifferent([]). alldifferent([Head|Tail]):- onediff(Head,Tail), alldifferent(Tail). onediff(El,[]). onediff(El,[Head|Tail]):- El #\= Head, onediff(El,Tail). Modelling with non-binary constraints

19. Modelling with non-binary constraints • It’s possible but it’s not very pleasant! • Nor is it very compact • alldifferent([X1,…Xn]) expands into n(n-1)/2 binary not-equals constraints, Xi #\= Xj • one non-binary constraint or O(n^2) binary constraints? And there exist very efficient algorithms for reasoning efficiently with many specialized non-binary constraints

20. Constraint solvers • Two main approaches • Systematic, tree search algorithms • Local search or repair based procedures • Other more exotic possibilities • Hybrid algorithms • Quantum algorithms

21. Systematic solvers • Tree search • Assign value to variable • Deduce values that must be removed from future/unassigned variables • Propagation to ensure some level of consistency • If future variable has no values, backtrack else repeat • Number of choices • Variable to assign next, value to assign Some important refinements like nogood learning, non-chronological backtracking, …

22. Local search • Repair based methods • Generate complete assignment • Change value to some variable in a violated constraint • Number of choices • Violated constraint, variable within it, … Unable to exploit powerful constraint propagation techniques

23. Constraint propagation • Heart of constraint programming • Most often enforce arc-consistency (AC) • A binary constraint r(X1,X2) is AC iff for every value for X1, there is a consistent value (often called support) for X2 and vice versa • A problem is AC iff every constraint is AC

24. Enforcing arc-consistency • X2 \= X3 is AC • X1 \= X2 is not AC • X2=1 has no support so can this value can be pruned • X2 \= X3 is now not AC • No support for X3=2 • This value can also be pruned Problem is now AC {1} X1 \= {1,2} {2,3} \= X2 X3

25. Enforcing arc-consistency • Remove all values that are not AC (i.e. have no support) • May remove support from other values (often queue based algorithm) • Best AC algorithms (AC7, AC-2000) run in O(ed^2) • Optimal if we know nothing else about the constraints

26. Properties of AC • Unique maximal AC subproblem • Or problem is unsatisfiable • Enforcing AC can process constraints in any order • But order does affect (average-case) efficiency

27. Non-binary constraint propagation • Most popular is generalized arc-consistency (GAC) • A non-binary constraint is GAC iff for every value for a variable there are consistent values for all other variables in the constraint • We can again prune values that are not supported • GAC = AC on binary constraints

28. GAC on alldifferent • alldifferent(X1,X2,X3) • Constraint is not GAC • X1=2 cannot be extended • X2 would have to be 3 • No value left then for X3 • X1={1} is GAC {1,2} X1 {2,3} {2,3} X2 X3

29. Enforcing GAC • Enforcing GAC is expensive in general • GAC schema is O(d^k) On k-ary constraint on vars with domains of size d • Trick is to exploit semantics of constraints • Regin’s all-different algorithm • Achieves GAC in just O(k^3/2 d) On k-ary all different constraint with domains of size d Based on finding matching in “value graph”

30. Other types of constraint propagation • (i,j)-consistency [due to Freuder, JACM 85] • Non-empty domains • Any consistent instantiation for i variables can be extended to j others • Describes many different consistency techniques

31. (i,j)-consistency • Generalization of arc-consistency • AC = (1,1)-consistency • Path-consistency = (2,1)-consistency • Strong path-consistency = AC + PC • Path inverse consistency = (1,2)-consistency

32. Enforcing (i,j)-consistency • problem is (1,1)-consistent (AC) • BUT is not (2,1)-consistent (PC) • X1=2, X2=3 cannot be extended to X3 • Need to add constraints: not(X1=2 & X2=3) not(X1=2 & X3=3) • Nor is it (1,2)-consistent (PIC) • X1=2 cannot be extended to X2 & X3 (so needs to be deleted) {1,2} X1 \= \= {2,3} {2,3} \= X2 X3

33. Other types of constraint propagation • Singleton arc-consistency (SAC) • Problem resulting from instantiating any variable can be made AC • Restricted path-consistency (RPC) • AC + if a value has just one support then any third variable has a consistent value • …

34. Comparing local consistencies • Formal definition of tightness introduced by Debruyne & Bessiere [IJCAI-97] • A-consistency is tighter than B-consistency iff If a problem is A-consistent -> it is B-consistent We write A >= B

35. Properties • Partial ordering • reflexive A  A • transitive A  B & B  C implies A  C • Defined relations • tighter A > B iff A  B & not B  A • incomparable A @ B iff neither A  B nor B A

36. Comparison of consistency techniques • Exercise for the reader, prove the following identities! Strong PC > SAC > RPC > AC NB gaps can reduce search exponentially!

37. Which to choose? • For binary constraints, AC is often chosen • Space efficient Just prune domains (cf PC) • Time efficient • For non-binary constraints GAC is often chosen • If we can exploit the constraint semantics to keep it cheap!

38. Why consider these other consistencies? • Promising experimental results • Useful pruning for their additional cost • Theoretical value • E.g. GAC on non-binary constraints may exceed SAC on equivalent binary model

39. Maintaining a local consistency property • Tree search • Assign value to variable • Enforce some level of local consistency • Remove values/add new constraints • If any future variable has no values, backtrack else repeat • Two popular algorithms • Maintaining arc-consistency (MAC) • Forward checking (only enforce AC on instantiated variable)

40. Modelling case study: all interval series Results due to Simonis, Puget & Regin

41. All interval series • Prob007 at www.csplib.org • Comes from musical composition • Traced back to Alban Berg • Extensively used by Ernst Krenek Op.170 “Quaestio temporis”

42. All interval series • Take the 12 standard pitch classes • c, c#, d, .. • Represent them by numbers 0, .., 11 • Find a sequence so each occurs once • Each difference occurs once

43. All interval series • Can generalize to any n (not just 12) Find Sn, a permutation of [0,n) such that |Sn+1-Sn| are all distinct • Finding one solution is easy

44. All interval series • Can generalize to any n (not just 12) Find Sn, a permutation of [0,n) such that |Sn+1-Sn| are all distinct • Finding one solution is easy [n,1,n-1,2,n-2,.., floor(n/2)+2,floor(n/2)-1,floor(n/2)+1,floor(n/2)] Giving the differences [n-1,n-2,..,2,1] Challenge is to find all solutions!

45. Basic recipe • Devise basic CSP model • What are the variables? What are the constraints? • Introduce auxiliary variables if needed • Consider dual or combined models • Break symmetry • Introduce implied constraints

46. Basic CSP model • What are the variables?

47. Basic CSP model • What are the variables? Si = j if the ith note is j • What are the constraints?

48. Basic CSP model • What are the variables? Si = j if the ith note is j • What are the constraints? Si in [0,n) All-different([S1,S2,… Sn]) Forall i<i’ |Si+1 - Si| =/= |Si’+1 - Si’|

49. Basic recipe • Devise basic CSP model • What are the variables? What are the constraints? • Introduce auxiliary variables if needed • Consider dual or combined models • Break symmetry • Introduce implied constraints

50. Improving basic model • Introduce auxiliary variables? • Are there any loose or messy constraints we could better (more compactly?) express via some auxiliary variables?