Decision Procedures for Linear Arithmetic

1. Decision Procedures for Linear Arithmetic Presented By Omer Katz 01/04/14 Based on slides by OferStrichman

2. Agenda • DPLL-T (a very short reminder) • What is Linear Arithmetic and why is it needed • Decision procedures for • Decision procedures for • A few preprocessing improvement steps • Given enough time: • Difference logic • Delayed Theory Combination • Improvement on Nelson-Oppen • Not related to Linear Arithmetic

3. Reminder: DPLL-T • Our goal is to solve SMT problems with formulas from a theory T • DPLL-T is the most common approach • Based on the DPLL algorithm for SAT problems • Combines a decision procedure for the theory T

4. Reminder: DPLL-T

5. Linear Arithmetic • Theory grammar: • Can be defined over Rational () or Integers () • Example:

6. Why is it needed? Given the following C code: The following assembly can be generated:

7. Why is it needed? • A possible optimization: • Read the value of a[j] only once • Need to verify that loop won’t change a[j] • Can be encoded as Linear Arithmetic problem • If no solution is found, optimization is safe

8. Decision Procedures for • Gaussian’s elimination • Fourier-Motzkin • Simplex

9. Gaussian’s elimination • A simple method for solving a set of equalities • Less suitable for inequalities • Given a linear system Ax = b A x = b

10. Gaussian’s elimination • Manipulate A|b to an upper-triangular form • Then, solve backwards from the k’s row according to:

11. Gaussian elimination - example And now… x3 = -1, x2 = 3, x1 = 1 problem solved.

12. Fourier-Motzkin Elimination • Earliest method for solving linear inequalities • Given linear non-strict inequalities: • Pick a variable and eliminate it • Continue until all variables but one are eliminated • If problem included equalities, eliminate by assignment

13. A system of conjoined linear inequalities Fourier-Motzkin Elimination mconstraints nvariables

14. Fourier-Motzkin Elimination • When eliminating xn, partition the constraints according to the coefficient ain: • ain > 0: upper bound • ain < 0: lower bound assumeai,n >0

15. Fourier-Motzkin Elimination • Example: (1) x1 – x2 ≤ 0 (2) x1 – x3 ≤ 0 (3) -x1 + x2 + 2x3 ≤ 0 (4) -x3 ≤ -1 Category? Upper bound Assume we eliminatex1. Upper bound Lower bound

16. Fourier-Motzkin Elimination • For each pair of a lower bound aln<0 andupper bound aun>0, we have • For each such pair, add a constraint

17. Fourier-Motzkin Elimination • Example: (1) x1 – x2 ≤ 0 (2) x1 – x3 ≤ 0 (3) -x1 + x2 + 2x3 ≤ 0 (4) -x3 ≤ -1 (5) 2x3 ≤ 0 (from 1 and 3) (6) x2 + x3 ≤ 0 (from 2 and 3) We eliminatex1.

18. Fourier-Motzkin Elimination • Example: (1) x1 – x2 ≤ 0 (2) x1 – x3 ≤ 0 (3) -x1 + x2 + 2x3 ≤ 0 (4) -x3 ≤ -1 (5) 2x3 ≤ 0 (from 1 and 3) (6) x2 + x3 ≤ 0 (from 2 and 3) (7) 0 ≤ -1 (from 4 and 5) We eliminatex3. Contradiction (the system is unsatisfiable)!

19. Complexity of Fourier-Motzkin • Worst-case complexity: m2n • Popular in compilers • Because of simplicity • Popular when the problems are small – then it can be the fastest. • Not suitable for large problems • Need another solution for big problems

20. Simplex • Simplex was originally designed for solving optimization problems: • Given coefficient matrix and bound vector find a satisfying assignment that maximizes a goal function described by • Also known as Linear Programming • We are only interested in the feasibility problem • If problem is feasible than it is satisfiable

21. General form • Given same input as Fourier-Motzkin, convert input to general form • General form: • A combination of: • Linear equalities of the form • Lower and upper bounds on variables.

22. Converting to General Form • Replace (where )with and • s1,..., sm are called the additional variables.

23. Example 1 • Convertto:

24. Example 2 • Convertto:

25. Same problem, geometrically

26. Matrix form • Due to the additional variables: • now A is an mx (n + m) matrix. xy s1 s2 s3

27. xy s1 s2 s3 xy s1 s2 s3 The tableau • The diagonal part is inherent to the general form • Marked section will always be there • We can instead write:

28. xy s1 s2 s3 The tableau • The tableau changes throughout the algorithm, but maintains its m x n structure • Distinguish between basic and nonbasic variables • Initially, basic variables = the additional variables.

29. The tableau • Denote by • B– Basic variables • N – Nonbasic variables • The tableau is simply a rewrite of the system: • The basic variables are also called the dependent variables. • Their value is determined by the values of the other variables

30. The general simplex algorithm • Simplex maintains and updates: • The tableau, • an assignment to all variables • The bounds • Two invariants: • All nonbasic variables satisfy their bounds

31. Invariants • Initially, • B = additional variables • N = problem variables • (xi) = 0 for i {1,...,n+m} • Trivial to see that initial state satisfies invariants

32. The simplex algorithm • The initial assignment satisfies • If the bounds of all basic variables are satisfied by , return `Satisfiable’ • Otherwise... choose a variable and pivot • Pivoting is the basic step of the algorithm

33. Pivoting • Find a basic variable xithat violates its bounds. • Suppose that (xi) < li • We need to fix the value ofxi • Find a nonbasic variable xj such that • aij > 0 and(xj) < uj, or • aij < 0 and(xj) > lj • Such a variable xj is called suitable. • If there is no suitable variable – return ‘Unsatisfiable’

34. Pivoting xiwithxj • Solve equation i for xj: From: To: • Swap xi and xj, and update the i-th row accordingly. From To:

35. Pivoting xi withxj • Update all other rows: • Replace xjwith its equivalent obtained from row i:

36. Pivoting • Update as follows: • Increase (xj) by • Now xj is a basic variable: it can violate its bounds. • Update (xi) to li • Update  for all other basic (dependent) variables.

37. Example • Recall the tableau and constraints in our example: • Initially  assigns 0 to all variables • Bounds of s1 and s3 are violated

38. Example • Recall the tableau and constraints in our example: • We will pivot on s1 • x is a suitablenonbasic variable for pivoting • It has no upper bound • So now we pivot s1 with x

39. Example • Recall the tableau and constraints in our example: • Solve 1st row for x: • Replace x with s1 in other rows:

40. Example • The new state: • Solve 1st row for x: • Replace x with s1 in other rows:

41. Example • The new state: • We should increase x by • Hence, (x) = 0 + 2 = 2 • Now s1is equal to its lower bound: (s1) = 2 • Update all the others

42. Example • The new state: • Now s3 violates its lower bound • yis a suitable nonbasic variable for pivoting

43. Example • The new state: • We should increase y by

44. Example • The final state: • All constraints are now satisfied

45. A few observations • The additional variables: • Only additional variables have bounds. • These bounds are permanent. • Additional variables exit the base only on extreme points (their lower or upper bounds). • When entering the base, they shift towards the other bound and possibly cross it (violate it).

46. A few observations • Can it be that we pivot(xi,xj) and then pivot(xj,xi) and enter a (local) cycle ? • No. • Is termination guaranteed ? • No. • Perhaps there are bigger cycles.

47. A few observations • In order to avoid circles, we use Bland’s rule: • determine a total order on the variables. • Choose the first basic variable that violates its bounds, and first nonbasic suitable variable for pivoting. • It can be proven that this guarantees that no base is repeated, which implies termination. • We won’t prove this

48. A few observations • Simplex is exponential in the worst case • However, considered very efficient on most real practical problems • Need for exponential number of steps is rare

49. Decision Procedures for • Decision problem for is NP-hard • Unlike • Branch & Bound • Cuts • Omega test • Based on Fourier-Motzkin elimination • Will not be discussed today

50. Decision Procedures for • Both Branch & Bound and Cuts rely on a solver to provide a (possibly non-integer) solution • For example: Simplex • All variables in the final solution must be integers • In the case of simplex, this does not includes the additional variables • The additional variables do not have to be a part of the outputted solution