Time-Space Tradeoffs in Proof Complexity: Superpolynomial Lower Bounds for Superlinear Space

1. Time-Space Tradeoffs in Proof Complexity:Superpolynomial Lower Bounds for Superlinear Space Chris Beck Princeton University Joint work with Paul Beame & Russell ImpagliazzoJakob Nordstrom & Bangsheng Tang

2. SAT • The satisfiability problem is a central problem in computer science, in theory and in practice. • Terminology: • A Clause is a boolean formula which is an OR of variables and negations of variables. • A CNF formula is an AND of clauses. • Object of study for this talk: CNF-SAT: Given a CNF formula, is it satisfiable?

3. Resolution Proof System • Lines are clauses, one simple proof step • Proof is a sequence of clauses each of which is • an original clause or • follows from previous ones via resolution step • a CNF is UNSAT iff can derive empty clause⊥

4. Proof DAG General resolution: Arbitrary DAG Tree-like resolution: DAG is a tree

5. SAT Solvers • Well-known connection between Resolution and SAT solvers based on Backtracking • These algorithms are very powerful • sometimes can quickly handle CNF’s with millions of variables. • On UNSAT formulas, computation history yields a Resolution proof. • Tree-like Resolution ≈ DPLL algorithm • General Resolution ≿ DPLL + “Clause Learning” • Best current SAT solvers use this approach

6. SAT Solvers • DPLL algorithm: backtracking search for satisfying assignment • Given a formula which is not constant, guess a value for one of its variables , and recurse on the simplified formula. • If we don’t find an assignment, set the other way and recurse again. • If one of the recursive calls yields a satisfying assignment, adjoin the good value of and return, otherwise report fail.

7. SAT Solvers • DPLL requires very little memory • Clause learning adds a new clause to the input CNF every time the search backtracks • Uses lots of memory to try to beat DPLL. • In practice, must use heuristics to guess which clauses are “important” and store only those. Hard to do well! Memory becomes a bottleneck. • Question: Is this inherent? Or can the right heuristics avoid the memory bottleneck?

8. Proof Complexity & Sat Solvers • Proof Size≤Timefor Ideal SAT Solver • Proof Space≤Memoryfor Ideal SAT Solver • Many explicit hard UNSAT examples known with exponential lower bounds for Resolution Proof Size. • Question: Is this also true for Proof Space?

9. Space in Resolution • Clause space. [Esteban, Torán ‘99] … Must be in memory Time step

10. Lower Bounds on Space? • Considering Space alone, tight lower and upper bounds of , for explicit tautologies of size . • Lower Bounds: [ET’99, ABRW’00, T’01, AD’03] • Upper Bound: All UNSAT formulas on 𝑛 variables have tree-like refutation of space ≤ 𝑛. [Esteban, Torán ‘99] • For a tree-like proof, Space is at most the height of the tree which is the stack height of DPLL search • But, these tree-like proofs are typically too large to be practical, so we should instead ask if both small space and time are simultaneously feasible.

11. Size-Space Tradeoffs for Resolution • [Ben-Sasson ‘01] Pebbling formulas with linear size refutations, but for which all proofs have SpacelogSize Ω(n/log n). • [Ben-Sasson, Nordström ‘10] Pebbling formulas which can be refuted in SizeO(n),SpaceO(n), but SpaceO(n/log n) Size exp(n(1)). But, these are all for Space < 𝑛, and SAT solvers generally can afford to store the input formula in memory. Can we break the linear space barrier?

12. Size-Space Tradeoffs • Informal Question: Can we formally show that memory rather than time can be a real bottleneck for resolution proofs and SAT solvers? • Formal Question (Ben-Sasson): “Does there exist a 𝑐 such that any CNF with a refutation of sizeT also has a refutation of sizeT𝑐 in spaceO(𝑛)?” • Our results: Families of formulas of size n having refutations in Time, Spacenk, but all resolution refutations have T > (n0.58 k/S)loglogn/logloglog n

13. Tseitin Tautologies Given an undirected graph and :𝑉→𝔽2 , define a CSP: Boolean variables: Parity constraints:(linear equations) When has odd total parity, CSP is UNSAT.

14. Tseitin Tautologies • When  odd, G connected, corresponding CNF is called a Tseitin tautology. [Tseitin ‘68] • Only total parity of  matters • Hard when G is a constant degree expander: [Urqhart 87]: Resolution size =2Ω(𝐸)[Torán 99]: Resolution space =Ω(E) • This work: Tradeoffs on 𝒏 × 𝒍grid, 𝒍 ≫ 𝒏, and similar graphs, using isoperimetry.

15. Tseitin formula on Grid • Consider Tseitin formula on 𝒏 × 𝒍grid, 𝒍=4 • How can we build a resolution refutation? n l

16. Tseitin formula on Grid • One idea: Mimic linear algebra refutation • If we add the lineqn’s in any order, get 1 = 0. • A linear equation on variables corresponds to clauses. Resolution is implicationally complete, so can simulate any step of this with -blowup. n l

17. Tseitin formula on Grid • One idea: Mimic linear algebra refutation • If we add the lineqn’s in column order: • Never have an intermediate equation with more thanvariables. Get a proof of Size , Space. n l

18. Tseitin formula on Grid • Different idea: Divide and conquer • In DPLL, repeatedly bisect the graph • Each time we cut, one of the components is unsat. Done after queries, tree-like proof with Space, Size . • Savitch-like savings in space, for . n l

19. Tseitin formula on Grid • “Chris, isn’t this just tree-width?” – Exactly. • Our work is about whether this can be improved. • The lower bound shows is that quasipolynomial blowup in size when the space is below is necessary for the proof systems we studied. • For technical reasons, work with “doubled” grid. n l

20. High Level Overview of Lower Bound • [BeamePitassi ’96] simplified previous Proof Size lower bounds using random restrictions. • A restriction to a formula is a partial assignment of truth values to variables, resulting in some simplification. We denote the restricted formula by . • If is a proof of , is a proof of .

21. High Level Overview of Lower Bound • Philosophy of [Beame, Pitassi’96] • Find such that any refutation contains a “complex clause” • Find and a random restriction which restricts to , but also kills any complex clause whp. • Modern form of “bottleneck counting” due to Haken. Can reinterpret this as finding a large collection of assignments, one extending each restriction, each passing through a wide clause.

22. High Level Overview of Lower Bound • This work: Multiple classes of complex clauses • Can state main idea as a pebbling argument. • Suppose G is a DAG with T vertices that can be scheduled in space S, and the vertices can be broken into 4 classes, . • Assume that for arcs , • ll sources in 1, all sinks in 4. Everything in 2 is a “complex clauses” of first type, everything in 3 is a “complex clauses” of second type.

23. High Level Overview of Lower Bound • Suppose that there is a distribution on source-to-sink paths and a real number p s.t.: • For any vertex , • For any two vertices . • Then,

24. High Level Overview of Lower Bound • Proof: • Let be sequence of pebbling steps using only S pebbles. Divide time into epochs, to be determined later. • Plot vs. time 4 3 2 1 Time

25. High Level Overview of Lower Bound • Let be any source-to-sink path. Claim: • Either, hits a vertex of level 2 and a vertex of level 3 during the same epoch • Or, hits a vertex of level 2 or 3 during one of the breakpoints between epochs. 4 3 2 1 Time

26. High Level Overview of Lower Bound • Now choose randomly. • Either, hits a vertex of level 2 and a vertex of level 3 during the same epoch Probability of this: By a union bound, at most 4 3 2 1 Time

27. High Level Overview of Lower Bound • Now choose randomly. • Or, hits a vertex of level 2 or 3 during one of the breakpoints between epochs. Probability of this: By a union bound, at most 4 3 2 1 Time

28. High Level Overview of Lower Bound • Now choose randomly. • Conclude • Optimizing yields • Previoussuch results only for expanders, etc. • Gets stronger when you have more levels. Later we will see that with levels, get

29. High Level Overview of Lower Bound • Goal for remainder of analysis: • Show that any proof of Tseitin on doubled grid has such a flow (using restrictions), for the bestand as many complex clause classes as possible. • It is enough that any k complex clauses from distinct classes have collective width for some large W. (For n x l grid, W will be n.) • How did we get complex clauses before in [BeamePitassi ‘96]? A “measure of progress” argument…

30. Progress Measure • The following progress measure construction appeared in [Ben-Sasson and Wigderson’01]. • Let be a partition of clauses of an unsat CNF. Assume it is minimally unsat. • For any clause C, define

31. Progress Measure • 𝜇 is a sub-additive complexity measure: • 𝜇(initial clause) = 1, • 𝜇(⊥) = m, • 𝜇(𝐶) ≤ 𝜇(𝐶1 ) + 𝜇(𝐶2), when 𝐶1, 𝐶2⊦ 𝐶. • In any refutation, there must occur a clause Cwith . In previous work, choose so that this implies Cis wide.

32. Progress Measure for Tseitin • For Tseitin tautologies, natural choice is to have one for each vertex v, containing all associated clauses. So is an 𝔽2 lin. eqn. • Claim: Let C be any clause, and let be any subset attaining the min, . Then, .

33. Progress Measure for Tseitin • Claim: Let C be any clause, and let , . Then . • Proof: • If , then is unsat linear system. So we can derive 1 = 0 by linear combinations. • If any equation from S has coeff 0, S wasn’t minimal. In sum of equations of S, noncancellingvars are exactly boundary edge vars. These must cancel with to get 1=0, so .

34. Isoperimetry → Lower Bounds • Consequence: Tseitin formula has minimum refutation width at least the balanced bisection width of graph G. • Corollary: Tseitin on a constant degree expander requires width . • Corollary: Ongrid, needswidth . • Here, “Complex clause” :=

35. Size Lower Bounds from Width • We have formulas with width lower bounds, can use XOR-substitution and restrictions to get size lower bounds. • For a CSP, [⨁] is the CSP over variable set with two copies of each var of , and each constraint of interpretted as constraining the parities rather than the variables . (Then expand as CNF.)

36. Size Lower Bounds from Width • [⨁] has a natural random restriction : • For each , independently choose one of to restrict to {0,1} at random. Substitute either or for the other, so that . • Previously used in [B’01]. Nice properties: • , always • Fairly dense, kills all wide clauses whp. • Black box, doesn’t depend on .

37. Size Lower Bounds from Width • When is a Tseitin tautology, is “multigraph-Tseitin” on same graph but where each edge has been doubled. • Suppose Tseitinon doubledgridhas a proof of size less than . Then by a union bound, some restriction kills all clauses of width , so Tseitin on the gridhas proof of width .Contradiction, for small enough c.

38. Additional Complex Clause Levels • Idea: Instead of just considering C with , what about [1/6, 1/3], [1/12, 1/16], etc.? • Gives us the levelledproperty we need. • Task: Show that clauses from k different levels are collectively wide. • Given the machinery we have, this is some “extended isoperimetry” property of the grid.

39. Extended Isoperimetry • Claim: Let be subsets of vertices, ,, of superincreasing sizes. Then, . • Several simple combinatorial proofs of this. • Implies clauses from distinct levels are collectively wide. n l

40. Main Lemma • For any set of clauses in doubled-Tseitin, Proof: For any k-tuple, opportunities for to kill at least one. Union bound over k-tuples.

41. Complexity vs. Time • Consider the time ordering of any proof, and plot complexity of clauses in memory v. time • Sub-additivity implies cannot skip over any [t, 2t] window of μ-values on the way up. ⊥ μ Time Input clauses

42. Complexity vs. Time • Consider the time ordering of any proof, and divide time into 𝑚equal epochs (fix 𝑚 later) Hi Med Low Time

43. Two Possibilities • Consider the time ordering of any proof, and divide time into 𝑚 equal epochs (fix 𝑚 later) • Either, a clause of medium complexity appears in memory for at least one of the breakpoints between epochs, Hi Med Low Time

44. Two Possibilities • Consider the time ordering of any proof, and divide time into 𝑚 equal epochs (fix 𝑚 later) • Or, all breakpoints only have Hi and Low. Must have an epoch which starts Low ends Hi, and so has clauses of all log⁡ 𝑛 Medium levels. Hi Med Low Time

45. Analysis • Consider the restricted proof. With probability 1, one of the two scenarios applies. • For first scenario, clauses, • For second scenario, epochs with clauses each. Union bound and main lemma:

46. Analysis • Optimizing yields something of the formfor • Can get a nontrivial tradeoff this way, but need to do a lot of work on constants.

47. Better tradeoff • Don’t just divide into epochs once • Recursively divide proof into epochs and sub-epochs where each sub-epoch contains 𝑚 sub-epochs of the next smaller size • Prove that, if an epoch does a lot of work, either • Breakpoints contain many complexities • A sub-epoch does a lot of work

48. An even better tradeoff • If an epoch contains a clause at the end of level , but every clause at start is level , (so the epoch makes progress), • and the breakpoints of its children epochs contain together complexity levels, • then some child epoch makes progress. a

49. Internal Node Size = mS Leaf Node Size = T/m^(h-1) Previous Slide shows:Some set has complexities, where …

50. An even better tradeoff • Choose , have , so • Choose so all sets are the same size:, so all events are rare together. • Have sets in totalFinally, a union bound yields