Stronger Connections Between Circuit Analysis and Circuit Lower Bounds, via PCPs of Proximity

1. Stronger Connections Between Circuit Analysis and Circuit Lower Bounds, via PCPs of Proximity Lijie Chen Ryan Williams

2. Context: The Algorithmic Method for Proving Circuit Lower Bounds Proving limitations on non-uniform circuits is extremely hard. Prior approaches (restrictions, polynomial approximations, etc.) face barriers (Relativization, Algebrization, Natural Proofs). • Algorithmic Method • Non-trivial circuit-analysis algorithm Circuit Lower Bounds. • Breakthroughs where previous approaches failed (NEXP ACC0). • Believed to be possible for strong circuits (even ).

3. Context: A Frontier of Circuit Complexity, Depth-2 Threshold Circuits THR gates : , . MAJ gates : when ’s and are bounded by poly(n). THRTHR We can also define THR THR THR THR

4. Context: A Frontier of Circuit Complexity, Depth-2 Threshold Circuits Exponential Lower Bounds are known for [Hajnal-Maass-Pudlák-Szegedy-Turán’93] [Nisan’94] [Forster-Krause-Lokam-Mubarakzjanov-Schmitt-Simon’01] NEXP Non-deterministic Exponential Time. Frontier Open Question:Is NEXP ? Potential Approaches in this talk.

5. Motivation: Apply the Algorithmic Method to THR of THR? What Circuit-Analysis Tasks? Non-trivial Circuit- Analysis Algorithms Circuit Lower Bounds -SAT -CAPP Derandomization!! Estimate quantity , with additive error x s.t. : constant or inverse polynomial time? ?

6. Motivation: Apply the Algorithmic Method to THR of THR? Most previous work on the algorithmic method exploits SAT algorithms. Problem SAT of THR of THR is probably very hard. A special case is MAX--SAT, for which no non-trivial ( time) algorithm is known for and clauses. Considered to be a barrier for the Algorithmic Approach. THRTHR THR THR THR THR MAX--SAT MAJ

7. Motivation: Apply the Algorithmic Method to THR of THR? • From Derandomization (CAPP) • Circuit Lower Bounds • For a circuit class , • -time CAPP for () [Williams’13/14, Santhanam Williams’14, Ben-Sasson Viola’14] • -time CAPP for () can’t be -approximated by [R. Chen Oliveira Santhanam’18] • -time CAPP for () [Murray Williams’18] • -time CAPP for () can’t be -approximated by [L. Chen’19] SAT of THR of THR : probablyvery hard But derandomization is widely believed to be possible. NQP Non-deterministic Quasi-Polynomial Time.

8. Back to THR of THR SAT of THR of THR : probablyvery hard To show , we need to derandomize , which could be harder. Our result 1 It suffices to derandomize . Our result 2 Surprisingly, it indeed only suffices to derandomize or !

9. General Result: A Stronger Connection Between Circuit-Analysis Algorithms and Circuit Lower Bounds • For a circuit class : • -time CAPP for , , or • . • -time CAPP for , , or • . • Why the constant “2”? • Short answer: A PCP system needs to make at least queries. • Long answer: See the paper

10. Tighter Connections for Algorithms/Lower Bounds for THR of THR -time CAPP algorithm for . Luckily, the “2” doesn’t matter for  -time CAPP algorithm for . : depth-d, poly-size, linear threshold circuits

11. Let Us Make Our Life Even Easier Poly-size and are equivalentfor Non-Trivial ( time) CAPP Algorithms when THR MAJ THR THR MAJ MAJ THR MAJ Proved by new structure lemmas for

12. Let Us Make Our Life Even Easier Poly-size and are equivalentfor Non-Trivial ( time) CAPP Algorithms for any constant ! THR THR THR THR MAJ MAJ THR MAJ Proved by new structure lemmas for

13. Corollary If there are -time CAPP for with , or a -time CAPP for with constant , then .

14. Another Application: Inapproximability by Depth-2 Neural Networks Thm For every and constant , there is a function such that cannot be approximated by Depth-2 Neural Networks of size Depth-2 Neural Network THR THR THR Improved [Wil’18],which proved that there is such an which cannot be exactly computed by Depth-2 Neural Networks of size . ReLU ReLU ReLU

15. PhilosophyUsing PCP Algorithmically to Prove Circuit Lower Bounds (Remember: PCPs are algorithms!) If you want to prove , then PCPs should make your life much easier (now you only need an algorithm for -approximation to 3-SAT!) [Håstad’97] (Well, I don’t really believe in .) We only want to derandomize circuits. But PCPs still make our life easier (though in a more indirect way)

16. Starting Point: Non-deterministic Derandomization Suffices for Circuit Lower Bounds -GAP-TAUT (tautology) [Wil’13] time non-deterministic algorithm for GAP-TAUT on poly-size general circuits with . Distinguish between (Yes Case) .(No Case) Non-deterministic Algorithm for GAP-TAUT Given a general circuit , we want a time non-deterministicalgo, such that: If is a tautology, then accepts on some guesses. If , rejects on all guesses.

17. Proof Overview: Outline Starting Point [Wil’13] time non-deterministic algorithm for GAP-TAUT on poly-size general circuits with . Key point: make use of this assumption as much as possible! Assume Contradiction! non-deterministic GAP-TAUT for Think of as Non-trivial CAPP on with constant

18. Goal: Designing the Algorithm under Assumption Assume non-deterministic GAP-TAUT on Think of as It is universal Non-trivial CAPP on with constant Goal Given an circuit , under the two assumptions, design a time non-deterministicalgo, such that: If is a tautology, then accepts on some guesses. If , rejects on all guesses.

19. Review: Approach of [Wil’14]Guess-and-Verify-Equivalence implies collapses to . That is, under assumption, the given general circuit has an equivalent circuit . If we can find , then we can derandomize instead, where we have algorithms! Problem: How to find ? Allowed to use non-determinism so one can guess . But still have to verify is equivalent to , which seems HARD. Solution Well, just guess more circuits!

20. Review: Approach of [Wil’14]Guess-and-Verify-Equivalence Suppose has gates, let be the corresponding sub-circuits. is the output gate. are inputs. implies collapses to . We guess circuits , hoping that . We wish to check. To do this, for each , suppose gate- has inputs from gate- and gate-. We verify. Then run CAPP on . Problem Checking for all requires solving SAT for .

21. A Local-checkable Proof System View Problem: the previous approach requires solving SAT for . Let This is a Claimed Proof for by giving values at all gates. Intuitively, it is supposed to be the computation history of on input . What is so good about this proof ? • Local checks on • For each , .

22. A Local-checkable Proof System View Let A Claimed Proof for by giving values at all gates. • One can get functions on • , such that • Each is an of 3 bits (or their negations) from . • If on the correct guesses , all ’s are satisfied by . (Completeness) • If , for all possible at least one is notsatisfied by . (Soundness)

23. An Attempt • Guess circuits , let • Estimate . (.) • (:number of ’s) • If is a tautology. Then on the correct guess, • If then on all guesses,. • To distinguish the above two cases, we need a CAPP algo with error . • But we only assume a CAPP algo with constant error!

24. What Went Wrong? Proof System View : a claimed proof of : local check of the verifier • One can get functions on , such that • Each is an of 3 bits (or their negations) from . • If on the correct guess , all ’s are satisfied by . (Completeness is 1) • If , for all possible at least one is not satisfied by . (Soundness is ) If there is a verifier who picks a random and checks whether . She detects an error only with probability when . This is an extremely``bad’’ PCP! Why not just use the PCP theorem?

25. Issues When Applying PCPs Directly Use PCPs of Proximity! Like PCPs but both input and proof are given as oracles. Recall that in the end we want to estimate Key properties being used in previousattempt: These local checks (verifier’s queries positions) do not depend on the input ! PCPs PCPs of Proximity (input) (input) Unlimited access V V 3 queries in total (proof) (proof) 3 queries Now, can depend on many bits of . Therefore, we want a proof system for verifying , such that given the random bits, verifier queries both input and proof . If , exists , such that always accept. If , for all ,rejects w.h.p.

26. Issues When Applying PCP Directly Therefore, we want a proof system for verifying , such that given the random bits, verifier queries both input and proof . If , , such that always accept. If , , rejects w.h.p. Counter-example? Suppose computes the parity. Parity changes if we flip a random bit of . The verifier can’t distinguish unless she queried that bit. Solution Give access to an error correcting code of !

27. Combing PCP of Proximity and ECCs • PCP of Proximity • Verifier is given both the input () and the proof as oracles and makes queries. • accepts w.p. 1, when ; • accepts w.p., when makes robustlyoutput ( is zero in a small hamming ball around ). (like property testing) How it avoids the parity counter example? No inputs can make parity robustly output ! (input) V 3 queries in total (proof)

28. PCP of Proximity with ECCs • Verifier is given both the encoded input () and the proof as oracles and makes queries. • accepts w.p. 1, when ; • accepts w.p., when . Use of Proximity for verifying , makes robustly output when ! DEC(corrupted ) is still 

29. Final Algorithm Guess circuits , let Fix to be -linear. That is, is a parity on a subset of bits in . Suppose there is uniform parity circuit in for now (this assumption can be avoided) Now constant error CAPP algo for suffices! • Estimate . (.). • If is a tautology. Then on the correct guesses, • If then on all guesses,.

30. Future Work NEW Building on the PCPP based approach, [Alman Chen’19] give a construction of Razborov-rigid matrices in . Can we find non-trivial CAPP algorithms for or to prove circuit lower bounds for ? Recall: we know exponential lower bounds for these two models! Can we ``mine’’ some algorithms from these proofs?

31. Thank You