Complexity Theory Lecture 1

1. Complexity TheoryLecture 1 Lecturer:Moni Naor

2. Computational Complexity Theory • Study the resources needed to solve computational problems • Computer time • Computer memory • Communication • Parallelism • Randomness • … • Identify problems that are infeasible to compute by any reasonable machine • Taxonomy: classify problems into classes with similar properties wrt the resource requirements • Help find the most efficient algorithm for a problem

3. Goal of Complexity Theory • Given a computational problem – determine the best method of solving it • For a given computational setting – determine which computation tasks can be executed • State of the art: very far from these goals • Cannot provide unconditional interesting lower bounds for almost any interesting problem • But:…

4. Complexity, Algorithms and Cryptography • Complexity Theory has developed a taxonomy allowing to determine for many problems Complexity Cryptography Algorithms

5. Brief History of Complexity Theory • Logical foundations • 1930s: Computability Theory: • Kurt Gödel • Alan Turing • Alonzo Church • 1940s 1950s: work on circuit complexity (Shannon) • 1960s: first papers dealing explicitly with complexity issues: • Definitions of polynomial time (Cobham, Edmonds, Peterson) • Time hierarchies, Speedup Theorems, Abstract Complexity Measures • Early 1970s: formulation of the P=NP? question, NP-Completeness an industry (Cook, Karp, Levin) • Since then – an explosion Developments in the Soviet Union • 1956 Gödel’s letter to von Neumann with the P=NP question

6. What will we learn in this course • Time and space hierarchies • Non-determinism and NP-Completeness • Relativization (oracles) • Alternation • Space Complexity • Randomness • Interactive Proofs • PCP • Complexity of counting • Concrete Complexity models: • Communication Complexity • Pebbling • Branching Programs • Circuit Complexity • P, RP, BPP BQP • NP, co-NP, PH, #P, PSPACE, • IP, AM, PCP • NC, AC, L, NL

7. Course Information Complexity Theory  - Winter 2004/5 Instructor: Moni Naor Grader: Asaf Nussbaum When:     Thursdays, 14:00--17:00 (3 points) Where:    Ziskind 1 • Course web page: www.wisdom.weizmann.ac.il/~naor/COURSE/complexity.html • Prerequisites: familiarity with algorithms, data structures, probability theory, and linear algebra, at an undergraduate level; a basic course in computability is assumed. • Requirements: • Homework: There will be around ten homework assignments and a final test. • Homework assignments should be turned in on time (usually two weeks after they are given)! • Try and do as many problems from each set. • You may discuss the problems with other students, but the write-up MUST be individual. • You may look up any material, but should cite any external source. • Exam : The exam will be in class.

8. Bibliography No textbook for the course. A lot of background and relevant material is available in  • Christos Papadimitriou , Computational Complexity , Cambridge, 1994 • Michael Sipser, Introduction to the Theory of Computation, 1997 • Garey and  Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness. New York: W. H. Freeman, 1979.      Two online  courses close in nature to our course: • Lecturers: Steven Rudich and Avrim Blum, University: CMU • Lecturer: Luca Trevisan, Computational Complexity, University : UC Berkeley.

9. Things you should already know • Turing Machine • One-tape vs. Multi-tape • Defined by (,Q,) •  finite alphabet including blank symbol • State space including qstart, qaccept and qreject • transition function :  X Q  X Q X {left,right} • A configuration is represented as a string of symbols from [ Q 12 … q … m • Multi-tape machine has • Read-only input tape • Write-only output tape • Several working tapes

10. Things you should already know • Recognition or decision of language L by a TM M • if Maccepts every string in L and rejects all strings not in L • A language L is recursive or decidable if there is a TM M which recognizes it • Acceptance of language L by a TM M • if Maccepts every string in L and does not accept strings not in L • does not necessarily halt on all inputs • A language L is recursive enumerable (r.e.) if there is a TM M which accepts it

11. Turing Machines are general Truing Machines can simulate many other models and machines • RAM – Random Access Machines • Java.. • Circuits – if the circuit itself is uniform • Various logical systems • Turing machines: the language Lu ={<M,x>|M accepts x} is r.e. – there is a (universal) TM accepting it Simulation are pretty efficient Counterexample: quantum

12. Time and Space Let f:N N by any function. Time: L 2 TIME(f(n)) if there is a (multi-tape) TM which recognizes L and uses no more than f(|x|) steps on input x Space: L 2 SPACE(f(n)) if there is a (multi-tape) TM which recognizes L and uses no more than f(|x|) cells of its work tapes during the computation on input x • Cell is used if it scanned by the tape head Time and space can be defined for function computation as well No point in accuracy in the time function Theorem (Linear Speedup): for any L 2 TIME(f(n)) and >0 there is an equivalent TM M’ requiring only f(n)+n steps Conclusion: lets talk about O(f(n))

13. Proper complexity functions • f(n) is a proper complexity function if • f(n)≥ f(n-1) • There is a TM which on input x outputs a string of length exactly f(|x|) while running it time O(|x|+f(|x|)) and space O(f(|x|)) • Exercise: show that if f(n) and g(n) are proper complexity functions, then so are • f(n)+g(n) • f(n)g(n) Clocked Simulation Theorem: for every proper complexity function f There exists a TM Mfrunning in time f3(|Mx|) that on input (M,x) accepts iff M(x) accepts within f(|x|) steps Can improve f3(n) to f(n) log f(n)

14. Hierarchy Theorems and Diagonalization • Diagonalization – proof technique invented by Cantor to prove the uncountability of the real numbers Theorem (Time Hierarchy): for any proper complexity functions f(n) and g(n) ≥ f(n) log(f(n)) TIME(f(n)) is properly contained in TIME(g(2n)) Proof: Recall Mf. Consider L ={<M|Mf(MM) rejects} We have L2TIME(g(2n)) but L is not in TIME(f(n)) Corollary: TIME(nk) is properly contained in TIME(nk+1)

15. Alice Bob Communication Complexity x2X y2Y • Let f:X xY Z • Input is split between two participants • Want to compute outcome: z=f(x,y) • while exchanging as few bits as possible

16. Alice: 0 Bob: 1 Alice: 0 Bob: 1 A protocol is defined by the communication tree z5 z0 z1 z2 z3 z4 z5 z6 z7 ...

17. A Protocol A protocol P over domain X x Y with range Z is a binary tree where • Each internal node v is labeled with either • av:X {0,1} or • bv:Y {0,1} • Each leaf is labeled with an element z 2 Z • The value of protocolP on input (x,y) is the label of the leaf reached by starting from the root and walking down the tree. • At each internal node labeled avwalk • left if av(x)=0 • right if av(x)=1 • At each internal node labeled bvwalk • left if bv(y)=0 • right if bv(y)=1 • The cost of protocolP on input (x,y) is the length of the path taken on input (x,y) • The cost of protocolPis the maximum path length

18. Motivation for studying communication complexity • Originally for studying VLSI questions • Connection with Turing Machines • Data structures and the cell probe model • Boolean circuit depth • …

19. Communication Complexity of a function • For a function f:X x Y Z the (deterministic) communication complexity of f (D(f)) is the minimum cost of protocol P over all protocols that compute f Observation: For any function f:X x Y Z D(f) ≤ log |X| + log |Z| Example: let x,y µ {1,…,n} and let f(x,y)=max{x [ y} Then D(f) · 2 log n

20. Median let x,y µ {1,…,n} and let MED(x,y) be the median of the multiset x [ y If the size is even then element ranked |x[ y|/2 Claim:D(MED) is O(log2 n) protocol idea: do a binary search on the value, each party reporting how many are above te current guess Homework:D(MED) is O(log n) protocol idea: each party proposes a candidate See which one is larger - no need to repeat bits

21. Combinatorial Rectangles • A combinatorial rectangle in X x Y is a subset R µ X x Y such that R= A x B for some A µ X and B µ Y Proposition: R µ X x Y is a combinatorial rectangle iff (x1,y1) 2 R and (x2,y2) 2 R implies that (x1,y2) 2 R For Protocol P and nodevlet Rv be the set of inputs(x,y) reaching v Claim: For any protocol P and nodevthe set Rv is a combinatorial rectangle Claim: For any given the transcript of an exchange between Alice an Bob possible (but not x and y) possible to determine z=f(x,y)

22. Fooling Sets • For f:X x Y Z a subset R µ X x Y is f-monochromatic if f is fixed on R • Observation: any protocol P induces a partition of X x Y into f-monochromatic rectangles. The number of rectangles is the number of leaves in P • A set Sµ X x Y is a fooling set for f if there exists a z 2 Z where • For every(x,y) 2 S, f(x,y)=z • For everydistinct (x1,y1), (x2,y2) 2 S either • f(x1,y2)≠z or • f(x2,y1)≠z Property: no twoelements of a fooling set S can be in the same monochromatic rectangle Lemma: if f has a fooling set of size t, then D(f) ≥ log2 t

23. Applications Equality: Alice and Bob each hold x,y 2 {0,1}n • want to decide whether x=y or not. • Fooling set for Equality S={(w,w)|w 2 {0,1}n } Conclusion: D(Equality) ¸ n Disjointness: let x,y µ {1,…,n} and let • DISJ(x,y)=1 if |x  y|¸ 1 and • DISJ(x,y)=0 otherwise • Fooling set for Disjointness S={(A,comp(A))|A µ {1,…,n} } Conclusion: D(DISJ) ¸ n

24. Applications to Time-Space lower bounds in Turing Machines Let Palindromes={xxR| x 2*} • Palindromes 2 TIME(n) • Palindromes 2 SPACE(log n) But what about simultaneous linear time and log space? Lemma: let f: {0,1}nx {0,1}n {0,1} be a function with communication complexity D(f) . For any multi-tape TM running in time T(n) and space S(n)accepting all string {x0ny| |y|=|x|=n, f(x,y)=1} and rejecting all strings {x0ny| |y|=|x|=n, f(x,y)=0} We have D(f) = O(T(n)S(n)/n) Proof: joint simulation of the TM by Alice and Bob. To get time-space lower bound on Palindromes use Equality Consider {x0nxR| x 2*, |x|=n}

25. Rank lower bound • For f:X x Y {0,1} let Mf tbe the |X| x |Y| matrix where entry f(x,y) has value f(x,y). The rank of f, rank(f), is the rank of Mf over the reals. Theorem: For any f:X x Y {0,1} D(f) ≥ log2 rank(f) Proof: For any protocol PMf =ℓ leaf Mℓ Mℓ is the matrix corresponding to the rectangle of leaf ℓ Examples: • Equality MEquality is the identity matrix – rank is 2n

26. Inner Product • Let x,y 2 {0,1}n IP(x,y)= i=1n xi yi mod 2 What is rank(MIP)? Let N=MIP MIP. Then entry (x,y) in N is z 2 {0,1}n <x,z> <y,z> which is # z’s where <x,z>=<y,z>=1 • 2n-2if x≠y • 2n-1if x=y • 0 if x or y is 0 Hence rank(N) ¸ 2n-1. Since rank(MIP MIP) · min{rank(MIP},rank(MIP}} rank(MIP} ¸ 2n-1 and D(IP) ¸ n

27. Non-determinism and covers • Can a protocol tree be very unbalanced? • Technique for balancing: given a protocol with t leaves there is a protocol with communication complexity O(log t) Is the monotone rectangles lower bound tight? • Consider instead of partition into monochromatic rectangles a cover by monochromatic rectangles The rectangles are not necessary disjoint For z 2 Z a z-cover handles only the inputs (x,y) where f(z,y)=z This corresponds to non-deterministic complexity A non-deterministic protocol for verifying that f(x,y)=z: Alice: Guess a rectangle R intersecting row x, send name to Bob Bob: verify that R intersects column y tell Alice Accept only if Bob approve Complexity:log # of z-rectangles