480 likes | 665 Vues
Information Complexity Lower Bounds. Rotem Oshman, Princeton CCI Based on: Bar-Yossef,Jayram,Kumar,Srinivasan’04 Braverman,Barak,Chen,Rao’10. Communication Complexity. = ?. Yao ‘79, “Some complexity questions related to distributive computing ”. Communication Complexity.
E N D
Information Complexity Lower Bounds Rotem Oshman, Princeton CCI Based on: Bar-Yossef,Jayram,Kumar,Srinivasan’04 Braverman,Barak,Chen,Rao’10
Communication Complexity = ? Yao ‘79, “Some complexity questions related to distributive computing”
Communication Complexity • Applications: • Circuit complexity • Streaming algorithms • Data structures • Distributed computing • Property testing • …
Deterministic Protocols • A protocol specifies, at each point: • Which player speaks next • What should the player say • When to halt and what to output • Formally, what we’ve said so far who speaks next: Alice, Bob, = halt what to say/output
Randomized Protocols • Can use randomness to decide what to say • Private randomness: each player has a separate source of random bits • Public randomness: both players can use the same random bits • Goal: for any compute correctly with probability • Communication complexity: worst-case length of transcript in any execution
Randomness Can Help a Lot • Example: Equality • Input: • Output: is ? • Trivial protocol: Alice sends to Bob • For deterministic protocols, this is optimal!
Equality Lower Bound #rectangles
Randomized Protocol • Protocol with public randomness: • Select random • Alice sends • Bob accepts iff • If : always accept • If : • Reject with probability non-zero vector
Set Disjointness • Input: • Output: ? • Theorem [Kalyanasundaran, Schnitger ‘92, Razborov ‘92]: randomized CC = • Easy to see for deterministic protocols • Today we’ll see a proof by Bar-Yossef, Jayram, Kumar, Srinivasan ‘04
Application: Streaming Lower Bounds • Streaming algorithm: • Example: how many distinct items in the data? • Reduction from Disjointness [Alon, Matias, Szegedy ’99] How much space is required to approximate f(data)? algorithm data
Reduction from Disjointness: • Fix a streaming algorithm for Distinct Elements with space , universe size • Construct a protocol for Disj.with elements: algorithm ⇔ #distinct elements in is State of the algorithm and (#bits = )
Application 2: KW Games • Circuit depth lower bounds: • How deep does the circuit need to be?
Application 2: KW Games • Karchmer-Wigderson’93,Karchmer-Raz-Wigderson’94: find such that
Application 2: KW Games • Claim: if has deterministic CC , then requires circuit depth . • Circuit with depth protocol with length
Some Basic Concepts from Info Theory • Entropy of a random variable: • Important properties: • is deterministic • = expected # bits needed to encode
Some Basic Concepts from Info Theory • Conditional entropy: • Important properties: • are independent • Example: • If then , if 1then
Some Basic Concepts from Info Theory • Mutual information: • Conditional mutual information: • Important properties: • are independent
Some Basic Concepts from Info Theory • Chain rule for mutual information: • More generally,
Information Cost of Protocols • Fix an input distribution on • Given a protocol , let also denote the distribution of ’s transcript • Information cost of : • Information cost of a function :
Information Cost of Protocols • Important property: • Proof: by induction. Let . • : what we know after r rounds what we knew after r-1 rounds what we learn in round r, given what we already know
Information vs. Communication • Want: • Suppose is sent by Alice. • What does Alice learn? • is a function of and so • What does Bob learn?
Information vs. Communication • Important property: • Lower bound on information cost ⇒ lower bound on communication complexity • In fact, IC lower bounds are the most powerful technique we know
Information Complexity of Disj. • Disjointness: is ? • Strategy: for some “hard distribution” , • Direct sum: • Prove that .
Hard Distribution for Disjointness • For each coordinate :
Let be a protocol for on • Construct for as follows: • Alice and Bob get inputs • Choose a random coordinate , set • Sample and run • For each ,
Let be a protocol for on • Construct for as follows: • Alice and Bob get inputs • Choose a random coordinate , set • Bad idea: publicly sample Suppose in , Alice sends . In , Bob learns one bit in he should learn bit But if is public Bob learns 1 bit about !
Let be a protocol for on • Construct for as follows: • Alice and Bob get inputs • Choose a random coordinate , set • Another bad idea: publicly sample , Bob privately samples given • But the players can’t sample , independently…
Let be a protocol for on • Construct for as follows: • Alice and Bob get inputs • Choose a random coordinate , set Publicly sample Privately sample Privately sample Publicly sample
Direct Sum Theorem • Transcript of • Need to show:
Information Complexity of Disj. • Disjointness: is ? • Strategy: for some “hard distribution” , • Direct sum: • Prove that .
Hardness of And transcript on should be “very different”
Hellinger Distance • Examples: • If have disjoint support,
Hellinger Distance • Hellinger distance is a metric • , with equality iff • Triangle inequality:
Hellinger Distance • If for some we have then
Hellinger Distance vs. Mutual Info • Let be two distributions • Select by choosing , then drawing • Then
Hardness of And Same for Bob until Alice acts differently Same for Alice until Bob acts differently
“Cut-n-Paste Lemma” • Recall: • Enough to show: we can write
“Cut-n-Paste Lemma” • We can write • Proof: • induces a distribution on “partial transcripts” of each length : probability that first bits are • By induction: • Base case: • Set
“Cut-n-Paste Lemma” • Step: • Suppose after it is Alice’s turn to speak • What Alice says depends on: • Her input • Her private randomness • The transcript so far, • So • Set
The Coordinator Model sites • bits …
Multi-Party Set Disjointness • Input: • Output: is ? • Braverman,Ellen,O.,Pitassi,Vaikuntanathan’13: lower bound of bits
Reduction from Disjtograph connectivity • Given we want to • Choose vertices • Design inputs such that is connected iff
Reduction from Disjtograph connectivity (Players) (Elements) input graph connected
Other Stuff • Distributed computing
Other Stuff • Compressing down to information cost • Number-on-forehead lower bounds • Open questions in communication complexity