Communication Complexity

1. Communication Complexity Guy Feigenblat Based on lecture by Dr. Ely Porat Some slides where adapted from various sources Complexity course Computer science department , Bar-Ilan university January 2008

2. The Model • 2 Computers : Alice (A) ,Bob (B) • All calculations for A are free • All calculations for B are free • Algorithm costs are measured by cost of communications. • Cost is measured per bits • Motivation – Distributed models

3. (98) 00100110 (48) 01110110 The Model 14, 29,53,28,284,348 39, 67,98,22,35,253 B A Communication Complexity

4. Our goal Minimize the communication between A,B while calculating functions on their inputs

5. First Problem • Input A has array of n numbers B has array of n numbers • Output median of all 2n numbers • Numbers are O(log n) bits long

6. Naive Solution • A sends all of his numbers to B • B calculates median of all 2n numbers • Cost • Each number is O(log n) bits • n numbers are sent • Total cost is O(n*log n) bits

7. (284) 00011111 (348) 001110101 (53) 101011 (29) 10111 (14) 001110 Naive Solution 14, 29,53,28,284,348 39, 67,98,22,35,253 B A Communication Complexity • Total cost is O(n*log n) bits

8. Better algorithm • A sorts his array and sends his median ( ) to B • B sorts his array and sends his median ( ) to A. Exercise : define : r = real median b = MAX{ } s = MIN { } prove :

9. B A Better algorithm A sort his array and find his median B sort his array and find his median 14, 28,29,53,284,348,500 22, 35,39,67,98,253,300 Communication Complexity

10. 53 67 B A Better algorithm B send his median to A A send his median to B 14, 28,29,53,284,348,500 22, 35,39,67,98,253,300 Communication Complexity

11. Better algorithm • 4. If = then return ( = ) • 5. If > then A throws top (n/2) elements B throws low (n/2) elements • 6. Otherwise, vice versa • We reduces the size of the problem by half • 7. Back to step 1, until size of arrays = 1

12. 14, 28,29 ,98,253,300 ,98,253,300 B A Better algorithm 53<67 Then A throws the small half of his array 67>53 Then B throws the big half of his array ,53,284,348,500 22, 35,39,67 Communication Complexity

13. 14, 28,29 ,98,253,300 ,98,253,300 B A Better algorithm We will repeat this algorithm until the size of the array will be 1, while every loop the array is cut in half, and log n bits transferred ,53,284,348,500 22, 35,39,67 • Total cost is • CCmid = O (log2n) bits Communication Complexity

14. Even Better algorithm Exercise: Try reducing the communication complexity to O(log n) bits

15. B A Communication Complexity EQ The previous subject talked about problem of finding median of a divided array. Now we consider a new problem : Each side has a number and we want to know if the numbers are equal. ? X=Y Y X

16. Deterministic Algorithm Send all the data We can’t avoid it !! Think why ?

17. Probabilistic Algorithm

18. Analysis Like Co-RP

19. How can we lower the failure probability ? • Run the experience few times • Use larger “q” (i.e. q = n10 )

20. G – Communication Complexity • Till now we had to send the Random number. • Consider a model in which A,B use a third party in order to synchronize random numbers. • A,B use exactly the same random “Alise on the moon, Bob on earth, both take random from the sun” Hard to “understand” – Remember this is just a model!!

21. Protocol

22. Analysis • Completeness – Perfect Completeness If A=B then all i times (a,r) = (b,r) • Soundness – If A≠B the probability that all i times (a,r) = (b,r) is 2-i • Communication Complexity

23. Exercise Define LE to be : LE(x,y) = 1 x ≤ y 0 otherwise Show that

24. Theorem: We prove that we can give up synchronized random with an overhead of O(logn) communication bits. We will choose the same random numbers (n2) using deterministic machine for both A,B

25. Protocol A choose and send it to B

26. If algorithm exists for L, we will run it 3 times and use it.

27. As for the proof… We have perfect completeness in But, we need to prove soundness For the chosen i

28. It is equivalent to argue: By union bound:

31. We have proved that there exist a group of There is a deterministic algorithm that can find them. Remember, both A and B have unlimited computational power.