Randomized Algorithms

1. Randomized Algorithms Morteza ZadiMoghaddam Amin Sayedi

2. Types of Randomized algorithms • Las Vegas • Monte Carlo

3. Las Vegas • Always gives the true answer. • Running time is random. • Running time is bounded. • Quick sort is a Las Vegas algorithm.

4. Monte Carlo • It may produce incorrect answer! • We are able to bound its probability. • By running it many times on independent random variables, we can make the failure probability arbitrarily small at the expense of running time.

5. Monte Carlo Example • Suppose we want to find a number among n given numbers which is larger than or equal to the median.

6. Monte Carlo Example Suppose A1 < … < An. We want Ai, such that i ≥ n/2. It’s obvious that the best deterministic algorithm needs O(n) time to produce the answer. n may be very large! Suppose n is 100,000,000,000 !

7. Monte Carlo Example • Choose 100 of the numbers with equal probability. • find the maximum among these numbers. • Return the maximum.

8. Monte Carlo Example • The running time of the given algorithm is O(1). • The probability of Failure is 1/(2100). • Consider that the algorithm may return a wrong answer but the probability is very smaller than the hardware failure or even an earthquake!

9. Monte Carlo • Suppose the output is Yes or No. • One sided error. • Two sided error.

10. RP Class( randomized polynomial ) • Bounded polynomial time in the worst case. • If the answer is Yes; Pr[ return Yes] > ½. • If the answer is No; Pr[ return Yes] = 0. • ½ is not actually important.

11. PP Class( probabilistic polynomial ) • Bounded polynomial time in worst case. • If the answer is Yes; Pr[ return Yes] > ½. • If the answer is No; Pr[ return Yes] < ½. • Unfortunately the definition is weak because the distance to ½ is important but is not considered.

12. Routing Problem • There are n computers. • Each computer has a packet. • Each packet has a destination D(i). • Packets can not follow the same edge simultaneously. • An oblivious algorithm is required.

13. Routing Problem • For any deterministic oblivious algorithm on a network of N nodes each of outdegree d, there is an instance of permutation routing requiring (N/d) ½.

14. Routing Problem • Pick random intermediate destination. • Packet i first travels to the intermediate destination and then to the final destination. • With probability at least 1-(1/N), every packet reaches its destination in 14n of fewer steps in Qn. • The expected number of steps is 15n.

15. Maximum Satisfiability • You have m clauses and n boolean variables. • Each clause contains some of variables or some of complements. • A clause is satisfied if at least one of it’s variables are satisfied. • We want to set the variables such that the number of satisfied clauses is maximized.

16. Example for Maximum Sat • There are 3 variables A, B and C. • M1 = (A) or (B) • M2 = (A) or (not B) or (not C) • M3 = (C) • M4 = (B) or (not C) • M5 = (not C)

17. Example of Maximum Sat • Set A = True • Set B = True • Set C = False • Four of the clauses are satisfied.

18. Maximum Sat • This problem is a famous problem which has no polynomial time algorithm yet. It’s NP-hard.

19. Maximum Sat • For any set of m clauses, there is truth assignment for the variables that satisfies at least m/2 clauses.

20. Maximum Sat • Let Zi =1 if the i-th clause is satisfied and 0 otherwise. • Set the variables in a random way. • The probability of a clause with k variables to be true is 1- (1/(2k)) >= ½. • So E[Z1]+…+E[Zm] >= ½. • Thus there exist at least one assignment such that Z1+…+Zm >= ½.

21. Maximum Sat algorithm • This problem is NP-hard so we seek for approximation algorithms. • We have an algorithm that produces an answer which is at least ½ of the best answer. • If all clauses have at least 2 literals then we have an algorithm that produces an answer which is at least ¾ of the best one.

22. Maximum Sat algorithm • We want to maximize Z1+…+Zm. • We have some inequalities: • ∑ yi (if Xi is in Zj and is uncomplemented) • ∑ (1-yi) (if Xi is in Zj and is complemented). • This inequality must be hold: • ∑ yi + ∑ (1-yi) >= Zj • This problem could be solved using integer linear programming. • We have to use linear programming.

23. Maximum Sat • Solve the problem using linear programming. • You get a real number for each yi or zi. • Assign Xi true with the probability yi. • The expected number of clauses that are satisfied is (1- 1/e) of the best answer.

24. Maximum Sat algorithm • Using both algorithms and choosing the better answer gives us an answer which is at least ¾ of the best answer!!! Which is better than ½ and 1- 1/e.

25. 2-Sat • Every clause has at most 2 literals. • We want to check if all clauses can be satisfied. • It has polynomial algorithm. • Assign random values to the variables. • If all clauses are satisfied we are finished. • If there is an unsatisfied clause, the value of one of it’s literals is different from the best answer. • Change the value of one of the variables in this clause. You may make a good change or bad one.

26. 2-Sat • You are walking on a path. • If you are on 0 you go to 1. • If you are on i you go to i+1 or i-1 with equal probability. • The expected number of steps to reach the end of the path is O(n2). • So the given algorithm is O(n3).

27. Graph Connectivity • You want to check if two vertices u and v are in the same connected component. • Start a random walk from v. • Have a random walk of length 2n3. • If you haven’t visited u, the probability of u to be in this component is less than ½. • By repeating this algorithm, you can make the probability of failure arbitrarily small.

28. Graph Connectivity • Running time of algorithm is O(n3). • Required space is O(logn).

29. Diameter of a Point Set • You want to find the diameter of set of n points in the space. • Suppose I(x) is the convex body formed by the intersection of n sphere centered at n points with radius x. • F(p) is distance between p and the point in the set that is farthest from p.

30. Diameter of a Point Set • Consider I(x) when x=F(p). • For any q in S, if q is in I(x) then F(q)<F(p). • And if q is not in I(x) then F(p)<F(q).

31. Diameter of a Point Set • Pick a point p in s at random. Computer F(p). [O(n)] • Set x=F(p). Compute I(x). [O(n logn)] • Find the points outside I(x). Call this set T. [O(n logn)] • If T is empty return x as the answer, else continue on T.

32. Diameter of a Point Set • The running time of algorithm above is O(n log n). • In each step, all points that have smaller F(x) than the chosen point are removed.

33. All-Pairs Shortest Paths • Let G(V,E) be an undirected, connected graph with V={1,…,n} and |E|=m. • The adjacency matrix A is an n  n 0-1 matrix with Aij=Aji=1 if and only if the edge (i,j) is present in E. • We are going to compute matrix D which Dij equals the length of a shortest path from vertex i to vertex j.

34. All-Pairs Distances • Z A2 • Compute matrix A’ such that A’ij=1 if and only if i ≠ j and (Aij=1 or Zij>0) • If A’ij=1 for all i ≠ j then return D = 2A’-A. • Recursively compute the APD matrix D’ for the graph G’ with adjacency matrix A’. • S AD’ • Return matrix D with Dij=2D’ij if Sij≥D’ijZii , otherwise Dij=2D’ij-1.

35. APSP • The APD algorithm computes the distance matrix for an n-vertex graph in time O(MM(n)log(n)) using integer matrix multiplication algorithm. • Matrix multiplication algorithm running in time O(n2.376).

36. Boolean Product Witness Matrix • Suppose A and B are nn boolean matrices and P=AB is their product under Boolean matrix multiplication. • A witness for Pij is an index k  {1,…,n} such that Aik=Akj=1. Observe that Pij=1 if and only if it has some witness k.

37. BPWM • W -AB • 2. for t=0,…, log(n) do • 2.1. r 2t • 2.2. Repeat 3.77log(n) times • 2.2.1. Choose random R  {1,…,n} with |R|=r . • 2.2.2. Compute AR and BR . • 2.2.3. Z ARBR . • 2.2.4. for all (i,j) do • if Wij < 0 and Zij is witness then Wij Zij • 3. for all (i,j) do • if Wij < 0 then find witness Wij by brute force.

38. BPWM • The BPWM algorithm is a Las Vegas algorithm for the BPWM problem with expected running time O(MM(n)log2(n)). • The probability that no witness is found for Pij before the end of Step 2 is at most • (1-1/2e)3.77log(n) 1/n.

39. Determining Shortest Path • A successor matrix S for an n-vertex graph G is an n  n matrix such that Sij is the index of a neighbor of vertex i that lies on a shortest path from i to j.

40. APSP • Compute the distance matrix D=APD(A). • for s={0,1,2} do • Compute 0-1 matrix Dkj(s)=1 if and only if Dkj+1 = s (mod 3) • Compute the witness matrix W(s)=BPWM(A,D(s)). • Compute successor matrix S for G.

41. APSP • Algorithm APSP computes the successor matrix for an n-vertex graph G in expected time O(MM(n)log2(n)).

42. Algorithm contract • H G • While H has more than 2 vertices do • Choose an edge (x,y) uniformly at random from the edges in H. • F F  {(x,y)}. • H H / (x,y). • (C,V/C) the sets of vertices corresponding to the two meta-vertices in H=G/F.

43. FastCut • n |v| • if n  6 then compute min-cut of G by brute-force enumeration else • t 1+n/2 • Using Algorithm Contract, perform two independent contraction sequences to obtain graphs H1 and H2 each with t vertices. • Recursively compute min-cuts in each of H1 and H2. • Return the smaller of the two min-cuts.

44. Fastcut • Algorithm Fastcut succeeds in finding a min-cut with probability (1/log(n)). • Algorithm Fastcut runs in O(n2log(n)) time and uses O(n2) space.

45. MST • Finding MST in a graph with n vertices and m edges has a Las Vegas algorithm which has the expected running time O(n+m). • But We don’t have enough time to Explain it !!!

46. Research problems • Devise an algorithm for the all-pairs shortest paths problem that does not use matrix multiplication and runs in time O(n3-) for a positive constant . • Devise an algorithm for computing the diameter of an unweighted graph that does not use matrix multiplication and runs in time O(n3-) for a positive constant .

47. Research problems • Devise a Las Vegas or a deterministic algorithm for min-cuts with running time close to O(n2). • Is there a randomized algorithm for min-cuts with expected running time close to O(m)?

48. Have a nice randomized life