1 / 63

Something for almost nothing: Advances in sublinear time algorithms

Something for almost nothing: Advances in sublinear time algorithms. Ronitt Rubinfeld MIT and Tel Aviv U. Algorithms for REALLY big data. No time. What can we hope to do without viewing most of the data?. Small world phenomenon. The social network is a graph: “ node’’ is a person

emile
Télécharger la présentation

Something for almost nothing: Advances in sublinear time algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Something for almost nothing: Advances in sublinear time algorithms RonittRubinfeld MIT and Tel Aviv U.

  2. Algorithms for REALLY big data

  3. No time What can we hope to do without viewing most of the data?

  4. Small world phenomenon • The social network is a graph: • “node’’ is a person • “edge’’ between people that know each other • “6 degrees of separation’’ • Are all pairs of people connected by path of distance at most 6?

  5. Vast data • Impossible to access all of it • Accessible data is too enormous to be viewed by a single individual • Once accessed, data can change

  6. The Gold Standard • Linear time algorithms: • for inputs encoded by n bits/words, allow cn time steps (constant c) • Inadequate…

  7. What can we hope to do without viewing most of the data? • Can’t answer “for all” or “exactly” type statements: • are all individuals connected by at most 6 degrees of separation? • exactly how many individuals on earth are left-handed? • Compromise? • is there a large group of individuals connected by at most 6 degrees of separation? • approximately how many individuals on earth are left-handed?

  8. What types of approximation?

  9. Property Testing • Does the input object have crucial properties? • Example Properties: • Clusterability, • Small diameter graph, • Close to a codeword, • Linear or low degree polynomial function, • Increasing order • Lots and lots more…

  10. “In the ballpark” vs. “out of the ballpark” tests • Property testing: Distinguish inputs that have specific property from those that arefarfrom having that property • Benefits: • Can often answer such questions much faster • May be the natural question to ask • When some “noise” always present • When data constantly changing • Gives fast sanity check to rule out very “bad” inputs (i.e., restaurant bills) • Model selection problem in machine learning

  11. Examples • Can test if a function is a homomorphism in CONSTANT TIME[Blum LubyR.] • Can test if the social network has 6 degrees of separation in CONSTANT TIME [Parnas Ron]

  12. Constructing a property tester: • Find characterization of property that is • Efficiently (locally) testable • Robust - • objects that have the property satisfy characterization, • and objects far from having the property are unlikely to PASS Usually the bigger challenge

  13. Example: Homomorphism property of functions • A “bad” testing characterization: ∀x,y f(x)+f(y) = f(x+y) • Another bad characterization: For most x f(x)+f(1) = f(x+1) • Good characterization: For most x,yf(x)+f(y) = f(x+y)

  14. Example: 6 degrees of separation • A “bad” testing characterization: For every node, all other nodes within distance 6. • Another bad one: For most nodes, all other nodes within distance 6. • Good characterization: For most nodes, there are many other nodes within distance 6.

  15. An example in depth

  16. Monotonicity of a sequence • Given: list y1 y2 ...yn • Question: is the list sorted? • Clearly requires n steps – must look at each yi

  17. Monotonicity of a sequence • Given: list y1 y2 ...yn • Question: can we quickly test if the list close tosorted?

  18. What do we mean by ``quick’’? • query complexity measured in terms of list size n • Our goal (if possible): • Very smallcompared to n, will go for clog n

  19. What do we mean by “close’’? Definition:a list of size nis .99-close to sorted if can delete at most .01n values to make it sorted. Otherwise, .99-far. Sorted: 1 2 4 5 7 11 14 19 20 21 23 38 39 45 Close:1 4 2 5 7 11 14 19 20 39 23 21 38 45 1 4 5 7 11 14 19 20 23 38 45 Far:45 39 23 1 38 4 5 21 20 192 7 11 14 1 4 5 7 11 14 Requirements for algorithm: • pass sorted lists • if list passes test, can change at most .01 fraction of list to make it sorted

  20. yi i An attempt: • Proposed algorithm: • Pick random i and test that yi≤yi+1 • Bad input type: • 1,2,3,4,5,…j, 1,2,3,4,5,….j,1,2,3,4,5,…j, 1,2,3,4,5,…,j • Difficult for this algorithm to find “breakpoint” • But other algorithms work well

  21. yi i A second attempt: • Proposed algorithm: • Pick randomi<j and test that yi≤yj • Bad input type: • n/m groups of melements m,m-1,m-2,…,1, 2m,2m-1,2m-2,…m+1, 3m,3m-1,3m-2,…, • must pick i,j in same group • need at least (n/m)1/2 choices to do this

  22. A test that works • The test: (for distinct yi) • Test several times: • Pick random i • Look at value of yi • Do binary search for yi • Does the binary search find any inconsistencies? If yes, FAIL • Do we end up at location i? If not FAIL • Pass if never failed • Running time: O(log n) time • Why does this work? • If list is in order, then test always passes • If the test passes on choice iand j, then yi and yj are in correct order • Since test usually passes, most yi’sin the right order

  23. Many more properties studied! • Graphs, functions, point sets, strings, … • Amazing characterizations of problems testable in graph and function testing models!

  24. Properties of functions • Linearity and low total degree polynomials [Blum Luby R.] [Bellare Coppersmith Hastad Kiwi Sudan] [R. Sudan] [AroraSafra] [Arora Lund Motwani Sudan Szegedy] [Arora Sudan] ... • Functions definable by functional equations – trigonometric, elliptic functions [R.] • All locally characterized affine invariant function classes! [Bhattacharyya Fischer HatamiHatami Lovett] • Groups, Fields [Ergun Kannan Kumar R. Viswanathan] • Monotonicity [EKKRV] [GoldreichGoldwasser Lehman Ron Samorodnitsky] [DodisGoldreich Lehman Ron RaskhodnikovaSamorodnitsky][Fischer Lehman Newman Raskhodnikova R. Samorodnitsky] [ChakrabartySeshadri]… • Convexity, submodularity[Parnas Ron R.] [Fattal Ron] [SeshadriVondrak]… • Low complexity functions, Juntas [Parnas Ron Samorodnitsky] [Fischer Kindler Ron SafraSamorodnitsky] [Diakonikolas Lee MatulefOnak R. Servedio Wan]…

  25. Properties of sparse and general graphs • Easily testable dense and hyperfinitegraph properties are completely characterized! [Alon Fischer Newman Shapira][Newman Sohler] • General Sparse graphs: bipartiteness, connectivity, diameter, colorability, rapid mixing, triangle free,… [Goldreich Ron] [Parnas Ron] [BatuFortnow R. Smith White] [Kaufman Krivelevich Ron] [Alon Kaufman Krivelevich Ron]… • Tools: Szemeredi regularity lemma, random walks, local search, simulate greedy, borrow from parallel algorithms

  26. Some other combinatorial properties • Set properties – equality, distinctness,... • String properties – edit distance, compressibility,… • Metric properties – metrics, clustering, convex hulls, embeddability... • Membership in low complexity languages – regular languages, constant width branching programs, context-free languages, regular tree languages… • Codes – BCH and dual-BCH codes, Generalized Reed-Muller,…

  27. “Traditional” approximation • Output number close to value of the optimal solution(not enough time to construct a solution) • Some examples: • Minimum spanning tree, • vertex cover, • max cut, • positive linear program, • edit distance, …

  28. Example: Vertex Cover • Given graph G(V,E), a vertex cover (VC) C is a subset of V such that it “touches” every edge. • What is minimum size of a vertex cover? • NP-complete • Poly time multiplicative 2-approximation based on relationship of VC and maximal matching

  29. Vertex Cover and Maximal Matching • Maximal Matching: • ME is amatchingif no node in in more than one edge. • M is a maximal matching if adding any edge violates the matching property • Note: nodes matched by M form a pretty good Vertex Cover! • Vertex cover must include at least one node in each edge of M • Union of nodes of M form a vertex cover • So |M| ≤ VC ≤ 2 |M|

  30. “Classical” approximation examples • Can get CONSTANT TIME approximation for vertex cover on sparse graphs! • Output y which is at most 2∙ OPT + ϵn How? • Oracle reduction framework [Parnas Ron] • Construct “oracle” that tells you if node u in 2-approx vertex cover • Use oracle + standard sampling to estimate size of cover But how do you implement the oracle?

  31. Implementing the oracle – two approaches: • Sequentially simulate computations of a local distributed algorithm [Parnas Ron] • Figure out what greedy maximal matching algorithm would do on u [Nguyen Onak]

  32. Greedy algorithm for maximal matching • Algorithm: • For every edge (u,v) • If neither of u or v matched • Add (u,v) to M • Output M • Why is M maximal? • If e not in M then either u or v already matched by earlier edge

  33. Implementing the Oracle via Greedy • To decide if edge e in matching: • Must know if adjacent edges that come before e in the ordering are in the matching • Do not need to know anything about edges coming after • Arbitrary edge order can have long dependency chains! Odd or even steps from beginning? 1 2 4 8 25 36 47 88 89 110 111 112 113

  34. Breaking long dependency chains[Nguyen Onak] • Assign random ordering to edges • Greedy works under any ordering • Important fact: random order has short dependency chains

  35. Better Complexity for VC • Always recurse on least ranked edge first • Heuristic suggested by [Nguyen Onak] • Yields time poly in degree [Yoshida Yamamoto Ito] • Additional ideas yield query complexity nearly linear in average degree for general graphs[Onak Ron Rosen R.]

  36. Further work • More complicated arguments for maximum matching, set cover, positive LP… [Parnas Ron + Kuhn MoscibrodaWattenhofer] [Nguyen Onak] [Yoshida Yamamoto Ito] • Even better results for hyperfinite graphs [Hassidim Kelner Nguyen Onak][Newman Sohler] • e.g., planar Can dependence be made poly in average degree?

  37. No samples What if data only accessible via random samples?

  38. Distributions

  39. Play the lottery?

  40. Is the lottery unfair? • From Hitlotto.com:Lotteryexperts agree, past number histories can be the key to predicting future winners.

  41. True Story! • Polish lottery Multilotek • Choose “uniformly” at random distinct 20 numbers out of 1 to 80. • Initial machine biased • e.g., probability of 50-59 too small • Past results: http://serwis.lotto.pl:8080/archiwum/wyniki_wszystkie.php?id_gra=2

  42. Thanks to Krzysztof Onak (pointer) and Eric Price (graph)

  43. New Jersey Pick 3,4 Lottery • New Jersey Pick k (=3,4) Lottery. • Pick k digits in order. • 10kpossible values. • Assume lottery draws iid • Data: • Pick 3 -8522results from 5/22/75 to 10/15/00 • 2-test gives 42% confidence • Pick 4 -6544 results from 9/1/77 to 10/15/00. • fewer results than possible values • 2-test gives no confidence

  44. Distributions on BIG domains • Given samples of a distribution, need to know, e.g., • entropy • number of distinct elements • “shape” (monotone, bimodal,…) • closeness to uniform, Gaussian, Zipfian… • No assumptions on shapeof distribution • i.e., smoothness, monotonicity, Normal distribution,… • Considered in statistics, information theory, machine learning, databases, algorithms, physics, biology,…

  45. Key Question • How many samples do you need in terms of domain size? • Do you need to estimate the probabilities of each domain item? • Can sample complexity be sublinear in size of the domain? Rules out standard statistical techniques, learning distribution

  46. Our Aim: Algorithms with sublinear sample complexity

  47. Similarities of distributions Are p and q close or far? • p is given via samples • q is either • known to the tester (e.g. uniform) • given via samples

  48. Is p uniform? • Theorem:([Goldreich Ron] [BatuFortnow R. Smith White] [Paninski]) Sample complexity of distinguishing fromis • Nearly same complexity to test if p is any known distribution [Batu Fischer Fortnow Kumar R. White]: “Testing identity” p samples Test Pass/Fail?

  49. Upper bound for L2 distance [GoldreichRon] • L2distance: • ||p-U||22= S(pi -1/n)2 = Spi2 - 2Spi/n + S1/n2======== = Spi2 - 1/n • Estimatecollision probability to estimate L2 distance from uniform

  50. Testing uniformity [GR, Batu et. al.] • Upper bound: Estimate collision probability and use known relation between betweenL1 and L2norms • Issues: • Collision probability of uniform is 1/n • Use O(sqrt(n)) samples via recycling • Comment: [P] uses different estimator • Easy lower bound: (n½) • Can get  (n½/2) [P]

More Related