Master Theorem: Understanding Recurrence Relations for Algorithm Analysis
Learn how the Master Theorem simplifies solving recursive algorithms by analyzing cases and bounding asymptotic functions. Discover examples and applications in computational analysis.
Master Theorem: Understanding Recurrence Relations for Algorithm Analysis
E N D
Presentation Transcript
Master Method Some of the slides are from Prof. Plaisted’s resources at University of North Carolina at Chapel Hill
The Master Method • Based on the Master theorem. • “Cookbook” approach for solving recurrences of the form T(n) = aT(n/b) + f(n) • a 1, b > 1 are constants. • f(n) is asymptotically positive. • n/b may not be an integer, but we ignore floors and ceilings. • Requires memorization of three cases. COT5407
The Master Theorem • Theorem 4.1 • Let a 1 and b > 1be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b. T(n) can be bounded asymptotically in three cases: • If f(n) = O(nlogba–) for some constant > 0, then T(n) = (nlogba). • If f(n) = (nlogba), then T(n) = (nlogbalg n). • If f(n) = (nlogba+) for some constant > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b) c f(n), then T(n) = (f(n)). COT5407
Recursion tree view f(n) a f(n/b) f(n/b) … f(n/b) a a a … … … f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) a a a a a a … … … … … … Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) f(n) af(n/b) a2f(n/b2) (nlogba) Total: COT5407
The Master Theorem • Theorem 4.1 • Let a 1 and b > 1be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b. T(n) can be bounded asymptotically in three cases: • If f(n) = O(nlogba–) for some constant > 0, then T(n) = (nlogba). • If f(n) = (nlogba), then T(n) = (nlogbalg n). • If f(n) = (nlogba+) for some constant > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b) c f(n), then T(n) = (f(n)). COT5407
Master Method – Examples • T(n) = 16T(n/4)+n • a = 16, b = 4, nlogba = nlog416 = n2. • f(n) = n = O(nlogba-) = O(n2- ), where = 1 Case 1. • Hence, T(n) = (nlogba ) = (n2). • T(n) = T(3n/7) + 1 • a = 1, b=7/3, and nlogba = nlog7/31 = n0 = 1 • f(n) = 1 = (nlogba) Case 2. • Therefore, T(n) = (nlogba lg n) = (lg n) COT5407
Master Method – Examples • T(n) = 3T(n/4) + n lg n • a = 3, b=4, thus nlogba = nlog43 = O(n0.793) • f(n) = n lg n = (nlog43 + )where 0.2 Case 3. • Therefore, T(n) = (f(n)) = (n lg n). • T(n) = 2T(n/2) + n lg n • a = 2, b=2, f(n) = n lg n, and nlogba = nlog22 = n • f(n)is asymptotically larger thannlogba, but not polynomially larger. The ratio lg n is asymptotically less than n for any positive . Thus, the Master Theorem doesn’t apply here. COT5407
Master Theorem – What it means? • Case 1:If f(n) = O(nlogba–) for some constant > 0, then T(n) = (nlogba). • nlogba = alogbn: Number of leaves in the recursion tree. • f(n) = O(nlogba–) Sum of the cost of the nodes at each internal level asymptotically smaller than the cost of leaves by a polynomial factor. • Cost of the problem dominated by leaves, hence cost is (nlogba). COT5407
Master Theorem – What it means? • Case 2:If f(n) = (nlogba), then T(n) = (nlogbalg n). • nlogba = alogbn : Number of leaves in the recursion tree. • f(n) = (nlogba) Sum of the cost of the nodes at each level asymptotically the same as the cost of leaves. • There are (lg n) levels. • Hence, total cost is (nlogba lg n). COT5407
Master Theorem – What it means? • Case 3:If f(n) = (nlogba+) for some constant > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b) c f(n), then T(n) = (f(n)). • nlogba = alogbn: Number of leaves in the recursion tree. • f(n) = (nlogba+) Cost is dominated by the root. Cost of the root is asymptotically larger than the sum of the cost of the leaves by a polynomial factor. • Hence, cost is (f(n)). COT5407
Master Theorem – Proof for exact powers • Proof when n is an exact power of b. • Three steps. • Reduce the problem of solving the recurrence to the problem of evaluating an expression that contains a summation. • Determine bounds on the summation. • Combine 1 and 2. COT5407
Iterative “Proof” of the Master Theorem • Using iterative substitution, let us see if we can find a pattern: • We then distinguish the three cases as • The first term is dominant • Each part of the summation is equally dominant • The summation is a geometric series
f(n) a f(n/b) f(n/b) … f(n/b) a a a … … … f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) a a a a a a … … … … … … Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) f(n) af(n/b) a2f(n/b2) (nlogba) Total: COT5407
Integer Multiplication • Algorithm: Multiply two n-bit integers I and J. • Divide step: Split I and J into high-order and low-order bits • We can then define I*J by multiplying the parts and adding: • So, T(n) = 4T(n/2) + n, which implies T(n) is O(n2). • But that is no better than the algorithm we learned in grade school.
An Improved Integer Multiplication Algorithm • Algorithm: Multiply two n-bit integers I and J. • Divide step: Split I and J into high-order and low-order bits • Observe that there is a different way to multiply parts: • So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23), by the Master Theorem. • Thus, T(n) is O(n1.585).
