1. Computational Geometry Piyush Kumar (Lecture 5: Linear Programming) Welcome to CIS5930

2. Linear Programming • Significance • A lot of problems can be converted to LP formulation • Perceptrons (learning), Shortest path, max flow, MST, matching, … • Accounts for major proportion of all scientific computations • Helps in finding quick and dirty solutions to NP-hard optimization problems • Both optimal (B&B) and approximate (rounding)

3. Graphing 2-Dimensional LPs Optimal Solution y Example 1: 4 Maximize x + y 3 x + 2 y ³ 2 Subject to: Feasible Region x £ 3 2 y £ 4 1 x ³0y ³0 0 x 0 1 2 3 These LP animations were created by Keely Crowston.

4. Graphing 2-Dimensional LPs Multiple Optimal Solutions! y Example 2: 4 Minimize ** x - y 3 1/3 x + y £ 4 Subject to: -2 x + 2 y £ 4 2 Feasible Region x £ 3 1 x ³0y ³0 0 x 0 1 2 3

5. Graphing 2-Dimensional LPs y Example 3: 40 Minimize x + 1/3 y 30 x + y ³ 20 Subject to: Feasible Region -2 x + 5 y £ 150 20 x ³ 5 10 x ³ 0y ³ 0 x 0 Optimal Solution 40 0 10 20 30

6. Do We Notice Anything From These 3 Examples? Extreme point y y y 4 40 4 3 3 30 2 2 20 1 1 10 0 0 0 x x x 0 1 0 1 2 0 10 20 2 3 3 30 40

7. A Fundamental Point y y y 4 40 4 3 3 30 2 2 20 1 1 10 0 0 0 x x x 0 1 0 1 2 0 10 20 2 3 3 30 40 If an optimal solution exists, there is always a corner point optimal solution!

8. Graphing 2-Dimensional LPs Optimal Solution Second Corner pt. y Example 1: 4 Maximize x + y 3 x + 2 y ³ 2 Subject to: Feasible Region x £ 3 2 y £ 4 1 x ³ 0y ³ 0 Initial Corner pt. 0 x 0 1 2 3

9. And We Can Extend this to Higher Dimensions

10. Then How Might We Solve an LP? • The constraints of an LP give rise to a geometrical shape - we call it a polyhedron. • If we can determine all the corner points of the polyhedron, then we can calculate the objective value at these points and take the best one as our optimal solution. • The Simplex Method intelligently moves from corner to corner until it can prove that it has found the optimal solution.

11. But an Integer Program is Different y • Feasible region is a set of discrete points. • Can’t be assured a corner point solution. • There are no “efficient” ways to solve an IP. • Solving it as an LP provides a relaxation and a bound on the solution. 4 3 2 1 0 x 0 1 2 3

12. Linear Programs in higher dimensions maximize z = -4x1 + x2 - x3 subject to -7x1 + 5x2 + x3 <= 8 -2x1 + 4x2 + 2x3 <= 10 x1, x2, x3 0

13. In Matrix terms

14. LP Geometry • Forms a d dimensional polyhedron • Is convex : If z1 and z2 are two feasible solutions then λz1+ (1- λ)z2 is also feasible. • Extreme points can not be written as a convex combination of two feasible points.

15. LP Geometry • Extreme point theorem: If there exists an optimal solution to an LP Problem, then there exists one extreme point where the optimum is achieved. • Local optimum = Global Optimum

16. LP: Algorithms • Simplex. (Dantzig 1947) • Developed shortly after WWII in response to logistical problems:used for 1948 Berlin airlift. • Practical solution method that moves from one extreme point to a neighboring extreme point. • Finite (exponential) complexity, but no polynomial implementation known. Courtesy Kevin Wayne

17. LP: Polynomial Algorithms • Ellipsoid. (Khachian 1979, 1980) • Solvable in polynomial time: O(n4 L) bit operations. • n = # variables • L = # bits in input • Theoretical tour de force. • Not remotely practical. • Karmarkar's algorithm. (Karmarkar 1984) • O(n3.5 L). • Polynomial and reasonably efficientimplementations possible. • Interior point algorithms. • O(n3 L). • Competitive with simplex! • Dominates on simplex for large problems. • Extends to even more general problems.

18. LP: The 2D case Wlog, we can assume that c=(0,-1). So now we want to find the Extreme point with the smallest y coordinate. Lets also assume, no degeneracies, the solution is given by two Halfplanes intersecting at a point.

19. Incremental Algorithm? • How would it work to solve a 2D LP Problem? • How much time would it take in the worst case? • Can we do better?