Linear Programming & Flows

1. Linear Programming & Flows Instructor: YE, Deshi yedeshi@zju.edu.cn

2. Linear Programming • What is it? • Quintessential tool for optimal allocation of scarce resources, among a number of competing activities. (e.g., see ORF 307) • Powerful and general problem-solving method. • shortest path, max flow, min cost flow, generalized flow, multicommodity flow, MST, matching, 2-person zero sum games • Why significant? • Fast commercial solvers: CPLEX, OSL, Lindo. • Powerful modeling languages: AMPL, GAMS. • Ranked among most important scientific advances of 20th century. • Also a general tool for attacking NP-hard optimization problems. • Dominates world of industry. • ex: Delta claims saving $100 million per year using LP

3. Example

4. Geometry • Observation. Regardless of objective function (convex) coefficients, an optimal solution occurs at an extreme point.

5. Standard and slack forms • Standard form • In standard form, we are given n real numbers c1, c2, ..., cn; m real numbers b1, b2, ..., bm; and mn real numbers aij for i = 1, 2, ..., m and j = 1, 2, ..., n. We wish to find n real numbers x1, x2, ..., xn that

6. Standard and slack forms • Standard form • In standard form, we are given n real numbers c1, c2, ..., cn; m real numbers b1, b2, ..., bm; and mn real numbers aij for i = 1, 2, ..., m and j = 1, 2, ..., n. We wish to find n real numbers x1, x2, ..., xn that

7. Slack form • In standard form, Slack variable s, such that

8. The simplex algorithm • Simplex algorithm. (George Dantzig, 1947) • Developed shortly after World War II in response to logistical problems. • Used for 1948 Berlin airlift. • Generic algorithm. • Start at some extreme point. • Pivot from one extreme point to a neighboring one. • Repeat until optimal. • How to implement? Linear algebra. never decrease objective func

9. Illustration of Simplex • Step1: Convert the LP problem to a system of linear equations (slack form). • Step 2: Set up the initial tableau. • Step 3: Find the PIVOT • Step 4: Form RATIOS (quotients) for each row: divide the right-most number by the number in the pivot column of that row. • Step 5: The PIVOT ROW is the row with the smallest NON-NEGATIVE ratio (quotient). • Step 6: If all indicators (in the bottom row) are non-negative, STOP. • Step 7: Otherwise, goes to Step 3.

10. Step 1 Original problem: Slack form:

11. Step 2 SIMPLEX TABLEAU.Compare RED symbolswith Z = x1 + 2x2 - x3.

12. Step 3: Pivot Indicator row Pivot column

13. Step 3: Pivot ratios Ratios: 14÷1 =14 28÷2 =14 30÷5 =6 Indicator row Pivot column Pivot

14. Step 4: entering variable r1: R3=(r3 /5) r2: r3: r4:

15. Step 4: leaving variable R1=r1 - R3 R2=r2-2R3 R4=r4+2R3 r1: r2: R3: r4: R1: R2: R3: R4:

16. Step 3 again R1: R2: R3: R4: Indicator row Pivot column

17. Step 3 again Tie: choose arbitrary Ratios: 8÷(8/5) =5 R1: R2: 16÷(16/5) =5 R3: 6÷ (2/5) =15 R4: Indicator row Pivot column

18. Step 3 again R1: R2=(R2 /(16/5)) R2: R3: R4: R1: R2: R3: R4:

19. Step 3 again R1=R1 – (8/5)R2 R3=R3-(2/5)R2 R4=R4+(1/2)R2 R1: R2: R3: R4: R1: R2: R3: R4:

20. STOP x1 x2 x3 s1 s2 s3 R1: R2: R3: R4: Finally, the optimal solution of LP is x3=s2=s3=0 x1=5, x2=4, s1=0 z =13

21. History of LP • 1939. Production, planning. (Kantorovich, USSR) • Propaganda to make paper more palatable to communist censors. • Kantorovich awarded 1975 Nobel prize in Economics for • contributions to the theory of optimum allocation of resources. • Staple in MBA curriculum. • Used by most large companies and other profit maximizers.

22. History • 1939. Production, planning. (Kantorovich) • 1947. Simplex algorithm. (Dantzig) • 1950. Applications in many fields. • Military logistics. • Operations research. • Control theory. • Filter design. • Structural optimization.

23. History • 1939. Production, planning. (Kantorovich) • 1947. Simplex algorithm. (Dantzig) • 1950. Applications in many fields. • 1979. Ellipsoid algorithm. (Khachian) • Geometric divide-and-conquer. • Solvable in polynomial time: O(n4 L) bit operations. • – n = # variables • – L = # bits in input • Theoretical tour de force, not remotely practical.

24. History • 1939. Production, planning. (Kantorovich) • 1947. Simplex algorithm. (Dantzig) • 1950. Applications in many fields. • 1979. Ellipsoid algorithm. (Khachian) • 1984. Projective scaling algorithm. (Karmarkar) • O(n3.5 L). • Efficient implementations possible.

25. History • 1939. Production, planning. (Kantorovich) • 1947. Simplex algorithm. (Dantzig) • 1950. Applications in many fields. • 1979. Ellipsoid algorithm. (Khachian) • 1984. Projective scaling algorithm. (Karmarkar) • 1990. Interior point methods. • O(n3 L) and practical. • Extends to even more general problems.

26. Perspective • LP is near the deep waters of NP-completeness. • Solvable in polynomial time. • Known for less than 28 years. • Integer linear programming. • LP with integrality requirement. • NP-hard.

27. Formulation: The diet problem • n different foods and that the jth food sells at a price cjper unit. • m basic nutritional ingredients and, to achieve a balanced diet, each individual must receive at least biunits of the ith nutrient per day. • Finally, we assume that each unit of food j contains aij units of the ith nutrient. • Denote by xjthe number of units of food j in the diet, the problem then is to select the xj’s to minimize the total cost

28. Formulation: The diet problem • Min • Subject to the nutritional constraints • and the nonnegativity constraints

29. Formulation: Four-Step Rule • Sort out data and parameters from the verbal description • Define the set of decision variables • Formulate the objective function of data and decision variables • Set up equality and/or inequality constraints

30. Formulation : Air Traffic Control • Air plane j; j = 1,..., n arrives at the airport within the time interval [aj; bj ] in the order of 1; 2,..., n. The airport wants to find the arrival time for each air plane such that the minimal metering time (inter-arrival time between two consecutive airplanes) is the greatest. • Let tjbe the arrival time of the jth plane. Then, the problem is

31. Air Traffic Control continued • Rewrite the problem as an LP:

32. Formulation: Data Fitting I • Given data points aj, j = 1,..., n, and the observation value cjat data point aj, the least squares problem is to find y such that • is minimized. • Sometime, it is desired to minimize the p norm, where p = 1 or p = ∞

33. Data Fitting II • Suppose we want to minimize • This is equivalent to • It is a conic linear program.

34. Sensor Network Localization • There are ndistinct sensor points in Rdwhose locations are to be determined, • Given m fixed points (called the anchor points) whose locations are known as a1, a2,...,am. • dijthe Euclidean distance between the ith and jth sensor points, and the distance between the ith sensor and kth anchor point

35. Formulation: Transportation/Supply Chain Problem • Quantities siare to be shipped from m supply locations and received in amounts djin n demand locations, respectively. Associated with the shipping of a unit of product from origin i to destination j is a unit shipping cost cij.

36. Ex. • You have $12,000 to invest, and three different funds from which to choose. The municipal bond fund has a 7% return, the local bank's CDs have an 8% return, and the high-risk account has an expected (hoped-for) 12% return. To minimize risk, you decide not to invest any more than $2,000 in the high-risk account. For tax reasons, you need to invest at least three times as much in the municipal bonds as in the bank CDs. Assuming the year-end yields are as expected, what are the optimal investment amounts? Maximize Y = 0.07x + 0.08y + 0.12(12 – x – y) =1.44 – 0.05x – 0.04y, subject to:x> 0 y> 0 y> –x + 10 y< –x + 12 y< ( 1/3 )x

37. Maximizing Ad-auctions revenue • There is a set I of n buyers, each buyer i has a known daily budget of B(i). • Upon arrival of a product j, each buyer provides a bid b(i, j) for buying item j • The revenue received from each buyer is defined to be the minimum between the sum of the costs of the products allocated to a buyer (times the fraction allocated) and the total budget of the buyer. • The objective is to maximize the total revenue of the seller.

38. Maximizing Ad-auctions revenue • Let y(i, j) denote the fraction of product j allocated to buyer i

39. Where xj indicate jth item is include in the knapsack 0-1 Knapsack problem • Given a set N = {1,2,...,n} of n items, each item i having positive integer weight wiand profit pi and a knapsack of equal capacity c.

40. Multiple Knapsack Problem • Given a set N = {1,2,...,n} of n items, each item i having positive integer weight wiand profit pi and m knapsacks of equal capacity c

41. Formulating problems as linear programs • Maximum flow: • we are given a directed graph G = (V, E) in which each edge (u, v) ∈ E has a nonnegative capacity c(u, v) ≥ 0, and two distinguished vertices, a sink s and a source t. The goal is to find s-t flow of maximum value.

42. Dynamic TCP-Acknowledgement • A stream of packets arrives at a destination from a source. • The source needs to get an acknowledgement for each of the packets, however, it is possible to acknowledge several packets by a single acknowledgement message. • The objective function is to minimize the number of acknowledgement messages sent along with the sum of latencies of the packets

43. Dynamic TCP-Acknowledgement • Let M be the set of packets. For each packet j in M, let t(j) be the time of arrival at the destination. • Assume now that packets can only arrive in discrete times of 1/d. • xt=1 sends an acknowledgement message at t • Variable z(j, t) which is set to 1 if packet j is delayed between time t and time t + 1/d

44. China Rail Network

45. Maximum Flow and Minimum Cut • Max flow and min cut. • Two very rich algorithmic problems. • Cornerstone problems in combinatorial optimization. • Beautiful mathematical duality.

46. Minimum Cut Problem • Flow network. • Abstraction for material flowing through the edges. • G = (V, E) = directed graph, no parallel edges. • Two distinguished nodes: s = source, t = sink. • c(e) = capacity of edge e. 2 5 9 10 15 15 10 4 source sink 5 s 3 6 t 8 10 15 4 6 10 15 capacity 4 7 30

47. Cuts • Def. An s-t cut is a partition (A, B) of V with s  A and t B. • Def. The capacity of a cut (A, B) is: 2 5 9 10 15 15 10 4 5 s 3 6 t 8 10 A 15 4 6 10 15 Capacity = 10 + 5 + 15 = 30 4 7 30

48. Cuts • Def. An s-t cut is a partition (A, B) of V with s  A and t B. • Def. The capacity of a cut (A, B) is: 2 5 9 10 15 15 10 4 5 s 3 6 t 8 10 A 15 4 6 10 15 Capacity = 9 + 15 + 8 + 30 = 62 4 7 30

49. Minimum Cut Problem • Min s-t cut problem. Find an s-t cut of minimum capacity. 2 5 9 10 15 15 10 4 5 s 3 6 t 8 10 15 4 6 10 A 15 Capacity = 10 + 8 + 10 = 28 4 7 30

50. 2 5 9 10 15 15 10 4 5 s 3 6 t 8 10 15 4 6 10 15 4 7 30 Flows • Def. An s-t flow is a function that satisfies: • For each e  E: (capacity) • For each v  V – {s, t}: (conservation) • Def. The value of a flow f is: 0 4 0 0 0 4 0 4 4 0 0 0 0 capacity flow 0 0 Value = 4