Explorations in Artificial Intelligence

1. Explorations in Artificial Intelligence Prof. Carla P. Gomes gomes@cs.cornell.edu Module 7 Part 3 Integer Programming

2. Divisibility Decision variables in an LP model are allowed to have any values, including noninteger values, that satisfy the functional and nonnegativity constraints. i.e., activities can be run at fractional levels. What to do when divisibility assumption violated: realm of integer programming!!!

3. Revisiting the TBA Airlines Problem An Example where Integrality Matters

4. The TBA Airlines Problem • TBA Airlines is a small regional company that specializes in short flights in small airplanes. • The company has been doing well and has decided to expand its operations. • The basic issue facing management is whether to purchase more small airplanes to add some new short flights, or start moving into the national market by purchasing some large airplanes, or both. Question: How many airplanes of each type should be purchased to maximize their total net annual profit?

5. Data for the TBA Airlines Problem

6. Linear Programming Formulation Let S = Number of small airplanes to purchaseL = Number of large airplanes to purchaseMaximize Profit = S + 5L ($millions)subject to Capital Available: 5S + 50L ≤ 100 ($millions) Max Small Planes: S ≤ 2and S ≥ 0, L ≥ 0.

7. Graphical Method for Linear Programming

8. Violates Divisibility Assumption of LP • Divisibility Assumption of Linear Programming: Decision variables in a linear programming model are allowed to have any values, including fractional values, that satisfy the functional and nonnegativity constraints. Thus, these variables are not restricted to just integer values. • Since the number of airplanes purchased by TBA must have an integer value, the divisibility assumption is violated.

9. Integer Programming Formulation Let S = Number of small airplanes to purchaseL = Number of large airplanes to purchaseMaximize Profit = S + 5L ($millions)subject to Capital Available: 5S + 50L ≤ 100 ($millions) Max Small Planes: S ≤ 2and S ≥ 0, L ≥ 0S, L are integers.

10. Graphical Method for Integer Programming

11. Graphical Method for Integer Programming • When an integer programming problem has just two decision variables, its optimal solution can be found by applying the graphical method for linear programming with just one change at the end. • We begin as usual by graphing the feasible region for the LP relaxation, determining the slope of the objective function lines, and moving a straight edge with this slope through this feasible region in the direction of improving values of the objective function. • However, rather than stopping at the last instant the straight edge passes through this feasible region, we now stop at the last instant the straight edge passes through an integer point that lies within this feasible region. • This integer point is the optimal solution.

12. Why integer programs? • Advantages of restricting variables to take on integer values • More realistic • More flexibility • Disadvantages • More difficult to model • Can be much more difficult to solve

13. Integer Programming • When are “non-integer” solutions okay? • Solution is naturally divisible • e.g., $, pounds, hours • Solution represents a rate • e.g., units per week • Solution only for planning purposes • When is rounding okay? • When numbers are large • e.g., rounding 114.286 to 114 is probably okay. • When is rounding not okay? • When numbers are small • e.g., rounding 2.6 to 2 or 3 may be a problem. • Binary variables • yes-or-no decisions

14. Types of Integer Programming Problems • Pure integer programming problems are those where all the decision variables must be integers. • Mixed integer programming problems only require some of the variables (the “integer variables”) to have integer values so the divisibility assumption holds for the rest (the “continuous variables”). • Binary variables are variables whose only possible values are 0 and 1. • Binary integer programming(BIP) problems are those where all the decision variables restricted to integer values are further restricted to be binary variables. • Such problems can be further characterized as either pure BIP problems or mixed BIP problems, depending on whether all the decision variables or only some of them are binary variables.

15. Examples of Applications of Binary Variables • Making “yes-or-no” type decisions • Build a factory? • Manufacture a product? • Do a project? • Assign a person to a task? • Logical constraints • Alternative constraints • Conditional constraints • Representing non-linear functions • Fixed Charge Problem • If a product is produced, must incur a fixed setup cost. • If a warehouse is operated, must incur a fixed cost. • Piecewise linear representation • Diseconomies of scale • Approximation of nonlinear functions • Set-covering, and set partitioning • Make a set of assignments that “cover” a set of requirements. • Partition a set into subsets meeting given requirements

16. StockCompany ExampleCapital Budgeting Allocation Problem StockCompany is considering 6 investments. The cash required from each investment as well as the NPV of the investment is given next. The cash available for the investments is $14,000. Stockco wants to maximize its NPV. What is the optimal strategy? An investment can be selected or not. One cannot select a fraction of an investment.

17. Data for the StockCompany Problem Investment budget = $14,000

18. Integer Programming Formulation What are the decision variables? Objective and Constraints? Max 16x1+ 22x2+ 12x3+ 8x4+ 11x5+ 19x6 5x1+ 7x2+ 4x3+ 3x4+ 4x5+ 6x6  14 xje {0,1} for each j = 1 to 6

19. Capital Budgeting Allocation Problem (one resource) Knapsack Problem • Why is a problem with the characteristics of the previous problem called the Knapsack Problem? • It is an abstraction, considering the simple problem: A hiker trying to fill her knapsack to maximum total value. Each item she considers taking with her has a certain value and a certain weight. An overall weight limitation gives the single constraint. Practical applications: Project selection and capital budgeting allocation problems Storing a warehouse to maximum value given the indivisibility of goods and space limitations Sub-problem of other problems e.g., generation of columns for a given model in the course of optimization – cutting stock problem (beyond the scope of this course)

20. The previous constraints represent “economic indivisibilities”, either a project is selected, or it is not. There is no selecting of a fraction of a project. • Similarly, integer variables can model logical requirements (e.g., if stock 2 is selected, then so is stock 1.)

21. How to model “logical” constraints • Exactly 3 stocks are selected. • If stock 2 is selected, then so is stock 1. • If stock 1 is selected, then stock 3 is not selected. • Either stock 4 is selected or stock 5 is selected, but not both.

22. Formulating Constraints • Exactly 3 stocks are selected x1+ x2+ x3+ x4+ x5+ x6=3

23. A 2-dimensional representation Stock 2 Stock 1 The integer programming constraint: If stock 2 is selected then so is stock 1 x1  x2

24. A 2-dimensional representation Stock 3 Stock 1 The integer programming constraint: If stock 1 is selected then stock 3 is not selected x1 + x3  1

25. A 2-dimensional representation stock 5 stock 4 The integer programming constraint: Either stock 4 is selected or stock 5 is selected, but not both. x4 + x5 = 1

26. California Manufacturing Company • The California Manufacturing Company is a diversified company with several factories and warehouses throughout California, but none yet in Los Angeles or San Francisco. • A basic issue is whether to build a new factory in Los Angeles or San Francisco, or perhaps even both. • Management is also considering building at most one new warehouse, but will restrict the choice to a city where a new factory is being built. Question: Should the California Manufacturing Company expand with factories and/or warehouses in Los Angeles and/or San Francisco?

27. Data for California Manufacturing

28. Binary Decision Variables

29. Algebraic Formulation Let x1 = 1 if build a factory in L.A.; 0 otherwisex2 = 1 if build a factory in S.F.; 0 otherwisex3 = 1 if build a warehouse in Los Angeles; 0 otherwisex4 = 1 if build a warehouse in San Francisco; 0 otherwiseMaximize NPV = 8x1 + 5x2 + 6x3 + 4x4 ($millions)subject to Capital Spent: 6x1 + 3x2 + 5x3 + 2x4 ≤ 10 ($millions) Max 1 Warehouse: x3 + x4 ≤ 1 Warehouse only if Factory: x3 ≤ x1x4 ≤ x2andx1, x2, x3, x4 are binary variables. Resource Availability Mutually exclusive decisions Contingent decisions

30. Mpl Model Max 8x1 + 5x2 + 6x3 + 4x4; subject to 6x1 + 3x2 + 5x3 + 2x4 <= 10; x3 + x4 <= 1; x3 <= x1; x4 <= x2; BINARY x1; x2; x3; x4;

31. Modeling Fixed Charge Problems If a product is produced, must incur a fixed setup cost. If a warehouse is operated, must incur a fixed cost. • The problem is non-linear. x – quantity of product to be manufactured x = 0  cost =0; x > 0  cost = C1x + C2 • How to model it? Using an indicator variable y y = 1  x is produced; y = 0  x is not produced Objective function becomes  C1x + C2y Additional Constraint  x ≤ My

32. Wyndor with Setup Costs (Variation 1) Suppose that two changes are made to the original Wyndor problem: • For each product, producing any units requires a substantial one-time setup cost for setting up the production facilities. • The production runs for these products will be ended after one week, so D and W in the original model now represent the total number of doors and windows produced, respectively, rather than production rates. Therefore, these two variables need to be restricted to integer values.

33. Graphical Solution to Original Wyndor Problem

34. Net Profit for Wyndor Problem with Setup Costs

35. Feasible Solutions for Wyndor with Setup Costs Optimal solution

36. Algebraic Formulation Let D = Number of doors to produce,W = Number of windows to produce,y1 = 1 if perform setup to produce doors; 0 otherwise,y2 = 1 if perform setup to produce windows; 0 otherwise .Maximize P = 300D + 500W – 700y1 – 1,300y2subject to Original Constraints: Plant 1: D ≤ 4 Plant 2: 2W ≤ 12 Plant 3: 3D + 2W ≤ 18 Produce only if Setup: Doors: D ≤ My1 Windows: W ≤ My2andD ≥ 0, W ≥ 0, y1 and y2 are binary.

37. Wyndor with Mutually Exclusive Products(Variation 2) Suppose that now the only change from the original Wyndor problem is: • The two potential new products (doors and windows) would compete for the same customers. Therefore, management has decided not to produce both of them together. • At most one can be chosen for production, so either D = 0 or W = 0, or both.

38. Feasible Solution forWyndor with Mutually Exclusive Products(for non-binary variables)

39. Algebraic Formulation Let D = Number of doors to produce,W = Number of windows to produce,y1 = 1 if produce doors; 0 otherwise,y2 = 1 if produce windows; 0 otherwise.Maximize P = 300D + 500Wsubject to Original Constraints: Plant 1: D ≤ 4 Plant 2: 2W ≤ 12 Plant 3: 3D + 2W ≤ 18 Auxiliary variables must =1 if produce any: Doors: D ≤ My1 Windows: W ≤ My2 Mutually Exclusive: y1 + y2 ≤ 1 andD ≥ 0, W ≥ 0, y1 and y2 are binary.

40. Wyndor with Either-Or Constraints(Variation 3) Suppose that now the only change from the original Wyndor problem is: • The company has just opened a new plant (plant 4) that is similar to plant 3, so the new plant can perform the same operations as plant 3 to help produce the two new products (doors and windows). • However, management wants just one of the plants to be chosen to work on these new products. The plant chosen should be the one that provides the most profitable product mix.

41. Data for Wyndor with Either-Or Constraints(Variation 3)

42. Graphical Solution with Plant 3 or Plant 4

43. Algebraic Formulation Let D = Number of doors to produce,W = Number of windows to produce,y = 1 if plant 4 is used; 0 if plant 3 is usedMaximize P = 300D + 500Wsubject to Plant 1: D ≤ 4 Plant 2: 2W ≤ 12 Plant 3: 3D + 2W ≤ 18 + My Plant 4: 2D + 4W ≤ 28 + M(1 – y) andD ≥ 0, W ≥ 0, y is binary.

44. Applications of Binary Variables • Making “yes-or-no” type decisions • Build a factory? • Manufacture a product? • Do a project? • Assign a person to a task? • Fixed costs • If a product is produced, must incur a fixed setup cost. • If a warehouse is operated, must incur a fixed cost. • Either-or constraints • Production must either be 0 or ≥ 100. • Subset of constraints • meet 3 out of 4 constraints.

45. Special Kinds of Integer Programming Models • Knapsack Problem • Set Covering Problem • Set Partitioning Problem • Set Packing Problem • The Traveling Salesman Problem • The Quadratic Assignment Problem

46. Set Covering Problem • We are given a set of objects S = {1, 2, 3, …, n}. • We are also given a set of subsets of S, S. Each subset has a cost associated with it. • Problem: • to “cover” all the members of S at the minimum cost using members of S. • Properties: • The problem is a minimization and all constraints are >=; • All RHS coefficients are 1; • All other matrix coefficients are 0 or 1.

47. Fire Station ProblemSet Covering Problem Locate fire stations so that each district has a fire station in it, or next to it. 1 2 3 5 6 7 Minimize the number of fire stations needed. 4 8 9 11 12 13 10 14 15 16

48. Representation as Set Covering Problem Set Covers 1 2 3 1 1, 2, 4, 5 2 1, 2, 3, 5, 6 5 6 7 3 2, 3, 6, 7 4 8 9 11 12 13 10 16 13, 15, 16 14 15 16

49. 1 1 2 3 7 5 6 7 4 8 9 11 11 12 13 15 10 14 15 16 Representation as Integer program xj = 1 if node j is selectedxj = 0 otherwise Minimize x1 + x2 + … + x16 s.t. x1 + x2 + x4 + x5  1 x1 + x2 + x3 + x5 + x6  1 x13 + x15 + x16  1 xj {0, 1} for each j.

50. Southwestern Airways Crew Scheduling • Southwestern Airways needs to assign crews to cover all its upcoming flights. • We will focus on assigning 3 crews based in San Francisco (SFO) to 11 flights. Question: How should the 3 crews be assigned 3 sequences of flights so that every one of the 11 flights is covered?