Explorations in Artificial Intelligence

1. Explorations in Artificial Intelligence Prof. Carla P. Gomes gomes@cs.cornell.edu Module 5 Intro to Linear Programming

2. Mathematical Programming Constraint Programming • 1- Problem Formulation: • A problem is a finite set of constraints involving a finite set of variables. • Constraint Satisfaction Problems (CSP)  feasibility problem only and --- SAT is a particular case of CSP; • Constraint Optimization Problems (COP) if in addition the solution is required to maximize an objective function • 2- Problem Solution: • Domain specific methods • General Solution Methods

3. Mathematical Program • Optimization problem in which the objective and constraints are given as mathematical functions and functional relationships. • Optimize: Z = f(x1, x2, …, xn) • Subject to: g1(x1, x2, …, xn) = , , b1 g2(x1, x2, …, xn) = , , b2 … gm(x1, x2, …, xn) = , , bm Linear Programming problem  special type of a mathematical programming problem (all functions are linear)

4. Linear Programming (LP) • One of the most important scientific advances of the 20th century • A variety of applications: • Financial planning, Marketing, E-business, Telecommunications, Manufacturing, Transportation Planning, System Design, Health Care • Remarkably efficient solution procedures to solve LP models – simplex method and interior point methods --- • Very fast LP solvers (CPLEX – from 1981-2001 2,000,000X faster!)

5. Linear Programming • Significance. • Powerful tool for optimal allocation of scarce resources, among a number of competing activities. • Powerful model generalizes many classic problems: • shortest path, max flow, multicommodity flow, MST, matching, 2-person zero sum games • Ranked among most important scientific advances of 20th century. • accounts for a major proportion of all scientific computation • Helps find "good" solutions to NP-hard optimization problems. • optimal solutions (branch-and-cut) • provably good solutions (randomized rounding)

6. Linear Programming (LP) • Linear – all the functions are linear (f and g functions are linear. Ex:f (x1, x2, …, xn)= c1x1 + c2 x2 + … cn xn • Programming – does not refer to computer programming but rather “planning” - planning of activities to obtain an optimal result i.e., it reaches the specified goal best (according to the mathematical model) among all feasible alternatives.

7. Prototype Example – Wyndor Glass (Hillier and Liebermnan)

8. Wyndor Glass Co. Product Mix Problem • Wyndor Co. has developed the following new products: • An 8-foot glass door with aluminum framing. • A 4-foot by 6-foot double-hung, wood-framed window. • The company has three plants • Plant 1 produces aluminum frames and hardware. • Plant 2 produces wood frames. • Plant 3 produces glass and assembles the windows and doors.

10. Wyndor Glass Co. Product Mix Problem • Questions: • Should they go ahead with launching these two new products? • If so, what should be the product mix? How to formulate this problem as an Linear Programming problem?

11. Steps in setting up a LP • Determine and label the decision variables. • Determine the objective and use the decision variables to write an expression for the objective function. • Determine the constraints - feasible region. • Determine the explicit constraints and write a functional expression for each of them. • Determine the implicit constraints (e.g., nonnegativity constraints).

12. Algebraic Model for Wyndor Glass Co. Let D = the number of doors to produce W = the number of windows to produce Maximize P = 3 D + 5 W subject to D ≤ 4 2W ≤ 12 3D + 2W ≤ 18 and D ≥ 0, W ≥ 0.

13. Finding the Optimal Solution Our objective function is: maximize 3D+5W The vector representing the gradient of the objective function is: The line through the origin that contains this vector is: Maximize P = 3 D + 5 W subject to D ≤ 4 2W ≤ 12 3D + 2W ≤ 18 and D ≥ 0, W ≥ 0. isoprofit line

14. y Extreme points y z z Convex Not convex LP: Geometry • Geometry. • Forms an n-dimensionalpolyhedron. • Convex: if y and z are feasible solutions, then so is ½y + ½z. • Extreme point: feasible solution x that can't be written as ½y + ½z for any two distinct feasible solutions y and z.

15. LP: Geometry • Extreme Point Theorem. If there exists an optimal solution to standard form LP (P), then there exists one that is an extreme point. • Only need to consider finitely many possible solutions. • Greed.Local optima areglobal optima.

16. Graphical Method • Draw the constraint boundary line for each constraint. Use the origin (or any point not on the line) to determine which side of the line is permitted by the constraint. • Find the feasible region by determining where all constraints are satisfied simultaneously. • Determine the slope of one objective function line (perpendicular to its gradient vector). All other objective function lines will have the same slope. • Move a straight edge with this slope through the feasible region in the direction of improving values of the objective function (direction of the gradient). Stop at the last instant that the straight edge still passes through a point in the feasible region. This line given by the straight edge is the optimal objective function line. • A feasible point on the optimal objective function line is an optimal solution.

17. Terminology and Notation • Resources – m (plants) • Activities – n (2 products) • Wyndor Glass problemoptimal product mix --- allocation of resourcesto activities i.e., choose the levels of the activities that achieve best overall measure of performance

18. Terminology and notation (cont.) • Z – value of the overall measure of performance; value of the objective function, • xj – level of activity j (for j = 1, 2, …, n) decision variables • cj – increase in Z for each unit increase in the level of activity j; coefficient of objective function associated with activity j • bi – amount of resource i that is available (for i=1,2,…, m). Right-hand-side of constraint associated with resource i. • aij – amount of resource i consumed by each unit of activity j.  Technological coefficient. • (The values of cj, bi, and aij are the input constants for the model  the parameters of the model. )

19. Standard form of the LP model • xi >= 0 , (i =1,2,…,n) • Other forms: • Minimize Z (instead of maximizing Z) •    Some functional constraints have signs >= (rather than <=) •    Some functional constraints are equalities •    Some variables have unrestricted sign, i.e., they are not subject to the non-negativity constraints

20. Terminology of solutions in LP model • Solution – not necessarily the final answer to the problem!!! • Feasible solution – solution that satisfies all the constraints • Infeasible solution – solution for which at least one of the constraints is violated • Feasible region – set of all points that satisfies all the constraints (possible to have a problem without any feasible solutions) • Binding constraint – the left-hand side and the right-hand side of the constraint are equal, I.e., constraint is satisfied in equality. Otherwise the constraint is nonbinding. • Optimal solution – feasible solution that has the best value of the objective function. • Largest value  maximization problems • Smallest value  minimization problems • Multiple optimal solutions • No optimal solutions • Unbounded Z

21. Corner Point Solutions • Corner-point feasible solution – special solution that plays a key role when the simplex method searches for an optimal solution. Relationship between optimal solutions and CPF solutions: • Any LP with feasible solutions and bounded feasible region • (1)  the problem must possess CPF solutions and at least one optimal solution • (2)  the best CPF solution must be an optimal solution • If the problem has exactly one optimal solution it must be a CFP solution • If the problem has multiple optimal solutions, at least two must be CPF solutions

22. Wyndor Glass Let D = the number of doors to produce W = the number of windows to produce Maximize P = 3 D + 5 W Z=30 1 2 Z=36 Edge of Feasible region 3 Z=27 Z=0 0 CPF

23. No Feasible Solutions – Why? Maximize P = 3 D + 5 W subject to D ≤ 4 2W ≤ 12 3D + 2W ≤ 18 3 D + 5 W 50 and D ≥ 0, W ≥ 0. Previous Feasible Region

24. Multiple Optimal Solutions. Why? Maximize P = 3 D + 2 W subject to D ≤ 4 2W ≤ 12 3D + 2W ≤ 18 and D ≥ 0, W ≥ 0. Every point on this line is An optimal solution with P=18

25. Unbounded Objective Function. Why? (4, ) P= Maximize P = 3 D + 2 W subject to D ≤ 4 and D ≥ 0, W ≥ 0. (4,8) P=28 (4,4) P=20 (4,2) P=16

26. Sensitivity Analysis Our objective function is: maximize 3x1+c2x2 Production rate W for windows 8 P = 3600 = 300D + 500W Optimal solution c2>2 How does the optimal solution change as c2 changes? x*=(2, 6) P = 3000 = 300D + 500W 6 Feasible Multiple Optimal solution c2=2 P=18 x* = (2,6) ; x*=(4,3) And any convex combination 4 region P = 1500 = 300D + 500W Optimal solution 0 <c2<2 x*=(4,3) 2 Multiple Optimal solution c2=0 P=12 x*=(4,3) and (4,0) and any Convex combination 2 0 4 6 8 D 10 Production rate for doors Optimal solution c2<0 x*=(4,0)

27. LP Assumptions PProportionality – The contribution of each activity to the value of the objective function Z is proportional to the level of the activity xj as represented by the cjxj term; The contribution of each activity to the left-hand side of each functional constraint is proportional to the level of the activity xj as represented by the term aij. This assumption implies that all the x terms of the linear equations cannot have exponents greater than 1. Note: if there is a term that is a product of different variables, even though the proportionality assumption is satisfied, the additivity assumption is violated.

28. LP Assumptions • Additivity • The contribution of all variables to the objective function and to the left-hand side of the functional constraints has to be additive, i.e., it has to be the sum of the individual contributions of the respective activities – therefore cross-products of variables are ruled out.

29. LP Assumptions • Certainty • The parameters of the model, (coefficients of the objective function and of the functional constraints, and the right-hand sides of the functional constraints) are assumed to be known constants. • Rarely the case – sometimes we use approximations  important to perform sensitivity analysis to identify sensitive parameters (the parameters that cannot be changed without changing the value of the objective function). • What to do when certainty assumption violated: • treat parameters as random variables

30. LP Assumptions • Divisibility • Decision variables in an LP model are allowed to have any values, including noninteger values, that satisfy the functional and nonnegativity constraints. i.e., activities can be run at fractional levels. • What to do when divisibility assumption violated: • realm of integer programming!!!

31. Examples of Different Categories of LP problems: Resource-Allocation Problems Cost-benefit-trade-off problems Distribution-Network Problems

32. Algebraic Model for Wyndor Glass Co. Let D = the number of doors to produce W = the number of windows to produce Maximize P = 3 D + 5 W subject to D ≤ 4 2W ≤ 12 3D + 2W ≤ 18 and D ≥ 0, W ≥ 0.

33. Examples of Different Categories of LP problems: Resource-Allocation Problems

34. Resource Allocation Problem Wyndor Glass • Resources – m (plants) • Activities – n (2 products) • Wyndor Glass problemoptimal product mix --- allocation of resourcesto activities i.e., choose the levels of the activities that achieve best overall measure of performance with available resources

35. Finding the Optimal Solution Our objective function is: maximize 3D+5W The vector representing the gradient of the objective function is: The line through the origin that contains this vector is: Maximize P = 3 D + 5 W subject to D ≤ 4 2W ≤ 12 3D + 2W ≤ 18 and D ≥ 0, W ≥ 0. isoprofit line

36. Cost-benefit-trade-off problems

37. Cost-benefit-trade-off problems • The mix of levels of various activities is • chosen to achieve minimum acceptable levels • for various benefits at a minimum cost.

38. The Profit & Gambit Co.(Hillier & Hillier): • Management has decided to undertake a major advertising campaign that will focus on the following three key products: • A spray prewash stain remover. • A liquid laundry detergent. • A powder laundry detergent. • The campaign will use both television and print media • The general goal is to increase sales of these products. • Management has set the following goals for the campaign: • Sales of the stain remover should increase by at least 3%. • Sales of the liquid detergent should increase by at least 18%. • Sales of the powder detergent should increase by at least 4%. • Question: how much should they advertise in each medium to meet the sales goals at a minimum total cost?

39. Activity 1 – advertise on television • Activity 2 – advertise in the print media • Benefit 1 – increases sales of stain remover • Benefit 2 – increases sales of liquid detergent • Benefit 3 – increases sales of powder detergent

40. Profit & Gambit Co. Data

41. Algebraic Model for Profit & Gambit Let TV = the number of units of advertising on television PM = the number of units of advertising in the print media Minimize Cost = TV + 2PM (in millions of dollars) subject to PM ≥ 3 3TV + 2PM ≥ 18 –TV + 4PM ≥ 4 and TV ≥ 0, PM ≥ 0.

42. Algebraic Model for Profit & Gambit Let TV = the number of units of advertising on television PM = the number of units of advertising in the print media Minimize Cost = TV + 2PM (in millions of dollars) subject to Stain remover increased sales: PM ≥ 3 Liquid detergent increased sales: 3TV + 2PM ≥ 18 Powder detergent increased sales: –TV + 4PM ≥ 4 and TV ≥ 0, PM ≥ 0.

43. Applying the Graphical Method

44. The Optimal Solution

45. Summary of the Graphical Method • Draw the constraint boundary line for each constraint. Use the origin (or any point not on the line) to determine which side of the line is permitted by the constraint. • Find the feasible region by determining where all constraints are satisfied simultaneously. • Determine the slope of one objective function line. All other objective function lines will have the same slope. • Move a straight edge with this slope through the feasible region in the direction of improving values of the objective function. Stop at the last instant that the straight edge still passes through a point in the feasible region. This line given by the straight edge is the optimal objective function line. • A feasible point on the optimal objective function line is an optimal solution.

46. Union Airways Personnel Scheduling • Union Airways is adding more flights to and from its hub airport and so needs to hire additional customer service agents. • The five authorized eight-hour shifts are • Shift 1: 6:00 AM to 2:00 PM • Shift 2: 8:00 AM to 4:00 PM • Shift 3: Noon to 8:00 PM • Shift 4: 4:00 PM to midnight • Shift 5: 10:00 PM to 6:00 AM • Question: How many agents should be assigned to each shift, so that the cost is minimized?

48. LP Formulation Let Si = Number working shift i (for i = 1 to 5),Minimize Cost = $170S1 + $160S2 + $175S3 + $180S4 + $195S5subject toS1 ≥ 48S1 + S2 ≥ 79S1 + S2 ≥ 65S1 + S2 + S3 ≥ 87S2 + S3 ≥ 64S3 + S4 ≥ 73S3 + S4 ≥ 82S4 ≥ 43S4 + S5 ≥ 52S5 ≥ 15andSi ≥ 0 (for i = 1 to 5)

49. LP Formulation Let Si = Number working shift i (for i = 1 to 5),Minimize Cost = $170S1 + $160S2 + $175S3 + $180S4 + $195S5subject to Total agents 6AM–8AM: S1 ≥ 48 Total agents 8AM–10AM: S1 + S2 ≥ 79 Total agents 10AM–12PM: S1 + S2 ≥ 65 Total agents 12PM–2PM: S1 + S2 + S3 ≥ 87 Total agents 2PM–4PM: S2 + S3 ≥ 64 Total agents 4PM–6PM: S3 + S4 ≥ 73 Total agents 6PM–8PM: S3 + S4 ≥ 82 Total agents 8PM–10PM: S4 ≥ 43 Total agents 10PM–12AM: S4 + S5 ≥ 52 Total agents 12AM–6AM: S5 ≥ 15andSi ≥ 0 (for i = 1 to 5)

50. Work-scheduling problem