AM 121: Intro to Optimization Models and Methods

1. AM 121: Intro to OptimizationModels and Methods Lecture 2: Intro to LP, Linear algebra review. Ozlem Ergun SEAS/Georgia Tech TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA

2. Lecture 2: Lesson Plan • What is an LP? • LP in matrix form. Matrix review. • Graphical and algebraic correspondence • Problems in canonical form Jensen & Bard: 2.1-2.3, 2.5, 3.1 (can ignore the two definitions for now),3.2 Recommended text is available at Gordon McKay library (3rd floor of Piece Hall).

3. Linear Programming • Maximizing (or minimizing) a linear function subject to a finite number of linear constraints n j=1

4. Terminology • Decision variables: xj • Parameters: cj, aij, bi • Fixed, known • Objective function • Constraints

5. A Little History • The field of linear programming started in 1947 when George Dantzig (1914-2005) designed the “simplex method” for solving LP formulations of U.S. Air Force planning problems • Dantzig was deciding how to use the limited resources of the Air Force • planning == programming • “program” was a military term that referred to plans or proposed schedules for training, logistical supply, or deployment of combat units. • Some later came to call this “Dantzig’s great mistake”

6. LP in Matrix Form

7. Review: Matrices (1/4) • Matrix: rectangular array of numbers • Dimension: m ✕n, (m rows and n columns) • k✕1: column vector 1 ✕k : row vector • B = A = A, scalar : aij= bij • A B = C • c x = i cixi“inner product” (m x p) (p x n) (m x n) (1 x n) (n x 1)

8. Review: Matrices (2/4) • AT transpose: aTij = aji • Partitions (“usual rules of matrix algebra”)

9. Review: Matrices (3/4) • Square matrix: m ✕ m • Identity matrix: square matrix w/ diagonal elements all 1 and all non-diagonal are 0. • I2, I3, … • For square A, let inverse A, A-1 = B BA = AB = Im

10. Review: Matrices (4/4) • Given Ax = b • A-1(Ax)=A-1b or • x = A-1b • We can find an unique solution to a square linear system as above.

11. Terminology for Solutions of LP • A feasible solution • A solution that satisfies all constraints • An infeasible solution • A solution that violates at least one constraint • Feasible region • The region of all feasible solutions • An optimal solution • A feasible solution that has the best objective function value

12. Example: Marketing Campaign • Comedy– 7 million high-income women, 2 million high-income men. Cost $50,000 • Football – 2 million high-income women and 12 million high-income men. Cost $100,000 • Goal: reach at least 28 million high-income women and 24 million high-income men at MINIMAL cost

13. Graphical representation of the problem Optimal solution: x1=3.6, x2=1.4, z = 320 How would you establish the optimality of this algebraically?

14. Example: Alt. Optimal Solutions Note that there ARE still extremal optimal solutions

15. Example: Unbounded Objective How would you show this algebraically?

16. Example: Infeasible Problem • ..\..\Desktop\17.bmp How would you show this algebraically?

17. Thinking Algorithmically • canonical form • basic feasible solutions • solution improvement

18. Canonical Form • All decision variables are non-negative • All constraints are equalities • The RHS coefficients are all non-negative • One decision variable is “isolated” in each constraint with a +1 coefficient. These variables do not appear in any other constraint and have a zero coefficient in objective function Why might this be useful??

19. Basic Feasible Solution Have an associated “basic feasible solution” in which the isolated decision variables (basic) are non-zero and the rest (non-basic) are zero. Here, set x1 = 6, x2=4, x3=0, x4=0. Clearly optimal in this example as well. (Why?) Optimality criterion: “if every non-basic variable has a non-positive coefficient in the objective function”

20. Solution Improvement Current basic feasible solution: x1 = 6, x2=4, x3=0, x4=0. Now, increase x4 and decrease x2 (keep x3=0) until second constraint becomes binding. Obtain new solution: x1=3, x2=0, x3=0, x4=1. Can transform into a new canonical form in which the isolated variables are x1 and x4. (“pivot on x4 in the second constraint”). “pick something to come in, something is forced to leave”

21. New Canonical Form After linear transformations: Basic feasible solution x1=3, x2=0, x3=0, x4=1 is optimal

22. Geometric Interpretation of Solution Improvement x1 = 6, x2=4, x3=0, x4=0 x1=3, x2=0, x3=0, x4=1

23. Can anything be put in canonical form? (a) maximization, (b) positive RHS, (c) equality constraints, (d) non-negative variables, (e) isolated variables

24. Reduction to canonical form (I) • min z = max –z • if a RHS is negative then multiply the constraint by -1

25. Reduction to canonical form (II) • Inequality constraints surplus variable slack variable

26. Reduction to canonical form (III) • Free variables (i.e. without non-negativity constraints) • y = u – v • u ≥ 0 • v ≥ 0 • whenever y ≥ 0, u = y and v = 0 • whenever y < 0, u = 0, v=-y • Replace y with (u-v) wherever it appears

27. Make sure one variable is “isolated” in each constraint with a +1 coefficient • Some will be already OK. E.g., • But, some not OK. E.g., • Solution: introduce a new artificial variable (make sure x6 = 0 in final solution) isolated and can function as an initial basic variable doesn’t work (coeff -1)

28. Next Time • Solution concepts