1. CS5321 Numerical Optimization 13 Linear Programming: The Simplex Method

2. The standard form • The standard form of linear programming is Minxz=cTx subject to Ax = b, x  0 • Matrix A is anmn matrix,where m is the number ofconstraintsand n is the number of variables. • We assume A has full row rank and m  n. • For Axb, add slack variables. Ax −s = b, s  0. • For Axb, subtract slack variables Ax + s=b, s  0.

3. Geometry of LP • Feasible region : the set of all feasible points • If  is empty, LP has no solution. (infeasible) • If  is nonempty, it is convex. • If the object function is unbounded on , LP has no solution. • If LP is bounded and feasible, it can have either one or infinitely many solutions.

4. Basic feasible point • A point x is a basic feasible point (or a vertex of feasible polytope) if it is feasible and is not a linear combination of any other feasible points. • If LP has solutions, at least one solution is a basic feasible point. (Theorem 13.2) • At a basic feasible point, at least n−m variables are zero. • The case it has more than n−m zero variables is called degenerate.

5. µ ¶ µ ¶ x c B B ¡ ¢ B N A x c = = = ; ; x c N N Basic variable and basis matrix • At a basic feasible point, variables that can be uniquely determined are called basic variables. • The other are called nonbasic variables. (set to zero.) • Let B, N be the index sets for basic/nonbasic variables. • Variable x, c, and A can be rearranged according to basic/nonbasic variables. • B, an nn nonsingular matrix, is called the basis matrix

6. T T T 1 ¡ ¡ ( ) ¸ B B T T 1 ¡ b B c c T T T T = = B z c x c b A B N B B = = B + + B B z c x c x c x c x x x x x 1 ¡ = = = = = = B N B B N B b B B N B µ ¶ µ ¶ x = B x c B B ¡ ¢ B N A x c = = = ; ; x c N N Simplex multiplier • Since xN=0 at a basic feasible point • Object function: • Constraints: • The basic variables are and therefore the object function • The simplex multiplier is

7. T T 1 ¡ ( ) ¸ B b A B N + 1 1 ¡ ¡ c T T T T T T T T T T T = 1 1 1 b x x x B ¡ B N ¡ ¡ ( ) ( ) = = B N ¡ b ¸ b B B N B N B T + + + ¡ + ¡ x x ¸ N = B N z z z c c c x x c c x x c c c x x c c x ¡ = = = = = B B N N N N s c B B B N N N B N B = N N Pricing • Object function could be decreased by changing nonbasic variables, xN. , • Plug in , • The vector is called pricing. • If sN(i)<0, z can be decrease by increasing xN(i). • If all sN(i)0, the optimal solution is founded.

8. 1 1 ¡ ¡ ( ) b l d B B N i i ¡ x x p r e v o u s s e 1 ¡ ½ ¾ = ( ) ( ) B N b B i . 1 ¯ ¡ ( ( ) ) ( ) 1 1 ¡ ¡ B N ( ) ( ) ( ( ) ) i i 0 ¸ b B B N 0 t ¡ p a r g m n : q = : q x q x e x c e p x q = = N N N ¯ ; ; . ( ( ) ) ( ) 1 ¡ B N i i 1 : q m = : : ; The ratio test • Select q s.t. sN(q)<0 is the smallest element in sN and increase xN(q) until one element in xB, say xB(p), becomes zero. (How to find p?) • Need xB(i)  0 for all i. If B−1N(:,q)(i)<0, then xB(i)>0 • Only need to consider i for B−1N(:,q)(i)0 • What if no such i? the unbounded case

9. T 1 1 ¡ ¡ ( ( ) ) 1 1 ¡ ¡ B N B ( ( ) ) ( ) ( ) = ( ) ¡ d b d B N B : q e e T + : q ° p p ( ( ) ) = = B B N B p ; + ¡ 1 1 + ¡ ¡ p ( ) ; ; : q e e B B = ¡ p ; = p T ( ( ) ) 1 ¡ B N 1 + ¡ e : q e p ; p Pivoting • Exchange p and q in B,N and update xB, xN and B. • Let • Update xB = xB− d and xN(q) =  • Update B: replace B(p) with N(q) (How about B−1?) • It is a rank-1 update. Let B+ be the updated one. • The Sherman-Morrison formula

10. The simplex method While (true) • Given B,N. xB=B−1b, xN = 0 (Basic feasible point) • =B−1cB, sN=cN−NT (Simplex multiplier, pricing) • If sN  0, stop (Found an optimal solution) • Select q s.t. sN(q)<0, and solve Bd=N(:,q) • If d  0, stop(Unbounded case) • Compute [,p]=mind(i)>0xB(i)/d(i) (Ratio test) • Update xB=xB− d and xN(p) =  (Pivoting) • Exchange p and q in B,N and update matrix B.

11. Remaining problems • How to find the initial basic feasible point? • Two phase algorithm: add more slack variables to make the trivial point (0,…,0) feasible, and solve it until all additional slack variables become zero. • How to resolve the degenerate case? • In degenerate case, the algorithm might pivot the same p and q repeatedly. • Perturb the constraints to avoid the degenerate case.

12. µ ¶ ! n n = ( ) ! ! ¡ m n m m Complexity • In each iteration, the most time consuming task is pricing, ratio test and update B. O(mn) • The number of iterations is less than or equals to the number of basic feasible points, which is • The worst case time complexity is exponential. • Try n=2m. The number of iterations > 2m. • But practically, it terminates in m to 3m iterations. • Average case analysis and smoothed analysis.