Discrete Optimization Lecture 6 – Part I

Discrete Optimization Lecture 6 – Part I M. Pawan Kumar pawan.kumar@ecp.fr Slides available online http://cvn.ecp.fr/personnel/pawan

Programming Assignment I Nikos’ course website. Stereo correspondence. Images provided. Expansion algorithm. Max-flow code provided. 2 students per group. Deadline: 2 weeks.

Recap Label l1 Label l0 Vb Vc Va da db dc Labeling f : VL D : Observed data (image) V : Unobserved variables L : Discrete, finite label set

Recap Label l1 Label l0 Vb Vc Va da db dc Labeling f : VL D : Observed data (image) V : Unobserved variables Va is assigned lf(a) L : Discrete, finite label set

Recap 0 6 2 0 4 Label l1 1 2 3 1 Label l0 1 5 0 3 2 bc;f(b)f(c) Vb Vc Va a;f(a) da db dc Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b)

Recap 0 6 2 0 4 Label l1 1 2 3 1 Label l0 1 5 0 3 2 Vb Vc Va da db dc f* = argminf Q(f; ) Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b)

Recap 0 6 2 0 4 Label l1 1 2 3 1 Label l0 1 5 0 3 2 Vb Vc Va f* = argminf Q(f; ) Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b)

Outline • Convex Optimization • Integer Programming Formulation • Convex Relaxations • Comparison • Generalization of Results

Mathematical Optimization min g0(x) Objective function s.t. gi(x) ≤ 0 Inequality constraints hi(x) = 0 Equality constraints x is a feasible point gi(x) ≤ 0, hi(x) = 0 x is a strictlyfeasible point gi(x) < 0, hi(x) = 0 Feasible region - set of all feasible points

Convex Optimization min g0(x) Objective function s.t. gi(x) ≤ 0 Inequality constraints hi(x) = 0 Equality constraints Feasible region is convex Objective function is convex Convex set? Convex function?

Convex Set Line Segment x1 x2 c x1 + (1 - c) x2 c  [0,1] Endpoints

Convex Set x1 x2 All points on the line segment lie within the set For all line segments with endpoints in the set

Non-Convex Set x1 x2

Examples of Convex Sets x1 x2 Line Segment

Examples of Convex Sets x1 x2 Line

Examples of Convex Sets Hyperplane aTx - b = 0

Examples of Convex Sets Halfspace aTx - b ≤ 0

Examples of Convex Sets t x2 x1 Second-order Cone ||x|| ≤ t

Examples of Convex Sets Semidefinite Cone {X | X0} aTXa ≥ 0, for all a Rn All eigenvalues of X are non-negative aTX2a ≥ 0 aTX1a ≥ 0 aT(cX1 + (1-c)X2)a ≥ 0

Operations that Preserve Convexity Intersection Polyhedron / Polytope

Operations that Preserve Convexity Intersection

Operations that Preserve Convexity Affine Transformation x  Ax + b

Convex Function g(x) x1 x2 x Blue point always lies above red point

Convex Function g(x) x1 x2 x g( c x1 + (1 - c) x2 ) ≤ c g(x1) + (1 - c) g(x2) Domain of g(.) has to be convex

Convex Function g(x) x1 x2 x g( c x1 + (1 - c) x2 ) ≤ c g(x1) + (1 - c) g(x2) -g(.) is concave

Convex Function Once-differentiable functions g(y) + g(y)T (x - y) ≤ g(x) g(x) (y,g(y)) g(y) + g(y)T (x - y) x Twice-differentiable functions 2g(x) 0

Convex Function and Convex Sets g(x) x Epigraph of a convex function is a convex set

Examples of Convex Functions Linear function aTx p-Norm functions (x1p + x2p + xnp)1/p, p ≥ 1 Quadratic functions xTQx Q 0

Operations that Preserve Convexity Non-negative weighted sum g1(x) g2(x) + w2 + …. w1 x x xTQx + aTx + b Q 0

Operations that Preserve Convexity Pointwise maximum g1(x) g2(x) , max x x Pointwise minimum of concave functions is concave

Convex Optimization min g0(x) Objective function s.t. gi(x) ≤ 0 Inequality constraints hi(x) = 0 Equality constraints Feasible region is convex  Objective function is convex 

Linear Programming min g0(x) Objective function s.t. gi(x) ≤ 0 Inequality constraints hi(x) = 0 Equality constraints min g0Tx Linear function s.t. giTx ≤ 0 Linear constraints hiTx = 0 Linear constraints

Quadratic Programming min g0(x) Objective function s.t. gi(x) ≤ 0 Inequality constraints hi(x) = 0 Equality constraints min xTQx + aTx + b Quadratic function s.t. giTx ≤ 0 Linear constraints hiTx = 0 Linear constraints

Second-Order Cone Programming min g0(x) Objective function s.t. gi(x) ≤ 0 Inequality constraints hi(x) = 0 Equality constraints min g0Tx Linear function Quadratic constraints s.t. xTQix + aiTx + bi ≤ 0 hiTx = 0 Linear constraints

Semidefinite Programming min g0(x) Objective function s.t. gi(x) ≤ 0 Inequality constraints hi(x) = 0 Equality constraints min Q  X Linear function s.t. X 0 Semidefinite constraints Ai X = 0 Linear constraints

Outline • Convex Optimization • Integer Programming Formulation • Convex Relaxations • Comparison • Generalization of Results

Cost of V1 = 1 Cost of V1 = 0 Integer Programming Formulation 2 0 4 Unary Cost Label ‘1’ 1 3 Label ‘0’ 5 0 2 V2 V1 Labeling = {1 , 0} ; 2 4 ] 2 Unary Cost Vector u = [ 5

V1= 1 V1 0 Integer Programming Formulation 2 0 4 Unary Cost Label ‘1’ 1 3 Label ‘0’ 5 0 2 V2 V1 Labeling = {1 , 0} ; 2 4 ]T 2 Unary Cost Vector u = [ 5 Label vector x = [ -1 1 ; 1 -1 ]T Recall that the aim is to find the optimal x

Integer Programming Formulation 2 0 4 Unary Cost Label ‘1’ 1 3 Label ‘0’ 5 0 2 V2 V1 Labeling = {1 , 0} ; 2 4 ]T 2 Unary Cost Vector u = [ 5 Label vector x = [ -1 1 ; 1 -1 ]T 1 Sum of Unary Costs = ∑iui (1 + xi) 2

Pairwise Cost Matrix P Cost of V1 = 0 and V1 = 0 0 Cost of V1 = 0 and V2 = 0 0 0 1 0 Cost of V1 = 0 and V2 = 1 0 1 0 0 3 0 0 0 Integer Programming Formulation 2 0 4 Pairwise Cost Label ‘1’ 1 3 Label ‘0’ 5 0 2 V2 V1 Labeling = {1 , 0} 0 3 0

Pairwise Cost Matrix P 0 0 0 1 0 0 1 0 0 3 0 0 0 Integer Programming Formulation 2 0 4 Pairwise Cost Label ‘1’ 1 3 Label ‘0’ 5 0 2 V2 V1 Labeling = {1 , 0} Sum of Pairwise Costs 1 ∑ijPij (1 + xi)(1+xj) 0 3 0 4

Pairwise Cost Matrix P 0 0 0 1 0 1 = ∑ijPij (1 + xi + xj + Xij) 4 0 1 0 0 3 0 0 0 Integer Programming Formulation 2 0 4 Pairwise Cost Label ‘1’ 1 3 Label ‘0’ 5 0 2 V2 V1 Labeling = {1 , 0} Sum of Pairwise Costs 1 ∑ijPij (1 + xi +xj + xixj) 0 3 0 4 X = x xT Xij = xi xj

Uniqueness Constraint ∑ xi = 2 - |L| i  Va Integer Programming Formulation Constraints • Integer Constraints xi{-1,1} X = x xT

∑ xi = 2 - |L| i  Va Non-Convex Integer Programming Formulation 1 1 ∑ Pij (1 + xi + xj + Xij) x* = argmin + ∑ ui (1 + xi) 4 2 Convex xi{-1,1} X = x xT

Outline • Convex Optimization • Integer Programming Formulation • Convex Relaxations • Linear Programming (LP-S) • Semidefinite Programming (SDP-L) • Second Order Cone Programming (SOCP-MS) • Comparison • Generalization of Results

∑ xi = 2 - |L| i  Va LP-S Schlesinger, 1976 Retain Convex Part 1 1 ∑ Pij (1 + xi + xj + Xij) x* = argmin + ∑ ui (1 + xi) 4 2 Relax Non-Convex Constraint xi{-1,1} X = x xT

∑ xi = 2 - |L| i  Va LP-S Schlesinger, 1976 Retain Convex Part 1 1 ∑ Pij (1 + xi + xj + Xij) x* = argmin + ∑ ui (1 + xi) 4 2 xi[-1,1] Relax Non-Convex Constraint X = x xT

∑ Xij = (2 - |L|) xi j  Vb LP-S Schlesinger, 1976 X = x xT Xij[-1,1] 1 + xi + xj + Xij≥ 0

∑ xi = 2 - |L| i  Va LP-S Schlesinger, 1976 Retain Convex Part 1 1 ∑ Pij (1 + xi + xj + Xij) x* = argmin + ∑ ui (1 + xi) 4 2 xi[-1,1] Relax Non-Convex Constraint X = x xT

∑ xi = 2 - |L| i  Va ∑ Xij = (2 - |L|) xi j  Vb LP-S Schlesinger, 1976 Retain Convex Part 1 1 ∑ Pij (1 + xi + xj + Xij) x* = argmin + ∑ ui (1 + xi) 4 2 xi[-1,1], Xij[-1,1] LP-S 1 + xi + xj + Xij≥ 0

Discrete Optimization Lecture 6 – Part I

Télécharger la présentation

Plus de contenu associé