Créer une présentation
Télécharger la présentation

Télécharger la présentation
## Compressed Sensing Theory Geometric Interpretations

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Compressed Sensing TheoryGeometric Interpretations**April 7th 2016, Ilmenau Prof. Giovanni Del Galdo**Compressed Sensing TheoryGeometric Interpretations**April 7th 2016, Ilmenau Prof. Giovanni Del Galdo With contributions from… Dr. Florian RömerAnastasia LavrenkoAlexandra CraciunMagdalena PrusMohamed Gamal IbrahimRoman Alieiev Heavily inspired by… the tutorial of Dr. Dejan E. Lazichgiven at the 22nd meeting of the ITG section on “Applied Information Theory” on Oct. 7th, 2013**Goals of this lecture**• Answer the following questions: • Why is the desired sparse solution unique? • Why is the hyperplane of solutions tangent to the L1-ball at the right point? • What is the impact of measurement noise? ?**Complete Data Model**Reconstructionstrategy**Solution Set for the Equality Constraint**• x must lie on a line**Lp-Norms and Lp-Balls**• Definition of Lp-norm: • Lp-Balls for N = 2**Minimization Problem: p=0.5**L2 2D animation**Minimization Problem: p=0.5**L2 2D animation • The “arms” of the L0.5 ball reach out making sparse solutions favored wrt non-sparse**Minimization Problem: p=1**• Same effect as L0.5, although not as prominent**Minimization Problem: p=2**• This is the LS solution**Escape Velocities for the L1-ball**• Vertices move faster than edges, which move faster than sides • They correspond to 1-sparse, 2-sparse, and 3-sparse respectively**Choice of p-Norm**• p>1 lead to non-sparse solutions • p<=1 lead to sparse solutions • p=0 = cardinality • logical choice • gives the sparsest solution • NP-hard • p=1 • identical to p=0 for practical cases • linear and convex problem**Goals of this lecture**• Answer the following questions: • Why is the desired sparse solution unique? • Why is the hyperplane of solutions tangent to the L1-ball at the right point? • What is the impact of measurement noise? ?**Outline**• General geometrical considerations • affine subspace of solutions for a generalized System of Linear Equations (SLE) • subspaces spanned by sparse vectors • intersection between the subspace of solutions with the subspaces spanned by the sparse vectors, i.e., When does an SLE have sparse solutions? • Geometrical considerations specific to Compressed Sensing • solutions for a system originating from a compressed sensing scenario • impact of the noise: designing the sensing matrix • Conclusions**System of Linear Equations (SLE)**underdetermined overdetermined fully determined infinite number ofsolutions one unique solution no solutions ! assuming B to be full rank**Solutions to the i-th Linear Equation**solution to the homogeneous problem solution to the inhomogeneous problem**Solutions to the Homogeneous Linear Equation**• Any point x0 on the plane orthogonalto biis a solution to the homogeneous linearequation**Solutions to the Inhomogeneous Linear Equation**• Distance between the planes: • Any point on the affine planeis a solution to the inhomogeneous linearequation**Solutions to the Inhomogeneous SLE**• Each equation removes a degreeof freedom • There exists one unique solution**Shooting for Higher Dimensional Spaces**• In order to predict the characteristics of the affine subspace of solutions we cannot rely on intuition alone • Therefore, we need to introduce more powerful tools • the definition of affine subspaces • the rules applying to the intersection of affine subspaces ND 3D**K-Flats, Affine Subspaces in N-dimensional Ambient Space**• The origin-free generalization of points, lines, planes, and K-dimensional subspaces are termed: • affine subspaces of dimension 0, 1, 2, and K, respectively or • 0-flats, 1-flats, 2-flats, and K-flats, respectively • A K-flat is determined by K+1 linearly independent points • K points are linearly independent iff • taking a subset, no 3 points lie on a line (1-flat) • taking a subset, no 4 points lie on a plane (2-flat) • … • taking a subset, no k points lie on a (k-2)-flat, for 2<=k<=K • Ambient space of dimension N: the N-flat which contains the universeconsidered**K-Flats, Affine Subspaces in N-dimensional Ambient Space**• A line is a 1-flat and is determined by 2 linearly independent points • A plane is a 2-flat and is determined by 3 linearly independent points • 3 points are linearly independent if no 2 points overlap and no 3 points lie on a line**Intersection of Affine Subspaces**• Assume a p-flat and a q-flat in N-dimensional ambient space • Ignoring special cases, the intersection of the two flats is • (p+q-N)-flat, if p+q-N >= 0 • empty, if p+q-N < 0 • If two flats obey the rules above, they are said to be in • General Position • Randomly generated subspaces are in General Position with probability one**Intersection of Affine Subspaces: Examples**Two lines in 3D space p = 1 q = 1 N = 3 Intersection = empty …as 1 + 1 – 3 = -1 The lines are skew**Intersection of Affine Subspaces: Examples**Two lines in 2D space p = 1 q = 1 N = 2 Intersection = 0-flat …as 1 +1 – 2 = 0 The lines intersect in a point Degeneratecase: Twooverlappinglines Intersection = 1-flat**Intersection of Affine Subspaces: Examples**Two planes in 3D space p = 2 q = 2 N = 3 Intersection = 1-flat …as 2 + 2 – 3 = 1 The planes intersect in a line**Intersection of Affine Subspaces: Examples**A special case can be visualized! The rest of the plane is in the fourth dimension Two planes in 4D space p = 2 q = 2 N = 4 Intersection = 0-flat, i.e., a point …as 2 + 2 – 4 = 0 The two planes intersect in a point**Degenerate Cases: Examples**• Two 1-flats (lines) overlapping in 2D • The General Position would be an intersection in one point • Two 2-flats (planes) intersecting on a line in 5D • The General Position would be an empty intersection • Affine subspaces generated randomly are degenerate with probability zero**Flats, their Intersection for a SLE**• Each linear constraint gives rise to a (N-1)-flat • The subspace of solutions is the intersection of the M (N-1)-flats in N-dimensional space. It is an (N-M)-flat!**Recap: Geometrical Interpretation on the SLE**• Given a SLE featuring • an N dimensional unknown vector x • M linear equations • The solutions to each linear equation span an affine subspace of dimensionality N-1, i.e., an (N-1)-flat • The solutions to the whole SLE span the intersection of the M (N-1)-flats • Given an underdetermined system of equations • the affine subspace of solutions has size N-M • we call it: the “Solutions Monster” (N-M)-flat Solutions Monster**Outline**• General geometrical considerations • affine subspace of solutions for a generalized System of Linear Equations (SLE) • subspaces spanned by sparse vectors • intersection between the subspace of solutions with the subspaces spanned by the sparse vectors, i.e., When does an SLE have sparse solutions? • Geometrical considerations specific to Compressed Sensing • solutions for a system originating from a compressed sensing scenario • impact of the noise: designing the sensing matrix • Conclusions**Affine Subspaces for K-sparse Vectors**• The K-sparse vectors span specific affine subspaces in N-dimensional space • 1-sparse vectors span 1-flats**Affine Subspaces for K-sparse Vectors**• 2-sparse vectors span 2-flats**Outline**• General geometrical considerations • affine subspace of solutions for a generalized System of Linear Equations (SLE) • subspaces spanned by sparse vectors • intersection between the subspace of solutions with the subspaces spanned by the sparse vectors, i.e., When does an SLE have sparse solutions? • Geometrical considerations specific to Compressed Sensing • solutions for a system originating from a compressed sensing scenario • impact of the noise: designing the sensing matrix • Conclusions**Affine Subspaces for K-sparse Vectors**• In general: • K-sparse vectors span K-flats • Given a system of linear equations in General Position: • What is the probability of having sparse solutions? I.e., when will the solutions monster touch the flats containing sparse vectors? ? 1-flat (N-M)-flat Solutions Monster**Intersection: Solution Monster <-> Sparse Flats**• Assuming a SLE of M linear constraints in N-dimensional space • Assuming General Position the results in the table occur with Prob. = 1**Intersection: Solution Monster <-> Sparse Flats**2-flat 2-flat 1-flat 1-flat (N-M)-flat Solutions to the SLE 2-flat 1-flat • For K < M the Solutions Monster does not intersect with any K-flat carrying K-sparse solutions**Recap: Solution Monster vs. Sparse Flats**• In the General Position, the Solutions Monster dodges all sparse vectors up to K = M-1 Me too! (N-M)-flat Solutions to the SLE**Outline**• General geometrical considerations • affine subspace of solutions for a generalized System of Linear Equations (SLE) • subspaces spanned by sparse vectors • intersection between the subspace of solutions with the subspaces spanned by the sparse vectors, i.e., When does an SLE have sparse solutions? • Geometrical considerations specific to Compressed Sensing • solutions for a system originating from a compressed sensing scenario • impact of the noise: designing the sensing matrix • Conclusions**SLE in a Compressed Sensing Scenario**• Let us now take an underdetermined system of equations originating from a sparse solution x • The Solution Monster corresponding to the homogeneous system will bein General Position (N-M)-flat Solutions Monster