Lecture 1

1. Lecture 1 Linear Variational Problems (Part I)

2. 1. MotivationFor those participants wondering why we start a course dedicated to nonlinear problems by discussing a particular family of linear problems let us say that: (i) The Newton’s method provides a systematic way to reduce the solution of some nonlinear problems to the solution of a sequence of linear problems.

3. 1. MotivationFor those participants wondering why we start a course dedicated to nonlinear problems by discussing a particular family of linear problems let us say that: (i) The Newton’s method provides a systematic way to reduce the solution of some nonlinear problems to the solution of a sequence of linear problems. (ii) During this course, we will show that there are other methods than Newton’s which rely on linear solvers to achieve the solution of nonlinear problems. An example of such methods will be provided by the solution ofnonlinear problems in Hilbert spaces by conjugate gradient algorithms.

4. 1. MotivationFor those participants wondering why we start a course dedicated to nonlinear problems by discussing a particular family of linear problems let us say that: (i) The Newton’s method provides a systematic way to reduce the solution of some nonlinear problems to the solution of a sequence of linear problems. (ii) During this course, we will show that there are other methods than Newton’s which rely on linear solvers to achieve the solution of nonlinear problems. An example of such methods will be provided by the solution ofnonlinear problems in Hilbert spaces by conjugate gradient algorithms. We will start our discussion with the Newton’s method.

5. 2. The Newton’s method in Hilbert spaces Let Vbe a Hilbert space (real for simplicity); we denote by V’ the dual space of V and by <. , .>the duality pairing between Vand V’. Consider now an operator A(possibly nonlinear) mapping Vinto V’ and the following equation (E)A(u) = 0. Suppose that operator A is F-differentiable in the neighborhood of a solution uof (E); if v is sufficiently close to u we have then

6. 0 = A(u) = A(v + u – v) = A(v) + A’(v)(u – v) + o(||v – u||), with A’(v)( L( V, V’))thedifferentialofA at v. The abo- ve relation suggest the following algorithm, classically known as the Newton-Raphson method: (1) u0 is given in V ; then, for n≥ 0, un un+1by (2) un+1 = un – A’(un)–1A (un).

7. Suppose that A’(u)  Isom(V, V’) and that A’ is locally Lipschitz continuous, then if u0is sufficiently close to u we have limn+∞ un= u, the convergence being super-linear. From a ‘practical’ point of view it may be more appropriate to write (2) as (3)A’(un)(un+1– un) = – A(un), or even better (and equivalently) as: un+1– un V, (VF) <A’(un)(un+1– un),v> = –<A(un),v>,  v  V.

8. Many remarks are in order, but the only one we will make is that (VF) is a linear variational problem (in the sense of, e.g., J.L. LIONS) since the bilinear functional {v, w} < A’(un)v,w> : V×VR and the linear functional v < A(un),v> : VR are both continuous. 3. On a family of linear variational problems in Hilbert spaces: The Lax-Milgram Theorem We are going to focus our discussion on a particular family of linear problems, namely to those problems defined by

9. considering a triple {V, a, L} verifying the followingconditions: (i) Vis a real Hilbert space for the scalar (inner) product (.,.) and the associated norm || . ||. (ii) a: V×V Ris bilinear, continuous and V-elliptic (this last property meaning that there exists  > 0 such that a(v,v) ≥  ||v||2, v V). (iii) L: VRis linear and continuous. Remark 1:We do not assume that the bilinear functional ais symmetric.

10. Remark 2: If V is a finite dimensional space, the V-ellipticity of a the positive definiteness of a(not necessarily true if dim V is infinite). To {V, a, L}we associatethe following family oflinear (variational) problems: Find u  V such that (LVP) a(u,v) = L(v),  v  V.

11. Why do we consider such a family of linear problems ?Among the many reasons, let us mention the main ones:

12. Why do we consider such a family of linear problems ?Among the many reasons, let us mention the main ones: 1) Many applications in Mechanics, Physics, Control, Image Processing,…….. lead to such problems once the convenient Hilbert space has been identified (exemples of such situations will be encountered during this course).

13. Why do we consider such a family of linear problems ?Among the many reasons, let us mention the main ones: 1) Many applications in Mechanics, Physics, Control, Image Processing,…….. lead to such problems once the convenient Hilbert space has been identified (exemples of such situations will be encountered during this course). 2) Let us return to the solution of A(u) = 0 by the Newton’smethod and suppose that A = J’ where J is astronglyconvextwicedifferentiablefunctional (i.e.,   > 0, such that <J’(w) – J’(v), w – v> ≥ ||w – v||2,  v, w), then the linear problem that we have to solve at each iteration to obtain un +1belongs to the (LVP) family*.

14. Strictly speaking, a problem is (was) variational if it can be viewed as the Euler-Lagrange equation of some problem from the Calculus of Variations. In the particular case of (LVP) this supposes that a is symmetric; indeed, if ais symmetric, then (LVP) is equivalent to Find u  V such that (MP) J(u)  J(v),   V, with J(v) = ½ a(v,v) – L(v). Followingthe influence of J.L. Lions and G. Stampacchia (mid 1960’s) many authors use the terminology variational equations(and of course inequalities) even when a(.,.) is not symmetric.

15. The Lax-Milgram Theorem Suppose that the triple {V,a,L} verifies (i)-(iii) then (LVP) has a unique solution. Fora proof, see, e.g., Atkinson & Han (Springer). There are several proofs of the LM Theorem, our favorite one relies on the Banach fixed point theorem and can be generalized to variational inequalities. Actually, the LM Theorem can be generalized to complex Hilbert spaces by assuming that a(.,.) is sesquilinear, continuous and verifies Re a(v,v) ≥ ||v||2,  v  V.

16. Among the many comments which can be associated with the Lax-Milgram Theorem, we will focus on the following one, considering its computational implications : The fixed point based proof of the LM Theorem is based is based on the following observations:

17. Among the many comments which can be associated with the Lax-Milgram Theorem, we will focus on the following one, considering its computational implications : The fixed point based proof of the LM Theorem is based on the following observations: (1) From the Riescz representation Theorem, there exists a unique pair {A,l} such that: A  L(V,V), (Av,w) = a(v,w),  v, w  V,   ||A||, (l,v) = L(v),  v  V.

18. Among the many comments which can be associated with the Lax-Milgram Theorem, we will focus on the following one, considering its computational implications : The fixed point based proof of the LM Theorem is based on the following observations: (1) From the Riescz representation Theorem, there exists a unique pair {A,l} such that: A  L(V,V), (Av,w) = a(v,w),  v, w  V,   ||A||, (l,v) = L(v),  v  V. (2) If 0 < ρ < 2 ||A||– 2 the mapping v v – ρ(Av –l)is a uniformly strict contraction of V.

19. Among the many comments which can be associated with the Lax-Milgram Theorem, we will focus on the following one, considering its computational implications : The fixed point based proof of the LM Theorem is based on the following observations: (1) From the Riescz representation Theorem, there exists a unique pair {A,l} such that: A  L(V,V), (Av,w) = a(v,w),  v, w  V,   ||A||, (l,v) = L(v),  v  V. (2) If 0 < ρ < 2 ||A||– 2 the mapping v v – ρ(Av –l)is a uniformly strict contraction of V. (3) (LVP) and Au = lare equivalent.

20. It follows from observations (1)-(3) that if ρverifies0 < ρ < 2 ||A||– 2 we have geometric convergenceofthe following algorithm,  u0 V: (1)u0is given in V, n ≥ 0, un  un+1by (2)un+1 =un – ρ(Aun – l). The practical interest of (1)-(2) as written above is limited by the fact that in general A and lareunknown.

21. A more practical form of (1)-(2) is obtained by replacing relation (2) by the following equivalent one: un+1 V, (2)’ (un+1,v) = (un,v) – ρ[a(un,v) – L(v)],  v  V. Problem(2)’is also from the(LVP)family, with the role ofa(.,.)played by the V-scalar product(.,.). The fact thatρ is not known in general can be overcome by using various techniques using a sequence { ρn }n instead of a fixed ρ. If a(.,.) is symmetric, conjugate gradient provides a more efficient alternative to algorithm (1)-(2)’, at little extra cost. Conjugate gradient will be discussed next.