Inverse Multiscale Problems

1. Inverse Multiscale Problems Martin Burger Institut für Numerische und Angewandte Mathematik European Institute for Molecular Imaging (EIMI) Center for Nonlinear Science (CeNoS) Westfälische Willhelms-Universität Münster

2. Introduction Various processes in the natural, life, and social sciences involve multiple scales in time and space. An accurate description can be be obtained at the smallest (micro) scale, but the arising microscopic models are usually not tractable for simulations. In most cases one would even like to solve inverse problems for these processes (identification from data, optimal design, …), which results in much higher computational effort. BICS Workshop, Bath

3. Introduction In order to obtain sufficiently accurate models that can be solved numerically with reasonable effort there is a need for multiscale modelling. Multiscale models are obtained by coarse-graining of the microscopic description. The ideal result is a macroscopic model based on differential equations, but some ingredients in these models often remain to be computed from microscopic models. BICS Workshop, Bath

4. Introduction In many models some parameters (function of space, time, nonlinearities) are not accesible directly, but have to be identified from indirect measurements. For most processes one would like to infer improved behaviour with respect to some aspect – optimal design / optimal control For such identification and design tasks, a similar inverse multiscale modelling is needed. BICS Workshop, Bath

5. Electron / Ion Transport Transport of charged particles arises in many applications, e.g. semiconductor devices, ion channels, or nanopores The particles are transported along (against) the electrical field with additional diffusion. Self-consistent coupling with electrical field via Poisson equation. Possible further interaction of the particles at different scales: recombination, ionization, precipitation, size exclusion MOSFETS, from www.st.com Ion Channel Courtesy Bob Eisenberg BICS Workshop, Bath

6. Electron / Ion Transport Microscopic models from statistical physics (MD, Langevin, Boltzmann) or quantum mechanics (Schrödinger), coupled to Poisson (self-consistency) Coarse-graining to macroscopic PDE-Models classical research topic in applied math. Long hierarchy of models, well understood for semiconductors, not yet so well for channels and nanopores (due to crowding effects) Sketch of geometry of a MESFET Sketch of l-type CaChannel Mock 84, Markowich 86, Markowich-Ringhofer-Schmeiser 90, Jüngel 2002 Eisenberg et al 01-06 BICS Workshop, Bath

7. Electron / Ion Transport Other end of the hierarchy are Poisson-Drift-Diffusion / Poisson-Nernst-Planck equations (zero-th and first moment of Boltzmann-Poisson with respect to velocity) Poisson-Drift-Diffusion Poisson-Nernst-Planck BICS Workshop, Bath

8. Electron / Ion Transport Densities in a nanopore Ca2+ Na+ Cl- Size exclusion in ion channels significantly increases computational effort(e.g. nonlocal functionals in DFT) Densities and Potential in an L-type Ca channel (PNP-DFT) BICS Workshop, Bath

9. Electron / Ion Transport The main characteristics of the function of a device are current-voltage (I-V) curves (think of ion channels as a biological device). These curves are also the possible measurements (at different operating conditions, e.g. at different ion concentrations in channels) For semiconductor devices one can also measure capacitance-voltage (C-V) curves BICS Workshop, Bath

10. Electron / Ion Transport Inverse Problem 1: identify structure of the device (doping profiles, contact resistivity, relaxation times / structure of the protein, effective forces) from measurements of I-V Curves (and possibly C-V curves) Inverse Problem 2: improve performance (increased drive current at low leakage current, time-optimal behaviour / selectivity) by optimal design of the device (sizes, shape, doping profiles / proteins, nanopore geometry) BICS Workshop, Bath

11. Emergent Behaviour Many herding models can be derived from micro-scopic individual-agents-models, using similar paradigms as statistical physics. Examples are • Crowding effects in molecular biology (ion channels, chemotaxis) • Swarming / Herding / Schooling / Flocking of animals, humans (birds, fish, insect colonies, human crowds in evacuation and panic) • Traffic flow • Opinion formation • Volatility clustering, price herding in financial markets BICS Workshop, Bath

12. Emergent Behaviour Microscopic models can be derived in terms of SDEs, like Langevin equations for particle position / state Interaction kernels not determined by physics / natural laws in such applications A lot of macroscopic data collected BICS Workshop, Bath

13. Emergent Behaviour Coarse-graining to PDE-models similar to statistical physics (Boltzmann /Vlasov-type, Mean-Field Fokker Planck), but N smaller New effects yield also new types of interaction and advanced issues in PDE-models (general nonlocal interaction, scaling limits to nonlinear diffusion, ..) BICS Workshop, Bath

14. Emergent Behaviour Natural inverse problems in emergent behaviour (mostly future work): • Identification of interaction potentials from observation Bianchi et el 06, • Identification of dynamic parameters (effective diffusions, mobilities)McCarthy et al 07 • Optimal control (boundary or via external potentials) Lebdiez-Maurer 04, McCarthy et al 05 • Optimal shape / topology design (e.g. evacuation routes, traffic flow) BICS Workshop, Bath

15. Emergent Behaviour In some models blow-up is undesirable (e.g. chemotaxis and swarming due to finite size of individuals), in others it is wanted. E.g in opinion formation, the blow-up (as a concentration to delta-distributions) can explain the formation of extremist opinions (in stubborn societes) Blow-up is an enormous challenge with respect to the construction of stable numerical schemes and for inverse problems Porfiri, Stilwell, Bollt 2006 BICS Workshop, Bath

16. Swarming Example: swarming models without repulsive force (blowup) mb-Capasso-Morale 05 mb-DiFrancesco 06 BICS Workshop, Bath

17. Swarming Example: swarming models with local repulsive force (small nonlinear diffusion) mb-Capasso-Morale 05 mb-DiFrancesco 06 BICS Workshop, Bath

18. Chemotaxis Example: Chemotaxis models with Quorum sensing, Formation of clusters / coarsening BICS Workshop, Bath mb-Dolak-DiFrancesco, SIAP 07

19. Emergent behavious Inverse Problem 1: identify interaction or external potentials (or dynamic parameters like mobilities) from observations [mostly future work] Inverse Problem 2: design or control system to optimal behaviour [some results, a lot of future work] BICS Workshop, Bath

20. HR Molecular Imaging Molecular Imaging (PET, SPECT, …) techniques are usually based on some tracer that attaches to specific molecules Radioactive decay at some time, emitted photons (two directions) recorded outside the body Decay rate proportional to density inside the body, hence identification of right-hand side in transport equations BICS Workshop, Bath

21. HR Molecular Imaging Future clinical applications need increase of spatial resolution Current test setup – small animal PET Small scale effects become important: • Inaccuracy of decay position • Inaccuracy of emission axis • Scattering events BICS Workshop, Bath

22. HD Molecular Imaging Small animal pet: mouse heart (courtesy SFB 656 MoBiL,reconstruction by Coronal Sagittal Thomas Kösters) Transverse BICS Workshop, Bath

23. Summary of Issues Typical characteristics of the inverse problems are • huge amounts of data • low sensitivities of identification / design variables with respect to data nonetheless • simulation of data requires many solutions of forward model, high computational effort / memory • can be formulated as optimization problems (least-squares or optimal design) with model as constraints • sophisticated optimization models difficult to apply (even accurate computation of first-order variations might be impossible) BICS Workshop, Bath

24. Towards a General Theory ( ) F x y = Inverse problems techniques usually formulate a forward map F between the unknowns x and the data y Evaluating the map F(x) amounts to simulate the forward model for specific (given) x and compute macroscopic observables The inverse problem is formulated as or the associated least-squares problem / maximum likelihood estimation problem BICS Workshop, Bath

25. Towards a General Theory 0 ² ! 0 ( ( ) ) ² F F x x y y = = 0 ² In the multiscale case, we might think of a scale-dependent problem and a coarse-grained problem (maybe not completely scale-independent) related to BICS Workshop, Bath

26. Towards a General Theory How can we benefit from the coarse grained model ? • Replace e-problem by reduced problem • Preconditioning of reconstruction algorithms by coarse-grained solvers • Multiscale computations directly for the inversion • Interplay with regularization, appropriate coarse-graining for the unknown BICS Workshop, Bath

27. Multiscale Methods for the Inversion ( ) F B p u = ² ( ( ) ) E E 0 0 u u p p = = H H H ² ² ; ; ; Setup with u solving Multiscale scheme available BICS Workshop, Bath

28. Multiscale Methods for the Inversion ( ) ( ) ( ) D E R i 0 + u y u ® p p m n = ! ² ; ; u p ; Variational regularization of the inverse problem withconstraint BICS Workshop, Bath

29. Multiscale Methods for the Inversion ( ) ( ) ¤ @ @ D E 0 + u y u p w = u u ² ; ; ( ) ( ) ¤ @ R E 0 + ® p u p w = p ² ; ( ) E 0 u p = ² ; Schemes based on KKT-System Multiscale computation for last equation available, how to construct computation for first two ? BICS Workshop, Bath

30. Multiscale Methods for the Inversion ¢ ( ) ( ( ) ) ) k E B r r + ¤ ¢ ¢ u p u a u u p = = ² ; ² Example: source estimation in with macroscopic observation Homogenization techniques / multiscale FEM can be used BICS Workshop, Bath

31. Multiscale Methods for the Inversion ¢ ( ) ( ) ( ( ) ) ¤ ¤ @ @ E E r r ¢ ¢ u p w u p w a w w = = u ² p ² ; ; ; ² Adjoint problem Optimality Can again be solved with the same method Error estimates carry over BICS Workshop, Bath

32. Multiscale Methods for the Inversion ( ) ( ) @ f E t ¡ u p u u p = t ² ² ; ; ; Multiple Time Scales: highly-oscillatory ODE (similar schemes for SDES, see Kevrekidis) HMM for multiscale ODEs (Engquist-Tsai 05-07): • Macro time grid Tk and micro time grids Tk = tk,0 < tk,1 < .. < tk,n • Effective force estimation by local ODE integration on micro grid • Extrapolate to macro time step BICS Workshop, Bath

33. Multiscale Methods for the Inversion ( ) ( ) @ @ @ f E t ¡ u p w u p w = t u ² u ² ; ; ; ¡ Adjoint problem with final value w(T) = 0, backward in time Could try to apply same HMM scheme: • Macro grid Tk • Micro grid Tk = tk,0 > tk,1 > .. > tk,n BICS Workshop, Bath

34. Multiscale Methods for the Inversion HMM scheme for adjoint: • Needs to evaluate derivative of f at new micro time steps (values of u there !) • Corresponding solution is not the adjoint of the HMM scheme for the forward ODE • No convergence / error estimates for the regularized problem guaranteed • Not even existence of solution clear (even with regularization !) BICS Workshop, Bath

35. Multiscale Methods for the Inversion Alternative: use discrete adjoint to approximate adjoint equation • Backward integration on same micro/macro grid, no other values of u needed • Force correction rather than force estimation • Discrete optimality system, existence guaranteed, energy descent for time stepping • Estimation for approximation of dual variable w carries over BICS Workshop, Bath

36. Multiscale Methods for the Inversion 1 t ( ) ( ) ( ) f t ¡ u p a u p = ² ; ; ² ² Numerical experiment for linear model Coefficient a one-periodic e = 0.0002 Micro time step 0.00001 Macro time step 0.005 BICS Workshop, Bath

37. Multiscale Methods for the Inversion u v BICS Workshop, Bath

38. Multiscale Methods for the Inversion Multiscale Least-squares Reconstruction functional BICS Workshop, Bath

39. Multiscale Methods for the Inversion ¡ ² ¡ k k ( ) º C · + p p ® ¿ M S ¿ Error estimate for the (regularized) inverse problem and its multiscale approximation can be derived BICS Workshop, Bath

40. Interplay with regularization In molecular imaging, the main quantity of interest is the main variation of the density Small oscillations appear, but it is not realistic to find them from macroscopic data Look for structures with small total variation BICS Workshop, Bath

41. Interplay with regularization Z y X j j j l r + y o g ® u j ( ) F u j j Modelling via prior in the log-likelihood functional (Poisson distribution of noise) BICS Workshop, Bath

42. Interplay with regularization y 2 ( ) j Z Z ¡ 1 ( ) ¤ u u F = k 1 2 + u u = = k k j j 1 2 + r + ( ) F ® u u j 2 u k Two-step Algorithm for Minimization (EM-TV) uk+1as minimizer of Implemented by Alex Sawatzky BICS Workshop, Bath

43. Inversion and Cartooning 106 events standard EM EM-TV BICS Workshop, Bath

44. Inversion and Cartooning 250.000 events standard EM EM-TV BICS Workshop, Bath

45. Inversion and Cartooning 50.000 events standard EM EM-TV BICS Workshop, Bath

46. Identification of Doping Profiles Problem of highest technological importance is the identification of doping profiles (non-destructive device testing for quality control) In order to determine the doping profile many current measurements at different operating conditions are needed. Inverse problem is of the form (k=1,…,N) Fk(doping profile) = Current Measurementk Evaluating each Fkmeans to solve the model once mb-Engl-Markowich-Pietra 01 mb-Engl-Markowich 01, mb-Engl-Leitao-Markowich 04 BICS Workshop, Bath

47. Identification of Doping Profiles Sketch of a two-dimensional pn-diode Identification of a doping profile of a pn-diode by a nonlinear Kaczmarz-method BICS Workshop, Bath

48. Identification of Channel Structures Analogous problem in ion channels: identify permament charge of the channel More realistic: identify external potential (forces caused by the channel structure) acting on the permament charge distribution More data and higher sensitivity than for semiconductors, since concentrations can be varied Higher computational effort for the inverse problem BICS Workshop, Bath mb-Eisenberg-Engl, SIAP 07

49. Optimal Design of Doping Profiles Less operating conditions are of interest for optimal design problems (usually only on- and off-state), at most two different boundary conditions Possible non-uniqueness from primary design goal Secondary design goal: stay close to reference state (currently built design) Sophisticated optimization tools possible for Poisson-Drift-Diffusion models Hinze-Pinnau 02 / 06 mb-Pinnau 03 BICS Workshop, Bath

50. Optimal Design of Doping Profiles Fast optimal design by simple trick Instead of C, define new design variable as the total charge Q = -q(n-p-C) Partial decoupling, simpler optimality system Globally convergent Gummel method for design mb-Pinnau 03 / 07 BICS Workshop, Bath