1 / 77

Optimal Design For Large-Scale Ill-posed Problems

Optimal Design For Large-Scale Ill-posed Problems. Lior Horesh Joint work with E. Haber, L. Tenorio. IBM TJ Watson Research Center Large-Scale Inverse Problems and Uncertainty Quantification, Santa Fe, NM - May 2013. Introduction. Exposition - ill - posed Inverse problems.

gwylan
Télécharger la présentation

Optimal Design For Large-Scale Ill-posed Problems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Optimal Design For Large-Scale Ill-posed Problems LiorHoresh Joint work with E. Haber, L. Tenorio IBM TJ Watson Research Center Large-Scale Inverse Problems and Uncertainty Quantification, Santa Fe, NM - May 2013

  2. Introduction

  3. Exposition - ill - posed Inverse problems • Aim: infer model • Given • Experimental design y • Measurements • Observation model • Ill - posedness Naïve inversion ...

  4. How to solve ill-posed inverse problems ? • Naive strategy • Cast as an optimization problem

  5. How to improve model recovery ? • How can we ... • Provide more efficient optimization schemes ? • Improve observation model ? • Extract more information in the measurement procedure ? • Define appropriate distance measures / noise model ? • Incorporate more meaningful a-priori information ? • Supplement information from additional heterogeneous sources ? • Effectively and conclusively explore the posteriori space ? • Account for end decisions / control ?

  6. Evolution of inversionFrom simulation to Design • Forward problem (simulation, description) • Given: model m & observation model • Simulate: data d • Inverse problem (estimation, prediction) • Given: data d & observation model • Infer: model m (and uncertainties) • Design (prescription) • Given: inversion scheme & observation model • Find: the ‘best’ experimental settings y, regularization S, decision, …

  7. PART IOptimal experimental design

  8. Optimal experimental design - Motivation

  9. Which Experimental design is best ? Design 1 Design 2 Design 3

  10. Design questions – Diffuse Optical Tomography • Where light sources and optodes should be placed ? • In what sequence they should be activated ? • What laser source intensities should be used ? Arridge 1999

  11. Design questions –Electromagnetic inversion • What frequencies should be used ? • What trajectory of acquisition should be considered ? Newman 1996, Haber & Ascher 2000

  12. Design questions – Seismic inversion • How simultaneous sources can be used ? Clerbout 2000

  13. Design questions –Limited Angle Tomography • How many projections should we collect and at what angles ? • How accurate should the projection data be ?

  14. What can we Design ?

  15. Design experimental layout Stonehenge, 2500 B.C.

  16. Design experimental process Galileo Galilei, 1564-1642

  17. Respect experimental constraints… French nuclear test, Mururoa, 1970

  18. Optimal experimental design - Background

  19. Ill vs. Well - posed optimal experimental design • Previous work • Well-posed problems - well established [Fedorov 1997, Pukelsheim2006 ] • Ill-posed problems - under-researched [Curtis 1999, Bardow 2008 ] • Many practical problems in engineering and sciences are ill-posed What makes non-linear ill-posed problems so special ?

  20. Optimality criteria in well-posed problems • For linear inversion, employ Tikhonov regularized least squares solution • Bias - variance decomposition • For over-determined problems • A-optimal design problem

  21. Optimality criteria in well-posed problems • Optimality criteria of the information matrix • A-optimal design  average variance • D-optimality  uncertainty ellipsoid • E-optimality  minimax • Almost a complete alphabet… Stone, DeGroot and Bernardo

  22. The problem... • For non-linear ill-posed problems  none of these apply ! • Non-linearity  bias-variance decomposition is impossible • Ill-posedness controlling variance alone reduces mildly the error What strategy can be used ? Proposition 1 - Common practice so far Trial and Error…

  23. Experimental Design by Trial And Error • Pick a model • Run observation model of different experimental designs, and get data • Invert and compare recovered models • Choose the experimental design that provides the best model recovery

  24. The problem... • For non-linear ill-posed problems  none of these apply ! • Non-linearity  bias-variance decomposition is impossible • Ill-posedness controlling variance alone reduces mildly the error What other strategy can be used ? Proposition 2 - Minimize bias and variance altogether by some optimality criterion How to define the optimality criterion ? Haber, Horesh & Tenorio 2010 Horesh, Haber & Tenorio 2011

  25. Optimal experimental design - Statistical merit

  26. Optimality criteria for design • Loss • Mean Square Error • Bayes risk • Depends on the noise  • Depends on unknown model • Depends on unknown model • Computationally infeasible

  27. Optimality criteria for design • Bayes empirical risk • Assume a set of authentic model examples is available • Discrepancy between training and recovered models [Vapnik 1998 ] How can y and S be regularized ? • Regularized Bayesian empirical risk

  28. A quick survey – How Many Samples are needed? • What type of character are you ? 4 The Optimist The Realist The Pessimist

  29. Differentiable ObservationSparsity Controlled Design • Assume: fixed number of observations • Design preference: small number of sources / receivers is activated • The observation model • Regularized risk Horesh, Haber & Tenorio 2011

  30. Differentiable Observation Sparsity Controlled Design • Total number of observations may be large • Derivatives of the forward operator w.r.t. y • Effective when activation of each source and receiver is expensive Difficult… Horesh, Haber & Tenorio 2011

  31. Weights formulation Sparsity Controlled Design • Assume: a predefined set of candidate experimental setups is given • Design preference: small number of observations Haber, Horesh & Tenorio 2010

  32. Weights formulation Sparsity Controlled Design • Let be discretization of the space [Pukelsheim 1994] • Let • The (candidates) observation operator is weighted • w inverse of standard deviation • - reasonable standard deviation - conduct the experiment • - infinite standard deviation - do not conduct the experiment Haber, Horesh & Tenorio 2010

  33. Weights formulation Sparsity Controlled Design • Solution - more observations give better recovery • Desired solution many w‘s are 0 • Add penalty to promote sparsity in observations selection • Less degrees of freedom • No explicit access to the observation operator needed

  34. Optimal experimental design - Optimization framework

  35. The optimization problems • Leads to a stochastic bi-level optimization problem • Direct formulation • Weights formulation Haber, Horesh & Tenorio 2010, 2011

  36. The optimization problem • Bi-level optimization problem • Assuming the lower optimization level is: • Convex with a well defined minimum • With no inequality constraints Haber, Horesh & Tenorio 2010

  37. The optimization problem • m is eliminated from the equations and viewed as a function of • Compute gradient by implicit differentiation • The sensitivity • The reduced gradient Haber, Horesh & Tenorio 2010

  38. Optimal experimental design – Numerical studies

  39. Impedance Tomography – Observation model • Governing equations • Following Finite Element discretization • Given model and design settings • Find data , n < k • Design: find optimal source-receiver configuration

  40. Impedance Tomography – Designs comparison Naive design True model Optimized design Horesh, Haber & Tenorio 2011

  41. Magneto-tellurics Tomography – Observation model • Governing equations • Following Finite Volume discretization • Given: model and design settings y (frequency ) • Find: data , n < k • Design: find an optimal set of frequencies

  42. Magneto-telluricsTomography – Designs comparison True model Naive design Optimal linearized design Optimized non-linear design Haber, Horesh & Tenorio 2008 Haber, Horesh & Tenorio 2010

  43. The Pareto curve – A Decision making Tool • To drill or not to drill? Risk Haber, Horesh & Tenorio 2010

  44. Simultaneous source Seismic inversion Marmousi velocity model Reconstruction with all data Simultaneous source with random weights Simultaneous source with optimal weights Haber, Horesh & van den Doel (2013)

  45. PART IIregularization Design

  46. Regularization Design -Background

  47. Regularization approaches • Why regularization is necessary ? • Imposes a-priori information • Stabilizes the inversion process • Provides a unique solution • Several approaches • Explicit • Implicit - e.g. sparse representation

  48. The Black art of regularization- Time evolution • In the past several decades various regularizers were prescribed Energy Smoothness Adapt + Smooth Total-Variation • Compression algorithms as priors • Monte Carlo • … Robust Statistics Wavelet Sparsity Sparse & Redundant M. Elad 2006

  49. How to represent sparsely ? p l Over-complete Dictionary D • Principle of parsimony  True model can be represented by a small number of parameters • Each column Di is a prototype model  atom • Sparse representation vector u Sparse vector u Standard representation m

  50. Sparse representation • Ideally sparsest solution achieved by -’norm’ penalty • Non-convex  NP-hard combinatorial problem • Instead employ -norm Donoho 2006

More Related