Multilevel Monte Carlo Metamodeling

1. Multilevel Monte Carlo Metamodeling Imry Rosenbaum Jeremy Staum

2. Outline What is simulation metamodeling? Metamodeling approaches Why use function approximation? Multilevel Monte Carlo MLMC in metamodeling

3. Simulation Metamodelling • Simulation • Given input we observe . • Each observation is noisy. • Effort is measured by number of observations, . • We use simulation output to estimate the response surface . • Simulation Metamodelling • Fast estimate of given any . • “what does the response surface look like?”

4. Why do we need Metamodeling • What-if analysis • How things will change for different scenarios . • Applicable in financial, business and military settings. • For example • Multi-product asset portfolios. • How product mix will change our business profit.

5. Approaches • Regression • Interpolation • Kriging • Stochastic Kriging • Kernel Smoothing

6. Metamodeling as Function Approximation Metamodeling is essentially function approximation under uncertainty. Information Based Complexity has answers for such settings. One of those answers is Multilevel Monte Carlo.

7. Multilevel Monte Carlo Multilevel Monte Carlo has been suggested as a numerical method for parametric integration. Later the notion was extended to SDEs. In our work we extend the multilevel notion to stochastic simulation metamodeling.

8. Multilevel Monte Carlo In 1998 Stefan Heinrich introduced the notion of multilevel MC. The scheme reduces the computational cost of estimating a family of integrals. We use the smoothness of the underlying function in order to enhance our estimate of the integral.

9. Example • Let us consider and we want to compute For all . • We will fix a grid , estimate the respective integrals and interpolate.

10. Example Continued We will use piecewise linear approximation Where are the respective hat functions and are Monte Carlo estimate, i.e, . are iid uniform random variables.

11. Example Continued Let us use the root mean square norm as metric for error It can be shown that under our assumption of smoothness that at the cost of .

12. Example Continued • Let us consider a sequence of grids . • We could represent our estimator as . Where, is the estimation using the grid. • We define each one of our decision variables in terms of M, as to keep a fair comparison.

13. Example Continued • The variance reaches its maximum in the first level but the cost reaches its maximum in the last level.

14. Example Continued Let us now use a different number of observations in each level, thus the estimator will be We will use to balance between cost and variance.

15. Example Continued • It follows that the square root of the variance is while the cost is . • Previously, same variance at the cost of .

16. Generalization Let and be bounded open sets with Lipschitz boundary. We assume the Sobolev embedding condition .

17. General Thm Theorem 1 (Heinrich). Let Then there exist constants such that for each integer there is a choice of parameters such that the cost of computing is bounded by and for each with

18. Issues MLMC requires smoothness to work, but can we guarantee such smoothness? Moreover, the more dimensions we have the more smoothness that we will require. Is there a setting that will help with alleviating these concerns?

19. Answer The answer to our question came from the derivative estimation setting in Monte Carlo simulation. Derivative Estimation is mainly used in finance to estimate the Greeks of financial derivatives. Glasserman and Broadie presented a framework under which a pathwise estimator is unbiased. This framework will be suitable as well in our case.

20. Simulation MLMC Goal Framework Multi Level Monte Carlo Method Computational Complexity Algorithm Results

21. Goal Our goal is to estimate the response surface The aim is to minimize the total number of observations used for the estimator. Effort is relative to amount of precision we require.

22. Elements We will Need for the MLMC Smoothness provided us with the information how adjacent points behave. Our assumptions on the function will provide the same information. The choice of approximation and grid will allow to preserve this properties in the estimator.

23. The framework First we assume that our simulation output is a Holder continuous function of a random vector , Therefore, there exist and such that for all in

24. Framework Continued… Next we assume that there exist a random variable, with a finite second moment such that for all a.s. Furthermore, we assume that and that it is compact.

25. Behavior of Adjacnt Points bias of estimating using is It follows immediately that,

26. Multi Level Monte Carlo Let us assume that we have a sequence of grids with increasing number of points The experiment designed are structured such that the maximum distance between a point and point in the experiment design is , denoted by . Let denote an approximation of using the same at each design point.

27. Approximating the Response

28. MLMC Decomposition Let us rewrite the expectation of our approximation in the multilevel way . Let us define the estimator of using m observations, .

29. MLMC Decomposition Continued Next we can write the estimator in the multilevel decomposition, Do we really have to use the same for all levels?

30. The MLMC estimator We will denote the MLMC estimator as Where

31. Multilevel Illustration

32. Multi Level MC estimators Let us denote We want to consider approximation of the form of

33. Approximation Reqierments We assume that for each there exist a window size >) which is . Such that for each, we have and for each

34. Bias and Variance of the Approximation • Under these assumptions we can show that • Our measure of error is Mean Integrated Square Error • Next, we can use a theorem provided by Cliffe et al. to bound the computational complexity of the MLMC.

35. Computational Complexity Theorem Theorem. Let denote a simulation response surface and , an estimator of it using replications for each design point. Suppose there exist such that , and The computational cost of is bounded by

36. Theorem Continued… Then for every there exist values of and for which the MSE of the MLMC estimator is bounded by with a total computation cost of

37. Multilevel Monte Carlo Algorithm The theoretical results need translation into practical settings. Out of simplicity we consider only the Lipschitz continuous setting.

38. Simplifying Assumptions The constants and stated in the theorem are crucial in deciding when to stop. However, in practice they will not be known to us. If we can deduce that .

39. Simplifying Assumptions Continued Hence, we can use as a pessimistic estimate of the bias at level . Thus, we will continue adding level until the following criterion is met However, due to its inherent variance we would recommend using the following stopping criteria

40. The algorithm

41. Black-Scholes

42. Black-Scholes continued

43. Conclusion Multilevel Monte Carlo provides an efficient metamodeling scheme. We eliminated the necessity for increased smoothness when dimension increase. Introduced a practical MLMC algorithm for stochastic simulation metamodeling.

44. Questions?