How to Emulate: Recipes without Patronising

1. How to Emulate: Recipes without Patronising The MUCM Toolkit Dan Cornford, Aston University

2. Overview • What and why is the toolkit? • How is it delivered • Current toolkit contents • A (slightly contrived) tour through parts of the toolkit • What is the future of the toolkit? • What would you like to see in the toolkit?

3. What is the toolkit? A series of linked (web) pages: • Threads follow the derivation of major idea as a series of linked pages • Core threads cover main areas, variants cover specialisations • Procedures describe an operation or algorithm • provide sufficient information to allow the implementation of the operation • Discussions cover issues that may arise • during the implementation of a method, or other optional details • Alternatives present available options • when building a specific part of an emulator (e.g. choosing a covariance function) and provide some guidance for making the selection • Examples present how to use the techniques in practice • Definitions of a term or a concept • Meta any page that does not fall in one of the above categories • usually pages about the Toolkit itself

4. What are the main threads? • ThreadCoreGP - the core model, dealt with by fully Bayesian, Gaussian Process, emulation • ThreadCoreBL - the core model, dealt with by Bayes Linear emulation And to come … • ThreadVariantMultipleOutputs - variant of the core model in which we emulate more than one output of a simulator • ThreadGenericMultipleEmulators – dealing with multiple outputs from more than one emulator • ThreadVariantMultipleSimulators - variant of the core model: emulating outputs from more than one related simulator • ThreadVariantDynamic - a special case of multiple outputs as timeseries • ThreadVariantStochastic - variant of the core model in which the simulator output is random • ThreadVariantDerivatives - variant of the core model in which we also model derivatives of outputs

5. Example

6. Do I have to read it linearly? • Pages can be accessed individually or as part of a thread. • We will add cross-cutting threads, e.g. on design for computer models

7. How are we creating it? • The toolkit is built using a wiki • All the MUCM team contributes • Tony O’Hagan is the editor in chief, Yiannis Andrianakis is managing the overall technology • We release sections of the toolkit as they become mature to a web site • This allows us control over the quality of the content • We plan further enhancement to the presentation • More graphical presentation of the structure • Ability for users to add comments to pages

8. How to use the toolkit • I’ll use a scenario to motivate this. • A chemical engineer is working on an azoisopropane chemical process simulation. • The process involves two key chemicals, which react to produce 39 main chemicals, with 42 reactions possible. • Thus the simulator has 39+2*42 = 123 inputs. • For now the chemist is mainly interested in a single output, the main target azoisopropane concentration, 1 output! • I want to show how the toolkit can help here!

9. What does the chemist want to know? • There are many chemical reactions, but which are the most important for determining the output variation? • This is in essence a sensitivity analysis. • Not all the reaction rates and activation energies are perfectly known – many are not directly observable • Initial concentrations can be controlled • ThreadCoreGP is relevant here.

10. Exploratory analysis, prior judgements • The chemist expects only a few reactions to be important, and wants to know which these are • At present they use local estimates based on simulator Jacobians • The model is not too complex – typical evaluation takes a few tens of seconds, depending on target time • It is likely that reaction rate parameters within the model could lie in the range 0.5x to 2.0x where x is the specified value

11. ThreadCoreGP: how to emulate • ThreadCoreGP discusses all the issues that need to be tackled when undertaking emulation in the situation: • We are only concerned with one simulator • The simulator only produces one output • The output is deterministic • We do not have observations of the real world • We don’t make statements about the real world process • We cannot directly observe derivatives of the simulator • We’ll explore how we can use ThreadCoreGP

12. What is in ThreadCoreGP? • Definition of what a Gaussian process is • Discussion of the implications of using a Gaussian process • Alternatives to the ‘full Bayesian’ approach – Bayes Linear methods • Provides technical information and discusses alternatives for: • determining active inputs • mean functions and covariance functions • choice of prior distributions • experimental design of simulator runs • fitting the emulator • using the emulator • prediction, uncertainty analysis and sensitivity analysis

13. DiscGaussianAssumption – what is in there? • This discusses issues to do with representing beliefs about the simulator in terms of a Gaussian process • Why we use a Gaussian process • computation and simplicity; other approaches could be entertained • When a Gaussian process might be inappropriate • outputs constrained in a range (but not practically important if we have a good emulator) • What to do if Gaussian process is not appropriate • main solution is use transformations e.g. log • Also mentions Bayes Linear methods

14. AltMeanFunction – what is in there? • Discussion of the alternatives for the mean function: • mean function should be chosen to represent ‘the general shape of how the analyst expects the simulator output to respond to changes in the inputs’ • Typically a linear in parameters regression, with a prior over the parameters – AltGPPriors • Other forms possible but there is a price

15. AltCorrelationFunction – what is in there? • Discussion of the alternatives for choosing the covariance function • Gaussian (squared exponential), generalised Gaussian, Matern • Role of nuggets • Implications of choices • Other possible choices

16. OK time for you to take over • Rather than presenting this I now want to get you to do some work • I like volunteers to try and use the toolkit – let’s talk about your simulation problems as see if the toolkit has the answers • What problems made you sign up for today • I’ll try and find the answers in the toolkit or the experts

17. Toolkit development – the future • The toolkit is continually developing • By the end of MUCM there will be a complete description of most aspects of building and using emulators • MUCM2 will add more content, particularly accessible introductions and more examples • Have we missed something? • Please tell us! • Future releases should allow easy commenting

18. Summary • The toolkit will distil the combined knowledge of the MUCM team (and beyond) • We intend it to become the ‘emulationWikipedia’: • An accessible, free community resource which will outlive the project • We are releasing it in parts, and will continue to improve it within MUCM2