1. Dynamic Batch Bayesian Optimization JavadAzimi, Ali Jalali, Xiaoli Fern Oregon State University University of Texas at Austin In NIPS 2011, Workshop in Bayesian optimization, experimental design and bandits: Theory and applications

2. Bayesian Optimization (BO) • Finding the Maximizerof an unknown function by requesting a small set of • function evaluations (experiments) – experiments are costly • BO assumes prior over – select next experiment based on posterior Current Experiments Select Single/Multiple Experiment Gaussian Process Surface Run Experiment(s)

3. Traditional Approaches • Sequential: • Only one experiment is selected at each iteration • Pros: Performance is optimized • Cons: Can be very costly when running one experiment takes long time • Batch: • experiments are selected at each iteration • Pros: times speed-up comparing to sequential approaches • Cons: Can not performs as well as sequential algorithms

4. Batch Performance (Azimi et.al NIPS 2010) • Given a sequential policy, it chooses a batch of samples which are likely to be selected by the sequential policy. k=5 k=10

5. Motivation • Given a sequential policy, is it possible to simultaneously, • select a batch of experiments • approximately preserve the sequential policy performance. • Size of the batch can change at each time step • Dynamic batch size

6. Proposed Idea: Big Picture • Based on a given prior (blue circles) and an objective function (G), is selected • To select the next experiment, , we need, which is not available • The statistics of the samples inside the red circle are expected to change after observing at • Set the Gvalues for all samples inside the red circleas their upper bound value • If the next selected experiment is outside of the red circle, we claim it is independent from. x2 x3 x1

7. Problems • Which samples statistics are changed after selecting an/a set of experiment? • How can we upper bound the objective function G?

8. Gaussian Process (GP) • GP is used to model the posterior over the unobserved samples in BO • Statistical prediction for each point by a normal random variable rather than deterministic prediction • The posterior variance is independent from the observation

9. Definition • Unobserved set of points • Corresponding Outputs • Any point : z z z

10. GP Theorems

11. Expected Improvement (EI) • Our algorithm inputs a sequential policy to compete with. • We choose Expected Improvement (EI) as criterion • our approach extends to other policies • EI simply computes the expected improvement after sampling at each point

12. Dynamic Batch • samples are asked at each iteration. • if the selected samples are independent from each other. • The first selected sample, is the same as sequential • Choice of the second point depends on • Setting (maximum possible value) EI of the next step is upper bounded • The next sample is selected, if it is not inside the red circle (not significantly effected by )

13. Dynamic Batch: Algorithm

14. Experimental Results: Setting • GP with squared exponential kernel is used as the model • We set nl =20(total number of experiments), and nb=5 (maximum batch size) • The average regret over 100 independent runs is reported where regret is: • Speedup of each framework is reported which is the percentage of experiments asked in batch mode. • ε=0.02 for 2-3 dimensional and 0.2 for higher dimensional frameworks • An alternative and more realistic approach is to set M=(1+ α) ymwhich means (100* α)% improvement at each iteration.

15. Experimental Results: Results

16. Experimental Results: Speedup vs Budget

17. Conclusion and Future works • Conclusion • The proposed dynamic batch approach selects variable number of experiments at each step • The selected experiments are approximately independent from each other • The proposed approach approximately preserves the sequential performance • Future Works • Theoretical analysis of the distance between selected samples in batch and sequential approach. • The analysis of choose of epsilon in performance