FDA – A SCALABLE EVOLUTIONARY ALGORITHM FOR THE OPTIMIZATION OF ADDITIVELY DECOMPOSED FUNCTIONS

1. FDA – A SCALABLE EVOLUTIONARY ALGORITHM FOR THE OPTIMIZATION OF ADDITIVELY DECOMPOSED FUNCTIONS BISCuit EDA Seminar 2008. 06. 26.

2. Introduction • Optimization of additively decomposed functions (ADFs) • How to decompose a given function? – a factorization problem • Bayesian network • Optimization by discovering distribution of good solutions. • Distribution model – Boltzmann distribution • Selecting good solutions to build model • Theoretical and empirical results

3. Factorization Theorem • Additively decomposed function (ADF) • Boltzmann distribution The larger the function value f(x), the larger p(x).

4. Factorization Theorem • Factorization Theorem Given partition of variables, the joint distribution can be factorized into a product of marginal and conditional probabilities.

5. The Factorized Distribution Algorithm (FDA) Sampling Selection Build Model

6. The Factorized Distribution Algorithm (FDA) – Factorization Algorithm • Can compute an exact factorization for simple structures (chains, trees). • Computes an approximation factorization for complex structures (rings, torus).

7. How to generate initial population? • The factorization is computed beforehand. • Conditional probabilities are computed using the local fitness functions only. • The steepness of the distribution (favor of better solutions) is adjusted so that

8. How to generate initial population? • Example of OneMax • Factorization • Span = 1, thus u = 10. • It might not give a Boltzmann distribution. Therefore, half of the population is generated in this way, and the other half is generated randomly. For each site xi, 1 is more favored 10 times than 0.

9. Does FDA Converge? • Given the distribution of the selected individuals as At each generation, the dist. of selected individuals follow Boltzmann dist. In the end, the dist. of selected individuals converge to uniform dist. over optimal solutions.

10. Theoretical Analysis for Infinite Populations • Target functions Factorization

11. Theoretical Analysis for Infinite Populations

12. Theoretical Analysis for Infinite Populations Truncation Boltzmann

13. For infinite populations, strongest selection is the best. However, the optimal annealing schedule is very difficult for finite populations. • For OneMax, FDA with truncation selection can generate a Boltzmann distribution. • Theorem 6, 7, 8 gives the maximum generation for convergence for Int problem.

14. Analysis of FDA for Finite Populations • Due to the heavy dependency of Boltzmann selection to annealing schedule, EDA with truncation selection will be considered.

15. Analysis of FDA for Finite Populations

16. Analysis of FDA for Finite Populations

17. Numerical Results • Generations until convergence f2: order-3 OneMax f3(1,1,1) = 10, Otherwise, 0.

18. Numerical Results

19. Numerical Results

20. LFDA – Computing a Bayes Factorization • Given a population of selected points, what is a good Bayes factorization fitting the data? • Minimum description length (MDL) score • Bayesian Information Criterion

21. LFDA – Computing a Bayes Factorization • Finding Bayesian network structure (greedy) Due to the search for structure, LFDA is computationally very expensive!