Outline

1. Global Optimization: For Some Problems, There's HOPEDaniel M. DunlavySandia National Laboratories, Albuquerque, NM, USADianne P. O’LearyDept. of Computer Science and UMIACSUniversity of Maryland, College Park, MD, USACopper Mountain Conference on Iterative MethodsApril 3, 2006SAND2006-2001C Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

2. Outline • Unconstrained Minimization Problem • Homotopy Optimization Methods • Numerical Experiments • Protein Structure Prediction Problem • Numerical Experiments • Concluding Remarks

3. Problem • Solve the unconstrained minimization problem • Function Characteristics • Solution exists • Smooth ( ) • Multimodal, deep local minima • Good starting points unavailable

4. Some Related Methods • Stochastic search methods • Random perturbations • Simulated annealing • Perturbations, acceptance criterion, schedule • Evolutionary algorithms • Ensembles/populations • Smoothing methods • Deformation of function being minimized • Homotopy/continuation methods • Nonlinear equations,

5. Outline • Unconstrained Minimization Problem • Homotopy Optimization Methods • Numerical Experiments • Protein Structure Prediction Problem • Numerical Experiments • Concluding Remarks

6. Homotopy Optimization Method (HOM) • Goal • Minimize complicated nonlinear target function • Steps to solution • Easy template function: , is a known local min. • Define a continuous homotopy function: • Produce sequence of minimizers of w.r.t.starting at and ending at

7. Illustration of HOM

8. Homotopy Optimization using Perturbations & Ensembles (HOPE) • Improvements over HOM • Produce ensembles of local minimizers of by perturbing intermediate results • Increase likelihood of predicting a global minimizer • Algorithmic considerations • Maximum ensemble size • Determining ensemble members

9. Illustration of HOPEMaximum ensemble size = 2

10. Considerations using HOPE • Template function • Known solution, relation to target function • Homotopy function • Continuous, but smoothness may help • Parameters • Maximum ensemble size • Perturbation function/amount of perturbation • Number of perturbed versions produced • Number of steps taken in • Metric for choosing ensemble members

11. Convergence of HOPE • HOPE parameterized as existing methods • Probability-one homotopy methods for NLPs • Stochastic search methods • Pure Adaptive Search/Improving Hit-and-Run • Simulated Annealing • HOPE converges in probability • Smooth homotopy: • Closed form for probability • Conditions for convergence with probability one • Extension to general homotopy maps • One spatial dimension, • Basins of attraction must be known

12. Outline • Unconstrained Minimization Problem • Homotopy Optimization Methods • Numerical Experiments • Protein Structure Prediction Problem • Numerical Experiments • Concluding Remarks

13. Numerical Experiments • Test Problems • N-modal sine function: • Moré, Garbow, Hillstrom test functions • 5 functions where local methods failed • Pintér test function • Random multimodal functions, unique minimizer known • Standard homotopy functions • Convex, probability-one • Comparisons with local and stochastic search • More computation using HOM/HOPE • Better results and ensembles of local minimizers

14. Outline • Unconstrained Minimization Problem • Homotopy Optimization Methods • Numerical Experiments • Protein Structure Prediction Problem • Numerical Experiments • Concluding Remarks

15. Protein Structure Ala Arg Asp Asn Arg H O H N C C R Given the amino acid sequence of a protein (1D), is it possible to predict its native structure (3D)? Protein Structure Prediction Amino Acid Sequence

16. Protein Structure Prediction:Computational Methods • Molecular dynamics • Langevin dynamics approximated using stochastic differential equation • Bioinformatics • Sequence/structure matches to experimentally determined native structures • Comparative modeling (threading, homology modeling) • Energy Minimization • Find lowest energy conformation • Native structure [Anfinsen, 1973]

17. Some Existing Energy Minimization Methods • Local methods • Truncated Newton and quasi-Newton methods • Memory efficient (second derivatives not stored) • Global methods • Stochastic search, simulated annealing,evolutionary algorithms, and smoothing methods • Other methods • Convex global underestimation • Stochastic tunneling • Packet annealing • Derivative-free pattern search

18. Energy Minimization using HOM • Goal • Minimize energy function of target protein • Steps to solution • Energy of template protein: • Define a homotopy function: • Deforms template protein into target protein • Produce sequence of minimizers of starting at and ending at

19. Energy Minimization using HOPE • Extensions of HOM • Perturbations • Specific to protein structure • Bond length, bond angle, and particle perturbations • Ensembles • Ensembles chosen using homotopy function value • Benefits over existing minimization methods • Take advantage of sequence-related structural properties of template and target proteins

20. Outline • Problem and Existing Methods • Homotopy Optimization Methods • Numerical Experiments • Protein Structure Prediction Problem • Numerical Experiments • Concluding Remarks

21. Backbone Model: Particle Properties • Backbone model • Single chain of particles with residue attributes • Particles model C atoms in proteins • Properties of particles • Hydrophobic, Hydrophilic, Neutral • Diverse hydrophobic-hydrophobic interactions [Veitshans, et al., 1996.]

22. Backbone Model: Energy Function

23. Backbone Model: Energy Function

24. Loop Region ABCDE F GH I HydrophobicHydrophilic Neutral Backbone Model: Experiments 9 chains (22 particles) with known structure

25. Backbone Model: Experiments • 62 template-target pairs • 10 pairs had identical native structures • Methods • HOM vs. Newton’s method w/trust region (N-TR) • HOPE vs. ensemble-based simulated annealing (SA) • Different ensemble sizes (2,4,8,16) • Averaged over 10 runs • Perturbations where sequences differ • Measuring success • Structural overlap function: • Percentage of interparticle distances off by more than 20% of the average bond length ( ) • Root mean-squared deviation (RMSD)

26. Backbone Model: Results

27. Backbone Model: Results Success of HOPE and SA with ensembles of size 16 for each template-target pair. The size of each circle represents the percentage of successful predictions over the 10 runs. HOPE SA

28. Outline • Problem and Existing Methods • Homotopy Optimization Methods • Numerical Experiments • Protein Structure Prediction Problem • Numerical Experiments • Concluding Remarks

29. Conclusion • New homotopy optimization methods • HOM: sequence of minimizers vs. path tracing • HOPE: perturbations and ensembles • Convergence (cast as existing methods) • Numerical Experiments • HOM/HOPE outperform several standard methods • Standard test functions • Standard homotopy functions used • Protein structure prediction • Problem-specific homotopy functions • Take advantage of sequence-related protein properties

30. Future Directions • Protein structure prediction • More realistic energy functions (AMBER, CHARMM) • Protein Data Bank (templates) • Different size chains for template/target • HOPE for large-scale problems • Inherently parallelizable • Communication: initializing ensembles at each step • HOPE for other optimization problems • Constrained problems • HOPE for other applications

31. Thank You Daniel M. Dunlavy – HOPE dmdunla@sandia.gov Publications • D.M. Dunlavy, D.P. O'Leary, D. Klimov, and D. Thirumalai HOPE: A Homotopy Optimization Method for Protein Structure PredictionJ. Comput. Biol., 12(10):1275-1288. Dec. 2005 • D. M. Dunlavy and D.P. O'LearyHomotopy Optimization Methods for Global OptimizationSandia National Laboratories, SAND2005-7495. Dec. 2005 • D.M. DunlavyHomotopy Optimization Methods and Protein Structure PredictionPh.D. Thesis, University of Maryland, College Park, Aug. 2005 • D.M. DunlavyGlobal Optimization of a Simplified Protein Energy ModelIn preparation