Ab-initio Calculations of Microscopic Structure of Nuclei

1. Ab-initio Calculations ofMicroscopic Structure of Nuclei James P. Vary, Iowa State University Esmond G. Ng, Berkeley Lab Masha Sosonkina, Ames Lab April 17, 2008

2. Collaborators Nuclear Physics Iowa State University James Vary Pieter Maris (postdoc) Alina Negoita (grad student) Andrey Shirokov (international collaborator) Applied Math/CS Lawrence Berkeley National Laboratory Esmond G. Ng Chao Yang Philip Sternberg (postdoc) Ames Laboratory Masha Sosonkina Hung Viet Le (postdoc) Anurag Sharda (grad student)

3. MFDn • What is the science? • Determine the microscopic structure of nuclei with NN + NNN interactions using the “ab-initio” no-core shell model and full configuration interaction methods. • What does MFDn do and what are its kernels? • Evaluates the many-nucleon Hamiltonian in an m-scheme harmonic oscillator basis that eliminates spurious center of mass motion effects. • Solves an eigenvalue problem to obtain the low-lying spectra and wavefunctions, then evaluates a suite of 1-body and 2-body observables to compare with experiment. • MFDn has external field capabilities, and, when it is run in tandem with Navratil's TRDENS code, one obtains 1-body, 2-body, etc., density matrices for comparison with energy-density functionals.

4. MFDn • What are the major bottlenecks? • Size of eigenvalue problems. • 16O: matrix dimension ~ 109; # of nonzero entries ~ 21012. • Storage restrictions. • Speed of execution - including per processor performance (inner loops). • I/O burden with NNN interaction files. • Future directions in improving numerical convergence. • Limitations of the harmonic oscillator basis. • need more general basis spaces, resulting in more compute-intensive inner loops for generating matrix elements, which requires intelligent way of exploiting matrix kernels. • m-scheme inefficiencies. • need coupled J basis capability, resulting in denser matrices.

5. Applied Math/Comp Sci Contributions • Eigenvalue Calculations … • Drop tolerance to reduce # of nonzero entries in rank-3 Hamiltonians. • Reduced storage requirement and execution time without significantly affecting accuracies. • 15N: dim ~600,000; 60% reduction in storage; 15 mins vs 28 mins on 55 processors (Thunder) to converge to the 15 smallest eigenvalues. • Integration of an alternative eigensolver. • Incorporated an implicitly restarted Lanczos (PARPACK). • Reduced memory usage since fewer Lanczos vectors have to be stored. • Improved termination criterion. • Improving data structure for storing the Hamiltonian. • Incorporated a standard sparse matrix storage scheme. • Reduced integer and logical operations, at the expense of slight increase in memory usage.

6. Applied Math/Comp Sci Contributions • In generation of Hamiltonian … • Parallelization of the basis generating algorithm. • Development of a hierarchical blocking scheme to compute the structure of the Hamiltonian. • Exploited combinatorial properties of the index set of the basis in determining sparsity structure by identifying zero blocks of Hamiltonian. • Improved the sorting algorithm for the basis of the Hamiltonian. • Potentially improving matrix-vector multiplication kernel in eigensolver (future work). • Reducing I/O burden with NNN interaction files. • Explored approximation of NNN interaction files via singular value decomposition.

7. Preliminary Comparisons - Blocking Strategies fine fine on top of coarse coarse 6He: no blocking: ~ 43 minutes one level: 180 seconds multiple levels: 90 seconds Nonzero blocks Zero blocks Potentially nonzero blocks Potentially nonzero blocks in fine partitioning

8. Applied Math/Comp Sci Contributions • Explorations of “hidden” symmetries for the Hamiltonian in MFDn. • Several algebraic graph reorderings, such as Reverse-Cuthill McKee, have been applied to reduce the matrix bandwidth and search for a (approximate) block diagonal form. • Observed useful block structures, pointing the way to future optimization of matrix-vector multiplications. • Design MFDn interfaces with search programs to automate the basis space optimization especially important for external field applications. • Used the master-worker paradigm of the search code (VTDIRECT95 [Virginia Tech]) and introduced a second tier of workers for expensive function evaluations such as MFDn. • Found that assigning different numbers of processors to different MFDn executions, in accordance with the Hamiltonian size, reduces overall time.

9. Preliminary Comparisons: Before, Intermediate, Present

10. Future Work • Year 2 – • Continue to improve the many-body basis setup and Hamiltonian evaluation phases of MFDn. • Year 3 – • Improve I/O performance and reducing memory requirements in MFDn. • Year 4 – • Improve performance of eigenvalue calculations and overall performance of MFDn, and integrate parallel search algorithms with MFDn. • Years 3-5 – • Enhance capability of MFDn, such as alternative scalable eigenvalue solvers, algorithmic advances to enable scattering applications and coupled J basis.

11. Deliverables • Codes • Integrate improved/new math/CS tools into MFDn. • Explore opportunities to incorporate such tools into other ab-initio codes. • Talks/Papers • P. Sternberg, C. Yang, E.G. Ng, J.P. Vary, P. Maris, “High Performance Nuclear Structure Computation”. SIAM Conference on Parallel Processing for Scientific Computing. Atlanta (March 12-14, 2008). • P. Sternberg, C. Yang, E.G. Ng, P. Maris, J.P. Vary, “High Performance Nuclear Structure Calculations”. The 9th International Workshop on State-of-the-Art in Scientific and Parallel Computing. Trondheim, Norway (May 13-16, 2008). • M. Sosonkina, A. Sharda, A. Negoita, J.P. Vary, “Integration of Ab Initio Nuclear Physics Calculations with Optimization Techniques”. International Conference on Computational Science. Kraków, Poland (June 23-25, 2008). • P. Sternberg, P. Maris, E.G. Ng, M. Sosonkina, H.V. Le, J.P. Vary, C. Yang, “Accelerating Full Configuration Interaction Calculations for Nuclear Structure”. SC08, Austin (November 15-21, 2008). Submitted. • Possible SIAM SISC paper. Under preparation.