Geant4 Physics Validation

1. Geant4 Physics Validation K. Amako, S. Guatelli, V. Ivanchenko, M. Maire, B. Mascialino, K. Murakami, P. Nieminen, L. Pandola, S. Parlati, M.G. Pia, A. Ribon, T. Sasaki, L. Urban et al. Geant4 Space User Workshop Leuven, 5-7 October 2005

2. Geant4 Physics Models • Ample variety of physics models in the Geant4 Toolkit • complementary and alternative • Electromagnetic physics • Standard, LowEnergy, Muon, Optical • Hadronic physics • data driven, parameterised and theory driven models • Geant4 Physics Book • on-going project to document the performance of Geant4 physics against experimental data and in relevant experimental application domains

3. Validation process • Geant4 test process • Physics packages are subject to unit and system testing • Verification, validation of single processes/models performed by Working Groups • Validation process • systematic: cover all models of a given process • comparison to experimental data and established reference databases • rigorous software process to guarantee quality and reliability • statistical analysis • Goals • evaluate quantitatively the accuracy Geant4 physics models • document their respective strength • provide guidance to users to select the models to use in their applications based on sound, objective ground

4. G.A.P Cirrone, S. Donadio, S. Guatelli, A. Mantero, B. Mascialino, S. Parlati, M.G. Pia, A. Pfeiffer, A. Ribon, P. Viarengo “A Goodness-of-Fit Statistical Toolkit” IEEE- Transactions on Nuclear Science (2004), 51 (5): 2056-2063. Partly funded by ESA (SEPTIMESS Project)

5. GoF algorithms (currently implemented) • Algorithms for binned distributions • Anderson-Darling test • Chi-squared test • Fisz-Cramer-von Mises test • Tiku test(Cramer-von Mises test in chi-squared approximation) • Algorithms for unbinned distributions • Anderson-Darling test • Cramer-von Mises test • Goodman test(Kolmogorov-Smirnov test in chi-squared approximation) • Kolmogorov-Smirnov test • Kuiper test • Tiku test(Cramer-von Mises test in chi-squared approximation)

6. Overview of recent activities • Geant4 Physics Book: Electromagnetic Volume • comparison against the NIST databases • K. Amako, S. Guatelli, V. N. Ivanchenko, M. Maire, B. Mascialino, K. Murakami, P. Nieminen, L. Pandola, S. Parlati, M. G. Pia, M. Piergentili, T. Sasaki, L. UrbanComparison of Geant4 electromagnetic physics models against the NIST reference dataIEEE Trans. Nucl. Sci., Vol. 52, Issue 4, Aug. 2005, 910-918 • Current Physics Book projects (preliminary results) • Bremsstrahlung final state • Atomic relaxation and PIXE • Bragg peak • Radioactivity from rocks and sands • LCG Simulation Validation Project • focused on hadronic physics • see A. Ribons’s talk at EPS-HEP, Lisbon, July 2005

7. NIST Test • Photon Mass Attenuation Coefficient • Photon Partial Interaction Coefficient • related to the cross section of a specific photon interaction process • Electron CSDA range and Stopping Power • Proton CSDA range and Stopping Power • aCSDA range and Stopping Power Elements Be, Al, Si, Fe, Ge, Ag, Cs, Au, Pb, U (span the periodic element table) Geant4 models: electrons and photons Standard Low Energy EEDL/EPDL Low Energy Penelope Energy range photon 1 keV – 100 GeV electron 10 keV – 1 GeV proton 1 keV – 10 GeV a1 keV – 1 GeV Geant4 models: protons and a Standard Low Energy ICRU49 Low Energy Ziegler 1977 Low Energy Ziegler 1985 Low Energy Ziegler 2000 (Low Energy: free electron gas + parameterisations + Bethe-Bloch) Simulation configuration reproducing NIST conditions (ionisation potential, fluctuations, production of secondaries etc.)

8. Statistical analysis Goodness-of-Fit test (Statistical Toolkit) H0: Geant4 simulation = NIST data H1: Geant4 simulation ≠ NIST data Alternative hypotheses under test: Distance between Geant4 simulation and NIST reference data GoF test (χ2 test) Test result p-value GoF Toolkit The p-value represents the probability that the test statistics has a valueat leastas extreme as the one observed, assuming the null hypothesis is true 0 ≤ p ≤ 1 p < 0.05Geant4 simulation and NIST datadiffer significantly p > 0.05Geant4 simulation and NIST datado not differ significantly

9. Transmitted photons (I) Monochromatic photon beam (Io) Mass attenuation coefficient in Fe Geant4 LowE Penelope Geant4 Standard Geant4 LowE EPDL NIST - XCOM H0 REJECTION AREA Photon mass attenuation coefficient • Geant4 models: • Standard • Low Energy – EPDL • Low Energy – Penelope • Reference data: NIST - XCOM Experimental set-up Results All Geant4 models compatible with NIST Best agreement: Geant4 LowE models p-value stability study

10. p-value stability study Compton interaction coefficient in Ag Geant4 LowE Penelope Geant4 Standard Geant4 LowE EPDL NIST - XCOM H0 REJECTION AREA Compton interaction coefficient (cross section) • Geant4 models: • Standard • Low Energy – EPDL • Low Energy – Penelope • Reference data: NIST - XCOM Results All Geant4 models compatible with NIST Best agreement: Geant4 LowE-EPDL

11. p-value stability study Photoelectric interaction coefficient in Ge Geant4 LowE Penelope Geant4 Standard Geant4 LowE EPDL NIST - XCOM Geant4 LowE Penelope Geant4 Standard Geant4 LowE EPDL NIST - XCOM H0 REJECTION AREA Photoelectric interaction coefficient (cross section) • Geant4 models: • Standard • Low Energy – EPDL • Low Energy – Penelope • Reference data: NIST - XCOM Results All Geant4 models compatible with NIST Best agreement: Geant4 LowE models

12. Pair production interaction coefficient in Au p-value stability study Geant4 LowE Penelope Geant4 Standard Geant4 LowE EPDL NIST - XCOM p-value (pair production interaction coefficient test) H0 REJECTION AREA Pair production interaction coefficient (cross section) • Geant4 models: • Standard • Low Energy – EPDL • Low Energy – Penelope • Reference data: NIST - XCOM Results All Geant4 models compatible with NIST

13. Geant4 LowE Penelope Geant4 LowE EPDL NIST - XCOM Rayleigh interaction coefficient (cross section) Results The Geant4 Low Energy models look in disagreement with the reference data for some materials • Geant4 models: • Low Energy – EPDL • Low Energy – Penelope • Reference data: NIST - XCOM EPDL XCOM Penelope XCOM Rayleigh interaction coefficient in Be

14. Rayleigh interaction coefficient in Au NIST EPDL 97 Rayleigh interaction coefficient The disagreement is evident between 1 keV and 1 MeV photon energies For what concerns the Geant4 Low Energy EPDL model, the effect observed derives from an intrinsic inconsistency between Rayleigh cross section data in NIST-XCOM and the cross sections of EPDL97, on which the model is based Differences between EPDL97 and NIST-XCOM have already been highlighted in a paper by Zaidi, which recommends the Livermore photon and electron data libraries as the most up-to-date and accurate databases available for Monte Carlo modeling. Zaidi H., 2000, Comparative evaluation of photon cross section libraries for materials of interest in PET Monte Carlo simulation IEEE Transaction on Nuclear Science 47 2722-35

15. p-value stability study Geant4 LowE Penelope Geant4 Standard Geant4 LowE Livermore NIST - ESTAR H0 REJECTION AREA Experimental set-up Electron Stopping Power Electrons are generated with random direction at the center of the box and stop inside the box • Geant4 models: • Standard • Low Energy – EEDL • Low Energy – Penelope • Reference data: NIST – ESTAR (ICRU 37) Maximum step allowed in tracking particles was set about1/10 of the expected range value, to ensure the accuracy of the calculation Results All Geant4 models compatible with NIST and equivalent

16. CSDA range in U Geant4 LowE Penelope Geant4 Standard Geant4 LowE Livermore NIST - ESTAR CSDA: particle range without energy loss fluctuations and multiple scattering Electron CSDA Range • Geant4 models: • Standard • Low Energy – EEDL • Low Energy – Penelope • Reference data: NIST – ESTAR (ICRU 37) Results All Geant4 models compatible with NIST and equivalent p-value stability study H0 REJECTION AREA

17. Stopping power in Al Geant4 LowE Ziegler 1985 Geant4 LowE Ziegler 2000 Geant4 Standard Geant4 LowE ICRU 49 NIST - PSTAR + Proton stopping power - range Stopping power: p-value stability study H0 REJECTION AREA CSDA range: p-value stability study Results Ziegler parameterisations are as authoritative as ICRU 49 ones Comparison rather than validation H0 REJECTION AREA

18. CSDA range in Si Stopping power: p-value stability study Geant4 LowE Ziegler 1977 Geant4 Standard Geant4 LowE ICRU 49 NIST - ASTAR H0 REJECTION AREA a stopping power and range The complex physics modeling of ion interactions in the low energy range is addressed by the Geant4 Low Energy package and it represented one of the main motivations for developing this package The complex physics modeling of ion interactions in the low energy range is addressed by the Geant4 Low Energy package and it represented one of the main motivations for the developing of this package.

19. 3 sets of models: Standard: G4eBremsstrahlung Low Energy EPDL: G4LowEnergyBremsstrahlung Low Energy Penelope: G4PenelopeBremsstrahlung Bremsstrahlung 3 angular distributions:Tsai, 2BS, 2BN Penelope Standard Low Energy (default) Penelope TSAI (def) 2BS 2BN Angle (deg) Angle (deg) Angular distribution of photons is strongly model-dependent

20. Reference data Transmitted energy spectrum at two different emission angles for four materials (Al, Pb, W, Ag) Absolute yields are reported (= photons/primary), though with an “odd” normalization The absolute Bremsstrahlung cross section can be tested R. Ambrose et al., Nucl. Instr. Meth. B 56/57 (1991) 327

21. Relative comparison... Preliminary Low E EEDL - TSAI LowE-Penelope Intensity/Z (eV/sr keV) Intensity/Z (eV/sr keV) Photon energy (keV) Photon energy (keV) Relative comparison (45 deg direction) Shapes of the spectra are in good agreement Work in progress, will be published

22. Proton Bragg Peak Reference data from CATANA (INFN-LNS Hadrontherapy Group) Geant4 models: electromagnetic • Standard • Low Energy ICRU 49 • Low Energy Ziegler 1977 • Low Energy Ziegler 1985 • Low Energy Ziegler 2000 Geant4 models: hadronic • Precompound + default de-excitation • Precompound + GEM evaporation • with/without Fermi Break-up • Binary Cascade • (including Precompound + de-excitation) • Bertini Cascade • Parameterised Systematic test in progress Lot of work… Preliminary results

23. EM only – ICRU49 Preliminary

24. EM only – ICRU49 – GoF results Preliminary

25. EM only – Standard Preliminary

26. LowE + precompound default Preliminary

27. ICRU49 + precompound – GoF results Preliminary ICRU 49 only

28. Nuclear de-excitation alternative models Preliminary Work in progress, more to come…

29. simulation simulation sample detector data data source Radioactive spectrum Studies of environmental radioactivity from rocks and sands at the Gran Sasso Laboratory Geant4 (LowE EM) can reproduce very well the results of a calibration with a 60Co source (in the presence of the sample) Lower part of the histogram is not meaningful

30. Statistical analysis • Anderson-Darling test (for binned data) • First peak • A2 = 0.45 • p-value = 0.80 NO DIFFERENCE • Second peak • A2 = 1.05 • p-value = 0.33 NO DIFFERENCE • Both peaks • A2 = 0.80 • p-value = 0.48 NO DIFFERENCE

31. Conclusion • More results available, no time to show them all… • Systematic, quantitative validation of Geant4 physics in progress • all available models • rigorous statistical analysis • A lot of work! • first paper published • several on-going projects • limited resources available • The validation work provides valauable feedback for the improvement of Geant4 physics models