Software Development Infrastructure for the FAIR Experiments

1. Software Development Infrastructure for the FAIR Experiments Florian Uhlig (GSI-IT/CBM) Mohammad Al-Turany (GSI-IT) Denis Bertini (GSI-IT/R3B) Radek Karabowicz (GSI-IT/Panda)

2. Facility for Antiproton and Ion Research First beam planned in 2018 See plenary talk of Peter Malzacher (PL-09) CHEP 2010, Taipei

3. Experiments using FairRoot CBM Heavy Ion Physics PANDA Hadron Physics R3B Nuclear Structure Astrophysics MPD Heavy Ion Physics NICA@Dubna • FairRoot, a simulation and analysis framework, • is the base of the experiment specific software • Started as Framework for CBM • Other experiments followed • Provide base functionality and Software • Development Tools CHEP 2010, Taipei

4. User User View Version Control System CHEP 2010, Taipei

5. First Version Control System used was CVS Several drawbacks Losing file history during move or copy. No consistent history. Changes belonging to each other may have different revision numbers. No binary diff No atomic commits Natural choice for next Version Control System was SVN SVN support all required features (at least now) SVN is further development of CVS Easy conversion due to good converter Similar command line interface than CVS No problems for users to move from CVS to SVN Distributed VCS much more complex than SVN In my opinion to complicated for normal user Version Control System CHEP 2010, Taipei

6. Use of separate git repository for experienced users possible Fetch a copy of the SVN repository git svn clone <svn_repository_url> Host it somewhere (gitorious.org, ssh server) Create a working branch git branch <branchname> git checkout <branchname> Do your development and then merge the new branch with the master branch git commit (many times) git checkout master && git svn rebase git merge --squash <branchname> Commit to SVN repository git svn dcommit Distributed Version Control System? CHEP 2010, Taipei

7. Version Control System User User View Build & Test System CHEP 2010, Taipei

8. Started with handmade Makefiles Good in the beginning, but become hard to maintain. Not one Makefile but many files at different places which are included. Only one or (two?) person with expert knowledge. No support for automatic testing Next step to Autotools State of the art on *ix systems (at that time?) Easy for users (at least in theory) (./configure && make && make install) Complete tool chain of several programs, each with different "macro" syntax (quotes) (autcoconf, automake, libtool, m4, sh, make) Creates big build scripts and helper files even for a hello world example Hard to extend to non standard features e.g. creating the ROOT dictionaries Hard to understand error messages Build and Test System CHEP 2010, Taipei

9. Features Useful error messages Simple but powerful macro language Test framework CTest Package generation CPack (rpm, deb, dmg, Windows Installer ....) CMake • Makefiles • make • NMake • ...... • Project files for • Xcode • Eclipse • MS Visual Studio • ...... *ix systems Windows Mac OS CHEP 2010, Taipei

10. Ideal integration with CMake Different test scenarios possible Nightly Experimental Continuous Easy to setup new tests Trivial to setup new sites for Nightly tests A test can be anything which is executable Scripts (shell, perl ..) Executable Root macros Coverage and Memory checks possible Help to automatically identify problems when they occur Direct feedback for developers when they play with new features CTest CHEP 2010, Taipei

11. Continuous Integration Central SVN repository Dedicated test server 3b. Configure, build and test on local machine 2. SVN triggers test server 3a. Update of local copy 4. Send results automatically to central web page 5. Dashboard prepare and display results 1. Developer commit code 6. In case of problems Dashboard sends an E-mail to Developer and Administrator 7. Developer check results CHEP 2010, Taipei

12. Version Control System User User View Build & Test System Software Testing Server CMake CHEP 2010, Taipei

13. CDash • Web server receiving Data from many independent clients • php framework with MySQL database • Display results produced by clients • Output from configuration and build process • Results (passed/failed/timeout), the runtime and in case of • failure the output of the tests • Output of code coverage analysis (gcov) • Graphs to visualize trends • Deviations from mean runtime for tests • E-mail information in case of problems CHEP 2010, Taipei

14. CDash CHEP 2010, Taipei

15. CDash CHEP 2010, Taipei

16. CDash CHEP 2010, Taipei

17. CDash CHEP 2010, Taipei

18. CDash CHEP 2010, Taipei

19. CDash CHEP 2010, Taipei

20. Version Control System User User View Project Management Build & Test System Software Testing Server CMake CHEP 2010, Taipei

21. Looking for single tool only one entry point to access information Should allow easy cross references Revision number -> bug tracker / feature request Bug -> Wiki description Trac SVN Web front-end Ticket system (Bugs, Feature requests) Project Management (Timeline, Roadmap) More functionality via plug-in mechanism (not tested) Discussion Forum Doxygen ........ Alternatives Redmine (will be evaluated in near future) ...... Source Code/Project Management CHEP 2010, Taipei

22. Trac CHEP 2010, Taipei

23. Trac CHEP 2010, Taipei

24. Version Control System User Global View Project Management Build & Test System Software Testing Server RuleChecker CMake Doxygen CHEP 2010, Taipei

25. Aggregated Information CBM: 22 Panda: 55 ALICE: 82 different committers CHEP 2010, Taipei

26. Provide development and monitoring tools for all experiments using FairRoot Low manpower Reuse existing software as much as possible Use standard tools All tools are Open Source Extensible if needed Good support due to large user basis Train the user to use the aggregated information Use aggregated information to define/change best practice advices Summary and Outlook CHEP 2010, Taipei