1. Benchmarking, Performance Evaluation, Modeling and Prediction Erich Strohmaier

2. ICL Benchmarking Activities • Linpack benchmark • ParkBench • LLCbench • The Performance Database Server • TOP500 • Performance Analysis Tool for NAS

3. Linpack Benchmark • 3 main ‘flavors • 100x100 • 1000x1000 • NxN • R max from NxN used in the TOP500

4. ParkBench • Collection of • LowLevel benchmarks • sequential • MPI • PVM • Kernels • Linear Algebra • NAS PB Kernels • Applications • NAS PB (simulated applications) • PSTSWM

5. LLCbench • Low-Level Tests for various hardware aspects: • MPBench • Core MPI contructs • BLASBench • BLAS routines • CacheBench • In and out of cache data movement

6. Performance Database Server • Searchable Interface to DB of benchmark results such as: • Linpack • SPEC • (Dhrystone etc)

7. Basis for analysing the HCP market - Quantification of observations - Detection of trends • market, • architecture, • technology

8. TOP500 Procedure • Listing of the 500 most powerful • Computers in the World • Yardstick: Rmax/LINPACK • Updating halfyearly

10. TOP500 list - Data shown • Manufacturer Manufacturer or vendor • Computer Type indicated by manufacturer or vendor • Inst. Site Customer • Location Location and country • Year Year of installation/last major update • Field of Appl. Academic,Research,Industry,Vendor,Class. • # Proc. Number of processors • Rmax Maxmimal LINPACK performance achieved • Rpeak Theoretical peak performance • Nmax Problemsize for achieving Rmax • N1/2 Problemsize for achieving half of Rmax • Nworld Position within the TOP500 ranking

11. TOP10

12. Performance Development

13. Manufacturer

14. Architectures

15. Performance Development

16. Performance Development

17. Performance Development

18. www.top500.org.

19. NAS Performance Analysis Tool • To provide a tool to NAS users which allows • Access to performance data of NAS reference codes • Performing a variety of predefined performance tests on target systems in an easy and comparable ways • Analyzing user performance measurements using advanced statistical methods • Analyze scalability of his/her application • Compare its efficiency to other codes • Get a first prediction of attainable performance ranges on other NAS systems

20. Necessary Functionality • Broad variety of predefined performance tests • Simple interface do define custom performance tests • Reference measurement of NAS applications • Generation and maintenance of performance database

21. Necessary Functionality • Black Box performance analysis for individual codes • Cross-code and cross-platform performance analysis. • Tool for performance prediction for NAS applications • Detection and analysis of system and code performance signatures

22. System Architecture Test definition database - All predefined performance test with performance models GUI: - Test definition - Test suite setup Test suite setup file Benchmark Skeleton codes Create script - User data entry Test benchmarks Execute script - Result display - Result analysis using various methods Test results - Result commit to include in DB Performance DB Connected by same keys

23. Functions available for non-linar regression GUI - Analysis Component Input of measured data as: #proz time Data are NPB MG Class A on Steger Statistical output of analysis - best model Data and regression display