1 / 9

NCSA Environment Overview and the “Abe” Dell Resource

NCSA Environment Overview and the “Abe” Dell Resource. John Towns Director, Persistent Infrastructure NCSA jtowns@ncsa.edu. The short version of the story…. Compute resources: 6 compute systems, 146.5 TF total TOP500 entries in top 100: #8, Abe @ 62.68TF R max #47, T3 @ 16.68TF R max

ella
Télécharger la présentation

NCSA Environment Overview and the “Abe” Dell Resource

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NCSA Environment Overview and the “Abe” Dell Resource John Towns Director, Persistent Infrastructure NCSA jtowns@ncsa.edu National Center for Supercomputing Applications

  2. The short version of the story… • Compute resources: • 6 compute systems, 146.5 TF total • TOP500 entries in top 100: • #8, Abe @ 62.68TF Rmax • #47, T3 @ 16.68TF Rmax • #90, Tungsten @ 9.82 TF Rmax • Data Resources: • Archival: 5PB capacity • Disk: 1PB rotating disk on site • Databases: SMP and Oracle RAC cluster • Visualization Resources: • 30M pixel tiled display • Prism cluster National Center for Supercomputing Applications

  3. Compute Systems Strategy • ISV applications and low level parallel (up to ~32-way) codes • supported on copper and migrating to cobalt • Large shared memory codes • supported on cobalt; 4TB memory • Moderately parallel codes: O(100)-way parallel • supported on tungsten and mercury • Path to petascale: highly parallel codes, O(1,000)-way parallel • supported on abe National Center for Supercomputing Applications

  4. Major NCSA Allocated Systems Today • Distributed Memory Clusters • Abe (Dell, 2.33 GHz Xeon Quad-Core): • 1,200 blades/9,600 cores • 89.5 TF; 9.6 TB RAM; 120 TB disk • Tungsten (Dell, 3.2 GHz Xeon): • 2,950 processors • 16.4 TF; 4.4 TB RAM; 140 TB disk • Mercury (IBM, 1.3/1.5 GHz Itanium2): • 1,846 processors • 10 TF; 4.6 TB RAM; 90 TB disk • Shared Memory Clusters • Copper (IBM p690,1.3 GHz Power4): 12 x 32 processors • 2 TF; 64 or 256 GB RAM each; 35 TB disk • Cobalt (SGI Altix, 1.5 GHz Itanium2): 2 x 512 processors • 6.6 TF; 1 TB or 3 TB RAM; 250 TB disk National Center for Supercomputing Applications

  5. Major NCSA non-Allocated Systems • Tungsten-3 distributed memory cluster • Dell 2.66 GHz Woodcrest (dual core): 1,024 processors/2048 cores • 22 TF; 4.1 TB RAM; 20 TB disk • funded by State of Illinois and Private Sector Partner funds • supports Private Sector Partner projects National Center for Supercomputing Applications

  6. Cyber-resourcesMajor NCSA Storage Systems • Archival: SGI/Unitree (5 PB total capacity) • 72TB disk cache; 50 tape drives • currently 2.8PB of data in MSS • >1PB ingested in last 6 months • project ~3.2PB by end of CY2006 • licensed to support 5PB resident data • ~30 data collections hosted • Infrastructure: 394TB Fiberchannel SAN connected • Fiberchannel SAN connected; FC and SATA environments • Lustre, IBRIX, NFS filesystems • Databases: • 8 processor 12GB memory SGI Altix • 30TB of SAN storage • Oracle 10G, mysql, Postgres • Oracle RAC cluster • Single-system Oracle deployments for focused projects National Center for Supercomputing Applications

  7. Cyber-resourcesMajor Visualization Resources • 30M-pixel Tiled Display Wall • 8192 x 3840 pixels composite display • 40 NEC VT540 projectors, arranged in a 5H x 8W matrix • driven by 40-node Linux cluster • dual-processor 2.4GHz Intel Xeons with NVIDIA FX 5800 Ultra graphics accelerator cards • Myrinet interconnect • to be upgrade by early CY2007 • funded by State of Illinois • SGI Prisms • 8 x 8 processor (1.6 GHz Itanium2) • 4 graphics pipes each; 1 GB RAM each • InfiniBand connection to Altix machines National Center for Supercomputing Applications

  8. NCSA Facility - ACB • Advanced Computation Building • Three rooms, totals: • 16,400 sqft raised floor • 4.5 MW power capacity • 250 kW UPS • 1,500 tons cooling capacity • Room 200: • 7,000 sqft – no columns • 70” raised floor • 2.3 MW power capacity • 750 tons cooling capacity National Center for Supercomputing Applications

  9. Abe: 1955 blade cluster 2.33 GHz Cloverton Quad-Core 1,200 blades/9,600 cores 89.5 TF; 9.6 TB RAM; 120 TB disk Perceus management; diskless boot Cisco Infiniband 2 to 1 oversubscribed OFED-1.1 w/ HPSM subnet manager Lustre over IB 22 OSTs 2 9500 DDN controllers direct FC 10 FasT controllers on SAN fabric 8.4GB/s sustained 22 OSTs and 2 MDS w/complete auto failovers Power/Cooling 500 KW / 140 tons Use Model/Policy intended for highly parallel applications jobs at less than 1000-way parallel will be strongly disfavored by queuing system block reservations of system encouraged large fractions of the machine for days to weeks NCSA’s 4th Dell Cluster The largest Dell cluster!!! Highest ranked Intel system in TOP500!!! National Center for Supercomputing Applications

More Related