The following are technical specifications for Cardinal.
- Number of Nodes
-
378 nodes
- Number of CPU Sockets
-
756 (2 sockets/node for all nodes)
- Number of CPU Cores
-
39,312
- Cores Per Node
-
104 cores/node for all nodes (96 usable)
- Local Disk Space Per Node
-
- 1.6 TB for compute nodes
- 12.8 TB for GPU and Large mem nodes
- 3.2 TB for login nodes
- Compute, Large Mem & Login Node CPU Specifications
- Intel Xeon CPU Max 9470 HBM2e (Sapphire Rapids)
- 2.0 GHz
- 52 cores per processor (48 usable)
- GPU Node CPU Specifications
- Intel Xeon Platinum 8470 (Sapphire Rapids)
- 2.0 GHz
- 52 cores per processor
- Server Specifications
-
- 326 Dell PowerEdge C6620
- 32 Dell PowerEdge XE9640 (GPU nodes)
- 20 Dell PowerEdge R660 (largemem & login nodes)
- Accelerator Specifications
-
NVIDIA H100 (Hopper) GPUs each with 96 GB HBM2e memory and NVIDIA NVLINK
- Number of Accelerator Nodes
-
32 quad GPU nodes (4 GPUs per node)
- Total Memory
-
~281 TB (44 TB HBM, 237 TB DDR5)
- Memory Per Node
-
- 128 GB HBM / 512 GB DDR5 (compute nodes)
- 1 TB (GPU nodes)
- 128 GB HBM / 2 TB DDR5 (large mem nodes)
- 128 GB HBM / 1 TB DDR5 (login nodes)
- Memory Per Core
-
- 1.2 GB HBM / 4.9 GB DDR5 (compute nodes)
- 9.8 GB (GPU nodes)
- 1.2 GB HBM / 19.7 GB DDR5 (large mem nodes)
- 1.2 GB HBM / 9.8 GB DDR5 (login nodes)
- Interconnect
-
- NDR200 Infiniband (200 Gbps) (compute, large mem, login nodes)
- 4x NDR400 Infiniband (400 Gbps x 4) with GPUDirect, allowing non-blocking communication between up to 10 nodes (GPU nodes)
Service:
