64 node HGX B300 AI cluster, 512 GPUs
Sixty four HGX B300 servers with 512 GPUs, a non blocking 800 Gb/s fabric, CPU tier, six control plane nodes and 512 TB of flash, as one AI cluster.
GPU: NVIDIA HGX B300
- GPU nodes
- 64 x 8 GPU HGX B300 servers, 512 x B300, 134.4 TB HBM3e
- Fabric
- 512 x 800 Gb/s ports, leaf and spine, non blocking, rail optimized
- CPU tier
- 8 x dual socket EPYC 9005 nodes, 3,072 threads and 12 TB memory, 6 threads and 24 GB per GPU
- Control plane
- 6 x 1U single socket EPYC 9005 control plane servers
- Storage
- 512 TB usable all flash, 1 TB per GPU, 8 x NVMe storage nodes under a parallel file system
- Facility
- About 950 kW total, 22 to 24 racks air cooled or 10 to 12 racks liquid cooled
Overview
The 64 node cluster is a complete AI training and inference platform with 512 B300 GPUs and 134.4 TB of HBM3e, engineered as one system from the GPU nodes down to the out of band network. It is the scale at which fabric topology, storage throughput and facility power have to be designed together, and we deliver it with that design in hand.
Our sizing rules produce the supporting tiers: six threads and 24 GB per GPU gives a CPU tier of eight dual socket EPYC 9005 nodes with 3,072 threads and 12 TB; six control plane nodes run the scheduler quorum, fabric manager, monitoring and provisioning; and one terabyte of usable flash per GPU gives 512 TB across eight NVMe storage nodes under a parallel file system with GPUDirect Storage.
The fabric is rail optimized and non blocking with 512 ports at 800 Gb/s. Plan for about 950 kW, in 22 to 24 air cooled racks or 10 to 12 liquid cooled racks. We rack, cable, validate and document the cluster, deliver it in phases to your data center or colocation facility, and offer on site installation and commissioning so that the cluster is in production on the date your site is ready.
Use cases
- Frontier scale model training
- AI lab and foundation model infrastructure
- Sovereign and national AI compute
- Large scale inference for AI products
Specifications
| GPU nodes | 64 x 8 GPU HGX B300 AI server, air cooled 10U or liquid cooled 4U |
|---|---|
| GPUs | 512 x NVIDIA B300, 288 GB HBM3e each, 134.4 TB total (64 x 2.1 TB per node) |
| CPU tier | 8 x 1U dual socket EPYC 9005 compute servers, 384 threads and 1.5 TB each, 3,072 threads and 12 TB total |
| Control plane | 6 x 1U single socket EPYC 9005 control plane servers: scheduler quorum, fabric manager, monitoring, provisioning |
| Intel option | GPU node hosts, CPU tier and control plane nodes can be specified on Intel Xeon 6 instead of AMD EPYC 9005 |
| East west | 512 x 800 Gb/s, rail optimized leaf and spine, non blocking, InfiniBand XDR on Quantum-X800 class or 800 GbE on Spectrum-X class switches |
|---|---|
| Switch count | 8 x 144 port class leaf and 4 to 8 spine switches, or equivalent 64 port class Ethernet design |
| North south | 128 x 400 Gb/s BlueField-3 ports into a 400 GbE leaf and spine |
| Management | 1 GbE out of band network for every BMC, switch and PDU |
| Cabling | 800G OSFP transceivers and cables, structured per rail |
| Capacity | 512 TB usable minimum, 1 TB per GPU, expandable with 4U HDD capacity tier |
|---|---|
| Nodes | 8 x 2U 24 bay all NVMe storage servers with 2 x 400 Gb/s each, erasure coded |
| Software | Parallel file system with GPUDirect Storage, specified with the order |
| Racks | Air cooled: 22 to 24 racks at 3 GPU nodes per rack; liquid cooled: 10 to 12 racks at 6 to 8 GPU nodes per rack, plus storage, CPU and network racks |
|---|---|
| Planning power | About 950 kW total: 64 x 14 kW GPU nodes, 8 x 1 kW CPU, 6 x 0.5 kW control, 8 x 1.3 kW storage, switches and overhead |
| Cooling | Air with hot aisle containment, or direct liquid cooling with CDUs; facility water loop required for liquid |
| Standard | 64 x HGX B300 nodes, 512 GPUs, air or liquid cooled, deliverable now |
|---|---|
| Alternative platform | 7 x GB300 NVL72 racks, 504 GPUs in seven 72 GPU NVLink domains, liquid cooled hall required |
| Rubin nodes | HGX Rubin NVL8 nodes can replace or extend B300 nodes on the same fabric design when they ship (coming soon) |
| Rubin racks | Vera Rubin NVL72 racks as the rack scale successor to GB300 NVL72 when they ship (coming soon) |
Figures describe the system class. Exact configuration, weights and power draw are confirmed on the quotation.
What's included
- Rack elevation, cabling plan and power schedule
- Nodes, switches, cables and optics as one bill of materials
- Burn in of every node and fabric validation across the cluster, acceptance report
- NVIDIA end user registration handled with the order
- Deployment support and optional on site installation
- 3 year warranty, options to 5 years with advance replacement
- Fabric topology design and storage throughput model
Compliance
Related
Related systems
16 node HGX B300 AI pod, 128 GPUs
Allocation required- GPU nodes
- 16 x 8 GPU HGX B300 servers, 128 x B300
- Fabric
- 128 x 800 Gb/s ports, leaf and spine
- CPU and control
- 3 x CPU nodes for data services
- Chassis
- Multi rack, about 245 kW, air or liquid cooled
NVIDIA GB300 NVL72 rack
Allocation required- GPUs
- 72 x NVIDIA Blackwell Ultra, 288 GB HBM3e each
- CPUs
- 36 x NVIDIA Grace, 2,592 Arm Neoverse V2 cores
- NVLink
- 130 TB/s total NVLink bandwidth across all 72 GPUs
- Chassis
- Single rack, power not yet published, liquid cooled
1U dual socket EPYC 9005 server
- CPU
- 2 x AMD EPYC 9005 series
- Memory
- 24 x DDR5 RDIMM, 12 channels per socket
- Expansion
- 2 x PCIe Gen5 x16 slots, OCP 3.0 mezzanine
- Chassis
- 1U, about 1 kW, air cooled
2U 24 bay all NVMe storage server
- CPU
- 2 x AMD EPYC 9005 or 2 x Intel Xeon 6
- Memory
- 24 x DDR5 RDIMM, 512 GB to 1.5 TB
- Software
- Parallel file systems, NVMe over Fabrics
- Chassis
- 2U, about 1.3 kW, air cooled
Related guides
Air cooled vs liquid cooled AI servers
How to decide between air and direct liquid cooling for GPU servers, from the rack power limit up to NVL72 racks that only come liquid cooled.
In this guide
- The question is the rack, not the server
- How air cooling works at high density
Buying data center GPUs, export and registration
The compliance steps that come with every HGX, H200 NVL, Server Edition and NVL72 purchase, explained so your order ships on time.
In this guide
- Why a GPU purchase comes with paperwork
- Which products are covered
Choosing a GPU server for LLM inference
A practical method for sizing an inference server from the model, the quantization and the concurrency you expect, with worked examples.
In this guide
- Start with the model, not the server
- Step 1: how much GPU memory the weights need
HGX vs PCIe GPU servers
The real differences between an HGX node and a PCIe GPU server, and a simple rule for choosing between them.
In this guide
- Two kinds of 8 GPU server
- The interconnect is the point
Compare it with the next class up or down
The nearest classes we quote are the 16 node HGX B300 AI pod, 128 GPUs and the NVIDIA GB300 NVL72 rack. If neither is quite right, tell us the workload and we specify the system between them.
- (858) 613-1818
- sales@computermodules.com
- Monday to Friday, 8am to 5pm Pacific