16 node HGX B300 AI pod, 128 GPUs
Sixteen HGX B300 servers with 128 GPUs on a two tier 800 Gb/s fabric, with CPU, control plane and 128 TB of flash, air or liquid cooled, as one pod.
GPU: NVIDIA HGX B300
- GPU nodes
- 16 x 8 GPU HGX B300 servers, 128 x B300, 33.6 TB HBM3e
- Fabric
- 128 x 800 Gb/s ports, leaf and spine, non blocking, Quantum-X800 or Spectrum-X class
- CPU and control
- 3 x CPU nodes for data services, 6 x control plane nodes
- Storage
- 128 TB usable all flash minimum, 1 TB per GPU, 3 x NVMe storage nodes
- Racks
- 6 to 7 racks air cooled, or 3 to 4 racks liquid cooled, about 245 kW total
- Delivery
- Racked, cabled and fabric validated, with optional on site installation
Overview
The 16 node pod doubles the 8 node design and introduces a two tier leaf and spine fabric, which is the point at which fabric design, cable management and cooling choices start to matter. With 128 B300 GPUs and 33.6 TB of HBM3e it trains models in the few hundred billion parameter class and serves large inference fleets.
The supporting tiers follow the same rules: six threads and 24 GB per GPU across three dual socket CPU nodes, six control plane nodes, and 128 TB of usable flash across three NVMe storage nodes. The fabric is non blocking with 128 ports at 800 Gb/s, built on Quantum-X800 class InfiniBand or Spectrum-X class Ethernet, with a separate 400 GbE north south network.
At this scale we recommend the liquid cooled HGX B300 node where the facility allows, since it halves the rack count and lets GPUs run at full power. Either way we deliver the pod racked, cabled and validated to your data center or colocation facility, with an elevation and power schedule your site team can build against, and we time the shipment to your ready date.
Use cases
- Training models in the few hundred billion parameter class
- Large inference fleets
- Multi team enterprise AI platforms
- National and academic research clusters
Specifications
| GPU nodes | 16 x 8 GPU HGX B300 AI server, air cooled 10U or liquid cooled 4U |
|---|---|
| GPUs | 128 x NVIDIA B300, 288 GB HBM3e each, 33.6 TB total (16 x 2.1 TB per node) |
| CPU nodes | 3 x 1U dual socket EPYC 9005 compute servers, meeting 6 threads and 24 GB per GPU |
| Control plane | 6 x 1U single socket EPYC 9005 control plane servers |
| Intel option | GPU node hosts, CPU nodes and control plane nodes can be specified on Intel Xeon 6 instead of AMD EPYC 9005 |
| East west | 128 x 800 Gb/s, two tier leaf and spine, non blocking, InfiniBand XDR or 800 GbE |
|---|---|
| North south | 32 x 400 Gb/s BlueField-3 ports into a 400 GbE leaf pair |
| Management | 1 GbE out of band network for every BMC and switch |
| Cabling | 800G OSFP transceivers and cables, lengths per rack elevation |
| Capacity | 128 TB usable minimum, 1 TB per GPU, expandable |
|---|---|
| Nodes | 3 x 2U 24 bay all NVMe storage servers with 2 x 400 Gb/s each |
| Software | Parallel file system or NVMe over Fabrics, specified with the order |
| Racks | Air cooled: 6 to 7 racks at 3 GPU nodes per rack; liquid cooled: 3 to 4 racks at 6 to 8 GPU nodes per rack |
|---|---|
| Planning power | About 245 kW total: 16 x 14 kW GPU nodes, 3 x 1 kW CPU, 6 x 0.5 kW control, 3 x 1.3 kW storage, switches and overhead |
| Cooling | Air with hot aisle containment, or direct liquid cooling with rack CDUs |
| Standard | 16 x HGX B300 nodes, 128 GPUs, air or liquid cooled |
|---|---|
| Alternative platform | 2 x GB300 NVL72 racks, 144 GPUs in two 72 GPU NVLink domains, liquid cooled hall required |
| Choice | Decided with the fabric design and the facility survey |
Figures describe the system class. Exact configuration, weights and power draw are confirmed on the quotation.
What's included
- Rack elevation, cabling plan and power schedule
- Nodes, switches, cables and optics as one bill of materials
- Burn in of every node and fabric validation across the cluster, acceptance report
- NVIDIA end user registration handled with the order
- Deployment support and optional on site installation
- 3 year warranty, options to 5 years with advance replacement
Compliance
Related
Related systems
8 node HGX B300 AI pod, 64 GPUs
Allocation required- GPU nodes
- 8 x 8 GPU HGX B300 servers, 64 x B300
- Fabric
- 64 x 800 Gb/s ports, non blocking
- CPU and control
- 2 x CPU nodes for data services
- Chassis
- Multi rack, about 125 kW, air cooled
64 node HGX B300 AI cluster, 512 GPUs
Allocation required- GPU nodes
- 64 x 8 GPU HGX B300 servers, 512 x B300
- Fabric
- 512 x 800 Gb/s ports, leaf and spine
- CPU tier
- 8 x dual socket EPYC 9005 nodes
- Chassis
- Multi rack, about 950 kW, air or liquid cooled
8 GPU HGX B300 AI server, liquid cooled
Allocation required- GPUs
- 8 x NVIDIA B300, 288 GB HBM3e each
- CPU
- 2 x AMD EPYC 9005 or 2 x Intel Xeon 6
- Memory
- Up to 3 TB DDR5 RDIMM
- Chassis
- 4U, about 14 kW, liquid cooled
800G Spectrum-X Ethernet switch
- Ports
- 64 x 800 GbE OSFP
- Throughput
- 51.2 Tb/s aggregate switching
- AI features
- Spectrum-X RoCE with adaptive routing
- Form factor
- 2U, air cooled
Related guides
Air cooled vs liquid cooled AI servers
How to decide between air and direct liquid cooling for GPU servers, from the rack power limit up to NVL72 racks that only come liquid cooled.
In this guide
- The question is the rack, not the server
- How air cooling works at high density
Buying data center GPUs, export and registration
The compliance steps that come with every HGX, H200 NVL, Server Edition and NVL72 purchase, explained so your order ships on time.
In this guide
- Why a GPU purchase comes with paperwork
- Which products are covered
Choosing a GPU server for LLM inference
A practical method for sizing an inference server from the model, the quantization and the concurrency you expect, with worked examples.
In this guide
- Start with the model, not the server
- Step 1: how much GPU memory the weights need
HGX vs PCIe GPU servers
The real differences between an HGX node and a PCIe GPU server, and a simple rule for choosing between them.
In this guide
- Two kinds of 8 GPU server
- The interconnect is the point
Compare it with the next class up or down
The nearest classes we quote are the 8 node HGX B300 AI pod, 64 GPUs and the 64 node HGX B300 AI cluster, 512 GPUs. If neither is quite right, tell us the workload and we specify the system between them.
- (858) 613-1818
- sales@computermodules.com
- Monday to Friday, 8am to 5pm Pacific