(858) 613-1818Quote list 0
Allocation required

NVIDIA GB300 NVL72 rack

Liquid cooled rack with 72 Blackwell Ultra GPUs and 36 Grace CPUs in one NVLink domain, 20 TB HBM3e and 130 TB/s NVLink, with site engineering and fabric.

GPU: NVIDIA GB300 NVL72

Illustration of a rack of servers
GPUs
72 x NVIDIA Blackwell Ultra, 288 GB HBM3e each, 20 TB per rack, one NVLink domain
CPUs
36 x NVIDIA Grace, 2,592 Arm Neoverse V2 cores, 17 TB LPDDR5X
NVLink
130 TB/s total NVLink bandwidth across all 72 GPUs, 37 TB fast memory
Performance
1,440 PFLOPS sparse FP4, 1,080 PFLOPS dense, per NVIDIA
Scale out
ConnectX-8 at 800 Gb/s per GPU into Quantum-X800 or Spectrum-X fabrics
Facility
Fully liquid cooled rack, facility water loop and CDU required, high density power feed

Overview

GB300 NVL72 is a different class of system from an HGX server: an entire rack behaves as one computer with 72 Blackwell Ultra GPUs and 36 Grace CPUs in a single NVLink domain. NVIDIA publishes 20 TB of HBM3e at up to 576 TB/s, 130 TB/s of NVLink bandwidth, 37 TB of fast memory and 1,440 PFLOPS of sparse FP4 per rack.

The large NVLink domain is what makes it the platform for reasoning model inference with long context and for training runs that would otherwise be bound by inter node traffic. Several racks join over 800 Gb/s ConnectX-8 ports into Quantum-X800 or Spectrum-X fabrics to form larger systems, and we design that scale out fabric with the order.

The rack is fully liquid cooled and needs a facility water loop, a coolant distribution unit and a high density power feed, so every GB300 NVL72 project starts with a site survey. NVIDIA has not published a rack level power figure, and we plan power with the site survey and the system partner. The rack is sold as a complete system and delivered to your own liquid cooled hall or to a colocation facility that provides the water loop and power density it needs.

Use cases

  • Reasoning model inference at scale
  • Training with a large NVLink domain
  • AI factories and sovereign compute
  • Long context inference for AI products

Specifications

Compute
Configuration72 x Blackwell Ultra GPUs and 36 x Grace CPUs in 18 compute trays, per NVIDIA
GPU memory20 TB HBM3e, up to 576 TB/s
CPU memory17 TB LPDDR5X, 14 TB/s
FP4 performance1,440 PFLOPS sparse, 1,080 PFLOPS dense
FP8 performance720 PFLOPS sparse
Interconnect
NVLinkFifth generation NVLink, 130 TB/s total, 9 NVLink switch trays
Scale out72 x 800 Gb/s ConnectX-8 ports, InfiniBand XDR or Spectrum-X Ethernet
FrontendBlueField-3 DPUs for storage and north south traffic
Facility
Form factorSingle rack with compute trays, switch trays, power shelves and in rack coolant manifold
CoolingDirect liquid cooling to all GPUs, CPUs and NVLink switches; CDU and facility water loop required
PowerHigh density rack power feed; rack level figure not yet published by NVIDIA, planned with the site survey
DeliverySite survey, rack delivery and commissioning with the system partner

Figures describe the system class. Exact configuration, weights and power draw are confirmed on the quotation.

What's included

  • Rack elevation, cabling plan and power schedule
  • Nodes, switches, cables and optics as one bill of materials
  • Burn in of every node and fabric validation across the cluster, acceptance report
  • NVIDIA end user registration handled with the order
  • Deployment support and optional on site installation
  • 3 year warranty, options to 5 years with advance replacement
  • Site survey for liquid cooling and power
  • Commissioning with the system partner

Compliance

Compare it with the next class up or down

The nearest classes we quote are the NVIDIA Vera Rubin NVL72 rack and the 64 node HGX B300 AI cluster, 512 GPUs. If neither is quite right, tell us the workload and we specify the system between them.

Talk to an engineerRequest a quotation