(858) 613-1818Quote list 0
Available on request

8 GPU PCIe AI server with H200 NVL

Eight NVIDIA H200 NVL GPUs with 141 GB HBM3e each, NVLink bridged in fours, in a 4U PCIe server with dual EPYC 9005 or Xeon 6 and 400 Gb/s networking.

GPU: NVIDIA H200 NVL

Illustration of a 4U GPU server
GPUs
8 x NVIDIA H200 NVL, 141 GB HBM3e each, 1.1 TB total, two 4 way NVLink bridge groups
CPU
2 x AMD EPYC 9005 or 2 x Intel Xeon 6
Memory
Up to 2 TB DDR5 RDIMM
Storage
2 x M.2 boot, up to 8 x U.2 NVMe
Networking
4 x 400 Gb/s ConnectX-7 (one per GPU pair), 2 x 25 GbE
Power
Redundant PSUs, about 6.5 kW per node planning figure

Overview

This node brings Hopper HBM3e capacity to a PCIe server. Eight H200 NVL cards give 1.1 TB of GPU memory at 4.8 TB/s per card, and NVLink bridges join the cards in two groups of four at 900 GB/s per GPU, so a 70B or 180B class model runs inside one bridge group without crossing PCIe. The remaining traffic uses PCIe Gen5 switches and 400 Gb/s adapters.

It suits inference and fine tuning teams that want HBM bandwidth and large context windows in standard air cooled racks, and MIG support with up to seven instances per GPU makes it a strong multi tenant platform. Supply follows NVIDIA allocation for H200 NVL, so the class is listed as available on request and we confirm configuration and timing with each quotation.

Hosts are dual AMD EPYC 9005 or Intel Xeon 6 with DDR5 to 2 TB, mirrored boot and up to eight U.2 NVMe drives. Four ConnectX-7 adapters at 400 Gb/s provide GPUDirect RDMA into the fabric. Plan for about 6.5 kW per node in four rack units.

Use cases

  • Large context inference
  • Fine tuning on HBM memory
  • Multi tenant inference with MIG
  • RAG and retrieval pipelines

Specifications

Compute
GPUs8 x NVIDIA H200 NVL, dual slot passive, PCIe Gen5 x16
GPU memory141 GB HBM3e per GPU, 4.8 TB/s each, 1.1 TB per node
GPU interconnectTwo groups of 4 GPUs on NVLink bridges at 900 GB/s per GPU, PCIe Gen5 between groups
FP8 performance3,341 TFLOPS sparse per GPU
CPUs2 x AMD EPYC 9005 series or 2 x Intel Xeon 6
System memory24 or 32 x DDR5 RDIMM, up to 2 TB
Storage
Boot2 x M.2 NVMe, mirrored
DataUp to 8 x U.2 NVMe, 7.68 TB to 30.72 TB each
RAIDSoftware RAID or optional hardware RAID
Networking
GPU fabric4 x 400 Gb/s ConnectX-7, InfiniBand or RoCE, GPUDirect RDMA
Frontend2 x 25 GbE onboard, optional 2 x 100 GbE
ManagementDedicated 1 GbE BMC port, TPM 2.0
Physical
Form factor4U rackmount, air cooled, validated for 600 W PCIe GPUs
Power suppliesRedundant 2,700 W class PSUs, 200 to 240 V input
Planning powerAbout 6.5 kW per node
GPU powerConfigurable up to 600 W per GPU

Figures describe the system class. Exact configuration, weights and power draw are confirmed on the quotation.

What's included

  • Rails, power cords and cable management arm
  • Burn in with multi hour GPU stress and NVLink validation, acceptance report
  • NVIDIA end user registration handled with the order
  • Operating system imaging, driver and fabric manager baseline on request
  • 3 year warranty, options to 5 years with advance replacement

Compliance

Compare it with the next class up or down

The nearest classes we quote are the 8 GPU RTX PRO 6000 PCIe AI server and the 8 GPU HGX H200 AI server. If neither is quite right, tell us the workload and we specify the system between them.

Talk to an engineerRequest a quotation