Air cooled vs liquid cooled AI servers

The question is the rack, not the server

Every AI server can be built to run on air or, increasingly, on liquid. The choice is rarely about the server itself. It is about how much heat a rack can shed, what the room can deliver, and how hard you want to run the GPUs. An 8 GPU HGX B300 node is planned at around 14 kW on air. Three of them in a rack is 42 kW, which is beyond most enterprise data centers and routine for purpose built AI facilities. A GB300 NVL72 rack is well over 100 kW and is only available liquid cooled. This guide explains what each cooling approach needs and how to pick.

How air cooling works at high density

Air cooled GPU servers pull cold air from the front through the GPUs, CPUs and memory and push hot air out the back. At HGX densities the fans are large and loud, the chassis is 8 to 10U tall to make room for heatsinks, and the data center has to provide a lot of cold air at the front and remove a lot of hot air at the back. Hot aisle or cold aisle containment, raised floor or overhead supply, and rear door heat exchangers are the usual tools. With these, air handles 30 to 50 kW per rack, which is two or three HGX nodes, or eight to ten PCIe GPU servers.

Air has two advantages that keep it the default. It needs no plumbing, so any facility with the power and the airflow can host it. And it is simple to service: pull the node, replace the part, slide it back.

How direct liquid cooling works

Direct to chip liquid cooling puts a cold plate on each GPU, and often the CPUs and NVSwitches, and pumps a water and glycol mixture through them. The warm liquid leaves the server through quick disconnects into a rack manifold, then to a coolant distribution unit (CDU) that exchanges heat with the facility water loop. Components without cold plates, memory, drives, NICs and power supplies, are still cooled by air, so a liquid cooled rack still needs some airflow, though far less.

Liquid moves heat far more efficiently than air, so the server can be shorter, the fans smaller and quieter, and the GPU can run at its full power cap without throttling. Racks at 80, 120 and more kW become practical, which is how NVL72 systems exist at all. The cost is the plumbing: manifolds, CDUs, facility water, leak detection, and procedures for service.

Side by side

When air is the right answer

Most purchases. One to three HGX nodes, any number of PCIe GPU servers, workstations, and any deployment into an existing enterprise data center or colocation facility that was not built for liquid. The air cooled 8 GPU HGX B300 server runs at a lower GPU power cap than the liquid version, which costs a few percent of peak throughput and saves you the plumbing. For an 8 GPU PCIe server at 4 to 6 kW, air is not a compromise at all.

When liquid is the right answer

A training hall of many HGX nodes where rack count and floor space matter. Any deployment that wants the full B300 power cap and the throughput that comes with it. A facility that already has, or is building, water to the rack. And any NVL72 deployment, because the GB300 NVL72 rack is liquid cooled by design. The liquid cooled 8 GPU HGX B300 server is the building block for these rows. Liquid also reaches the desk: a liquid cooled 4 GPU workstation keeps four GPUs at full power in an office without the noise.

Performance at the power cap

A GPU's performance scales with the power it is allowed to draw, up to its maximum. Air cooled HGX nodes typically run the GPUs below the ceiling to keep temperatures inside the limit at the rack densities air can support; liquid cooled nodes run at the ceiling. The gap depends on the generation and the workload, usually a modest single digit to low double digit percentage of throughput. For a single node it rarely justifies plumbing. For a hall of them it adds up to whole nodes of throughput.

What to check before you order

Ask your facility for the rack power limit in kW, the cooling method and its limit, the floor loading, and whether water is available or planned. Ask for the inlet temperature and airflow per rack on air, and the supply temperature and flow rate per rack on liquid. Send those figures with your quotation request and we will specify the cooling to match. If the facility cannot take the density you want to buy, we can propose a lower density configuration that it can, or the air cooled variant of the same node.

FAQ

Can I convert an air cooled server to liquid later?

Generally no. The chassis, heatsinks and fan layout differ, and the GPU baseboard on HGX systems is specified for one or the other. Decide at purchase.

Does liquid cooling void warranties or raise failure rates?

No. Direct liquid cooling is a manufacturer supported configuration with the same warranty, and GPUs running cooler at a stable temperature tend to have lower failure rates. Leaks are rare with modern quick disconnects, and racks carry leak detection.

Is immersion cooling an option?

Immersion is a different approach, submerging whole servers in a dielectric fluid. It is used in some purpose built facilities but is not a standard configuration for the systems on this site, which are specified for air or direct to chip liquid.