Inside the DataCenter Liquid Cooling Industry
A tour of the space
Traditionally datacenter servers consumed a few kilowatts of power. Managing the heat generated by these units was relatively straightforward and could easily be done with air cooling based systems. However, with the rise of AI where a single Nvidia Blackwell GPU consumes around 1 kW, the power demand of an AI server has risen dramatically. For example, a single Nvidia GB200 rack can require a total of 140 kW of power. This has shifted demand for server thermal management towards liquid based systems, with direct-to-chip cooling as the current winner.
In a traditional air-cooled datacenter, the setup is reasonably simple. Basically heat is blown away from the chip, then out of the server and the server aisle, and finally out of the datacenter itself. Here’s how it works exactly. Critical chips like CPUs and GPUs have heat sink fins that absorb heat from the chips. Subsequently, fans blow cool air across these fins, moving the heat away from the chip. Next, each server has built-in fans that pull cool air in and push hot air out. The way this works is that servers are arranged in both hot and cold aisles, with the front of a server facing the cold aisle, pulling cold air in which later gets pushed out into the hot aisle, cooling all the server components in between. This cold air is generated by CRACs or Computer Room Air Conditioning units. Finally, the air in the hot aisles gets vented back to the CRACs to cool it down again or alternatively, the hot air is simply blown outside of the datacenter.
There are a number of further cooling methods in between, but the key for investors to understand is that with the buildout of AI datacenters around the world, there is an increasing need for the most advanced cooling techniques. With liquid-based cooling, effectively a plumbing system is introduced in the datacenter that pumps coolants to the servers. In direct-to-chip, the coolant flows through a cold plate that sits on the chip, pumping away the heat towards a chiller or heat exchanger. The liquid that goes in is about 25 degrees Celsius, or room temperature, and when it is pumped out of the cold plate again it now is about 45 degrees Celsius, or jacuzzi temperature. With next-generation immersion cooling, entire servers or computing components will be submerged in non-conductive liquids. This liquid absorbs heat directly from all the server components to be then pumped back to a heat exchanger to cool it down again.
This creates an interesting situation for investors. Currently 85 to 90 percent of datacenters around the world are still air cooled, and with the rise of AI the datacenter mix will shift considerably towards AI datacenters. In addition, there is attractive growth in the number of hyperscale datacenters around the world. For example, the public clouds alone are building hundreds of new datacenters each year for the foreseeable future.
The third growth pillar for investors is that ASPs of liquid-based cooling systems rise dramatically. For an Nvidia GB200 high-end rack, the cost of a liquid-based cooling is estimated to be about 15 to 20 times higher than the cost of an air-cooled system according to Morgan Stanley. And this is a field where further innovation will be needed such as with next-gen immersion cooling.
As GPU performance continues to rise exponentially, notice the logarithmic y-axis on the chart below, more advanced cooling techniques will be needed over time which likely will mean again higher ASPs. The reason for this need for more advanced cooling systems is simple. Due to the slowdown in Moore’s Law, increases in compute performance will have to increasingly come from more transistors in a single computing system, which means more heat generation. One of the great benefits of Moore’s law is that smaller transistor sizes means less generation. So the less we can stay on this trend, the more the semi industry will struggle with heat generation.
In this article we’ll get a deep understanding of the liquid cooling industry. We’ll talk to a variety of industry insiders ranging from executives at the most advanced liquid cooling companies to datacenter engineers at the hyperscalers, and all insiders are available over Tegus for those who want to follow up. We’ll get a good understanding of the competitive landscape, the best players in the field and emerging competitors with new and innovative cooling solutions. Finally, we’ll do an analysis of the stocks in the space with our thoughts on the best picks.