Traditionally, server rooms in data centers use an air-cooled system: the servers have their own fans, and hot and cold aisles are organized between the racks. But the capacities are growing, along with them the heat dissipation of processors and accelerators is growing. That is why, in the light of the emergence of new server processors from Intel and AMD, as well as accelerators NVIDIA A100 and AMD MI100, the owners of large data centers began to actively look at more advanced cooling methods – liquid.
The problem is not far-fetched: air cooling systems in data centers with traditional layout are already at their limit, and new generations of chips promise further heat generation. So, Intel Ice Lake processors, which will replace the Cooper Lake generation, will have a thermal package in the region of 250-300 watts against the previous 150-250 watts.
The third generation of AMD EPYC server processors, codenamed Milan, promises a performance boost of around 15-20%, but heat dissipation should also be expected. Already, the EPYC 7763 can deliver up to 280 watts. And the NVIDIA A100 accelerator will easily deliver all 400 watts. And that is why large data centers are looking towards alternative cooling methods.
Liquid cooling of servers can be implemented in different ways, but one of the simplest and most effective ways is to use immersion baths filled with a high molecular weight electrically neutral coolant. Such fluids, for example, are produced by 3M under the brands Novec and Fluorinert. According to the manufacturer’s assurances, these fluids are non-toxic, non-flammable and neutral to any materials, but at the same time have the thermal characteristics necessary for the implementation of high-performance LSS.
Alibaba, a giant Chinese concern, already uses submersible cooling systems for its servers, although not in all data centers. Google plans to accelerate work on this technology, and Microsoft and Facebook are still at the test stage (proof of concept), but also plan to begin introducing submersible cooling systems in their data centers this year, according to Digitimes. However, these are far from the first experiments. The same Facebook dealt with them back in 2012.
Such systems are simple: in fact, open servers are simply placed in a bath of liquid (not counting the second phase systems that transfer heat to the main air conditioners in the data center); at the same time, no water blocks, additional pumps, hoses and splitters are required in the primary circuit, which means that the maintenance process is simplified. The submersible system is the cheapest to implement among server LSS, which translates into significant savings for large data centers. This explains the interest of large IT companies in this kind of cooling systems. 2021 promises to be the year of the “bathing” of servers.