As demands for artificial intelligence (AI) and machine learning (ML) grow, the IT hardware, energy infrastructure, and data center architecture needed to support these systems will have to adapt to keep up with these advancements. High performance compute (HPC) will require chips and processors that operate differently than traditional chips used in most data centers. Many of these chips require more energy, release more heat, and are packed together in racks that make it difficult for traditional air-cooled technologies to dissipate the heat effectively.
As a result, new types of cooling methods including various variations of liquid cooling will have to be used to address the needs of high-performance computing. In addition to updating cooling technologies, data centers will also require a variety of energy systems to be deployed to meet the growing power needs. Estimates suggest that the power consumed by data centers globally could double by the end of the decade – potentially requiring more than 800TWh of power.
Ideally, much of this would be done through a mix of renewables such as wind and solar. However, standing up new projects and connecting sources of energy to data centers is a challenge. As a result, many data centers and utilities may turn towards natural gas to quickly meet power needs. Recently, Entergy announced that a 1.5GW natural gas plant could be used to power data centers in Louisiana. Although renewables like solar and wind were evaluated for the project, it was stated that the need for natural gas for backup power would have been required anyway.
As data centers demand more power, a mix of power solutions will be used to meet the growing needs. However, the immediacy of the demands may place more of a pull on natural gas projects that can be up and ready faster than renewables and overcome some of the challenges with reliability and intermittency.
However, many data centers will still look to reduce their energy consumption and carbon emissions through other means.
Reducing Emissions Through Improved Cooling Systems
Currently, 40% of a data center’s energy is used to cool down chips and IT infrastructure when heat is released. In most data centers, air cooling is used. Air cooling is currently used in 80% of data centers worldwide. Many of these technologies work well for existing data centers that have rack densities of 10-15kW per rack but as data centers deploy IT systems and chips that are required for AI and ML, these systems may require power over 40kW which cannot easily be cooled using air cooling technologies.
Liquid cooling technologies are more efficient at transferring heat since they use liquid to move heat away from the chips which cools the systems faster than air. These will likely be required as more and more data centers deploy powerful IT systems that require effective cooling systems.
Within liquid cooling systems, technologies can be broken down into various categories. These can include single-phase and two-phase immersion cooling and single-phase and two-phase direct-to-chip cooling. Two-phase systems are when the liquid undergoes a phase change into a gas and pulls more heat away from the systems during that phase change as opposed to single-phase systems where the liquid stays a liquid. Single-phase systems usually are simpler systems to operate but two-phase systems may have faster heat transfer rates due to the phase change. Two-phase immersion cooling is when the phase change occurs in a large tank where the computing systems are immersed in the liquid which is non-conductive.
Direct-to-chip cooling solutions use a cold plate to allow fluid to pass through regions where the chip is releasing heat and moves heat away from those components.
- Microchannel direct-to-chip cooling solutions, which constitute the majority of direct-to-chip cooling technologies, involve small channels on the cold plate that allow for fluid to pass through those sections.
- Another form of direct-to-chip cooling is two-phase direct-to-chip cooling which is being developed by companies like ZutaCore and Accelsius. These technologies use a non-conductive liquid that evaporates on the surface of the chip pulling away more heat than traditional direct-to-chip technologies. The added benefit of two-phase direct-to-chip cooling systems when compared to two-phase immersion is that they can be easier to integrate into existing data center systems. Additionally, two-phase liquids can operate with slower flow rates since heat is removed more rapidly. Also, if the liquid were to get on the chips themselves due to a break or leakage there would be fewer challenges and losses when compared to water based coolants.
Recent Activity
As a result of the growing energy and cooling needs for data centers there have been multiple investments and acquisitions in this space.
- Jetcool, which was just acquired by Flex, raised $17M in 2023 with investors Bosch Group, In-Q-Tel, Raptor Group, and Schooner Capital
- Accelsius, developer of two-phase, direct-to-chip liquid cooling systems raised $24M in their Series A round recently
- ZutaCore is also advancing two-phase direct-to-chip liquid cooling solutions and partnered with Wiwynn to expand and develop their technology
- Liquid Stack raised $35M in a Series B round led by Tiger Global for their full range of solutions including Direct-To-Chip Cooling, Single-Phase Immersion Cooling, and Two-Phase Immersion Cooling
- Submer raised $55.5M in a Series C round let by M&G Investments, Mundi Ventures, Planet First Partners and Norrsken VC. Funding will be used to expand operations in the U.S. and APAC regions
Venture Investments in Liquid Cooling
Data centers will require advanced cooling technologies to meet the needs of high-performance compute and the liquid cooling market will likely see a number of additional investments and acquisitions in this space.
