As compute-intensive activities such as artificial intelligence (AI) workloads have continued to grow in prominence worldwide, so have data center power requirements. While server energy consumption has grown steadily, the power required to cool those servers to keep them from overheating and failing is also a big piece of the power cost puzzle.¹ According to one McKinsey analysis, “Cooling accounts for some 40 percent of a data center’s energy consumption,” and “efficient cooling is therefore a crucial driver of a data center’s profitability.”² While no single cooling solution is right for every situation, air cooling, liquid cooling, or a combination of innovative cooling technology all offer unique advantages.

Dell Technologies^™ offers a wide range of cooling technologies to suit the needs of different organizations and workloads. By choosing new Dell PowerEdge^™ servers powered by AMD EPYC^™ processors, you also access unique innovations in air cooling and liquid cooling, with the option to configure the right cooling solution for your specific needs. In this paper, we review publicly available data to discuss the Dell portfolio of server cooling technologies, including rack and multi-rack cooling, and discuss various factors you might consider when crafting your approach to cooling.

Prioritize your needs with a range of cooling options for Dell PowerEdge servers

Cooling is a critical concern for businesses sustaining their own data centers. For perspective, let’s look at the power requirements of a single rack and the equivalent cooling it needs. Until recently, one fully populated rack, configured without power supply redundancy, might draw up to 20 kilowatts (kW) of power in a real-world data center environment.³ Past that density, air cooling can struggle to keep up. For reference, a typical US household requires 24 or 48 kW (100 or 200A at 240V) maximum feed.⁴ That means a trio of populated racks drawing 20 kW of power might draw as much as or more power than an entire household, not including the additional power draw that cooling creates.

But with AI driving dramatically higher power needs, we’re now moving far beyond that level of power consumption. While not all AI tasks consume high amounts of energy, tasks like generating a single image can require as much energy as charging a phone—energy consumption that stacks with each user.⁵ Morgan Stanley estimates that “by 2025, generative AI’s power demands alone could equal a third of what data centers needed for all of 2022.”⁶ A Data Center Dynamics article notes that for racks packed full of AI servers and powerful GPUs, power needs are likely to increase to between 40 and 60 kW—double or triple what we used to consider a rack’s power needs.⁷ At these levels of energy consumption, to maintain high density, new cooling solutions are required. Direct liquid cooling is one such solution. Significantly more energy-efficient than air cooling, direct liquid cooling can translate to a smaller datacenter footprint, lower energy costs, less data center noise, and better sustainability.⁸

Dell is ready. They are currently developing liquid-cooled solutions that will support greater than 400kW in a single rack, a 20x increase from the old 20 kW standard in the same data center footprint.⁹ To give just one example, by utilizing direct liquid cooling, the new Dell Integrated Rack 7000 Series would support up to 480kW per 21-inch ORv3 rack. With the potential to include up to 72 nodes of AMD EPYC 5^th Gen processors per rack, Michael Dell notes that “the system is engineered to capture nearly 100% of the heat produced within each rack.”¹⁰ With the dramatic increase in power and heat production from servers running heavy workloads, innovative cooling techniques will be essential for high-density/high-power deployments.

In this increasingly expensive space, administrators must make three overall choices when strategizing their approach to cooling: how to design the physical space of the data center, what systems to invest in solely for cooling (e.g., specialized in-row coolers), and what cooling technologies to choose inside the servers or racks themselves. While these decisions are interconnected, in this analysis we address the latter two questions, with a particular focus on the cooling technologies inside servers.

When deploying servers, you can choose air cooling, liquid cooling, or a combination of innovative cooling technology. Dell offers a wide range of server cooling technologies both inside its PowerEdge servers and at the rack and multi-rack level. Below, we dive into key options, arming you with the data and context you need to make informed decisions about the right cooling approaches for your workloads.

Air cooling

As you might expect, air cooling systems use air to cool servers and server components via fans, heat sinks, and other airflow management techniques. In a direct-air-cooled system, the server rack draws in ambient air (generally between 64 and 81 degrees Fahrenheit¹¹) from the front of the chassis; passes it through the entire system, during which it grows hotter due to the heat generated by the components in the system; and expels it from the rear of the chassis, transferring the heat to the environment behind the server rack. Data center designers can enhance the efficacy and efficiency of this approach with hot and cold aisle containment, which separates areas of warmer and cooler air and thus prevents the recycling of hot air into the server.

Some data centers that use air cooling pull in outside air to improve efficiency, particularly during colder months, while the traditional approach uses heat exchange systems that filter and condition the air to cool it. Newer design strategies such as variable-speed fans and precision airflow management systems can adjust airflow based on temperature and other factors.

Dell air cooling technologies

All mainstream Dell PowerEdge servers offer air cooling by default, though many servers incorporate different innovations in air cooling technologies. According to a Dell blog focused on PowerEdge servers with AMD processors, such as the PowerEdge R7625 with AMD EPYC 9754 CPUs, “Our unique Smart Cooling technology uses computational fluid dynamics to discover the optimal airflow configurations for our PowerEdge servers.”¹² Even very powerful Dell servers utilize air cooling: The Dell PowerEdge R7725, with two AMD EPYC 9005 series processors, is supported in an air-cooled configuration, while the PowerEdge XE9680, available with eight AMD Instinct MI300X GPUs, has a 5U footprint with air cooling as the default.¹³ PowerEdge servers also incorporate T-shaped motherboards, multi-vector cooling, thermal and power sensors, and airflow control algorithms to optimize air cooling.¹⁴ Dell notes that its fans and heatsinks “are qualified using mandated and extensive reliability and qualification processes to run at full speed for the life cycle of the server[,] minimizing costly downtime.”¹⁵ In one example, the modular AMD EPYC processor-powered Dell PowerEdge C6615 features an optimized thermal solution that allows for air-cooling configurations with up to 53% improved cooling performance compared to the previous generation chassis.¹⁶

For data centers that use hot/cold aisle configurations, some PowerEdge servers offer front input/output and power, allowing IT teams to access the servers from the cold aisle.¹⁷ This convenience enables IT to service equipment without the discomfort of standing in the full heat of thermal exhaust.

Smart Flow configurations

Some new Dell PowerEdge servers use Smart Flow configurations, which incorporate an airflow channel through the center of the server, where a middle storage slot would normally be.¹⁹ These servers, which also have redesigned backplanes to allow for more air intake, enable the use of more powerful CPUs and GPUs without liquid cooling. According to Dell, a PowerEdge R7625 server supporting high-powered processors like the AMD EPYC 9684x can achieve an approximately 17 percent higher rate of airflow, in cubic feet per minute (CFM), than a traditional 24x2.5-inch GPU chassis.²⁰ Figure 1 illustrates a standard PowerEdge server configuration compared to a Smart Flow configuration. According to Dell, Dell Smart Flow “reduces fan power by up to 52% compared to previous generation servers.”²¹

Liquid cooling ecosystem

Liquid cooling systems use either water or specialized coolants to keep server components cool. Cooling systems that incorporate both air and liquid cooling elements give customers control over the option that works best for their specific situations.

Dell embraces cooling solutions that make use of both direct liquid cooling and air cooling on different internal server components. In the Dell cooling ecosystem, direct liquid cooling is generally reserved for CPUs and GPUs via cold plates, while air cooling systems cool other server components such as storage drives, memory, and networking hardware. This reduces the complexity and maintenance requirements of the liquid cooling system and its internal fluid piping and cold plates. Dell internal modeling indicates that using direct liquid cooling (DLC) for CPUs and air cooling for everything else “can use just 3% to 4% more energy in cooling” than a fully direct-liquid-cooled solution, which they call a “cold plate everything approach,” while reducing cooling system design complexity and streamlining maintenance processes.²² This approach allows companies with less computationally intensive workloads to access many of the benefits of liquid cooling without the high cost of a fully direct-liquid-cooled solution.

Because liquids conduct thermal energy more effectively than air and can retain more energy by volume, they have the potential to remove heat from a system more effectively and enable greater density in data center deployments. Liquid cooling allows for more uniform distribution of cooling across hardware and eliminates the hot spots and other airflow control concerns associated with air-cooled data centers.

Two common liquid cooling methods are direct-to-chip cooling and immersion cooling. Direct-to-chip systems circulate liquid through cold plates attached to heat-generating components such as CPUs or GPUs, thereby transferring the heat to a separate component that cools it. Immersion cooling systems submerge server components in a non-conductive dielectric fluid that directly absorbs heat. In this paper, when we discuss liquid cooling, we will generally focus on direct-to-chip cooling. Dell offers immersion cooling solutions via Dell Technologies OEM Solutions.

Dell liquid cooling technologies

The Dell liquid cooling ecosystem uses direct-to-chip cold plates in conjunction with coolant distribution units (CDUs) to manage the coolant loop. In-rack CDUs provide enough flow to cool an entire rack, while in-row CDUs can support the coolant needs of multiple racks. Via the primary flow network, CDUs receive cool water from the facility’s water supply that controls coolant temperature via an internal heat exchanger. Then, the secondary flow network automatically distributes coolant in the appropriate volume and temperature to the coolant manifolds on the racks they support between the CDU and rack manifolds. They then use internal heat exchangers to cool the warm return coolant from the secondary flow network. CDUs also ensure that the coolant flowing to the rack coolant manifolds remains above the dew point temperature of the environmental air, to prevent condensation from forming inside the hardware.²³

Dell also offers rear-door heat exchangers (RDHx) as an alternative to or in conjunction with servers enabled with DLC. A radiator in the RDHx uses facility water to cool the hot air that the server expels and then releases the cooled air to the environment. These heat exchangers fit on the back of the server rack, while their CDUs can work with either a single rack or a row of racks.²⁴ RDHx provides customers a scalable and cost-effective option to integrate liquid cooling into their infrastructure without the disruption associated with retrofitting individual servers with direct-to-chip liquid cooling.

Cooling options by server model

Table 1 shows which 16^th Generation and 17^th Generation AMD EPYC processor-powered Dell PowerEdge server platforms support direct liquid cooling. (While some Dell solutions using these systems require a specific type of cooling, all of these AMD processor-powered PowerEdge models support air cooling.) We collected this data from the publicly available specification sheets for each server model. Unless we note otherwise, all models that support DLC are rack solutions that require rack manifolds and a CDU (in-rack or in-row) to operate.

Comparing server cooling methods

Determining the right cooling method for your situation comes down to several key factors, including data center footprint limitations, existing/available cooling infrastructure, site power limitations, environmental cooling options, and environmental commitments.

At a high level, if you have a large data center with significant available space and CPU-only workload needs, traditional air cooling is the simplest solution to engineer. More constrained for space or scaling up your GPU-heavy AI workloads? Use liquid cooling to manage the high-performance infrastructure in your data center. Need the ultimate in high-density, high-performance? Utilize immersion cooling—available via Dell Technologies OEM Solutions—to keep every component chilled to optimum temperatures.

Even the choice of individual components within servers can impact the decision of how you cool your servers. As just one example, consider CPUs. The cooling needs of a server using a lower thermal design power (TDP) processor, such as the 24-core AMD EPYC 9224 at 200W, are different than a sever using a higher TDP processor, like the 192-core AMD EPYC 9965 at 500W. Or how about GPUs? If you’re building multiple GPU-heavy racks for intensive AI workloads, your cooling needs will be dramatically greater than they would be for a rack of CPU-only servers running everyday database workloads.

In data center engineering, one size rarely fits all. You must evaluate these technologies—and figure out which is right for you—by balancing efficiency, scalability, reliability, cost effectiveness, thermal management, energy consumption, and environmental concerns.

Efficiency

Liquid cooling is typically more energy-efficient than air-cooled infrastructure. One scientific review of liquid cooling calls direct liquid cooling “a transformative technology for enhancing energy efficiency and operational safety in high-density computing environments” with “potential to substantially lower thermal resistances and improve energy utilization.”²⁷ Dell claims that using direct liquid cooling instead of traditional air cooling via perimeter air handlers can save 11 percent of total energy consumed by the rack, or as much as 16 percent using rear door heat exchangers and 18 percent using both DLC and RDHx in tandem.²⁸

Some new technologies can help to increase efficiency in air-cooled environments. For example, Dell Smart Cooling technology comprises “intelligent fan control algorithms that work in unison to maintain the lowest possible fan power state without sacrificing cooling or the reliability of the server.”²⁹ Dell also offers Smart Flow configurations that enable more airflow into the server, allowing for higher-powered processors that would otherwise require a DLC solution.³⁰

That said, air cooling solutions that dissipate more heat may also consume more energy. According to a white paper published by the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE), “The ability to move additional airflow and enabling higher rack heat loads is not without cost…increasing air cooling density not only increases the amount of power used to move that air, but also reduces cooling efficiency.”³¹

Scalability

Both air and liquid cooling can be used at scale, but environmental, facility, and density factors can affect how efficient or cost-effective they can be. Air cooling requires a larger facility footprint due to its inefficiencies at higher power densities and requires more energy than liquid cooling. As density increases, issues can arise in airflow management and excess heat, and noise levels and air velocity from air-cooled data centers can present complications at scale. Air cooling also faces limitations for the high-density, GPU-driven AI workloads that are growing more and more popular. According to one Data Center Dynamics article, “air cooling can support high-performance computing (HPC) and AI workloads to a degree” but traditional air cooling “may struggle to adequately dissipate the high heat density generated by AI workloads.”³²

Data center layout can also be a factor in the scalability of your cooling solution. For example, if there isn’t adequate roof space for the required number of condensers or chillers, waste heat from discharge vents can affect the efficiency of other units nearby.

Liquid cooling at scale presents its own challenges. The higher thermal efficiency of direct liquid cooling enables significantly greater power density than air cooling, which can make it difficult to physically route enough power to serve the data center’s needs. Liquid cooling also requires significant capital expenditure to facilitate, making it more expensive to scale.

Operational reliability

The reliability of a cooling solution is directly linked to its performance: Can it keep a server cool enough to continue functioning as expected? If a server gets too hot, its performance will suffer, or it may shut down entirely. We saw this occur in the high-temperature testing study we referenced earlier: In an HVAC malfunction scenario, the Supermicro server we tested experienced OS SSD failure and ultimately stopped responding.³³ There can also be more subtle costs to servers running at high temperatures. According to a Forbes article, “components that operate at cooler temperatures often experience less wear and tear, resulting in extended lifespan and reduced maintenance costs.”³⁴

Air cooling systems are typically more robust and have fewer points of failure than more complex liquid cooling solutions, at the cost of decreased thermal and energy efficiency. However, at higher density and thermal load, air cooling systems can encounter difficulties managing airflow and waste heat, leading to issues such as hot spots—areas in the data center where the cooling systems fail to adequately dissipate the heat generated by the infrastructure—which in turn impact the overall operational efficiency of the hardware.

Due to their increased complexity, liquid cooling solutions have more potential points of failure, so admins must monitor them for problems such as pump failures that could rapidly compromise the ability of the system to maintain safe operating temperatures. Admins must also ensure that liquid cooling systems maintain coolant temperatures above the dew point of the ambient air, or risk condensation forming on coolant system components, which could cause electrical shorts and damage hardware. Because liquid cooling involves the use of fluids, buyers may worry about that fluid leaking and damaging the server. To address this concern, Dell PowerEdge servers with liquid cooling include Leak Sense technology, which “detects and reports any fluid leak at the cold plate via iDRAC” and automatically powers down the server if a coolant leak occurs.³⁵

Cost-effectiveness

If acquisition cost is the driving factor in your server purchase, air cooling may be the right answer. As the default choice for many servers, air cooling is generally less expensive than liquid cooling. And if you aren’t pushing for the highest possible performance with heavy use of GPUs and powerful CPUs, such as the AMD EPYC 9755 processor, air cooling may be adequate. For “standard workloads with lower densities…air cooling remains a viable solution,” according to Dell, “keeping costs reasonable for moderate demands.”³⁶

Liquid cooling generally requires a greater up-front investment than air cooling, but it has the potential to reduce energy costs over time. Dell notes that choosing direct liquid cooling can reduce energy costs by up to 45 percent and, in some cases, pay back its cost within 1.3 years.³⁷

Thermal management capabilities

Air and liquid cooling options can both be effective methods for removing heat from server hardware. After doing so, however, you must remove it from the surrounding environment. Costs in this area center around the HVAC and fan operation electrical bills.

Air-cooled systems pull heated air back through air handlers to chill and redistribute it into the data center for cooling again. Chillers between racks or within racks provide supplemental cooling to high-density infrastructure to increase density capabilities, but don’t alter the equation for moving the air out of the data center. This also applies to servers with closed-loop direct liquid cooling.

When local environmental conditions allow, you can draw in fresh air from outside for cooling and move the hot air back outdoors. Some Dell systems are designed to operate in this type of extended thermal envelope, increasing the range of climates for this more cost-effective option.³⁸

Liquid cooling that the data center provides to the rack offers a more effective way to remove heat from the components, servers, and whitespace of a facility (the area of the data center that stores server racks and other equipment). This technique removes heat directly from the loop with radiators, a heat exchanger, a secondary cooling loop, or other liquid cooling systems. All these methods provide a direct conduit for the thermal energy to leave the building, leaving the data center itself quieter and cooler.

Energy consumption and environmental impact

According to an analysis by McKinsey and Company, cooling accounts for about 40 percent of energy consumption in a data center.³⁹ Data center energy consumption is set to more than double by 2028 due to increased data center capacities, driven in large part by AI.⁴⁰ To address this, new AMD EPYC processors deliver more core density than previous generations, resulting in higher performance per watt for better energy efficiency. Traditional air cooling options may still suffice for smaller, lower density workloads (10 to 15 kW per rack), especially with Dell Smart Flow technology. But as you scale up, air cooling can introduce problems such as increased cooling costs, meaning higher energy consumption and possible performance hits. Because liquid can transfer heat more efficiently than air, a dedicated liquid cooling solution can help decrease energy consumption by up to 40 percent versus traditional air-cooled solutions.⁴¹

New air cooling options are also increasing power efficiency. This means that it’s not necessary to fully switch from air cooling to liquid cooling to see results. In one study, the American Society of Mechanical Engineers found that gradually switching from a 100 percent air-cooled environment to a 75 percent water-cooled and 25 percent air-cooled facility decreased energy consumption by 27 percent.⁴²

These energy savings can decrease both data center operation costs and their carbon footprints. A report from Google showed that by using water-cooling solutions in its data centers, they were able to save approximately 10 percent in energy usage and reduce their carbon footprint by a similar 10 percent, leading to a reduction of 300,000 tons of CO2 emissions.⁴³

That said, liquid cooling also has an environmental downside. When data centers consume water, it sometimes becomes “unsuitable for human consumption or agricultural use,” noted a Dallas Morning News article. The article goes on to note that while dry air is ideal for data centers, building data centers in “water-stressed regions, such as the southwest United States,” also stresses those areas’ water supplies even further.⁴⁴

As AI continues to grow and demand more power, many organizations are seeking innovative solutions to minimize the environmental impact of the data center. There are even possibilities to turn the heat generated by data centers into a boon for communities: some Nordic countries have worked with data centers to use heat exchangers to turn excess data center heat into hot water for homes.⁴⁵

Conclusion

Keeping server components cool is essential to both short-term and long-term hardware health, ensuring that critical workloads stay up and running. To help data centers beat the increasing heat that AI and other demanding technologies produce, the Dell PowerEdge server portfolio offers a wide range of modern cooling technologies tailored to fit a variety of needs.

From traditional air-cooled servers including Dell Smart Flow configurations for less taxing workloads, to direct liquid cooling or innovative technologies that allow organizations to leverage the benefits of liquid cooling for GPU-dense AI workloads, the PowerEdge server portfolio is poised to help you meet the cooling demands of your data centers both today and into the future. By selecting a suitable cooling approach (or combination of approaches) from the Dell PowerEdge portfolio, you can successfully keep your data centers cool, ensure reliability, and minimize ongoing operating expenses related to cooling.