As AI continues to evolve and improve, businesses are identifying more use cases to facilitate implementation for everyday tasks. From streamlining workflows to automating repetitive tasks, AI has become an essential asset in empowering the workforce. This trend will only expand over time and necessitate significantly more computing power throughout the value chain.
The past few years have seen generative AI establish a predominant role in enterprise workflows, allowing businesses to maintain a competitive advantage to enhance productivity. While the casual user may experiment with free services, both for-profit and non-profit organisations must understand how generative AI can help drive the bottom line.
From a productivity perspective, the benefits are clear: faster completion of challenging or repetitive tasks is certain; however, businesses must also consider the bigger picture. Notably, the significant energy demands that AI computing requires, highlighting why improving efficiency should also be a clear priority for global economies.
AI energy reckoning
While the excitement of using AI is undeniable, there are steep costs associated with this technology, particularly the use of servers, storage, and networking equipment. Although the efficiency per watt of each generation of CPUs and GPUs is increasing significantly, the total cost of ownership (TCO) must still be considered and planned for. A major component of calculating the TCO is understanding the expense of electricity necessary to power AI infrastructure.
Sustainability is becoming increasingly important for data centres worldwide, as national and local bodies increase their scrutiny on environmental impact reporting. Potential customers are inquiring with cloud service providers about their power usage and the entire life cycle of the servers used. Enterprises that equip their own data centres are becoming more diligent in understanding the overall environmental cost of their on-premises data centres.
Each training run or inference workload requires power and generates heat from the CPUs, GPUs, storage, and network infrastructure. New cooling methods are necessary to maintain the performance of a data centre housing the latest AI technologies, both today and beyond.
Liquid cooling as a strategic choice
Cooling very hot CPUs and GPUs is critical for maintaining system operations. Liquid cooling has significantly greater heat removal capability than air, enabling higher wattage CPUs and GPUs to operate consistently at their specified TDP for maximum performance.
There are numerous benefits to using liquid cooling. For example, the power consumption at the server level is about 85% compared to an air-cooled server, as the fans do not need to run at full speed during intense workloads. Future technologies will significantly reduce fan requirements as more components can be liquid cooled. At the data centre level, the power reduction is around 40%, which decreases power costs or allows more servers to operate within a power envelope.
One measurement (there are a few) of data centre power efficiency is the power usage effectiveness, or PUE. This ratio compares the power delivered to the overall data ecosystem with the power used by IT equipment, including servers, storage, and networking infrastructure. A lower PUE value, such as 1.0, indicates that all energy delivered to the data centre is utilised for IT equipment, whereas a higher value signifies reduced efficiency, or less effective cooling. Modern data centres with liquid cooling are approaching a PUE of 1.0, whereas legacy ones typically range from 1.5 to 1.6.
Modelling the right sustainability outcomes
The choice of adopting a liquid-cooling can be complex and requires careful simulations. The specific climate in which the data centre is located, including humidity, and air temperature are important considerations. These can significantly impact water usage, and the energy needed to maintain the function of cooling towers. In addition, the costs of water, power, and land need to be considered.
From an environmental perspective, choosing liquid cooling will save a significant amount of energy in addition to lowering construction costs. In fact, the construction cost of a data centre is directly related to the power needed from the local utility. Therefore, when less power is required, the initial construction costs can also be reduced. The operating expenses will be lower, as significantly less power is needed to deliver the desired results to the end user.
Ultimately, all these considerations can fluctuate wildly and are hard to predict. Data centre designers need to establish a reliable testing mechanism where they can simulate, test and map how their technology will run, given the working environment. This will subsequently allow for them to find the right cooling solutions to fit their own needs.
What this means for the future of AI workloads
In summary, liquid cooling of the data centre will be a requirement, not an option, for AI factories. The roadmaps from the manufacturers of the CPUs and GPUs show a continued increase in the power needed for their future generations of hardware.
This has also been a central topic across the global technology stage. Whilst different regions have established frameworks around its use (such as the instatement of the EU AI Act), one consistent thread is the acknowledgement of AI’s importance in driving growth. However, a recent report from the International Energy Agency, projected that the energy need for AI training and inference use would quadruple by 2030.
While there will always be a need for air-cooled data centres, but the need to plan for liquid-cooling is approaching quickly. In alignment with this, the UK, as an example, has established the AI Energy Council, to keep power use in check, as well as ensure the UK’s energy infrastructure can keep pace with the rapidly innovating tech landscape.
