The hidden cost of scaling AI: infrastructure, energy, and the push for liquid cooling.
Updated
January 8, 2026 6:31 PM
The inside of a data centre, with rows of server racks. PHOTO: FREEPIK
As artificial intelligence models grow larger and more demanding, the quiet pressure point isn’t the algorithms themselves—it’s the AI infrastructure that has to run them. Training and deploying modern AI models now requires enormous amounts of computing power, which creates a different kind of challenge: heat, energy use and space inside data centers. This is the context in which Supermicro and NVIDIA’s collaboration on AI infrastructure begins to matter.
Supermicro designs and builds large-scale computing systems for data centers. It has now expanded its support for NVIDIA’s Blackwell generation of AI chips with new liquid-cooled server platforms built around the NVIDIA HGX B300. The announcement isn’t just about faster hardware. It reflects a broader effort to rethink how AI data center infrastructure is built as facilities strain under rising power and cooling demands.
At a basic level, the systems are designed to pack more AI chips into less space while using less energy to keep them running. Instead of relying mainly on air cooling—fans, chillers and large amounts of electricity, these liquid-cooled AI servers circulate liquid directly across critical components. That approach removes heat more efficiently, allowing servers to run denser AI workloads without overheating or wasting energy.
Why does that matter outside a data center? Because AI doesn’t scale in isolation. As models become more complex, the cost of running them rises quickly, not just in hardware budgets, but in electricity use, water consumption and physical footprint. Traditional air-cooling methods are increasingly becoming a bottleneck, limiting how far AI systems can grow before energy and infrastructure costs spiral.
This is where the Supermicro–NVIDIA partnership fits in. NVIDIA supplies the computing engines—the Blackwell-based GPUs designed to handle massive AI workloads. Supermicro focuses on how those chips are deployed in the real world: how many GPUs can fit in a rack, how they are cooled, how quickly systems can be assembled and how reliably they can operate at scale in modern data centers. Together, the goal is to make high-density AI computing more practical, not just more powerful.
The new liquid-cooled designs are aimed at hyperscale data centers and so-called AI factories—facilities built specifically to train and run large AI models continuously. By increasing GPU density per rack and removing most of the heat through liquid cooling, these systems aim to ease a growing tension in the AI boom: the need for more computers without an equally dramatic rise in energy waste.
Just as important is speed. Large organizations don’t want to spend months stitching together custom AI infrastructure. Supermicro’s approach packages compute, networking and cooling into pre-validated data center building blocks that can be deployed faster. In a world where AI capabilities are advancing rapidly, time to deployment can matter as much as raw performance.
Stepping back, this development says less about one product launch and more about a shift in priorities across the AI industry. The next phase of AI growth isn’t only about smarter models—it’s about whether the physical infrastructure powering AI can scale responsibly. Efficiency, power use and sustainability are becoming as critical as speed.
Keep Reading
AI growth is increasingly becoming a manufacturing, packaging and deployment challenge — not just a computing one.
Updated
May 26, 2026 5:28 PM
Taipei 101 and Taipei Nan Shan Plaza, viewed from Elephant Mountain. PHOTO: UNSPLASH
As AI companies continue scaling larger models and data centers, the pressure is no longer falling only on chip design. Manufacturing capacity, advanced packaging and infrastructure deployment are becoming equally important parts of the AI race. AMD’s latest investment announcement reflects how quickly that shift is accelerating.
The US chipmaker announced plans to invest more than US$10 billion across Taiwan’s semiconductor and manufacturing ecosystem to support next-generation AI infrastructure. The investment focuses on expanding partnerships and increasing advanced packaging capacity needed for future AI systems.
The announcement highlights a growing reality across the AI industry. Building powerful AI chips is no longer enough on its own. Companies now also need the manufacturing networks, packaging technologies and supply chain coordination required to deploy AI infrastructure at global scale.
AMD’s investments center heavily around advanced chip packaging, an area becoming increasingly critical as AI systems demand higher performance and greater power efficiency. Traditional chip architectures are struggling to keep pace with the size and complexity of modern AI workloads. Advanced packaging helps connect processors, memory and computing systems more efficiently while managing power and cooling limitations inside large-scale AI environments.
The company said it is working with Taiwan-based partners including ASE, SPIL and PTI to develop next-generation packaging technologies for its upcoming 6th Gen AMD EPYC processors, codenamed “Venice.” AMD also said it had qualified what it described as the industry’s first 2.5D panel-based EFB interconnect technology alongside PTI.
At the center of the broader strategy is AMD Helios, the company’s rack-scale AI infrastructure platform scheduled for deployment beginning in the second half of 2026. The platform combines AMD Instinct MI450X GPUs, 6th Gen EPYC CPUs, networking systems and AMD’s ROCm software stack into integrated AI infrastructure systems designed for hyperscale deployment.
Rather than selling individual processors alone, companies are increasingly building complete AI infrastructure platforms that combine hardware, software, cooling systems and power management into unified deployments. That transition is reshaping how AI infrastructure is designed, manufactured and delivered.
Taiwan is also becoming more deeply embedded in that process. AMD’s investment spans not only semiconductor packaging companies but also manufacturing and system integration partners including Sanmina, Wiwynn, Wistron and Inventec. The partnerships reflect Taiwan’s growing role as one of the operational centers of the global AI infrastructure economy.
Dr. Lisa Su, Chair and CEO of AMD, said: “As AI adoption accelerates, our global customers are rapidly scaling AI infrastructure to meet growing compute demand. By combining AMD leadership in high-performance computing with the Taiwan ecosystem and our strategic global partners, we are enabling integrated, rack-scale AI infrastructure that helps customers accelerate deployment of next-generation AI systems”.
Power efficiency is becoming another major challenge shaping AI infrastructure decisions. As AI workloads consume more electricity and generate more heat, infrastructure providers are increasingly being forced to rethink cooling systems, interconnect technologies and deployment economics.
AMD’s announcement signals how the AI competition is evolving beyond model development and raw computing power. The next stage may depend just as heavily on who can manufacture, package and deploy AI infrastructure fast enough to support global demand.