Simon Naylor, SVP APAC, Oper8 Global Group

At NVIDIA’s annual GTC conference earlier this year – you can watch the whole two hours here! –  CEO Jensen Huang announced that its Rubin Ultra NVL576 will be coming in H2 2027: 576 GPUs in a single rack consuming 600kW of power. That’s right, 600kW per rack.

We talk about high-performance computing today with high-density racks of up to 120kW; this will be a five-fold increase in density. Huang also said in his keynote that we should start preparing for racks that will require megawatts of power. That requires a rethink on everything from power grids to cooling systems to the very architecture of data centres.

NVIDIA needs this escalation in computational power, because its ultimate vision is that traditional data centres will increasingly become AI factories. While data centres are designed to handle general-purpose computing tasks across fields, an AI factory is specifically optimised for AI workloads, with a strong emphasis on AI inference performance and energy efficiency. The product of these factories is intelligence – AI tokens – the building blocks of AI. The higher the token throughput the greater the factory’s ability to generate actionable intelligence, driving innovation and competitive advantage for an individual business or industry sector.

To support these new data centre paradigms, we’re going to need new everything – which makes retrofitting a challenge. Instead, Oper8 Global is in the process of completely redesigning its modular and prefabricated data centre solutions to support the wave of computing innovation to come – not just from NVIDIA, but also from Google and others.

Cooling

Traditional air cooling won’t be able to handle the heat from these new racks. Advanced liquid cooling infrastructure will be essential. These Rubin Ultra racks are going to use full liquid cooling from the ground up. NVIDIA is already moving in that direction with its current GB200 rack-scale, liquid-cooled systems but this will need to be on another level entirely. With McKinsey estimating that cooling accounts for 40% of a data centre’s energy consumption, cooling efficiency is going to be essential to reduce costs, access sufficient energy supplies and also to meet sustainability goals.

We will need direct-to-chip liquid cooling systems that recycle water in a closed loop to avoid evaporation, reducing water consumption. We’ll also have to accommodate other advances like microfluidics, with coolant directly flowing through channels embedded inside the silicon chip.

That also adds complexity and precision to environmental monitoring, particularly leak protection – with a rapid proactive response needed to prevent catastrophic damage to hardware.

Power

I read a post on LinkedIn that we need to consider that “600KW in a single rack is the equivalent of approximately 30 domestic boilers”. A micro data centre usually has a maximum of ten racks, so we are talking about the equivalent of 300 boilers – can you imagine a hyperscale data centre full of these 600kW racks?

To support something like this, data centres are going to need completely new energy architecture and also reduce their reliance on the existing power grids. Otherwise, they will have a problem.

“Every single data centre in the future is going to be power-limited, and your revenue is limited if your power is limited,” said Wade Vinson, Nvidia’s Chief Data Centre Distinguished Engineer.

That means new substations, higher-capacity transformers, custom switchgear, battery backup systems, and potentially on-site generation – which is why a lot of operators are looking to SMRs – small modular reactors.

We will also have to be better at managing energy usage – “every watt matters” says Vinson – so increasingly we will need to deliver power beyond the server rack from sidecars or dedicated power racks. That way, we can couple these more easily with energy management systems to better regulate and optimise the power we are delivering to the servers and more precisely match the demands of the workloads running.

Networking

In this new AI-accelerated world, in-rack networking is going to need to scale with compute –meaning faster, denser, and likely specialised processors and photonic-based connections to reduce the massive energy draw required to move data between GPUs.

The Dell’Oro Group anticipates that there will be greater vendor diversity at both the chip and system levels, and photonic integration in switches will introduce a new dimension, potentially reshaping the dynamics of an already vibrant vendor landscape.

Industry impact

To meet the changes coming, our industry will need to rely on collaboration more than ever before, so the data centre partner ecosystem is critical – everything from our technology vendors for hardware, cooling and software all the way through to property developers, power and utilities providers, and building and construction contractors.

The future of AI infrastructure will be dense, hot, and power-hungry. And if you’re in the data centre space, now’s the time to start preparing.

Retrofitting won’t be enough for most — this is a ground-up redesign moment. The players who start investing in next-gen infrastructure now will be the ones powering and profiting from the AI gold rush of the next decade.