The War for AI Compute Has Left the Chip Behind
Trying to put 72 modern GPUs into a single rack is like trying to park a space shuttle at your house. You cannot just wheel it into your garage — you would have to re-engineer the entire building around it. NVIDIA and AMD took that exact approach, and in doing so have shifted the battleground for AI infrastructure from the chip to the rack.
For years, the industry measured competition by GPU performance numbers. Today, the battleground has expanded to rack-scale AI systems that integrate compute, memory, networking, and software into a single platform. AMD's Helios and NVIDIA's Vera Rubin NVL72 are the clearest expression of that shift — and they represent fundamentally opposite bets on how to build an AI factory.
Two Racks. Two Philosophies.
- ⚡72 Rubin GPUs + 36 Vera CPUs
- 🧠3.6 EFLOPS sparse NVFP4 inference
- 💾~21 TB HBM3e memory
- 🔗NVLink 6 fabric — proprietary
- ⚡190–230 kW power draw
- 🏗️800V DC architecture required
- 🔒Vertically integrated closed stack
- 🛠️CUDA + full software ecosystem
- ⚡72 Instinct MI455X GPUs + 6th Gen EPYC Venice CPUs
- 🧠">~2.4 EFLOPS (FP8 estimated)
- 💾31 TB HBM4 memory — 50% more
- 🔗UALink + open Ethernet scale-out
- ⚡~140 kW power draw
- 🏗️Meta Open Rack Wide (ORW) spec
- 🔓Open Compute Project (OCP) standard
- 🛠️ROCm + open software stack
Speed vs Memory: Two Different Bets on the AI Bottleneck
NVIDIA: Built for Speed, Unified Control
Vera Rubin is engineered for pure throughput and low-latency reasoning. By tying 72 Rubin GPUs together with 36 Vera CPUs over NVLink 6 — NVIDIA's proprietary copper spine fabric — the entire rack behaves as one massive, hyper-optimised compute engine. The hardware, interconnect, and CUDA software stack are co-designed from the ground up. That tight integration is precisely why NVIDIA can claim 3.6 EFLOPS of sparse NVFP4 inference. The tradeoff: you play entirely by NVIDIA's rules.
AMD: Built for Memory, Built for Openness
AMD made a different wager. Instead of competing solely on raw compute numbers, Helios bets that memory capacity is the real AI bottleneck — especially for large language models with massive context windows. At 31 TB of HBM4 across the rack and 432 GB per GPU, Helios offers 50% more memory than Vera Rubin. AMD also chose open standards throughout: Open Rack Wide physical spec, UALink interconnect, and standard Ethernet for scale-out — giving data centres the freedom to mix and match without vendor lock-in.
Memory Is the New Differentiator
For large language model inference — especially models requiring long context windows, multi-modal inputs, or large batch sizes — memory capacity has become just as important as raw floating-point throughput. A model that does not fit in GPU memory cannot run efficiently, regardless of how fast the compute units are.
288 GB per GPU
432 GB per GPU · +50% vs Vera Rubin
AMD's next-generation MI450 GPU is reportedly designed with 432 GB of HBM4, significantly ahead of the 288 GB expected on NVIDIA's Vera Rubin. For organisations running frontier models, agentic AI workloads, or memory-intensive inference pipelines, that extra capacity is not a marketing statistic — it directly determines what you can serve without model sharding or offloading.
What It Actually Takes to Run These Systems
Neither of these racks can be wheeled into a legacy raised-floor data centre. The facility requirements are substantial — and represent a real capital cost that belongs in any architecture decision.
NVIDIA's NVL72 draws 190–230 kW and drives an 800V DC architecture that forces electrical microgrid retrofits in most existing facilities. AMD's Helios, built on Meta's Open Rack Wide spec, draws roughly 140 kW and avoids the 800V mandate — a meaningful operational advantage for organisations deploying into existing infrastructure. The cooling story is unambiguous for both: air cooling is no longer viable at this density. Vera Rubin requires full liquid cooling; Helios uses a cooling manifold with quick-disconnect trays. What is striking is the roadmap: NVIDIA's Rubin Ultra, projected for 2027, is expected to draw approximately 600 kW per rack — a number that will force an entirely new generation of data centre design.
Walled Garden vs Open Coalition
The single biggest architectural difference between these two platforms is not the GPU count, the memory, or the power draw. It is the ecosystem philosophy — and that choice has long-term implications that extend well beyond the hardware purchase.
- CUDA remains the dominant AI software standard with the deepest library ecosystem
- NVLink provides unmatched intra-rack GPU-to-GPU bandwidth
- Grace CPU + Rubin GPU co-designed for unified memory access
- Validated reference architecture reduces deployment risk
- Omniverse, TensorRT, NeMo — a complete AI software platform
- Tradeoff: full vendor lock-in at every layer of the stack
- Open Compute Project (OCP) and UALink — no proprietary lock-in
- Standard Ethernet scale-out — mix and match networking vendors
- ROCm open-source software stack, increasingly PyTorch compatible
- Meta, Microsoft, Oracle and OpenAI already deploying Helios
- Pensando networking for programmable data-plane control
- Tradeoff: software maturity still catching CUDA in some areas
Microsoft's decision to deploy Helios across Azure — for frontier AI inference, Azure AI services, and enterprise workloads — is the most significant validation of AMD's open-stack bet to date. It joins Meta, OpenAI, and Oracle in adopting AMD's next-generation AI platform, signalling that the hyperscaler tier is no longer treating AMD as a backup option.
AI Is Reshaping the Semiconductor Supply Chain
The competition between these two racks is not just a story about two companies. It is a signal about how the entire AI supply chain is reorganising. Success in rack-scale AI now requires the ability to coordinate silicon design, advanced packaging, memory production, networking, and manufacturing at a scale that few organisations in the world can manage.
EPYC Venice + MI450
HBM4 Production
432 GB HBM4
Network Silicon
Meta ORW
Enterprise AI
Samsung and SK Hynix are both expanding HBM4 production capacity specifically to supply the next wave of AI accelerators. TSMC's advanced 2nm process will power AMD's EPYC Venice CPUs and MI450 accelerators. The ability to secure these inputs — and assemble them reliably at rack scale — is becoming as strategically important as the chip designs themselves.
Which One Is Right for You?
- ⚡You need the absolute fastest, tightly integrated, low-latency inference performance
- 🛠️Your teams are deeply invested in CUDA and the NVIDIA software ecosystem
- 🏗️You can retrofit your facility for 800V DC power infrastructure
- 📦You want a fully validated, single-vendor reference architecture with minimal deployment risk
- 🔬Your workloads are compute-bound rather than memory-bound
- 💾Memory capacity is your primary bottleneck — large models, long context, big batch sizes
- 🔓Vendor diversification and avoiding lock-in are strategic priorities
- ⚡Lower power draw and avoiding 800V facility upgrades matters operationally
- 🔧Your ops team values physical serviceability and the double-wide hot-swap design
- 💰Competitive pricing pressure against NVIDIA is a factor in your procurement
For most buyers today, NVIDIA Vera Rubin is the safer near-term choice — it offers the broadest software maturity, the most tuned reference architecture, and the clearest deployment path. Helios is the more compelling choice if you are thinking about where the constraints will be in 18–36 months: memory capacity, energy efficiency, openness, and the leverage that comes from not depending on a single vendor.
The Rack Has Become the Computer
What NVIDIA and AMD have built is not just a denser version of yesterday's server. It is a new unit of compute — one where the rack itself is the system, where the interconnect is as important as the processor, and where facility design, memory architecture, and software ecosystem are all part of the same purchasing decision.
The question for AI infrastructure teams in 2025 is not which GPU has the best benchmark. It is which rack-scale philosophy — closed and fast, or open and memory-rich — better matches the direction your workloads and your organisation are heading.
