NVIDIA's roadmap is a memory story, not a compute story

If you read NVIDIA's datacenter roadmap the way the keynote slides want you to — bigger number, faster chip, buy the new one — you'll make bad infrastructure decisions. The A100-to-Rubin arc isn't really a story about compute. It's a story about memory bandwidth, ever-smaller number formats, and the quiet redefinition of what you're actually buying, which is no longer a GPU at all. Once you see those three threads, the whole roadmap becomes legible, and so does the decision about what your team should be renting in late 2026.

The tell: H200 shipped zero new compute

The A100 (Ampere, 2020) to H100 (Hopper, 2022) jump was a genuine compute leap — the Transformer Engine introduced FP8, and dense math throughput roughly tripled. But look at what NVIDIA did next. The H200 is the same Hopper silicon with the same tensor cores: 3,958 TFLOPS of FP8, identical to the H100. All that changed was memory — 141GB of HBM3e at 4.8TB/s, versus the H100's 80GB at 3.35TB/s. And that memory-only refresh delivered up to 1.9x on Llama 2 70B inference, by NVIDIA's own numbers.

That's the tell. LLM inference is overwhelmingly memory-bound: you're streaming weights and an ever-growing KV cache past the compute units, which spend most of their time waiting. NVIDIA figured out that the cheapest performance win in the AI buildout wasn't more FLOPS — it was feeding the FLOPS it already had. Every generation since has followed the pattern: memory capacity and bandwidth scale faster than compute, and the headline compute gains come mostly from shrinking the numbers themselves. Blackwell's NVFP4 4-bit format roughly doubles throughput over FP8 not by adding silicon but by halving each operand. Rubin's advertised 50 petaflops is an FP4 figure. Compare generations at the same precision and the curve is much flatter than the keynotes suggest.

Blackwell changed what a "GPU" is

Blackwell's B200 (2024–25) is two dies fused into one 192GB HBM3e package at 8TB/s, with the Blackwell Ultra refresh (B300/GB300) pushing to 288GB. But the spec sheet is the least interesting part. The real product is the GB200 NVL72: 72 GPUs and 36 Grace CPUs in one liquid-cooled rack, wired into a single NVLink domain, drawing on the order of 120 kilowatts.

That rack is the unit of purchase now, and it quietly disenfranchised most of the market. A 120kW liquid-cooled rack isn't something you slot into a typical colo cage built for 10–15kW air-cooled racks. The buyers are hyperscalers, neoclouds, and frontier labs with purpose-built facilities — which is why the interesting competition in AI infrastructure has shifted from chip design to power procurement and cooling. If your mental model is still "we'll buy some GPUs and put them in our datacenter," Blackwell is where that model died for the frontier tier.

Rubin and the depreciation treadmill

Rubin (named for Vera Rubin, paired with the Arm-based Vera CPU) is slated for the second half of 2026 as the NVL144: 288GB of HBM4 per GPU with bandwidth reported around 22TB/s, dual-die packages on TSMC 3nm, and power per GPU heading toward 1.8–2.3kW. Rubin Ultra follows in 2027 with the Kyber NVL576 rack and a claimed 1TB of HBM4e per package; Feynman is penciled in for 2028. NVIDIA has publicly committed to this annual cadence, and — credit where due — it has actually hit its dates since Hopper.

Understand what an annual cadence does economically: it's a deliberate depreciation machine. H100 rental prices have already fallen from their 2023 highs of $8-plus to as low as $1.50–$3/hour on specialist clouds as of August 2026, and every new generation shoves the previous one further down the price curve. If you bought H100s outright at peak, that curve is your write-down schedule. If you rent, it's your discount schedule. The roadmap's cadence is the single strongest argument for renting compute rather than owning it, unless your utilization is near-constant and your power is cheap.

What to actually run on in late 2026

Here's the practical read, with rough August 2026 on-demand pricing from the specialist clouds: H100s at $1.50–$3/hour, H200s at $2.50–$4, B200s at $7–$9.

For most teams — fine-tuning open-weight models up to ~70B, serving them, running batch inference — the H200 is the sweet spot. The 141GB means a 70B model in FP8 fits on a single GPU with headroom for KV cache, where the same job on H100s forces tensor parallelism across two cards and eats the savings in interconnect overhead and orchestration complexity. You're paying roughly a 40–50% premium over H100 for the thing that actually bounds your workload.

The B200's premium only pays off if you can exploit what's new in it: FP4 inference through TensorRT-LLM, FP8 training via Transformer Engine, or workloads that genuinely need the giant NVLink domain. This is the part teams consistently miss — precision gains aren't free. Running the same BF16 vLLM deployment on a B200 that you ran on an H200 captures a fraction of the on-paper gain at twice the price. The roadmap's compute curve is increasingly a software opt-in, and if your stack hasn't opted in, buy the older chip.

And don't wait for Rubin. Its first year of production is spoken for by the frontier labs and hyperscalers, just as Blackwell's was; realistic broad availability for everyone else is 2027, by which time B200 rental prices will have sagged toward today's H200 rates. That's the stable strategy the roadmap actually rewards: let the frontier labs pay the launch premium, stay one generation behind, and spend your attention on the two numbers that predict your performance better than any architecture codename — memory bandwidth, and whether your software stack speaks the newest number format.