A three-year-old chip got more expensive while two newer generations flooded the market.
Here is a number that should not exist if GPUs aged like smartphones. Silicon Data's H100 neocloud rental index read $2.88 per GPU-hour on 24 October 2026, up from $2.00 in December 2025. That is a 44 percent rise in the same ten months that NVIDIA's Blackwell Ultra B300 went from launch to broad cloud availability. A three-year-old chip got more expensive while two newer generations flooded the market.
That tells you something most procurement decks never say out loud: the market does not price GPUs by generation. It prices them by the workloads they can run. A team choosing between the H100, H200, B200 and B300 today is not choosing old versus new. It is choosing between four different answers to one question: how much memory does the model need, and how fast does that memory have to move?
This guide works through that question with current specs, October 2026 rental prices, published benchmark data and a worked cost-per-token example, so you can match each generation to the job it actually does best and avoid paying Blackwell Ultra rates for a Hopper-sized problem.
Two years ago the choice was simple because there was barely a choice. Today all four generations rent side by side. GetDeploying's tracker listed B300 configurations from 37 providers on 27 October 2026, with a median on-demand price of $9.20 per GPU-hour across the 16 that publish one, up about 14 percent in 90 days.
Prices are moving up, not down. Nebius raised its on-demand B300 rate 21 percent on 1 October 2026, from $7.85 to $9.50 per GPU-hour, as reported by The Motley Fool. AWS lists its p6-b300.48xlarge instance at $142.42 per hour, or $17.80 per GPU.
The spread also depends on which index you read. Uvik's October 2026 survey put median on-demand prices at $4.20 for the H100 SXM, $5.70 for the H200, $8.19 for the B200 and $9.50 for the B300. Silicon Data's neocloud indexes, which track a narrower set of specialist providers, read $2.88, $3.48, $6.45 and $6.71 respectively. On the first index the H200 carries a 36 percent premium over the H100; on the second it is 21 percent. Your negotiating position depends on knowing which market you are actually buying in.
Marketing treats each generation as a clean step up. The engineering reality is three very different upgrades, and only one of them is a new architecture.
The H200 runs the same Hopper compute as the H100, with identical dense FP8 throughput of roughly 1.98 petaFLOPS. What changed is memory. NVIDIA's product page lists 141 GB of HBM3e at 4.8 TB/s, nearly double the H100's 80 GB and 1.4 times its 3.35 TB/s of bandwidth. Power stays at 700W and the software stack is unchanged, which makes the H200 a drop-in Hopper upgrade. Neither Hopper part supports FP4.
The B200 is where the real generational change happens. Blackwell is a dual-die design with 192 GB of HBM3e at 8 TB/s, native FP4 through a second-generation Transformer Engine, and fifth-generation NVLink at 1.8 TB/s per GPU, double Hopper's 900 GB/s. Dense FP8 throughput rises to roughly 4.5 petaFLOPS. The cost is power: up to 1,000W per GPU. One practical detail buyers miss: DGX and HGX B200 systems expose about 180 GB per GPU, not the 192 GB on the spec sheet.
NVIDIA's technical blog describes Blackwell Ultra as adding 50 percent more NVFP4 compute (15 petaFLOPS dense) and 50 percent more HBM per chip (288 GB) over Blackwell, plus doubled special function unit throughput that delivers up to twice the attention-layer performance. Memory bandwidth stays at 8 TB/s and power rises to 1,400W.
Read that list carefully. Inference provider fal notes that at BF16 or FP8, the precisions most production models still run at, the B300 has the same peak throughput as the B200. The B300 pays off when you quantize to NVFP4, serve attention-heavy long-context or reasoning workloads, or need 288 GB to keep a model on fewer GPUs. If none of those apply, you are renting a more expensive B200.
| GPU | Memory | Bandwidth | Max power |
|---|---|---|---|
| H100 (Hopper) | 80 GB HBM3 | 3.35 TB/s | 700W |
| H200 (Hopper) | 141 GB HBM3e | 4.8 TB/s | 700W |
| B200 (Blackwell) | 192 GB HBM3e | 8 TB/s | 1,000W |
| B300 (Blackwell Ultra) | 288 GB HBM3e | 8 TB/s | 1,400W |
| GPU | Dense FP8 | Dense FP4 | NVLink per GPU |
|---|---|---|---|
| H100 | ~1.98 PFLOPS | Not supported | 900 GB/s |
| H200 | ~1.98 PFLOPS | Not supported | 900 GB/s |
| B200 | ~4.5 PFLOPS | ~9 to 10 PFLOPS | 1.8 TB/s |
| B300 | Same as B200 | 15 PFLOPS | 1.8 TB/s |
Most LLM inference spends its time in the decode phase, where each new token requires reading the model's weights from memory. That makes memory capacity and bandwidth, not peak FLOPS, the binding constraint for the majority of serving workloads. Capacity also determines something more expensive: how many GPUs you need before the model even loads.
The math is simple. A parameter takes two bytes at FP16 and one byte at FP8. A 70B dense model at FP16 needs about 140 GB for weights alone, and a 671B mixture-of-experts model such as DeepSeek-R1 needs about 671 GB at FP8. Add a working reserve, round up to a tensor-parallel group size, and the GPU count falls out.
| GPU | 70B FP16 | 405B FP8 | 671B FP8 |
|---|---|---|---|
| H100 | 2 GPUs | 8 GPUs | 16 GPUs |
| H200 | 2 GPUs | 4 GPUs | 8 GPUs |
| B200 | 1 GPU | 4 GPUs | 8 GPUs |
| B300 | 1 GPU | 2 GPUs | 4 GPUs |
The 671B case is the one that changes architecture decisions. Eight H100s pool 640 GB, which cannot hold the FP8 weights, so the model spills across two nodes and inter-node networking becomes part of your latency budget. Eight H200s pool 1,128 GB and fit on one node. Four B300s pool 1,152 GB. Note that these counts cover weights only. Long-context and high-concurrency serving adds KV cache on top, and for reasoning models that generate thousands of output tokens per request, that cache can claim a large share of what is left.
Hourly price is what appears on the invoice, so it is what gets compared. It is also the least useful number in this decision. On Silicon Data's indexes the B200 costs 2.2 times the H100 per hour. Whether that is expensive depends entirely on how many tokens each GPU-hour produces.
The best public reference point is MLPerf Inference v5.1, which added a DeepSeek-R1 reasoning benchmark in September 2025. NVIDIA's published per-GPU figures for the Offline scenario were 1,253 tokens per second on a DGX H200, 4,024 on a GB200 NVL72 and 5,842 on a GB300 NVL72.
Divide the October 2026 neocloud index price by hourly token output and the order flips. The H200 works out to about $0.77 per million output tokens, the B200 class to about $0.45, and the B300 class to about $0.32. The GPU with the highest hourly price produces tokens roughly 2.4 times cheaper than the H200.
| GPU class | Price per GPU-hour | Tokens/sec/GPU | Cost per 1M tokens |
|---|---|---|---|
| H200 (DGX H200) | $3.48 | 1,253 | $0.77 |
| B200 (GB200 NVL72) | $6.45 | 4,024 | $0.45 |
| B300 (GB300 NVL72) | $6.71 | 5,842 | $0.32 |
Treat those figures as directional, and know why. They are vendor-submitted results from the company that sells the chips. NVIDIA notes that per-GPU throughput is not a primary MLPerf metric, and its Hopper figure was not verified by MLCommons. The Blackwell numbers come from 72-GPU NVL72 racks, whose single NVLink domain suits mixture-of-experts models far better than the 8-GPU HGX servers most teams actually rent, so an HGX B200 or B300 will likely land below them. Offline results also measure maximum batch throughput, not the latency-bound interactive serving that user-facing products need. Run your own model on your own stack before signing anything.
The annual view makes the stakes concrete. An 8-GPU node running around the clock for a year at Silicon Data's October rates costs about $201,830 on H100, $243,878 on H200, $452,016 on B200 and $470,237 on B300. If your workload cannot use the extra throughput, the Blackwell node simply doubles the bill.
And if it can, remember that a cheaper token is not a cheaper bill. Teams that move to Blackwell Ultra to cut cost per token rarely bank the savings. They spend them on longer reasoning traces, larger context windows and more agent calls, and total GPU spend keeps rising with adoption.
The H100 remains the right answer for models up to roughly 30B parameters at FP16 or 70B at FP8 with modest context, LoRA and QLoRA fine-tuning of 7B to 13B models, embedding generation, batch inference, computer vision and development environments. It has the lowest hourly price, the widest provider choice and the most mature software ecosystem. The catch is that its price is rising, which argues for pricing a 6 to 12 month reservation rather than riding on-demand rates up.
The H200 is the most underrated option on this list. It delivers 43 percent more bandwidth and 76 percent more capacity than the H100 for a 21 percent premium on Silicon Data's index. For decode-heavy serving of 70B-class models at longer context, that usually means more tokens per dollar with zero code changes. It also hosts a 671B FP8 model on a single 8-GPU node, which the H100 cannot.
The B200 earns its premium on training runs where dense FP8 compute and NVLink 5 bandwidth shorten time to result, and on high-volume serving where FP4 quantization is acceptable. It is the generation where the software investment starts to matter, because kernels and serving frameworks need Blackwell-specific tuning to reach published performance.
The B300 is built for reasoning models, agentic pipelines, long-context inference and models of 400B parameters and up served at NVFP4. Here is the counterintuitive part: on Silicon Data's indexes it costs only 4 percent more per hour than the B200 (16 percent on Uvik's medians). If you were going to rent a B200 for frontier inference, the B300 is usually the better buy at near-equal price. The exception is a stack locked to FP8 or BF16 that does not need 288 GB, where the extra spend buys nothing.
Maximum power per GPU climbs from 700W on both Hopper parts to 1,000W on the B200 and 1,400W on the B300. In the cloud that cost is folded into the hourly rate. For teams running on-premises, in colocation or on local metal deployments, it determines whether existing racks, power feeds and cooling can host the hardware at all. The GB300 NVL72 rack, for example, is a liquid-cooled system.
The newest generation is where broker clouds multiply, because scarce capacity is easy to resell. As Sunny Smith of Massed Compute put it on Episode 7 of the DataStorage.com Podcast, the first question to ask any provider is whether they own the GPU. He also warned that generic containers tuned for one GPU can leave 30 to 50 percent of performance on the table when deployed elsewhere. Moving from Hopper to Blackwell Ultra without retuning is the fastest way to turn a B300 into an expensive B200.
Chasing a GPU generation often means switching providers, and every switch drags data along. Moving a 500 TB training corpus or checkpoint archive out of AWS at standard internet egress rates costs about $29,482. At Silicon Data's October B300 rate, that is roughly 4,394 B300 GPU-hours, or one 8-GPU node for about 23 days, spent before a single token is generated. Keeping training data and checkpoints on zero-egress, S3-compatible storage such as Backblaze B2, Wasabi or Cloudflare R2 lets you follow GPU price and availability without paying a toll each time you move.
Stop asking which GPU is newest. Ask which GPU holds your model on the fewest chips at the lowest cost per token, and the generation picks itself.
Compare AWS, Google Cloud, Azure, and alternatives like Backblaze B2 Discover how much you could save in seconds
