At a 40 percent headline discount, on a training run that checkpoints hourly, anything more frequent than one interruption every 86 minutes means you are paying more per unit of finished work than you would have paid on demand.
A 40 percent spot discount on an H100 sounds like a straightforward win. Work out how often the job actually gets interrupted and the picture changes. Most teams never run that calculation. They see the discount, book the savings in the forecast, and never reconcile it against the compute they actually consumed.
Spot GPU capacity in 2026 is not the bargain it was in 2021. The discounts have compressed, the pricing has stopped behaving like a market, and on a growing number of workloads the effective cost has crossed above the on demand rate. This is what that looks like in numbers, and what to do about it.
Spot was one of the cleanest bargains in cloud. Providers had idle capacity, buyers had interruptible work, and the market cleared at a steep discount. AWS still advertises savings of up to 90 percent off on demand rates for EC2 Spot, and for general purpose CPU fleets in unfashionable regions that figure is often close to real.
GPUs never behaved that way, and in 2026 they behave less like it than ever. A spot pool exists because a provider is holding capacity nobody is paying full price for. Since 2023 that condition has almost never held for H100, H200 and B200 class hardware. Accelerators are the constrained resource in every hyperscaler fleet, and constrained resources do not get dumped into a discount bin.
What buyers see instead is a discount that has drifted away from the advertised ceiling into a band that commonly sits between 20 and 50 percent, moves without warning, and arrives with interruption rates that make the headline number close to meaningless. The interesting question is no longer whether spot is cheaper on the price sheet. It is whether the discount is still large enough to pay for the interruptions it brings with it.
Three structural changes explain most of the compression, and none of them are going to reverse in the next twelve months.
EC2 Capacity Blocks for ML let buyers reserve GPU capacity for a defined window at a defined price. Before that product existed, a team needing two weeks of accelerator time with no annual commitment had one realistic option: bid on spot and engineer around eviction. Now they have a first class product that is not spot. That is a better outcome for the buyer and a better revenue outcome for the provider, and it removes precisely the population whose bidding used to make the spot market liquid. Google Cloud and Azure have shipped comparable reservation mechanics. Spot is no longer the escape hatch for short term GPU access, and it is not priced as though it were.
The word spot still implies a live clearing price. It has not meant that for years. Google Cloud documents that Spot VM prices change at most once every 30 days. AWS smoothed its spot pricing long ago and caps it at the on demand rate, so the price moves in gradual administered steps rather than bidding wars. Azure Spot runs on price caps with eviction notices measured in seconds.
This matters more than it sounds. An auction transmits scarcity in both directions: when capacity frees up, the price falls fast. An administered discount does not. When accelerators are tight, the provider has no commercial reason to widen the spread, and the discount stays narrow long after the underlying supply picture has changed. Buyers who assume spot pricing tracks real time availability are reading a number that was set weeks ago.
There is genuine idle capacity in the market, but it is concentrated in older accelerators. V100 and A100 class hardware has a long rental tail and still runs near capacity years after launch, which means the discount available on the silicon you want is not the discount available on the silicon there is a surplus of. If your workload can run on an A100, spot economics still look reasonable. If your model needs the memory bandwidth of an H200 or a B200, you are shopping in the tightest part of the market and the spot pool reflects that.
Every interruption on a long running job levies three separate taxes. You lose the work completed since the last checkpoint. You pay GPU time to write checkpoints frequently enough to limit that loss. And you wait for capacity to come back, which costs wall clock time even when it does not cost compute.
That gives a simple model for what a spot hour is really worth. If C is the checkpoint interval in hours, t is the GPU time each checkpoint write costs, and M is the mean time between interruptions, then the share of paid GPU time that produces no finished work is roughly (C divided by 2, divided by M) plus (t divided by C). The first term is the work you redo, assuming an interruption lands on average halfway through a checkpoint interval. The second is the write overhead itself.
Effective cost per useful GPU hour is therefore the spot price divided by one minus that wasted share. Take realistic values for a multi node training run: a one hour checkpoint interval, three minutes of GPU time per checkpoint write, and restart latency excluded entirely, which is a deliberately generous assumption in favour of spot. Set the effective cost equal to the on demand rate and solve, and the break-even mean time between interruptions is 0.5 divided by (the discount minus 0.05).
| Headline discount | Spot price per GPU hour | Break-even interval | Realised discount |
|---|---|---|---|
| 90 percent | $1.23 | 35 min | 87% |
| 70 percent | $3.69 | 46 min | 62% |
| 50 percent | $6.15 | 67 min | 36% |
| 40 percent | $7.37 | 86 min | 23% |
| 30 percent | $8.60 | 120 min | 11% |
| 20 percent | $9.83 | 200 min | minus 2% |
The result is not linear, and that is the whole story. A 90 percent discount buys enough headroom to absorb an interruption every 35 minutes. A 40 percent discount collapses that tolerance to 86 minutes. A 20 percent discount means anything more frequent than one interruption every three hours and twenty minutes puts you underwater. Below roughly a 5 percent discount, checkpoint write overhead alone consumes the entire saving and spot can never break even no matter how stable the capacity is.
Run the same model the other way, holding interruptions at one every three hours, and the realised discount is consistently far below the advertised one.
Inversion is not a rhetorical flourish. It happens in three distinct and independently verifiable ways, and most teams only recognise the third one.
This is the common case and the invisible one. The invoice shows a discount. The unit economics do not, because a meaningful share of the GPU hours purchased were spent redoing work or writing checkpoints. Nothing in a standard billing export separates useful GPU hours from repeated ones, so the loss never surfaces in a cost report. It surfaces as a training run that took nine days instead of six.
On genuine bid based GPU marketplaces, the interruptible price is set by whoever is willing to pay most for the same physical host. During demand spikes the clearing bid can rise to meet, and occasionally exceed, the operator's own fixed on demand rate for equivalent hardware. This is a literal price inversion, visible on the screen, and it happens because interruptible buyers competing for scarce hosts are not bidding against a list price, they are bidding against each other. Anyone running automated bidding without a hard ceiling is exposed to it.
The largest inversion has nothing to do with interruptions. Hyperscaler spot pricing routinely sits above neocloud on demand pricing for the same accelerator. A p5.48xlarge carries a published on demand rate of $98.32 per hour for eight H100s, which is $12.29 per GPU hour. Even a 40 percent spot discount only brings that to $7.37. Published on demand list pricing from specialist GPU providers has been sitting near $2.99 per GPU hour for the same 8x H100 SXM configuration. The discounted hyperscaler price is more than twice the undiscounted specialist price, for identical silicon, with eviction risk attached.
| Purchase mode | Per GPU hour | vs AWS on demand |
|---|---|---|
| On demand, AWS p5.48xlarge, 8x H100 | $12.29 | baseline |
| Spot at 40 percent off, AWS p5.48xlarge | $7.37 | 40% lower |
| Spot at 20 percent off, AWS p5.48xlarge | $9.83 | 20% lower |
| On demand, specialist provider, 8x H100 SXM | $2.99 | 76% lower |
That comparison is the one worth running, and almost nobody runs it. Teams benchmark hyperscaler spot against hyperscaler on demand, congratulate themselves on the delta, and never test the assumption that the baseline is the right baseline. The hyperscaler premium survives the spot discount comfortably.
None of this means abandon spot. It means stop treating the headline discount as the decision input. A few rules hold up well in practice.
Calculate the break-even interruption interval before you bid, not after the run. It takes one line of arithmetic and it converts an abstract discount into a concrete reliability requirement you can actually test.
Measure your real mean time between interruptions before you commit a pipeline to spot. Run a throwaway job on the target instance type in the target region for a week and count the evictions. Provider published interruption frequencies are fleet wide averages and tell you very little about the specific accelerator and region you need.
Treat checkpoint size as a cost variable rather than a fixed property of the model. Halving checkpoint write time moves the break-even point materially, and sharded or asynchronous checkpointing is usually cheaper to implement than the compute it saves.
Never put latency sensitive inference on spot. The break-even model assumes work can be redone. Missed inference requests cannot be, and no discount prices that correctly.
Compare against the right baseline. If the effective spot price at your interruption rate lands above a specialist provider's on demand rate, the spot discount is not the decision, the provider is. And when the discount on offer is under 30 percent, a short committed term at a lower rate is almost always the better instrument: you get the price certainty and you drop the eviction engineering entirely.
Spot stopped being a market and became an administered discount. Price it like one: measure the interruptions, do the arithmetic, and compare against every provider rather than the one you already use.
Compare AWS, Google Cloud, Azure, and alternatives like Backblaze B2 Discover how much you could save in seconds
