Your AI infrastructure bill is not high because GPUs are expensive. It is high because the people configuring them are learning on the job, at production scale, on hardware billed by the hour.
Ninety eight percent of teams report a gap in AI infrastructure skills. As a statistic it is easy to wave away, because it is usually deployed to sell a training subscription. As a line on next month's invoice it is much harder to wave away, because the gap never appears in a budget labelled as a gap. It appears as idle GPU hours, oversized nodes, storage sitting in the wrong region, egress nobody modelled, and a three year commitment signed by someone who had never priced a spot market.
Those five items are the same five items every time, and all of them are measurable. What follows puts a number on each one using a 64 GPU fleet as the model, then compares that number against the fully loaded cost of simply hiring the people who would have prevented it. The answer is not close.
The shortage is real, but it is narrower than the headline suggests. Teams are not short of machine learning researchers. Models are commoditising and the talent pipeline for using them is healthy. What teams are short of is the layer between the model and the metal: people who know how a scheduler bin packs jobs onto eight GPU nodes, why a container tuned for one accelerator underperforms on another, and what happens to the bill when training data lives in a different region from the cluster reading it.
On the DataStorage.com Podcast, Sunny Smith, founder and CTO of Massed Compute, put the global population of people who can build AI infrastructure end to end at tens of thousands. Measured against the number of enterprises now trying to run inference in production, that is not a skills gap so much as a rationing system. Flexera's State of the Cloud research has listed lack of resources and expertise among the top cloud challenges for years running, and AI workloads have made that constraint sharper, not softer.
The practical consequence is that GPU and storage decisions get made by whoever happens to be nearest. Usually that is an application engineer who is competent, overloaded, and has never had a reason to care about the difference between an on demand GPU rate and a committed one. They are not the problem. The absence of anyone whose job it is to care is the problem.
Five leaks account for almost all of it. They are not exotic. Each one is the direct fingerprint of a missing skill, and each one is visible in billing data that most teams already have and do not read.
This is the largest leak and the least visible, because a rented GPU that is doing nothing looks identical on an invoice to a rented GPU that is saturated. Cast AI's optimization research has repeatedly found production clusters running at a fraction of provisioned capacity, with median GPU utilization in the low single digits. The pattern is always the same: a reservation is created for a training run, the run finishes, and nothing releases the reservation. Nobody owns expiry, so nothing expires.
A container image tuned against one accelerator does not carry that tuning to another. In his own testing, Smith found that generic containers leave 30 to 50 percent of available performance unused when deployed onto hardware they were not profiled against. That does not show up as a performance complaint, because the job still completes. It shows up as more GPU hours to reach the same result, which is the same thing as a higher rate with extra steps. It is the same quiet mechanism that has turned AI coding costs into an enterprise budget problem: nothing breaks, the meter simply runs longer.
Storage decisions made without reference to where the GPUs sit are the fastest way to build a bill that cannot be optimised later. Training and inference need high east to west bandwidth between the accelerator and the dataset, and once a few hundred terabytes have accumulated in one provider's region, moving it becomes a capital decision rather than an operational one. This is the practical reason storage is the anchor of the AI infrastructure stack rather than an afterthought bolted on at the end.
It is also where zero egress providers such as Backblaze B2, Wasabi and Cloudflare R2 stop being a procurement preference and start being an architecture decision. The entire point of a multi provider GPU strategy is the freedom to move, and egress is what prices that freedom.
The remaining two leaks are quieter. Wrong instance shape means an eight GPU node running a two GPU job because that was the default in the console. Avoidable egress means paying to pull the same dataset out of the same bucket every time a job runs in a different provider, instead of staging it once.
The model below assumes a 64 GPU H100 fleet rented on demand at $2.99 per GPU hour, which sits inside the mid 2026 on demand band for specialist providers such as CoreWeave, Lambda Labs and Nebius, and well below hyperscaler list rates for the same silicon. It assumes 730 hours in the month, 500 TB of hot object storage plus NVMe scratch at $12,500, and 220 TB of monthly egress at $0.09 per GB. That produces a gross monthly bill of $171,993.
The waste figures below describe an unmanaged fleet: one with no dedicated platform owner, no reservation expiry policy, and no storage placement review. A well run team recovers most of this. A team without the skills recovers none of it, month after month, because nobody is looking.
| Leak | Monthly cost | Share | Root cause |
|---|---|---|---|
| Idle and unscheduled GPU hours | $43,305 | 25% | No owner for reservation expiry |
| Untuned container overhead | $25,145 | 15% | Images not profiled against the target accelerator |
| Avoidable egress | $14,850 | 9% | Datasets pulled per job instead of staged once |
| Wrong instance shape | $12,572 | 7% | Eight GPU nodes running two GPU jobs |
| Cross region storage placement | $6,200 | 4% | Storage chosen without reference to cluster region |
| Total recoverable | $102,072 | 59% | Every one of them, a missing owner |
Total recoverable spend is $102,072 per month, or just under 60 percent of the bill. Annualised, that is $1,224,864. It is worth sitting with that number before reading the next section, because the comparison it invites is uncomfortable. If you want to run the storage and egress side against your own volumes, the Cloud Cost Calculator will do it in a couple of minutes.
The obvious response is to hire. It is the right response, and it is also slow. A senior platform engineer with real GPU scheduling experience takes four to six months to find and another quarter to become useful on an unfamiliar stack. During those nine months the leak continues at full rate, which in the model above costs more than $900,000.
There is a structural problem underneath the timeline. The people who can do this work already have jobs, and increasingly those jobs are at the providers themselves. Neoclouds are hiring the same profile enterprises need, and they can pay for it out of margin on the capacity they sell. Enterprises are competing for talent with their own suppliers.
Which leaves buying the expertise rather than employing it, and that is where provider selection stops being a pricing exercise. Smith's framing is the sharpest test available: ask any provider whether they own the GPU. Operators who own and run their fleet can debug at the level the problem actually lives at. Brokers reselling someone else's capacity cannot, and tickets get copy pasted between companies while the meter runs. The support tier attached to your compute contract is a skills purchase, whether it comes from AWS or from an operator you have never heard of, and it should be evaluated as one.
Most teams try to solve this with one hire and a dashboard. The functions below are separable, and three of the four can start as fractional or shared roles. What matters is that each leak has a named owner.
| Role | What it owns | Annual cost | Leak it closes |
|---|---|---|---|
| Platform engineer, GPU scheduling | Queueing, bin packing, node shapes, reservation expiry | $240,000 | Idle hours and wrong instance shape |
| Storage and data architect | Placement, tiering, replication, egress paths | $210,000 | Storage placement and avoidable egress |
| FinOps practitioner | Showback, anomaly alerts, commitment modelling | $165,000 | Waste found in month, not at renewal |
| Procurement lead, partial allocation | Contract terms, exit clauses, rate benchmarking | $70,000 | Above market rates and lock in |
| Total | Four named owners, one per leak | $685,000 | Against $1,224,864 recoverable |
Funded in full, those four roles cost $685,000 a year against $1,224,864 of recoverable spend on a single 64 GPU fleet. The team pays for itself and leaves $539,864 on the table. Most organisations running AI workloads have more than one fleet.
Nobody closes a skills gap in a quarter. Everybody can close the measurement gap in a quarter, and measurement is what turns an abstract shortage into a funded headcount request.
That last step is where the money comes back fastest, because a benchmarked utilization figure changes what you are willing to commit to. The choice between reserved, on demand and spot capacity is only answerable once you know how much of what you already bought is actually running.
The skills gap is not a line item in an HR plan. It is already a line item on your invoice, and you have been paying it every month without seeing it.
Compare AWS, Google Cloud, Azure, and alternatives like Backblaze B2 Discover how much you could save in seconds
