The network that connects your GPUs decides how much of that rented time does useful work, and most buyers never ask a single question about it.
You can negotiate an H100 rate down to the cent and still waste a fifth of every hour you pay for. The network that connects your GPUs decides how much of that rented time does useful work, and most buyers never ask a single question about it.
Meta learned this at scale. When its engineers stood up two 24,576-GPU training clusters, they reported that small clusters reached over 90 percent of expected communication bandwidth out of the box, while untuned large clusters swung anywhere from 10 to 90 percent (Meta Engineering, 2024). Same GPUs, same software. The difference was the fabric and how well it was tuned.
That fabric is now the subject of a genuine market war. On one side is InfiniBand, the purpose-built, NVIDIA-controlled interconnect that dominated AI training. On the other is Ethernet, rebuilt for AI by NVIDIA itself, Broadcom, Arista, Cisco and an industry consortium. This article explains what actually differs, who is winning, what the choice costs you per training hour, and the questions to put to any GPU provider before you sign.
For years the back-end network was an engineering detail buried inside a cluster quote. Two things changed in 2026. First, spending flipped. Dell'Oro Group reported in September 2026 that switch sales in AI back-end networks surpassed front-end data center networks for the first time in 2Q 2026, roughly three years after the category emerged. The network that links GPUs to each other is now a bigger market than the network that links everything else.
Second, the money is flowing to whoever sells it. NVIDIA, which sells both fabrics, reported second quarter fiscal 2027 revenue of $96.2 billion in August 2026, and told investors that networking revenue hit a record with Spectrum-X Ethernet growing 2.6 times year over year. A year earlier, in its second quarter of fiscal 2026, networking was already $7.3 billion a quarter, nearly doubling year over year.
When a cost line grows that fast, it ends up in your bill whether or not it appears as a separate item. If you rent GPUs, the fabric is priced into the node rate. If you build, it is a line item you have to defend. Either way, you are buying it.
Distributed training is a synchronisation problem. Every few hundred milliseconds, thousands of GPUs exchange gradients in collective operations such as all-reduce. The slowest link sets the pace for everyone. A fabric for AI therefore has to do two things well: move data without dropping packets, and keep tail latency (the slowest few percent of transfers) under control when thousands of flows collide.
InfiniBand uses credit-based flow control: a sender only transmits when the receiver has advertised buffer space, so packets are not dropped in normal operation. Remote Direct Memory Access (RDMA), which lets one machine read and write another machine's memory without involving the CPU, is native. NVIDIA's current Quantum-X800 generation runs at 800 Gb/s per port and adds SHARP, its in-network computing feature that performs reduction operations inside the switch rather than on the GPUs, according to NVIDIA's product documentation. The trade-off is that InfiniBand is effectively a single-vendor ecosystem, with its own management tooling and its own skills requirement.
Ethernet was built to tolerate loss, so AI Ethernet has to be engineered into losslessness. The standard approach is RDMA over Converged Ethernet version 2 (RoCEv2), held together by priority flow control, explicit congestion notification and NIC-side rate control. It works, but it has to be tuned, and badly tuned RoCE is where most horror stories come from.
The industry has spent two years closing that gap. The Ultra Ethernet Consortium, founded by AMD, Arista, Broadcom, Cisco, Eviden, HPE, Intel, Meta and Microsoft, published its 1.0 specification in June 2025, defining a modern RDMA transport and congestion control for multi-vendor AI networks. Broadcom began shipping Tomahawk 6 the same month, a single switch chip with 102.4 Tb/s of capacity, double any Ethernet switch then on the market and compliant with the UEC specification. NVIDIA's answer is Spectrum-X, which pairs its own Ethernet switches with its SuperNICs and adaptive routing.
On market share, the war looks settled. When Dell'Oro began tracking AI back-end networks in late 2023, InfiniBand held over 80 percent share. By the fourth quarter of 2025, Ethernet accounted for more than two-thirds of switch sales in AI clusters, for the quarter and for the full year. Ethernet kept roughly two-thirds in 1Q 2026 despite what Dell'Oro called a strong InfiniBand rebound, and extended its lead again in 2Q 2026.
Here is the part the headline misses: Ethernet winning does not mean NVIDIA losing. NVIDIA ships both fabrics, and on its third quarter fiscal 2026 call management said Ethernet GPU attach was roughly on par with InfiniBand, naming Meta, Microsoft, Oracle and xAI as builders on Spectrum-X. Dell'Oro's 2Q 2026 vendor ranking for Ethernet AI back-end switches had Celestica first and NVIDIA a close second. Choosing Ethernet over InfiniBand does not automatically get you out of a single-vendor stack. Choosing an Ethernet fabric built on merchant silicon and UEC-compliant NICs does.
The most-quoted Ethernet proof point is xAI's Colossus, a 100,000-GPU Hopper cluster built on Spectrum-X. NVIDIA says it sustained 95 percent data throughput, against 60 percent for what it calls standard Ethernet at that scale. That is a vendor comparing its premium product against an unnamed baseline, and it should be read as a marketing ceiling rather than an independent benchmark. The more useful evidence is Meta's, because Meta sells neither fabric: it ran comparable 24,576-GPU clusters on both InfiniBand and RoCE, and after significant tuning work reported that the network was no longer a meaningful bottleneck on either. The lesson is not that one fabric wins. It is that tuning wins, and tuning takes people.
Network performance matters to a buyer because a stalled GPU still bills. Take a concrete case. CoreWeave lists an on-demand HGX H100 node (eight GPUs) at $49.24 per hour on its pricing page as of October 2026. A 1,024-GPU training cluster is 128 nodes, or $6,302.72 per hour.
Now assume some share of each training step is spent with GPUs waiting on communication that a better fabric or better tuning would have hidden. At 5 percent, that is $315.14 per hour, or about $226,898 over a 30-day run. At 15 percent, it is roughly $680,694. At 20 percent, close to $907,592 a month disappears into the network.
| Stall share of step time | Wasted per hour | Wasted per 30 days |
|---|---|---|
| 5% | $315.14 | $226,898 |
| 10% | $630.27 | $453,796 |
| 15% | $945.41 | $680,694 |
| 20% | $1,260.54 | $907,592 |
Two conclusions follow. A premium for a better-performing fabric is usually small next to the cost of idle GPU time, so the cheapest network is rarely the cheapest cluster. And the reverse is equally true: an Ethernet fabric that saves capital but runs untuned can erase that saving within weeks. The cheaper token is not the cheaper bill, and the cheaper switch is not the cheaper cluster. Run your own rates through the GPU Price Explorer and substitute your measured stall rate; the shape of the result does not change.
There is no universal winner, but there are clear patterns. Fine-tuning jobs on two to eight nodes rarely saturate either fabric, so the choice barely matters and price should decide. Mid-size training jobs from a few hundred to a few thousand GPUs are where a well-run InfiniBand cluster from a neocloud is often the lowest-risk option, because the provider has already done the tuning you would otherwise have to staff. Frontier-scale training can go either way, and increasingly goes Ethernet, but only for organisations with a dedicated network engineering team. Multi-tenant inference platforms and multi-site "scale-across" deployments lean Ethernet, because isolation, existing operational tooling and long-distance links are Ethernet's home ground.
| Workload | Typical scale | Better fit | Why |
|---|---|---|---|
| Fine-tuning | 2 to 8 nodes | Either | Rarely saturates the fabric; price decides |
| Mid-size training | 256 to 4,096 GPUs | InfiniBand (provider-run) | Lossless by default; tuning already done |
| Frontier training | 10,000+ GPUs | Either | Decided by in-house network team capability |
| Multi-tenant inference | Any | Ethernet | Isolation, existing tooling, lower cost |
| Multi-site scale-across | Multiple data centers | Ethernet | Long-distance links are Ethernet territory |
The honest decision variable is not the protocol. It is who does the tuning. If the answer is "the provider," buy a fabric the provider has proven at your node count. If the answer is "us," buy the fabric your team already knows how to operate.
Every comparison above concerns the compute fabric, the GPU-to-GPU network. There is a second network underneath: the path between GPUs and storage that carries training data in and checkpoints out. It is usually separate, often Ethernet, and almost never benchmarked by buyers. A cluster with a superb InfiniBand back end can still stall every time it writes a multi-terabyte checkpoint to a storage tier that sits too far away.
Sunny Smith, founder and CTO of Massed Compute, made this point on Episode 7 of the DataStorage.com Podcast: storage has to sit adjacent to the GPUs with high east-west bandwidth, and moving a petabyte between data centers is prohibitive. That is data gravity in network form. Your fabric decision and your storage decision are the same decision.
It is also why egress belongs in a fabric discussion. GPU capacity shifts between providers, and every move drags the dataset with it. Keeping the master copy of training data in zero-egress, S3-compatible object storage such as Backblaze B2, Wasabi or Cloudflare R2 means a provider switch costs you transfer time rather than a toll. Some neoclouds, CoreWeave among them, now list free egress on their own storage, which is worth confirming in writing before a contract rather than after.
Use these in your next RFP or vendor call. A provider that cannot answer them precisely is telling you something.
For a broader view of how neoclouds differ from hyperscalers on exactly these points, Russ Artzt's conversation on Episode 5 of the DataStorage.com Podcast is a useful primer.
Speed transitions reshuffle markets, and one is underway. Dell'Oro expects 1.6 Tb/s switches, which began sampling in early 2026, to ramp in the second half of 2026, and projects the majority of AI back-end ports to be 1.6 Tb/s by 2027. Co-packaged optics, which move the optical transceiver onto the switch package to cut power, were expected to begin volume ramp on both InfiniBand and Ethernet switches in 2026. And Ethernet is pushing into scale-up, the GPU-to-GPU domain inside a rack where NVLink has dominated, through efforts such as Broadcom's Scale Up Ethernet framework. If Ethernet takes scale-up too, the fabric question becomes a vendor-stack question at every layer.
You are not renting GPUs. You are renting synchronised GPUs, and the network decides how many of them are actually working.
Compare AWS, Google Cloud, Azure, and alternatives like Backblaze B2 Discover how much you could save in seconds
