The savings were real. The plan to capture them was not.
The spreadsheet always looks great. Your team prices 200 TB of Amazon S3 Standard against an S3-compatible alternative, sees the storage line fall by roughly 70 percent, and books the migration for next quarter. Then the first nightly ETL run on the new bucket takes three times longer, a LIST call against a 40-million-object prefix times out, and a multipart upload library that worked for six years throws an error nobody has seen before. The savings were real. The plan to capture them was not.
The fix is not more price comparison. It is a benchmark that uses your object sizes, your request mix, and your network path, run before a single production byte moves. This playbook walks through how to build one: what to measure, which tools to use, how to design the test matrix, which failure modes matter more than raw speed, and how to turn results into a go or no-go decision with a dollar figure attached.
Object storage pricing moved twice in 2026, and both moves pushed buyers back into their migration models. Backblaze raised B2 pay-as-you-go storage from $6 to $6.95 per TB on May 1, 2026, while making standard API calls free and keeping free egress up to three times stored volume (Backblaze, March 2026). Wasabi raised its pay-as-you-go and reserved capacity base price to $7.99 per TB on July 1, 2026 (Wasabi pricing notice, May 2026). Amazon S3 Standard still lists at $0.023 per GB, or $23 per TB, for the first 50 TB in US East.
That gap is still roughly three to one, so the economic case for moving hot data off a hyperscaler survives both increases. Meanwhile the pressure to act is rising: Flexera's 2026 State of the Cloud Report found estimated wasted IaaS and PaaS spend climbed to 29 percent, the first increase in five years, with AI workloads blamed for the reversal. Finance teams want the savings. Engineering teams own the risk. A benchmark is the document that lets both sign off.
Every provider publishes throughput numbers. They are measured on the provider's hardware, from clients sitting next to the storage, using object sizes chosen to flatter the result. None of that describes your workload.
Even the hyperscaler's own guidance shows how conditional these numbers are. AWS documents that an application can achieve at least 3,500 PUT, COPY, POST or DELETE requests and 5,500 GET or HEAD requests per second per partitioned prefix, and that latency-sensitive applications can see small-object and first-byte latencies of roughly 100 to 200 milliseconds (AWS S3 performance documentation, 2026). Note the words "partitioned" and "can." S3 partitions prefixes in response to sustained load and returns HTTP 503 Slow Down responses while it scales. Your current S3 performance is the product of years of that adaptation. A new provider starts cold.
So the question is never "which provider is fastest." It is "will my workload, on my network path, at my concurrency, behave acceptably on the target, and what does it cost to find out."
Teams that skip profiling end up benchmarking 1 GB objects at 32 threads because that is the tool's default, then migrate a workload made of 40 KB thumbnails. Pull five numbers from production before you open a benchmark tool.
Object size distribution. Use S3 Inventory to export every key and size, then bucket them: under 64 KB, 64 KB to 1 MB, 1 MB to 64 MB, over 64 MB. Small objects are latency and request-rate bound. Large objects are bandwidth bound. Most real buckets are bimodal, and you need to test both modes.
Request mix. S3 server access logs or CloudTrail data events give you the ratio of GET, PUT, HEAD, LIST and DELETE. A backup target is write-heavy and LIST-light. A media origin is GET-heavy. A data lake feeding Spark or a training pipeline is LIST-heavy, and LIST is where S3-compatible providers differ most.
Peak concurrency and throughput. Find the busiest 15 minutes in the last 90 days, not the average. That window is your pass or fail target.
Keys per prefix. Note the largest prefix by object count. Listing a prefix with a few thousand keys tells you nothing about listing one with 50 million.
Delete churn. Measure how many objects are deleted within 30 and 90 days of creation. This is a billing variable, not just a performance one, because some providers charge minimum storage durations.
Four options cover nearly every migration test. Pick by workload, not by popularity.
MinIO warp is the default for S3 API load testing. It runs mixed GET, PUT, DELETE and STAT workloads against any S3 endpoint, supports distributed client mode so one coordinator drives many load generators, and writes per-request CSV data you can re-analyze later. It is licensed AGPL-3.0, which matters only if you plan to modify and redistribute it.
elbencho tests file, block and object storage from one binary and can measure GPU data paths through NVIDIA CUDA and GPUDirect Storage. If the bucket you are moving feeds training jobs, elbencho lets you test the storage-to-GPU path rather than the storage-to-CPU path, which is the number your ML team actually cares about.
COSBench, originally from Intel, still appears in older vendor comparisons and RFP templates. Intel's own repository states it is not a product and has no full-time support team, so treat it as a legacy option rather than a default.
A custom SDK harness built with boto3 or the AWS SDK for your language is the only way to test your actual code paths: your retry settings, your multipart thresholds, your presigned URL flow. Run it alongside warp, not instead of it. Synthetic load proves capacity. Your own client proves compatibility.
| Tool | Best for | Distributed | Watch out for |
|---|---|---|---|
| MinIO warp | Mixed S3 API load on any endpoint | Yes | AGPL-3.0 license; defaults to random data |
| elbencho | Storage feeding GPUs; file, block and object in one tool | Yes | S3 support needs the S3 build or static binary |
| COSBench | Legacy RFP templates | Yes | Intel states no full-time support team |
| Custom SDK harness | Your real code paths and retries | Optional | Client CPU can cap results |
A useful benchmark is a grid, not a single run. Cross object size with concurrency and run each cell for long enough to reach steady state.
| Object size | Concurrency | Run time per cell | What it reveals |
|---|---|---|---|
| 64 KB | 16, 64, 256, prod peak | 10 min, drop first 2 | Request rate and latency ceiling |
| 1 MB | 16, 64, 256, prod peak | 10 min, drop first 2 | Typical app object behavior |
| 16 MB | 16, 64, 256, prod peak | 10 min, drop first 2 | Throughput per stream |
| 256 MB, multipart | 16, 64, 256, prod peak | 10 min, drop first 2 | Bandwidth ceiling and multipart compatibility |
Test at least four object sizes drawn from your profile, typically 64 KB, 1 MB, 16 MB and 256 MB, with the largest forced through multipart upload. Test concurrency at 16, 64 and 256 parallel requests, then at your measured production peak. Run each cell for at least 10 minutes and discard the first two, which absorb connection setup and any provider-side warm-up.
Run the load generators where the workload will run after migration. If your compute stays in AWS us-east-1 and the storage moves to another provider, test from us-east-1 and you will see the real cross-provider latency. If training will run on a neocloud GPU cluster, test from that cluster. Testing from a laptop or a single small VM measures the client, not the storage. AWS guidance suggests roughly one concurrent request for every 85 to 90 MB/s of target throughput, so a 10 Gbps target needs around 14 concurrent streams before the client NIC becomes the ceiling.
Record p50 and p99 latency per operation type, time to first byte for GETs, sustained throughput in MB/s, operations per second, and the error rate split by status code. A 503 or 429 rate above zero after warm-up is a finding. A p99 that is ten times the p50 is a finding even if the average looks fine, because tail latency is what stalls batch jobs and GPU data loaders.
Raw throughput rarely kills a migration. These five things do, and generic benchmarks skip all of them.
Load a test prefix with the same key count as your largest production prefix, then time a full paginated listing. Data lake engines and backup catalogs list constantly. A provider that is fast on GET but slow on LIST will look excellent in warp's default mixed test and fail in production.
S3-compatible means the common calls work. It does not guarantee identical behavior for multipart edge cases, checksum headers, object lock and retention modes, versioning with delete markers, lifecycle rules, conditional writes, or presigned URL expiry. Run your own client test suite against the target bucket and log every non-2xx response.
Ramp from zero to peak in one minute and watch for 503 or 429 responses. Then confirm your SDK retry configuration recovers cleanly instead of amplifying the load.
Wasabi pay-as-you-go applies a 90-day minimum storage duration, so objects deleted early are still billed for the remainder. Backblaze states that standard B2 has no minimum storage duration fee. If your profile shows heavy 30-day churn, as rotating backups often do, model it before you commit.
This is where storage meets GPU strategy. If your compute moves between hyperscalers and neoclouds as capacity shifts, every move drags data with it, and egress on that data can erase the savings that motivated the move. Backblaze offers unlimited free egress to approved neocloud, CDN and compute partners, and Cloudflare R2 charges no egress at all. Benchmark the actual route from the target bucket to the GPU cluster you expect to use, because adjacency and bandwidth decide whether expensive accelerators sit idle waiting for data.
Performance results only matter once they are priced. Model the full monthly bill, not the storage line. Here is a worked example for a workload storing 200 TB, sending 40 TB per month out to the internet or other clouds, and making 50 million writes and 500 million reads per month, at October 2026 US list prices.
Amazon S3 Standard comes to about $8,341 per month: $4,450 for storage, $3,441 for egress and $450 for requests. Backblaze B2 comes to $1,390, because the egress sits inside the three-times-stored allowance and API calls are now free. Wasabi comes to $1,598, since egress stays below stored volume and there are no API fees. Cloudflare R2 comes to about $3,397, with no egress charge but $397 in Class A and Class B operations.
| Provider | Storage | Egress + requests | Monthly total |
|---|---|---|---|
| Amazon S3 Standard | $4,450 | $3,891 | $8,341 |
| Cloudflare R2 | $3,000 | $397 | $3,397 |
| Wasabi | $1,598 | $0 | $1,598 |
| Backblaze B2 | $1,390 | $0 | $1,390 |
Then add the one-time exit cost. Moving 200 TB out of S3 at standard tiered transfer rates would cost about $13,791. AWS waives data transfer out to the internet for customers moving all of their data off AWS, through a support request and credits (AWS, 2024, updated 2025), but a partial migration does not qualify. At list rates, the move to B2 in this example pays back in roughly 2.0 months.
Budget the benchmark itself too. One hour at 2,000 PUTs and 10,000 GETs per second costs about $50.40 in request fees on S3 and about $45.36 on R2, and nothing on B2 since its API calls became free. Cheap, but a full matrix runs dozens of hours, so tear down test buckets the day you finish.
Write the pass criteria before the first test runs, so results cannot be rationalized afterward. A defensible scorecard has six lines.
Pass all six and you have a migration plan engineering and finance can both sign. Fail one and you have a specific, fixable problem instead of a surprise in production.
The price sheet tells you what a migration could save. The benchmark tells you whether you will actually keep it.
Compare AWS, Google Cloud, Azure, and alternatives like Backblaze B2 Discover how much you could save in seconds
