Pricing pages leave out key details. They list on-demand rates and sometimes a spot estimate, but not what a month of workloads costs across regions, A100 and H100 capacity, and shifting spot markets. We measured actual spend. The provider gap was larger than expected, and the cheapest provider was not always the one most people would assume.
These findings come from internal benchmark runs during Q3-Q4 2024. We ran identical fine-tuning jobs on 8-GPU instances, using A100 80GB and H100 80GB where available, across AWS, GCP, and Azure in their primary US regions. Each workload used the same base model, dataset, batch size, precision settings, and training steps. We recorded billed cost, not estimates. For spot capacity, we submitted jobs and logged what we paid rather than scraping the pricing API, since submission-time rates differ from page averages.
One caveat: spot rates fluctuate. These figures reflect the period measured, while your results depend on submission time, region, and account quota tier. Treat them as directional benchmarks with a reasonable range, not exact current prices. Provider rates change, so the specific figures may have moved by the time you read this.
A100 80GB On-Demand Baseline
On-demand rates for 8xA100 80GB instances in primary US regions during the measurement period:
| Provider | Instance Type | Region | On-Demand $/hr (8 GPUs) | $/GPU-hr |
|---|---|---|---|---|
| AWS | p4d.24xlarge | us-east-1 | $32.77 | $4.10 |
| GCP | a2-highgpu-8g | us-central1 | $29.39 | $3.67 |
| Azure | Standard_ND96asr_v4 | eastus | $27.20 | $3.40 |
Azure had the lowest on-demand A100 rate during this period, at roughly 17% less per GPU-hour than AWS. GCP was in the middle. This is an observation, not a recommendation, since most fine-tuning should not run at on-demand rates. Spot prices and interruption overhead matter more.
A100 80GB Spot Shows the Spread
Spot rates for these instances came from multiple job submissions during the measurement period, rather than point-in-time snapshots:
| Provider | Avg Spot $/hr (8 GPUs) | Avg Spot $/GPU-hr | Discount vs On-Demand | Observed Interruption Rate |
|---|---|---|---|---|
| AWS us-east-1 | $9.85 | $1.23 | 70% | ~8% per 4hr window |
| AWS us-west-2 | $11.40 | $1.43 | 65% | ~5% per 4hr window |
| GCP us-central1 | $8.73 | $1.09 | 70% | ~12% per 4hr window |
| Azure eastus | $13.60 | $1.70 | 50% | ~4% per 4hr window |
GCP had the lowest average A100 spot rate in us-central1, but also the highest interruption rate during our measurement period. AWS us-east-1 offered a similar discount with fewer interruptions. Azure's spot discount was smaller, though interruptions occurred less often.
The interruption figures need context. An "8% per 4hr window" rate means an 8-hour AWS us-east-1 A100 spot job has roughly a 15-16% chance of at least one interruption. With checkpointing, each event costs at most the checkpoint interval in recomputation, not the whole job. Without checkpointing, the full job may need to run again.
H100 80GB Changes the Picture
H100 capacity had a different cost pattern during the benchmark. These newer instances were generally more available from AWS than from GCP or Azure across US regions in Q4 2024. That supply gap appeared in spot pricing.
| Provider | Instance Type | On-Demand $/GPU-hr | Avg Spot $/GPU-hr | Spot Discount |
|---|---|---|---|---|
| AWS | p5.48xlarge (8x H100) | $9.86 | $2.96 | 70% |
| GCP | a3-highgpu-8g (8x H100) | $10.08 | $4.03 | 60% |
| Azure | Standard_ND96isr_H100_v5 | $9.18 | $5.06 | 45% |
During our test period, AWS H100 spot was notably cheaper than GCP or Azure, with a 70% discount versus 45-60% elsewhere. Azure H100 spot availability was limited, and some requests failed during parts of the measurement period. Those failures are absent from interruption rates, but waiting for capacity still costs time.
Effective Cost With Interruptions
Spot price alone is incomplete because interruptions require recomputation. A job saving checkpoints every 15 minutes and interrupted once incurs about 15 minutes of extra GPU time. A 24-hour job interrupted three times with a 20-minute checkpoint interval incurs roughly an extra hour at the spot rate. Include this overhead in effective cost comparisons.
At GCP's interruption rate of ~12% per 4hr window, a 48-hour job statistically encounters 1-2 interruptions. With 20-minute checkpoints, recomputation adds roughly 20-40 minutes of GPU time at $1.09/GPU-hr. For an 8-GPU job, that means $1.45-$2.90 in overhead against roughly $420 total at GCP A100 spot rates. The overhead is under 1% of total cost. Even at higher interruption rates, checkpoint recovery costs far less than relying on on-demand capacity to avoid interruptions.
Regional Differences Within Providers
Regional variation within one provider was substantial. AWS us-east-1 A100 spot averaged $1.23/GPU-hr during our measurement period, while us-west-2 averaged $1.43/GPU-hr. That is a 16% gap for the same instance type on the same provider. Restricting jobs to one region can waste money just as surely as restricting them to one provider.
GCP showed even greater regional variation. During parts of our measurement period, Europe-west4 (Netherlands) A100 spot averaged roughly 8% less than us-central1. The tradeoff was higher latency to US training data in GCS, an important consideration for data-heavy pretraining workloads.
How to Use These Results
The lesson is not simply to choose GCP A100 spot or AWS H100 spot. Prices, regional availability, and provider spot markets change independently. A routing layer that checks current prices across selected providers and regions before each submission can capture savings that a fixed provider choice misses.
For a team using 150-200 GPU-hours per day on A100 or H100 capacity, multi-cloud routing that selects the cheapest available option at submission time typically saves 15-25% in absolute dollars versus single-provider spot at average rates, based on the patterns measured. The range reflects data variability. Teams open to European regions and interruption-tolerant workloads may reach the upper end. Teams limited to one geography or running latency-sensitive jobs may save less.
These benchmarks come from our internal measurement period. We publish them to give ML infrastructure teams a firmer basis for evaluating routing strategies than pricing-page estimates. They are not a performance guarantee. Check your own workloads, regions, and quota tiers before making infrastructure decisions from this or any benchmark.