Cloud comparisons often defend a decision that has already been made. This analysis comes from a team that sends jobs across AWS, GCP, and Azure, observing GPU access, spot interruption patterns, network speed, and developer experience over time. The view is limited to particular regions and workload types. Geography, preferred instance families, and job design will change your results. Treat these findings as recurring observations, not a final ranking.
GPU Availability: The Biggest Practical Surprise
Price usually dominates cloud comparisons, but availability causes more day-to-day frustration. A low spot quote offers little value when the required instance cannot be obtained.
AWS offers the broadest A100 and H100 capacity across US regions, supported by its longer ML compute history and larger cloud footprint. We generally see strong spot access to p4d.24xlarge (8xA100) in us-east-1 and us-west-2, with few failed requests during ordinary demand. Access to p5.48xlarge (8xH100) improved through 2024, though it remains tighter than A100 supply, especially beyond primary regions.
In us-central1, GCP can match AWS for single-zone A100 requests, while availability is thinner elsewhere. GCP a3 H100 instances were newer and less available in several regions through Q3 2024. Europe is a notable exception: europe-west4 has often offered strong A100 spot access at competitive prices, helping teams that can place jobs flexibly.
Azure can support moderate GPU workloads, but our observations found tighter supply than AWS or GCP for high-demand types during several periods. Its quota rules are also more cautious. New accounts typically start with lower GPU quotas, and increases take longer than on AWS or GCP. Teams beginning with Azure should submit quota requests early.
Availability keeps shifting as providers add capacity and demand changes. Conditions from six months ago may no longer apply. Checking live capacity when submitting a job is the dependable approach.
Spot Interruptions: Distribution Beats the Average
Provider averages can hide important interruption patterns. A 5% daily average could mean 5% of jobs stop randomly, or that 95% finish without interruption while 5% stop several times. Those outcomes require different planning.
Our observations place AWS A100 spot interruptions in high-availability regions around 3-8% per 8-hour window during ordinary demand. Frequency rises during busy periods, such as customer product launches and end-of-quarter billing pressure, then falls during off-peak times, including overnight US hours and weekends.
Based on our measurements, GCP A100 spot interruptions in us-central1 occur more often, at about 8-15% per 8-hour window. GCP does provide clearer capacity signals through its Spot VM pricing API, giving teams more information for submission timing.
Azure GPU spot interruptions have varied the most in our observations. Some periods show very few interruptions, followed by denser interruption periods, possibly reflecting changes in Azure capacity management and demand across its customer base.
These findings cover a limited window in mid-to-late 2024, particular regions, and particular instance families. Providers do not publish interruption rates, and unseen factors can change them substantially. Measure interruption behavior for your own jobs and locations.
Networking Throughput for Multi-Node Training
When multi-node training depends on inter-GPU communication, a provider's network design can matter more than its spot price.
AWS p4d and p5 instances use EFA, or Elastic Fabric Adapter, for node-to-node traffic and provide 400 Gbps in cluster placement groups. The design suits distributed ML training and gives NCCL collective operations consistent throughput. Because every node must share an availability zone, placement groups reduce the spot pool compared with zone-flexible single-node jobs.
GCP A3 H100 instances provide 10x200 Gbps intra-node NVLink and 800 Gbps inter-node GPU-to-GPU bandwidth through Google's custom network. A2 A100 instances provide 100 Gbps per node for inter-node traffic. For large multi-node jobs, GCP's network design competes closely with AWS EFA.
Azure ND instances with A100 and H100 GPUs use InfiniBand for node-to-node traffic, with 200-400 Gbps depending on the family. Azure InfiniBand has long performed well in HPC, which also benefits multi-node ML training.
For single-node training, meaning one 8-GPU instance, inter-node networking has no role. All 2 providers use the same underlying NVLink hardware because they purchase comparable NVIDIA DGX-style configurations. Network differences emerge only when a job spans multiple nodes.
Developer Experience and Ecosystem Fit
This category is subjective, but it affects team productivity. AWS has a mature ML toolset, including SageMaker, Deep Learning AMIs with preinstalled frameworks, and close S3 integration for training data. Teams already running other workloads on AWS may find its ML tools familiar.
Cloud TPUs distinguish GCP when a workload suits TPU architecture, particularly transformer training with JAX. For GPU jobs, GCP tools are capable but less prescriptive than SageMaker. GCS is fast and works well with PyTorch distributed filesystem abstractions.
Azure fits teams already using Microsoft's stack through Azure ML, VS Code extensions, and GitHub Actions. Its container support is strong, while AKS, or Azure Kubernetes Service, connects cleanly to GPU jobs when the training platform already uses Kubernetes.
Matching Providers to Workloads
Single-node experimental fine-tuning: GCP us-central1 A100 spot capacity is often cheapest during off-peak periods. AWS us-east-1 is usually the safer choice when dependable access matters more than the lowest price.
Multi-node distributed training: AWS EFA and placement groups offer predictable networking for NCCL-heavy jobs. GCP a3 is a strong alternative for very large jobs requiring H100-class hardware.
Large batch inference: Azure can offer competitive on-demand GPU prices in several regions, along with close integration with its API management and monitoring tools for Microsoft's ecosystem.
Running jobs across AWS, GCP, and Azure shows that no provider leads for every workload. Treating provider choice as permanent optimizes for a market snapshot that may already be outdated. Spot prices, capacity, and availability move over time. Infrastructure that can change providers dynamically has the practical advantage.