AI GPU Infrastructure Sizing Calculator

Calculate GPU count and cost for LLM training & inference

v3.13.1 (26/10/27)

Based on NVIDIA Official Benchmarks + MLCommons Standards 🏆

This calculator uses NVIDIA official benchmarks (NIM, 🥇 MLPerf™ v6.1, TensorRT-LLM) and 🏆 MLCommons Standards.

Select Workload Type

Training Parameters

Tier 1: Megatron-Bridge official — LLaMA3-8B: micro_batch=2, 405B: micro_batch=1. Global batch = micro × GPUs × grad_accum.

All model weights updated. Highest memory usage.

Total tokens = dataset tokens × epochs

Not used — applied from NVIDIA's official scaling result (MLPerf Training v5.1: 512 → 5,120 GPUs at 85%)

Longer context = more activation memory

Reduces per-GPU memory. Requires NVLink for TP4+

Splits model layers across GPU groups (e.g. NVIDIA's official Llama 3 70B H100 recipe uses PP8).

Why GPUSIZE?

Uses curated NVIDIA NIM, MLPerf, and TensorRT-LLM benchmark data — not just model-size estimates.

Calculates training time and GPU count from official Megatron-Bridge measurements (or MFU back-calculated from them) and NVIDIA's official scaling — not just VRAM sizing.

Shows transparent source badges so you know whether results are benchmark-based or estimated.

Separates MoE weight memory from active compute for accurate sizing of DeepSeek, Mixtral, and Qwen models.

GPU Comparison

Grouped by architecture and precision support. Sorted by performance within each group.

Blackwell Ultra (FP16 / FP8 / FP4) Blackwell (FP16 / FP8 / FP4) Hopper / CDNA4 (FP16 / FP8) Ampere / Ada (FP16 only or FP8)
GPU VRAM FP16 TFLOPS (sparse) Mem BW FP16 FP8 FP4 $/GPU-hr Architecture
🆕 Rubin (Preview) — FP16 / FP8 / NVFP4
Vera Rubin NVL72 Preview 288 GB HBM4/GPU
20.7 TB system
4,000
dense
19,200 GB/s ✓ ✓ ✓ Contact for pricing Rubin
⚡ Blackwell Ultra — FP16 / FP8 / FP4
GB300 NVL72 🚀 Top 279 GB/GPU
20 TB system (NVIDIA datasheet)
5,000 8,000 GB/s
576 TB/s ÷ 72
✓ ✓ ✓ Contact for pricing Blackwell Ultra
🔥 Blackwell — FP16 / FP8 / FP4
GB200 NVL72 186 GB/GPU
13.4 TB system
5,000 8,000 GB/s ✓ ✓ ✓ $10.50 Blackwell
B300 DGX (8-GPU) 🆕 MLPerf v6.1 270 GB/GPU
2.1 TB system
4,500 7,700 GB/s
NVIDIA datasheet
✓ ✓ ✓ Contact for pricing Blackwell Ultra
B200 DGX (8-GPU) 180 GB/GPU
1.4 TB system
4,500 7,700 GB/s
NVIDIA datasheet
✓ ✓ ✓ $8.60 Blackwell
RTX PRO 6000 Blackwell Server Edition 96 GB 1,000 1,597 GB/s ✓ ✓ ✓ $2.50 Blackwell
Hopper — FP16 / FP8
H200 SXM 141 GB 1,979 4,800 GB/s ✓ ✓ — $6.31 Hopper
H100 SXM 80 GB 1,979 3,350 GB/s ✓ ✓ — $6.16 Hopper
H100 PCIe 80 GB 1,513 2,000 GB/s ✓ ✓ — $6.16* Hopper
Ada Lovelace — FP16 / FP8
L40S 48 GB 733 864 GB/s ✓ ✓ — $2.25 Ada
L4 24 GB 242 300 GB/s ✓ ✓ — Contact for pricing Ada
Ampere — FP16 only
A100 80GB SXM 80 GB 624 2,039 GB/s ✓ — — $2.70 Ampere
A100 40GB 40 GB 624 1,555 GB/s ✓ — — $2.70* Ampere

Prices = the per-GPU-hour values used in cost calculations: CoreWeave on-demand list price (coreweave.com/pricing, checked 2026-10-27). Contact for pricing = no official on-demand list price (Vera Rubin, GB300, B300, L4). * = price of the same-chip SKU (H100 PCIe: HGX H100, A100 40GB: A100 80GB). GB300/GB200 TFLOPS and bandwidth are per-GPU figures; total system performance scales with 72 GPUs. Source: NVIDIA official specs + MLPerf v6.1.