Calculate GPU count and cost for LLM training & inference
This calculator uses NVIDIA official benchmarks (NIM, 🥇 MLPerf™ v6.1, TensorRT-LLM) and 🏆 MLCommons Standards.
Tier 1: Megatron-Bridge official — LLaMA3-8B: micro_batch=2, 405B: micro_batch=1. Global batch = micro × GPUs × grad_accum.
All model weights updated. Highest memory usage.
Larger r = more trainable params = more memory. Default r=16 is standard for most tasks.
Total tokens = dataset tokens × epochs
Not used — applied from NVIDIA's official scaling result (MLPerf Training v5.1: 512 → 5,120 GPUs at 85%)
Longer context = more activation memory
Reduces per-GPU memory. Requires NVLink for TP4+
Splits model layers across GPU groups (e.g. NVIDIA's official Llama 3 70B H100 recipe uses PP8).
More micro-batches = less pipeline bubble (Narayanan et al. 2021, NVIDIA SC'21). Shown only — not applied to throughput: the official measurements already include their recipes' bubble (0.3–10.9%).
Uses curated NVIDIA NIM, MLPerf, and TensorRT-LLM benchmark data — not just model-size estimates.
Calculates training time and GPU count from official Megatron-Bridge measurements (or MFU back-calculated from them) and NVIDIA's official scaling — not just VRAM sizing.
Shows transparent source badges so you know whether results are benchmark-based or estimated.
Separates MoE weight memory from active compute for accurate sizing of DeepSeek, Mixtral, and Qwen models.
Grouped by architecture and precision support. Sorted by performance within each group.
| GPU | VRAM | FP16 TFLOPS (sparse) | Mem BW | FP16 | FP8 | FP4 | $/GPU-hr | Architecture |
|---|---|---|---|---|---|---|---|---|
| 🆕 Rubin (Preview) — FP16 / FP8 / NVFP4 | ||||||||
| Vera Rubin NVL72 Preview | 288 GB HBM4/GPU 20.7 TB system |
4,000 dense |
19,200 GB/s | ✓ | ✓ | ✓ | Contact for pricing | Rubin |
| ⚡ Blackwell Ultra — FP16 / FP8 / FP4 | ||||||||
| GB300 NVL72 🚀 Top | 279 GB/GPU 20 TB system (NVIDIA datasheet) |
5,000 | 8,000 GB/s 576 TB/s ÷ 72 |
✓ | ✓ | ✓ | Contact for pricing | Blackwell Ultra |
| 🔥 Blackwell — FP16 / FP8 / FP4 | ||||||||
| GB200 NVL72 | 186 GB/GPU 13.4 TB system |
5,000 | 8,000 GB/s | ✓ | ✓ | ✓ | $10.50 | Blackwell |
| B300 DGX (8-GPU) 🆕 MLPerf v6.1 | 270 GB/GPU 2.1 TB system |
4,500 | 7,700 GB/s NVIDIA datasheet |
✓ | ✓ | ✓ | Contact for pricing | Blackwell Ultra |
| B200 DGX (8-GPU) | 180 GB/GPU 1.4 TB system |
4,500 | 7,700 GB/s NVIDIA datasheet |
✓ | ✓ | ✓ | $8.60 | Blackwell |
| RTX PRO 6000 Blackwell Server Edition | 96 GB | 1,000 | 1,597 GB/s | ✓ | ✓ | ✓ | $2.50 | Blackwell |
| Hopper — FP16 / FP8 | ||||||||
| H200 SXM | 141 GB | 1,979 | 4,800 GB/s | ✓ | ✓ | — | $6.31 | Hopper |
| H100 SXM | 80 GB | 1,979 | 3,350 GB/s | ✓ | ✓ | — | $6.16 | Hopper |
| H100 PCIe | 80 GB | 1,513 | 2,000 GB/s | ✓ | ✓ | — | $6.16* | Hopper |
| Ada Lovelace — FP16 / FP8 | ||||||||
| L40S | 48 GB | 733 | 864 GB/s | ✓ | ✓ | — | $2.25 | Ada |
| L4 | 24 GB | 242 | 300 GB/s | ✓ | ✓ | — | Contact for pricing | Ada |
| Ampere — FP16 only | ||||||||
| A100 80GB SXM | 80 GB | 624 | 2,039 GB/s | ✓ | — | — | $2.70 | Ampere |
| A100 40GB | 40 GB | 624 | 1,555 GB/s | ✓ | — | — | $2.70* | Ampere |
Prices = the per-GPU-hour values used in cost calculations: CoreWeave on-demand list price (coreweave.com/pricing, checked 2026-10-27). Contact for pricing = no official on-demand list price (Vera Rubin, GB300, B300, L4). * = price of the same-chip SKU (H100 PCIe: HGX H100, A100 40GB: A100 80GB). GB300/GB200 TFLOPS and bandwidth are per-GPU figures; total system performance scales with 72 GPUs. Source: NVIDIA official specs + MLPerf v6.1.