Best GPUs to rent for LLM inference
For LLM inference, VRAM decides which model fits and price-per-hour decides your cost per token. A 24GB RTX 4090 serves 7B–14B models (or 4-bit 34B) at the best tokens-per-dollar on the market. Models around 70B need ~48GB in 4-bit — one L40S/RTX 6000 Ada or two 24GB cards. Latency-critical or high-batch production workloads still favor H100/A100 with HBM bandwidth.
Prices below are live minimum / median USD per hour on the GPUniq marketplace, as of 2026-07-21 00:37 UTC. Refreshed every 5 minutes.
| # | GPU | VRAM | Min $/hr | Median $/hr | Available | Why this card |
|---|---|---|---|---|---|---|
| 1 | RTX 4090 | 24GB GDDR6X | $0.30 | $0.56 | 1613 | Best tokens-per-dollar for 7B–14B (and 4-bit 34B) models; huge availability. |
| 2 | RTX 5090 | 32GB GDDR7 | $0.44 | $0.70 | 2780 | 32GB GDDR7 — fits bigger quantized models than the 4090 with a healthy speed bump. |
| 3 | RTX 3090 | 24GB GDDR6X | $0.15 | $0.22 | 1128 | Same 24GB as the 4090 at a steep discount — the value pick for steady-state serving. |
| 4 | L40S | 48GB GDDR6 | $0.70 | $1.21 | 68 | 48GB in one card — 70B models in 4-bit without multi-GPU complexity. |
| 5 | A100 | 40–80GB HBM2e | $0.80 | $1.31 | 199 | HBM bandwidth for high-batch serving; 80GB fits 70B in 8-bit. |
| 6 | H100 | 80GB HBM3 | $2.61 | $3.42 | 84 | Lowest latency and highest throughput when SLAs matter more than $/hr. |
Source: GPUniq live marketplace analytics (2026-07-21 00:37 UTC). Each linked page shows price history for 24h/7d/30d/365d.
Frequently asked questions
What GPU do I need to run a 70B model?
In 4-bit quantization a 70B model needs roughly 40–48GB of VRAM: one L40S / RTX 6000 Ada (48GB), or two 24GB cards (2× RTX 4090/3090) with tensor parallelism. For 8-bit you need ~80GB — an A100/H100 80GB or H200.
What is the cheapest way to serve a 7B–13B model?
A single RTX 3090 or RTX 4090 (24GB) running vLLM covers 7B–13B in fp16 and up to ~34B in 4-bit. On GPUniq these cards rent from well under a dollar per hour — check the live minimum price in the table above, which updates every 5 minutes.
Do I need an H100 for inference?
Only when latency SLAs, very large batch sizes, or 100B+ models demand HBM bandwidth. For most self-hosted assistants and batch pipelines, consumer 24GB cards deliver several times more tokens per dollar.