Can the A100 run Llama 3.3 70B?
Yes, with 80 GB or more. Llama 3.3 70B needs about 45 GB and the A100 has 80 GB of VRAM. Expect about 20 tok/s, about reading speed. The 40 GB version doesn't have room.
ollama run llama3.3:70bFirst time? Install Ollama, then paste that into a terminal.
Llama 3.3 70B needsAbout 45 GB
A100 has40 or 80 GB VRAM
Bandwidth1555 GB/s
Estimated speed~20 tok/s
By memory size
Each A100 version
- 40 GBToo big
- 80 GBUsable ~20 tok/s
Want it faster?
Speed comes from memory bandwidth: the card reads all 43 GB of active weights for every token. On the A100, gpt-oss 120B runs at about 190 tok/s if you can trade some quality for speed. Cards that run Llama 3.3 70B faster:
How we estimate
Speed is roughly memory bandwidth divided by the weights read per token, calibrated on our own rack. Sizes are Ollama's default 4-bit downloads plus about 1.2 GB for context. Real results are usually within about a third. Measured it yourself? Send us the number.
Keep going