Can the Jetson AGX Orin run Qwen3 32B?
Yes. Qwen3 32B needs about 22 GB and the Jetson AGX Orin with 32 GB of unified memory leaves about 24 GB for models. Expect about 6 tok/s, slow, but it works.
ollama run qwen3:32bFirst time? Install Ollama, then paste that into a terminal.
Qwen3 32B needsAbout 22 GB
Jetson AGX Orin has32 or 64 GB unified memory
Bandwidth205 GB/s
Estimated speed~6 tok/s
By memory size
Each Jetson AGX Orin version
- 32 GB (about 24 GB usable)Slow ~6 tok/s
- 64 GB (about 48 GB usable)Slow ~6 tok/s
Want it faster?
Speed comes from memory bandwidth: the card reads all 20 GB of active weights for every token. On the Jetson AGX Orin, Qwen3 30B-A3B runs at about 36 tok/s if you can trade some quality for speed. Cards that run Qwen3 32B faster:
How we estimate
Speed is roughly memory bandwidth divided by the weights read per token, calibrated on our own rack. Sizes are Ollama's default 4-bit downloads plus about 1.2 GB for context. Real results are usually within about a third. Measured it yourself? Send us the number.
Keep going