Qwen3 32B
The strongest dense model that fits on one 24 GB card. Slower than the 30B MoE, but more careful on hard problems and long answers.
ollama run qwen3:32bNeeds about 22 GB of graphics memory, or unified memory on a Mac. First time? Install Ollama, then paste that into a terminal.
Download20 GB
Memory to runAbout 22 GB
Read per token20 GB
LicenseApache 2.0
Context32K tokens, 128K with YaRN
ReleasedApril 2025
Good for
- Serious coding work
- Long, detailed answers and reports
- Hard reasoning with thinking mode
Not the best pick for
- Slower cards: as a dense model it reads all 20 GB per token
57 GPUs and Macs
What runs Qwen3 32B
Estimated speed on each card at the smallest memory size that fits it, from memory bandwidth. As a dense model it reads all its weights for every token, so memory bandwidth sets the speed.
Runs great 7 cards
Usable 35 cards
- Instinct MI5032 GB · ~28 tok/s
- RTX 3090 Ti24 GB · ~28 tok/s
- RTX 409024 GB · ~28 tok/s
- RTX 6000 Ada48 GB · ~26 tok/s
- RX 7900 XTX24 GB · ~26 tok/s
- RTX 309024 GB · ~26 tok/s
- A3024 GB · ~26 tok/s
- Tesla V10032 GB · ~25 tok/s
- RTX 5090 Laptop24 GB · ~25 tok/s
- RTX Pro 4500 Blackwell32 GB · ~25 tok/s
- L40S48 GB · ~24 tok/s
- Radeon Pro W790048 GB · ~24 tok/s
- Apple M3 Ultra96 GB · ~23 tok/s
- Apple M1 Ultra64 GB · ~22 tok/s
- Apple M2 Ultra64 GB · ~22 tok/s
- RTX A500024 GB · ~21 tok/s
- RTX A550024 GB · ~21 tok/s
- RTX A600048 GB · ~21 tok/s
- A4048 GB · ~19 tok/s
- Titan RTX24 GB · ~18 tok/s
- Quadro RTX 600024 GB · ~18 tok/s
- Quadro RTX 800048 GB · ~18 tok/s
- RTX Pro 4000 Blackwell24 GB · ~18 tok/s
- Radeon AI Pro R970032 GB · ~18 tok/s
- A1024 GB · ~17 tok/s
- RTX 5000 Ada32 GB · ~16 tok/s
- Radeon Pro W780032 GB · ~16 tok/s
- Apple M4 Max36 GB · ~15 tok/s
- Radeon Pro W680032 GB · ~14 tok/s
- Apple M5 Max36 GB · ~13 tok/s
- Arc Pro B6024 GB · ~13 tok/s
- RTX 4500 Ada24 GB · ~12 tok/s
- Apple M1 Max32 GB · ~11 tok/s
- Apple M2 Max32 GB · ~11 tok/s
- Apple M3 Max36 GB · ~11 tok/s
Slow 15 cards
- Tesla P4024 GB · ~10 tok/s
- Apple M5 Pro48 GB · ~8 tok/s
- L424 GB · ~8 tok/s
- Tesla M4024 GB · ~8 tok/s
- Apple M4 Pro48 GB · ~8 tok/s
- NVIDIA DGX Spark128 GB · ~8 tok/s
- Ryzen AI Max+ 395 (Radeon 8060S)64 GB · ~7 tok/s
- Ryzen AI Max 390 (Radeon 8050S)32 GB · ~7 tok/s
- Jetson AGX Orin32 GB · ~6 tok/s
- Apple M1 Pro32 GB · ~6 tok/s
- Apple M2 Pro32 GB · ~6 tok/s
- Apple M532 GB · ~4 tok/s
- Apple M3 Pro36 GB · ~4 tok/s
- Apple M432 GB · ~3 tok/s
- Radeon 890M64 GB · ~3 tok/s
Cheapest build from our list that runs it well: Used workstation + 2× Tesla P100, from $750 used, at about 20 tok/s. See the builds.
Learn
Read more about Qwen3 32B
- A local coding assistant in VS Code: Continue, Cline and OllamaSet up a private AI coding assistant in VS Code with Ollama and Continue or Cline. Which model to pick for your card, and what to expect from it.
- AMD GPUs for local AI: ROCm, Vulkan and what works in practiceCan you run local LLMs on an AMD Radeon card? Which tools support it, ROCm vs Vulkan, the best AMD cards for AI, and the catches compared to NVIDIA.
- Building a quiet, always-on home AI serverHow to build a home AI server that's quiet enough to live with and cheap to leave on. Hardware picks, noise and power tips, and sharing it on your network.
- Buying a used RTX 3090 for AI: what to check and what to payWhy the RTX 3090 is still the default card for local AI, what a fair used price is, how to spot a bad one, and what to pair it with.
- Context length and the KV cache: why long chats use more VRAMWhy a model that fits can still run out of memory on a long document, how to work out the KV cache size, and three ways to shrink it.
- Does PCIe bandwidth matter for LLMs? x1 risers, x4 slots and x16When a narrow PCIe link slows a local model down and when it doesn't, from loading time to multi-GPU and CPU offload, and how to check your link.
- Fix CUDA out of memory errors in Ollama and llama.cppWhy local models run out of GPU memory, and the fixes in order - smaller quant, shorter context, a quantized KV cache, fewer parallel slots, partial offload.
- Free up disk space in Ollama: how to update and delete modelsSee how much space your Ollama models use, delete the ones you don't need, update the rest, and clean up after LM Studio and Hugging Face too.
- How much RAM do you need for local AI? (System RAM, not VRAM)How much system memory a local AI machine needs when the model is on the GPU, when it spills onto the CPU, on a Mac, and with no GPU at all.
- How much VRAM do you need to run an LLM locally?A simple rule for how much graphics memory a model needs, what fits in 8, 12, 16, 24 and 48 GB, and why context length adds more.
- Is running AI locally worth it? Local vs ChatGPT, honestlyWhat you gain and give up by running open models on your own hardware instead of paying for a cloud AI: privacy, cost, quality, and speed.
- Llama 3.3 70B hardware guide: what you need to run it locallyLlama 3.3 70B needs about 45 GB at 4-bit. The setups that run it, two 3090s, a 48 GB card or a big Mac, with estimated speeds and used prices.
- Local AI glossary: tokens, context, quants and GGUF in plain EnglishThe words you'll meet when running AI models locally, explained in a sentence or two each: tokens, context, VRAM, quantization, GGUF, MoE and more.
- Ollama on Linux with an NVIDIA GPU: drivers, setup and fixesInstall the NVIDIA driver, install Ollama as a service, confirm it's using the GPU, and fix the problems that send it to the CPU instead.
- Power supplies for multi-GPU AI rigs: how many watts you needHow to size a power supply for two or more GPUs, the cables each card needs, wall circuit limits, power capping, and what our builds draw under load.
- Prompt processing vs generation speed: prefill and decode explainedWhy a local model can take ages to start answering a long prompt and then write quickly, what limits each speed, and which hardware helps.
- Quantization explained: Q4_K_M vs Q8 vs FP16What the letters and numbers in model file names mean, how much quality you lose at each size, and which one to download.
- Qwen3 32B vs Llama 3.3 70B: is the bigger model worth it?Qwen3 32B fits one 24 GB card. Llama 3.3 70B needs about 45 GB. What you gain and lose with each, with estimated speeds and hardware costs.
- Qwen3 vs Gemma 3: which local model should you run?Qwen3 and Gemma 3 compared for local use. Reasoning, images, writing, licenses, memory and estimated speeds on 12 and 24 GB cards.
- RTX 4090 vs two RTX 3090s for local AI: which should you buy?One 24 GB RTX 4090 or two used 3090s with 48 GB? Speeds, what each can run, power, cost and which is right for chat, 70B models and image generation.
- Speculative decoding: make local LLMs faster with a draft modelHow speculative decoding uses a small draft model to speed up a big one with no quality loss, when it helps, and how to turn it on in llama.cpp.
- The best GPU for local AI under $1,000What to buy for local AI under $1,000 now that a used RTX 3090 costs more, how Tesla P100s and the RTX 3060 compare to new 16 GB cards and the RTX A4500, and what to buy for each kind of use.
- The best local LLM for 24 GB of VRAM (RTX 3090, 4090, 7900 XTX)What to run on a 24 GB card, the community's favorite size for local AI. Picks for speed, smarts, images and code, with estimated speeds.
- The best local LLM for 32 GB, 48 GB and more (multi-GPU included)What to run with 32 GB, 48 GB or 64 GB+ of GPU memory, from an RTX 5090 to two 3090s to a server full of P100s. Picks, estimated speeds and our rack data.
- The best local LLM for coding, by how much memory you haveWhich open models are worth using for code on 8, 12, 16, 24 GB and up, what they're good and bad at, and how to wire them into your editor.
- The best local LLM for reasoning and mathWhich open models reason best through math, logic and multi-step problems on 8 GB to 64 GB+, how thinking mode works, and why speed matters more than you'd expect.
- The best local model for agents and tool callingWhich open models call tools reliably on 12 GB to 64 GB+, how tool calling works with Ollama, and what to expect from local agents compared with cloud models.
- The RTX 5090 for local AI: what 32 GB and 1.8 TB/s get youHow fast the RTX 5090 runs local LLMs, what fits in 32 GB, what doesn't, the power you need, and when a used 3090 or two is the smarter buy.
- The Tesla P40 for local LLMs: 24 GB cheap, but is it fast enough?What the 24 GB Tesla P40 runs, how fast, how it compares to the P100, and the cooling, power and software catches before you buy one.
- Thinking models explained: reasoning mode, /think and effort levelsWhat reasoning mode does, when it's worth the wait, how to turn it on and off in Qwen3 and gpt-oss, and what it costs in speed and context.
- Tokens per second explained: what's a good speed for a local LLM?What tok/s means, how fast is fast enough for chat, coding and thinking models, and how to measure your own speed in Ollama.
- Two GPUs for local LLMs: layer split vs tensor parallel explainedWhat a second GPU gets you, why two cards usually run at one card's speed, when tensor parallel is faster, and what we measured on two-card rigs.
- Used workstations for local AI: Dell T7810, HP Z840 and similarWhy old Xeon workstations like the Dell T7810 and HP Z840 make cheap multi-GPU AI boxes, what to check before buying, and how to fit GPUs in them.
- vLLM vs Ollama: when to move to vLLM for a teamOllama is right for one person. vLLM is built for many users at once. When the switch is worth it, what hardware vLLM needs, and how to start it.
- Why memory bandwidth sets LLM speed (and how to estimate tokens per second)The one spec that predicts how fast a GPU runs a local model, the formula we use, and why a 2016 data-center card can beat a new gaming card.
Compare
Other models
- Qwen3 4BAbout 4 GB · Quick everyday chat and summaries
- Qwen3 8BAbout 7 GB · A solid all-rounder for chat and writing
- Gemma 3 12BAbout 10 GB · Reads images as well as text
- Qwen3 14BAbout 11 GB · Noticeably smarter at reasoning and code
- gpt-oss 20BAbout 16 GB · OpenAI's open model. Fast, strong reasoning
- Gemma 3 27BAbout 19 GB · Google's best open model for its size
- Qwen3 30B-A3BAbout 21 GB · 30B smarts at small-model speed
- Llama 3.3 70BAbout 45 GB · Close to the big cloud models
- gpt-oss 120BAbout 67 GB · OpenAI's largest open model
- Qwen3 235B-A22BAbout 144 GB · Alibaba's flagship open model, for multi-GPU rigs
- DeepSeek R1 671BAbout 406 GB · The full DeepSeek R1, for the biggest rigs