Qwen3 30B-A3B
The best speed for its smarts on this list. It knows about as much as a 30B model but reads only 3B of weights per token, so it is fast even on older cards, if they have the memory to hold it.
ollama run qwen3:30bNeeds about 21 GB of graphics memory, or unified memory on a Mac. First time? Install Ollama, then paste that into a terminal.
Download19 GB
Memory to runAbout 21 GB
Read per token2 GB (mixture of experts)
LicenseApache 2.0
Context32K tokens, 128K with YaRN
ReleasedApril 2025
Good for
- Fast, capable chat on 24 GB cards
- Coding help at small-model speed
- Running on used data-center cards like the P100
Not the best pick for
- Cards under 24 GB: all 30B have to fit, even though few are used per token
57 GPUs and Macs
What runs Qwen3 30B-A3B
Estimated speed on each card at the smallest memory size that fits it, from memory bandwidth. As a mixture-of-experts model it reads only part of its weights per token, so it runs much faster than its size suggests.
Runs great 53 cards
- H200141 GB · ~840 tok/s
- H10080 GB · ~590 tok/s
- RTX 509032 GB · ~310 tok/s
- RTX Pro 6000 Blackwell96 GB · ~310 tok/s
- A10040 GB · ~270 tok/s
- RTX Pro 5000 Blackwell48 GB · ~240 tok/s
- Instinct MI10032 GB · ~220 tok/s
- Instinct MI5032 GB · ~180 tok/s
- RTX 3090 Ti24 GB · ~180 tok/s
- RTX 409024 GB · ~180 tok/s
- RTX 6000 Ada48 GB · ~170 tok/s
- RX 7900 XTX24 GB · ~170 tok/s
- RTX 309024 GB · ~160 tok/s
- A3024 GB · ~160 tok/s
- Tesla V10032 GB · ~160 tok/s
- RTX 5090 Laptop24 GB · ~160 tok/s
- RTX Pro 4500 Blackwell32 GB · ~160 tok/s
- L40S48 GB · ~150 tok/s
- Radeon Pro W790048 GB · ~150 tok/s
- Apple M3 Ultra96 GB · ~140 tok/s
- Apple M1 Ultra64 GB · ~140 tok/s
- Apple M2 Ultra64 GB · ~140 tok/s
- RTX A500024 GB · ~130 tok/s
- RTX A550024 GB · ~130 tok/s
- RTX A600048 GB · ~130 tok/s
- A4048 GB · ~120 tok/s
- Titan RTX24 GB · ~120 tok/s
- Quadro RTX 600024 GB · ~120 tok/s
- Quadro RTX 800048 GB · ~120 tok/s
- RTX Pro 4000 Blackwell24 GB · ~120 tok/s
- Radeon AI Pro R970032 GB · ~110 tok/s
- A1024 GB · ~110 tok/s
- RTX 5000 Ada32 GB · ~100 tok/s
- Radeon Pro W780032 GB · ~100 tok/s
- Apple M4 Max36 GB · ~96 tok/s
- Radeon Pro W680032 GB · ~90 tok/s
- Apple M5 Max36 GB · ~81 tok/s
- Arc Pro B6024 GB · ~80 tok/s
- RTX 4500 Ada24 GB · ~76 tok/s
- Apple M1 Max32 GB · ~70 tok/s
- Apple M2 Max32 GB · ~70 tok/s
- Apple M3 Max36 GB · ~70 tok/s
- Tesla P4024 GB · ~61 tok/s
- Apple M5 Pro48 GB · ~54 tok/s
- L424 GB · ~53 tok/s
- Tesla M4024 GB · ~50 tok/s
- Apple M4 Pro48 GB · ~48 tok/s
- NVIDIA DGX Spark128 GB · ~48 tok/s
- Ryzen AI Max+ 395 (Radeon 8060S)64 GB · ~45 tok/s
- Ryzen AI Max 390 (Radeon 8050S)32 GB · ~45 tok/s
- Jetson AGX Orin32 GB · ~36 tok/s
- Apple M1 Pro32 GB · ~35 tok/s
- Apple M2 Pro32 GB · ~35 tok/s
Usable 4 cards
Cheapest build from our list that runs it well: Used workstation + 2× Tesla P100, from $750 used, at about 130 tok/s. See the builds.
Learn
Read more about Qwen3 30B-A3B
- A local coding assistant in VS Code: Continue, Cline and OllamaSet up a private AI coding assistant in VS Code with Ollama and Continue or Cline. Which model to pick for your card, and what to expect from it.
- AMD GPUs for local AI: ROCm, Vulkan and what works in practiceCan you run local LLMs on an AMD Radeon card? Which tools support it, ROCm vs Vulkan, the best AMD cards for AI, and the catches compared to NVIDIA.
- Batch jobs with a local model: process a folder of files with OllamaRun a local model over hundreds or thousands of files with a short Python script and Ollama's API. Summarize, tag or extract, with no per-token bill.
- Building a quiet, always-on home AI serverHow to build a home AI server that's quiet enough to live with and cheap to leave on. Hardware picks, noise and power tips, and sharing it on your network.
- Buying a used RTX 3090 for AI: what to check and what to payWhy the RTX 3090 is still the default card for local AI, what a fair used price is, how to spot a bad one, and what to pair it with.
- CPU offloading explained: running a model bigger than your VRAMWhat happens when a model doesn't fit in GPU memory, how partial offload works in Ollama and llama.cpp, how slow it gets, and when it's worth it.
- GGUF vs safetensors vs MLX: which model file format do you need?What the three common model file formats are, which tools read each one, and which to download for an NVIDIA card, an older GPU, or a Mac.
- gpt-oss 20B vs Qwen3 30B-A3B: two fast MoE models comparedOpenAI's gpt-oss 20B and Qwen3 30B-A3B both run fast on home hardware. Which fits your card, which is smarter where, and estimated speeds.
- How to cool a passive Tesla card: shrouds, fans and airflowData-center GPUs like the Tesla P100 and P40 have no fans. How to cool them in a normal PC with a blower shroud, how to check temperatures, and what we run.
- How to judge a new local model in one eveningA new open model just dropped. A practical checklist to decide in a few hours whether it fits your hardware, works with your tools, and beats what you run now.
- How to read a model card: what matters for running it locallyParameters, active parameters, context, license, quant files and benchmarks. Which parts of a model card tell you if it will run on your hardware.
- Import a GGUF file into Ollama with a ModelfileRun any GGUF model in Ollama. Pull straight from Hugging Face, or write a three-line Modelfile for a local file, with templates, context and system prompts.
- llama.cpp server guide: run llama-server with the flags that matterInstall llama.cpp, download a model, and serve it with llama-server. The handful of flags that matter for GPU layers, context and parallel users.
- Local AI agents: tool calling, MCP and what works on small modelsHow agents and tool calling work with local models in Ollama, what MCP adds, which models handle tools reliably, and how to keep small models on track.
- Local AI for small business: a private assistant for your teamWhat a small business can do with AI running on its own hardware, what it costs, which build to buy, and how to share one machine with the whole team.
- Local AI glossary: tokens, context, quants and GGUF in plain EnglishThe words you'll meet when running AI models locally, explained in a sentence or two each: tokens, context, VRAM, quantization, GGUF, MoE and more.
- Mini PCs and AMD Strix Halo for local AI: unified memory on x86Can a mini PC run local LLMs? What ordinary mini PCs manage, why AMD's Strix Halo chips with up to 128 GB of shared memory are different, and the trade-offs.
- Mining cards for AI: do NVIDIA CMP cards run local LLMs?Can ex-mining NVIDIA CMP cards run local AI? What works, the limits to check before buying, and what we learned running them in our 90-GPU rack.
- Mixture of experts explained: why a 30B model can run like a 3BWhat MoE models are, what "active parameters" means, and why models like Qwen3 30B-A3B and gpt-oss are so fast on home hardware.
- Ollama commands cheat sheet: every command you'll actually useThe Ollama commands for running, downloading, listing, stopping and deleting models, plus the in-chat slash commands and the settings worth knowing.
- Ollama on a Mac: setup for Apple silicon, step by stepInstall Ollama on an M-series Mac, pick a model that fits your memory, check it's using the GPU, and change the settings that matter.
- Ollama on Linux with an NVIDIA GPU: drivers, setup and fixesInstall the NVIDIA driver, install Ollama as a service, confirm it's using the GPU, and fix the problems that send it to the CPU instead.
- Power and heat of local AI: watts and electricity cost per million tokensHow much power a local AI rig draws, how to work out your cost per million tokens, what our 90-GPU rack paid, and how to cut watts without losing speed.
- Prompt processing vs generation speed: prefill and decode explainedWhy a local model can take ages to start answering a long prompt and then write quickly, what limits each speed, and which hardware helps.
- Qwen3 vs Gemma 3: which local model should you run?Qwen3 and Gemma 3 compared for local use. Reasoning, images, writing, licenses, memory and estimated speeds on 12 and 24 GB cards.
- RTX 4090 vs two RTX 3090s for local AI: which should you buy?One 24 GB RTX 4090 or two used 3090s with 48 GB? Speeds, what each can run, power, cost and which is right for chat, 70B models and image generation.
- Serve a team on used GPUs: what our 90-card rack taught usHow to serve a local model to a team on cheap used GPUs. Our rack ran a billion tokens in 2.9 days on P100s, A4500s and mining cards. Builds and setup.
- The 90-GPU rack: what we learned running a billion tokens on used cardsA lab note from our rack of 90 used GPUs. What it's made of, what we measured, what it cost to power, and the lessons for anyone building with used cards.
- The best GPU for local AI under $1,000What to buy for local AI under $1,000 now that a used RTX 3090 costs more, how Tesla P100s and the RTX 3060 compare to new 16 GB cards and the RTX A4500, and what to buy for each kind of use.
- The best GPU for local AI under $300The best graphics cards for running local LLMs on a $300 budget, what each one runs, estimated speeds, and which to pick for easy setup or raw value.
- The best laptop for local AI: what to look for and what to avoidHow to pick a laptop for running local LLMs. Why laptop GPU names mislead, how much VRAM you need, estimated speeds, and when a MacBook is the better choice.
- The best local LLM for 24 GB of VRAM (RTX 3090, 4090, 7900 XTX)What to run on a 24 GB card, the community's favorite size for local AI. Picks for speed, smarts, images and code, with estimated speeds.
- The best local LLM for 32 GB, 48 GB and more (multi-GPU included)What to run with 32 GB, 48 GB or 64 GB+ of GPU memory, from an RTX 5090 to two 3090s to a server full of P100s. Picks, estimated speeds and our rack data.
- The best local LLM for a Mac with 64 GB, 96 GB or 128 GB+What to run on a Mac Studio or MacBook Pro with 64 GB or more of unified memory, from Qwen3 30B-A3B to Llama 3.3 70B and gpt-oss 120B, with estimated speeds.
- The best local model for agents and tool callingWhich open models call tools reliably on 12 GB to 64 GB+, how tool calling works with Ollama, and what to expect from local agents compared with cloud models.
- The RTX 5090 for local AI: what 32 GB and 1.8 TB/s get youHow fast the RTX 5090 runs local LLMs, what fits in 32 GB, what doesn't, the power you need, and when a used 3090 or two is the smarter buy.
- The Tesla P100 for local LLMs: 16 GB of HBM2 for under $100What we learned running a rack of 2016 data-center P100s for AI. Real speeds, what it takes to cool and power them, and the software catch.
- The Tesla P40 for local LLMs: 24 GB cheap, but is it fast enough?What the 24 GB Tesla P40 runs, how fast, how it compares to the P100, and the cooling, power and software catches before you buy one.
- Tokens per second explained: what's a good speed for a local LLM?What tok/s means, how fast is fast enough for chat, coding and thinking models, and how to measure your own speed in Ollama.
- Two GPUs for local LLMs: layer split vs tensor parallel explainedWhat a second GPU gets you, why two cards usually run at one card's speed, when tensor parallel is faster, and what we measured on two-card rigs.
- Use your local AI from anywhere: Tailscale or Cloudflare TunnelReach Ollama or Open WebUI on your home GPU from your phone or laptop anywhere, without opening router ports. Tailscale vs Cloudflare Tunnel, set up.
- Used workstations for local AI: Dell T7810, HP Z840 and similarWhy old Xeon workstations like the Dell T7810 and HP Z840 make cheap multi-GPU AI boxes, what to check before buying, and how to fit GPUs in them.
- vLLM vs Ollama: when to move to vLLM for a teamOllama is right for one person. vLLM is built for many users at once. When the switch is worth it, what hardware vLLM needs, and how to start it.
- Why memory bandwidth sets LLM speed (and how to estimate tokens per second)The one spec that predicts how fast a GPU runs a local model, the formula we use, and why a 2016 data-center card can beat a new gaming card.
Compare
Other models
- Qwen3 4BAbout 4 GB · Quick everyday chat and summaries
- Qwen3 8BAbout 7 GB · A solid all-rounder for chat and writing
- Gemma 3 12BAbout 10 GB · Reads images as well as text
- Qwen3 14BAbout 11 GB · Noticeably smarter at reasoning and code
- gpt-oss 20BAbout 16 GB · OpenAI's open model. Fast, strong reasoning
- Gemma 3 27BAbout 19 GB · Google's best open model for its size
- Qwen3 32BAbout 22 GB · Strong coding and long answers
- Llama 3.3 70BAbout 45 GB · Close to the big cloud models
- gpt-oss 120BAbout 67 GB · OpenAI's largest open model
- Qwen3 235B-A22BAbout 144 GB · Alibaba's flagship open model, for multi-GPU rigs
- DeepSeek R1 671BAbout 406 GB · The full DeepSeek R1, for the biggest rigs