← HomeThe RunItLocal.io podcast

All things AI. Run it local.

Conversations about the newest models, the hardware that runs them, and doing it yourself, from the people who run a rack of 90 used GPUs. The first episode is on its way. Sign up and get it the day it drops, plus our lab notes.

  • New open models, tested on real hardware
  • Used GPUs and what to buy
  • Agents and local tools
  • Owning your data
  • What broke on the rack this week
From the lab

What we tested this week

Short write-ups from the rack: what came out, whether it runs on used hardware, and what broke. The longer story is on the show.

Lab note

A billion tokens on 90 used GPUs: 2.9 days

Qwen3.6-35B-A3B on Tesla P100s, RTX A4500s and ex-mining CMP cards. The 2016 P100s never throttled, and the A4500s won every speed test while sitting on their power cap.

2× P10053 tok/s
2× A4500110 tok/s
Power per 1M tokens6 to 8 cents
Read the full bench
Testing now

NVIDIA's Nemotron 3.5 Bolt squeezes a 74B model into 24 GB

A 74B mixture-of-experts model with 9B active, trained to run at 2-bit, with vision built in. Open weights, commercial use allowed.

MemoryAbout 24 GB
Cheapest build2× P100 or one 3090
Our testQueued on P100, A4500, CMP
CatchNVIDIA's own file needs Ada or Blackwell. Use the Unsloth GGUF on older cards.
Model files on Hugging Face
New to this? Start here

Run it local in six steps

From your first model on the computer you already own to a server that runs your whole team. Every step has guides in Learn.

  1. 01Run your first model

    Install Ollama, then run one command. Works on most laptops.

    ollama run qwen3:8b
  2. 02Know what matters

    Memory, quantization, and why one card is fast and another isn't.

  3. 03Pick your build

    Match a budget to the models you want. Use the build finder.

  4. 04Build it

    Parts, drivers, cooling, and the cables nobody tells you about.

  5. 05Serve it

    Ollama, then llama.cpp, then Open WebUI for a ChatGPT-style app.

  6. 06Make it useful

    Chat with your documents, run agents, add voice. Or try it now in the Studio.