Hi guys,

We’ve implemented a one-click app for OpenClaw with Local Models built in. It includes TurboQuant caching, a large context window, and proper tool calling. It runs on mid-range devices. Free and Open source.

The biggest challenge was enabling a local agentic model to run on average hardware like a Mac Mini or MacBook Air. Small models work well on these devices, but agents require more sophisticated models like QWEN or GLM. OpenClaw adds a large context to each request, which caused the MacBook Air to struggle with processing. This became possible with TurboQuant cache compression, even on 16gb memory.

We found llama.cpp TurboQuant implementation by Tom Turney. However, it didn’t work properly with agentic tool calling in many cases with QWEN, so we had to patch it. Even then, the model still struggled to start reliably. We decided to implement OpenClaw context caching—a kind of “warming-up” process. It takes a few minutes after the model starts, but after that, requests are processed smoothly on a MacBook Air.

Recently, Google announced the new reasoning model Gemma 4. We were interested in comparing it with QWEN 3.5 on a standard M4 machine. Honestly, we didn’t find a huge difference. Processing speeds are very similar, with QWEN being slightly faster. Both give around 10–15 tps, and reasoning performance is quite comparable.

Final takeaway: agents are now ready to run locally on average devices. Responses are still 2–3 times slower than powerful cloud models, and reasoning can’t yet match Anthropic models—especially for complex tasks or coding. However, for everyday tasks, especially background processes where speed isn’t critical, it works quite well. For a $600 Mac Mini, you get a 24/7 local agent that can pay for itself within a few months.

Is anyone else running agentic models locally on mid-range devices? Would love to hear about your experience!

Sources:

OpenClaw + Local Models setup. Gemma 4, QWEN 3.5
https://github.com/AtomicBot-ai/atomicbot
Compiled app: https://atomicbot.ai/

Llama CPP implementation with TurboQuant and proper tool-calling:
https://github.com/AtomicBot-ai/atomic-llama-cpp-turboquant