Saw a post about running hermes agent locally with gemma4 through ollama. zero api costs, unlimited tokens, full privacy. spent a weekend setting it up.

Install is straightforward. brew install ollama, pull gemma4:4eb (9.6gb, took about 2 hours), configure hermes to use local endpoint instead of deepseek api. it works, model responds, does basic tasks.

But the quality gap between local and cloud frontier models for agentic tasks is massive. not 10-20% worse, more like a different category.

Tested three things:

Simple file organization script: gemma4 handled it fine. 40 seconds vs 5 on cloud claude. acceptable.

Refactoring a react component with complex state: local model got the structure right but missed two edge cases cloud models catch consistently.

Multi step task planning: asked it to break down a feature with dependencies. output was generic, missed project context entirely. same task in verdent with cloud models gives me clarifying questions about my codebase and catches dependency conflicts. night and day.

Speed compounds too. 15-20 tps on m2 pro. for chat its fine. for agentic loops where the model iterates 5-6 times, latency adds up fast.

Where local actually shines: privacy sensitive review, offline dev, cheap first pass before sending complex stuff to cloud. my deepseek bill dropped from $30/month to $8 by offloading simple queries locally.

Worth setting up as a complement, not a replacement. the "token freedom" pitch is technically true but quality tradeoff is significant for anything beyond basics