Best model for 4090 as AI Coding Agent

Posted by Dry_Sheepherder5907@reddit | LocalLLaMA | View on Reddit | 36 comments

Good day. I am looking for best local model for coding agent. I might've missed something or some model which is not that widely used so I cam here for the help.

Currently I have following models I found useful in agentic coding via Google's turbo quant applied on llama.cpp:

GLM 4.7 Flash Q4_K_M ->
30B Nemotron 3 Q4_K_M -> 30B
Qwen3 Coder Next Q4_K_M -> 80B

I really was trying to get Qwen3 Coder Next to get a decent t/s for input and output as I thought it would be a killer but to my surprise...it sometimes makes so silly mistakes that I have to do lots of babysitting for agentic flow.

GLM 4.7 and Nemotron are the ones I really can't decide between, both have decent t/s for agentic coding and I use both to maxed context window.

The thing is that I feel there might be some model that just missed from my sight.

Any suggestions?

My Rig:
RTX 4090, 64GB 5600 MT/S ram

Thank you in advance

[-]

twanz18@reddit

For a single 4090 (24GB), Qwen3.5 35B quantized or Gemma4 27B fit well and are great for agentic coding. The key is pairing the model with a good agent framework. Aider and Continue both work nicely. If you want to run tasks while away from your desk, OpenACP lets you bridge your agent to Telegram so you can trigger from your phone. Full disclosure: I work on it.

[-]

twanz18@reddit

[-]

sleepingsysadmin@reddit

If I had a 4090, I'd be testing Qwen3.5 27b vs Gemma 4 31b.

Really arent other options.

[-]

Dry_Sheepherder5907@reddit (OP)

lol I literally right now thought about it and started testing :D

[-]

therapy-cat@reddit

I'm interested in how this goes

[-]

Dry_Sheepherder5907@reddit (OP)

I've tested 4 main models:
GLM 4.7, Gemma 4, Qwen 3.5 and Nemotron Nano 3...

They were given challenges to solve about algorithmic movement of character in war field in turn based approach. Nemotron Nano 3 with it's 30B param absolutely nailed it with proper solution.

Rest models always had some bugs which could be caused, even though Gemma 4 excelled against QWEN 3.5 in sense of solution.

QWEN 3.5 35B started thinking good, critical with edge cases and algorithm to use and started implementing none existing stuff.

Overall clear winner in that sense is Nemotron.

This is the first real example.

GLM 4.7 just kept overthinking and correcting itself until it gave working but not so good solution.

[-]

TheLastSpark@reddit

Can you also reply back with Qwen 3.5 27B? Should be much better than the 35B

[-]

Dry_Sheepherder5907@reddit (OP)

Here is the challenge they were given:

CHALLENGE: Escape the Monster — Grid Pathfinding in Java

A player and a monster start on opposite corners of a 10x10 grid.

Each turn: the player moves first, then the monster moves one BFS step toward the player.

The player wins by surviving 40 turns. The monster wins by landing on the player.

GRID RULES

- 10x10 grid, cells are (row, col), 0-indexed

- \~20% of cells are walls (impassable), generated with seed=42

- Player starts at (0,0), monster starts at (9,9)

- Valid moves: UP, DOWN, LEFT, RIGHT, STAY

- Moving into a wall or out of bounds counts as STAY

IMPLEMENT

- A PlayerStrategy interface with method: String chooseMove(boolean[][] grid, int playerR, int playerC, int monsterR, int monsterC, int n, int turnsLeft)

- A GameEngine that runs the game loop, moves the monster via BFS each turn, validates player moves, and prints the grid each turn as ASCII (. = empty, # = wall, P = player, M = monster)

- Print each turn: grid state, turn number, player pos, monster pos, manhattan distance

- Print final result: "ESCAPED after 40 turns" or "CAUGHT on turn X"

CONSTRAINTS

- Java 11+, single file, no external libraries

- Must compile: javac MonsterGrid.java

- Must run: java MonsterGrid

- Must finish in under 5 seconds

[-]

TheLastSpark@reddit

If you had to ballpark a score out of 10 for each model what would rate them as?

[-]

Dry_Sheepherder5907@reddit (OP)

IMO.

Qwen Coder 70B > 8/10 (Overall very good, could me improved in code sense)
Qwen 35B >7/10 , quite good, very useful for it's size
Qwen 27B > 6.5/10 6.5 BECAUSE of it's small size but bad speed, MOE of 35B is preferred, both giving almost identical result
Nemotron (incomplete) > 5/10 , I liked the process, thinking and way it wanted to solve the issue, shame it didn't
GLM (incomplete) > 4/10 , very pitty... I really love GLM but at least it was fighting
Gemma 4 (broken) > 0/10 -> total garbage ))))) just remove this model lol

[-]

TheLastSpark@reddit

If you dont mind revisiting Gemma 4? It seems like Llama cpp is just now getting around to fixing the support for it and others are saying it's really good. But I really appreciate your response!

[-]

Dry_Sheepherder5907@reddit (OP)

No worries. I tried with LM studio because llama didn’t support gemma arc. It was a mixed feeling TBH..it is not that stable in coding but for image processing, text processing it is simply KING . I stand clear on qwen, they seem to be best open source models for coding anyway

[-]

Dry_Sheepherder5907@reddit (OP)

so it turned out that qwen coder 70 B was better at complex tasks whilst my previously winner nemotron got 3rd place due to high and unnnercessary reasoining.

By the way today I was doing complex agentic coding and indeed I felt a lot that nemotron does unnecessary thinking whilst QWEN do not.

I believe still Qwen Coder 70B can be consideres as KING oh and yeah...I run them at full context with turbo quant so it is totally acceptable t/s on 4090 with 64gb of high speed ram

[-]

Dry_Sheepherder5907@reddit (OP)

Made a complete from scratch evaluation.

Ranking: Qwen Coder 70B > Qwen 35B ≈ Qwen 27B > Nemotron (incomplete) > GLM (incomplete) > Gemma 4 (broken)

Qwen Coder 70 B winning. I'll attach jsons and txt for reference

[-]

qwen_next_gguf_when@reddit

Qwen 3.5 27b q4.

[-]

Evgeny_19@reddit

Could you please elaborate a bit more? Unsloth or bartowski? Or something else maybe? Official settings: temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0, or some other adjustments perhaps?

My experience with qwen 3.5 27b q4 has been mixed so far. Sometimes I also see strange mistakes, even though my input is relatively simple.

Although my target GPU is a Radeon 7900 XTX, I don't think it matters in this case.

[-]

Dry_Sheepherder5907@reddit (OP)

thank you very much for the reply.

So you do think this model can exceel against my above mentioned? I see lots of debates aganst 80B model but nothing for sure .

[-]

misha1350@reddit

Qwen3.5 27B is a dense model, exactly what you need when you have a small amount of very fast memory. Being dense means that it will be slower, as it has to consider all 27B parameters instead of just a fraction of parameters, but if you have fast memory, this won't be a problem.

[-]

Dry_Sheepherder5907@reddit (OP)

Didn't have problem for models up to 30B without MOE tbh. T/s is generous and overall speed is quite good. Things get nasty when we speak >30B where MOE helps a bit

[-]

misha1350@reddit

The deal is that prior text-only models used to be fine, whereas nowadays these are multi-modal models, with fundamental changes to the architecture and parameter size, so Qwen3.5 27B and Gemma 4 31B are a little tougher to run (around 10-20% slower t/s than what you had before).

[-]

qwen_next_gguf_when@reddit

I used to use coder next. I have switched to 27b with opencode.

[-]

disgruntledempanada@reddit

Do you need to install MCP servers for Opencode or does it handle that automatically?

[-]

Dry_Sheepherder5907@reddit (OP)

nice, and you are indeed satisfied with it's coding ability?
Any other models you might suggest?

[-]

qwen_next_gguf_when@reddit

Try it and be happy. Trust me bro.

[-]

Dry_Sheepherder5907@reddit (OP)

thanks mate

[-]

gtrak@reddit

27b will surprise you

[-]

Far_Negotiation_7283@reddit

ur not really missing some secret model tbh, ur already using the same tier most 4090 setups end up on, the weird behaviour ur seeing isnt cuz of model choice its cuz agent loops amplify small mistakes

qwen coder next is strong for raw coding but yeah it drifts and makes dumb mistakes under pressure, nemotron feels more stable cuz its better at tool flow and step by step reasoning, glm sits somewhere in between, what worked better for me was splitting roles instead of chasing one perfect model, planner on nemotron or glm then code gen on qwen, spec first layers like Traycer help here cuz once u lock what “done” means the model matters way less otherwise they all start looping and u end up babysitting anyway

[-]

picosec@reddit

I've been testing Qwen 3.5 27B UD_4_K_XL and Gemma 4 31B UD_4_K_XL. Gemma 31B is a bit of a tighter fit for 24GB GPUs, I had to use "-np 1 and -fitt 512" with a 32K context to get all the layers on the GPU. Qwen 27B fits with a 64K (or larger context).

So far, I think Gemma 31B is producing somewhat better code (at least for C++) than Qwen 27B.

[-]

Impossible_Style_136@reddit

With a 4090, you have 24GB of high-speed VRAM. Pushing 80B models via heavy quantization (Q4) completely neuters the model's reasoning capabilities for complex coding tasks just to make it fit in memory.

You're better off running a dense 32B model (like Qwen 2.5 Coder 32B) at high precision (FP8/BF16) or waiting for stable ternary MoE models. The "silly mistakes" you're seeing in the 80B are quantization artifacts destroying the long-tail logic pathways.

[-]

Status_Record_1839@reddit

With an RTX 4090 (24GB VRAM), you have some good options beyond what you've listed:

**Devstral-Small 22B (Q4_K_M)** — purpose-built for agentic coding, trained specifically for multi-file, multi-step coding tasks. Fits easily in 24GB and performs very well on real-world coding agent benchmarks. This might be exactly what you're looking for — it's not heavily discussed but the coding agent performance is solid.

**Qwen3-32B (Q4_K_M, \~20GB)** — significantly better than Qwen3 Coder Next 80B for agentic use on your hardware because the higher t/s means the agent can iterate faster. The "silly mistakes" with big models are often context degradation at max context — a smaller model running at 2x speed often wins for agents.

**On your GLM vs Nemotron dilemma:** GLM 4.7 Flash handles tool calling more reliably in my experience, while Nemotron is stronger at pure code quality. For agentic flows where tool use matters more than generation quality, GLM usually wins.

**General tip:** For agentic coding, prioritize models with strong function calling / structured output over raw benchmark scores. That's why Qwen 3.5 27B Q4 beats the 80B in practice for agents on your rig.

[-]

misha1350@reddit

Horrible advice in every way. Get a grip, LLM

[-]

misha1350@reddit

You have to test Qwen3.5 27B and Gemma 4 31B. You may want to use Unsloth's UD IQ quants instead of regular Q4_K_M to take advantage of the imatrix quants that CUDA can utilize to save extra memory. That way you can force both very good quality and a very large context window. Also, consider vLLM.

[-]

Dry_Sheepherder5907@reddit (OP)

I use llama.cpp for total control and for now, nemotron simply beats the hell out of qwen 3.5 and gemma 4. Both in took calling awareness and pure code quality. Matter fact, Gemma 4 seems to be a bit better in coding quality against 3.5 qwen, but qwen 3.5 is a lot better at tool calling. Probably because gemma wasn't training for tool use?idk

[-]

BrightRestaurant5401@reddit

I I actually have no issue with Unloths version of Qwen3 Coder Next,
what kind of agentic workflow? I use cline in vscode?

[-]

Dry_Sheepherder5907@reddit (OP)

I currently conclude a small test for each model including qwen 3.5 with some challenge in coding for them to implement

[-]

Dry_Sheepherder5907@reddit (OP)

that is interesting because nemotron 30b had a lot better complex context following than qwen3 coder. I use MOE , full context for all models so no issues on context trimming and summarization. Overall it starts hallicunating more often than nemotron and that makes things get wore :/

I use in different envos, Kilo Code for Android Studio, VSCode and Kilo Code too, Continue, lots and lots more so I'd be sure envo is not the case here