Which one is the best open-source model for coding/SWE-tasks?

Posted by fazkan@reddit | LocalLLaMA | View on Reddit | 14 comments

The last time I checked llama-coder was the only one specialized. Also which benchmarks are relevant here. SWE-bench is the only one that I know of.

Reply to Post

14 Comments

[-]

fazkan@reddit (OP)

update, found out codeqwen claims to outperform all benchmarks [https://qwenlm.github.io/blog/qwen2.5-coder-family/](https://qwenlm.github.io/blog/qwen2.5-coder-family/)

[-]

nuclearbananana@reddit

I've found it performs quite poorly the moment you use a lesser known language/env/library

[-]

Brilliant-Day2748@reddit

R1. Been using it lately - handles everything from debugging to unit tests pretty well. Performance on SWE-bench is good, but also MultiPL-E and HumanEval for real-world performance metrics.

[-]

Deepseek R1 and Deepseek v3 are the top scoring open-weights models for coding benchmarks. If you need a smaller model that can fit in one GPU then qwen2.5-coder-32b is considered the best open weight. Aider has a number of different of benchmarks for their toolset: https://aider.chat/docs/leaderboards/

[-]

KillerX629@reddit

Is there an r1 version of quen coder 32 b?

[-]

spookperson@reddit

When I ran the aider benchmarks from their GitHub against regular qwen2.5-coder-32b and the r1-distilled-qwen2.5-32b (which is not the coder variant as far as I know) the coder version scored about 4% higher (this is the older, easier python-only benchmark) than the R1 distillation (and the regular coder model gets through test cases much much faster than the reasoning model that generates tons of tokens talking to itself).

[-]

KillerX629@reddit

I think which model to use depends on the complexity of the use case. For easier questions i always try to google/use online sources, including language models. Local models having reasoning tokens may allow you to give more elaborate questions, and at least approximate an answer... That's what I think anyways.

[-]

spookperson@reddit

Aider also has [an architect mode](https://aider.chat/2024/09/26/architect.html) and at least in the case of the full R1 it has been shown that using a reasoning LLM as architect and then a proficient coder LLM as an implementor can yield higher scores than any single LLM: [https://aider.chat/2025/01/24/r1-sonnet.html](https://aider.chat/2025/01/24/r1-sonnet.html)

[-]

Which one is the best open-source model for coding/SWE-tasks?

Reply to Post

14 Comments

fazkan@reddit (OP)

Key-Actuator2196@reddit

nuclearbananana@reddit

Brilliant-Day2748@reddit

spookperson@reddit

KillerX629@reddit

spookperson@reddit

KillerX629@reddit

spookperson@reddit

AdCreative8703@reddit

raqqun0101@reddit

AdCreative8703@reddit

random-tomato@reddit

Recoil42@reddit